Underlying the seemingly conventional surface of the ocean of C code that constitutes OpenAMQ swims a fast and furious host of abstractions, tools, and methods. We started to stop writing C code by hand in 1992, and we finished this work in February 2004. Only 0.21% of OpenAMQ's 310k lines of C code are written by hand. The rest are generated from models, using iMatix's Model Oriented Programming, or MOP. MOP turns high-level XML models into executable code. What models do we use? We use state machines, class systems, grammars, and meta-models. We quite routinely plug one end of a MOP tool into another, creating models that build models that build models, which eventually produces code that we can compile and run, and documentation we can print and share. It sounds impossible, and possibly it is, but it produces code with very few bugs, at a fraction of the conventional cost.
OpenAMQ sits on several technology layers, the largest of which is ASL, which stands for "Abstract Server Language".
ASL is a tool we built specifically to take AMQP-style grammars (AMQP is a MOP model, of course, since we were mainly responsible for designing the protocol and writing the standard) and turn them automatically into client and server frameworks. It happened thus, one summer afternoon when we decided to throw out the fast but inelegantly single-threaded OpenAMQ/0.8 and start on a brand new multithreaded version, based on a new, elegantly-abstracted protocol design. "OK, how are we going to build this server?" "Let's make a tool that generates it" "Great idea" Again, it sounds impossible, and it definitely caused some grief to our patient patron, but this was one of the most brilliant ideas in this project and today all AMQ implementations do some form of code generation.
Take arbitrary AMQP-style protocol written as an ASL model, push button, and after about five to ten seconds of grinding, ASL gives you a full working client, a server framework, large parts of your documentation, and a protocol scripting language. Which itself is a MOP tool, but you had guessed that already. ASL lets us take any past or future version of the AMQP protocol and generate a complete client API. This about this next time you read a protocol specification and wonder where to start.
SMT, the State Machine Threadlets system, is what makes OpenAMQ into a real server. It's a multithreading network I/O platform, supporting C and C++. We invented SMT in 1996 or so, based on our experience writing multithreaded servers for Digital VMS in 1991 (systems that still run today, 15 years later). We used it successfully as the basis for the Xitami web server. Today we're on the fifth major version of SMT. It builds single-threaded or multithreaded C applications that do their own virtual threading. We're thinking about the sixth generation.
SMT hides a lot, but not all, of the pain of writing multithreaded code in C, a language that has no language support for concurrency. C++ is no better, just more complex. We paid a heavy price - several months of delay - for going natively multithreaded. But the future runs on many cores, and we did not want to have to re-design OpenAMQ ever again.
The main problem with SMT is that contention is still handled using mutexes, which is slow. OpenAMQ is fast enough for most users but it should be much faster. One day we'll fix this in SMT and then all software built on it will magically move into top gear where it belongs.
Beneath SMT lie iPR and iCL, the MOP-driven class libraries that hold our universe together. If you want an example of how elegant and simple a nice MOP model can be, read ipr_stat.icl or ipr_token.icl.
The granddaddy of the MOP family is XNF, the XML Normal Form, which we invented in 1999 when we got tired of writing MOP tools by hand. XNF generates MOP tools, from grammars. XNF builds itself, of course, like all good compilers do.
Finally, the language that makes it all possible is GSL, the iMatix code generator scripting language, which began life as a perl script in 1995, and evolved into a real though slow language by 1998. About 45,000 lines of GSL turn the MOP tools from pretty but lifeless XML data into living tools. GSL is not, sadly, written using our modern MOP tools, but still rests on substrata of older iMatix tools - Libero, SFL, and a non-MOP version of SMT.
This web site, however, is built using MOP. We hope that settles any doubts you have about whether or not MOP produces readable output.