The ESB cookbook

Service Oriented Architecture

SOA is a shortcut for Service oriented architecture, well-known and popular term, especially in business environment. SOA architecture attempts to decompose a complex application into small units (usually called services), each of them providing some simple operation. Moreover, the services are spread over the network at different locations, running on different platforms and often written in different programming languages. Services themselves are quite useless without the customers. Customers, consumers of the services are usually called clients. They are not providing services, just using them.

In order for clients to connect to services and use them well-defined communication protocol should be used over a network. In ideal case all the clients and all the services should be using the same protocol. The base information carrier in such a environment is called message.

At this point everything looks simple - there is protocol, messages, services and clients that can communicate seamlessly. However, in the case of big amount of services and clients, communication problems are arising. Main problem is with the number of connections. In the case of 10 clients where each of them is using 10 services, 100 connections have to exist on a network and each client has to manage all 10 connections with respect to availability, security, queueing, load balancing etc. This is simply not acceptable and therefore a centralised highly-available component has to exist on a network which is "delivering" messages from source to destination(s). "Delivery" can have a lot of meanings and it usually refers to quite a complex algorithm implemented by so called message broker to form what is know as Enterprise Service Bus (ESB).

The simplest possible architecture consist of one service, one client and a broker which is managing the communication between them (Fig. 1a). If high-availability of service is required or there is a need for load balancing, multiple services instances can exist at the same time. Also, there can be different services running at a single moment (Fig. 1.b). Figure 1.c shows a complex architecture with different clients accessing different services.

soa.png

Fig. 1 Service oriented achitecture examples.

OpenAMQ RPC implementation

Now lets have a look at how simple RPC should be implemented. RPC stands for remote procedure call. It allows you to invoke methods across the network using AMQP as a communication protocol. RPC is basic building block of service oriented architecture. The procedure on the remote machine is known as a service in SOA terminology. For simplicity's sake we are going to use only a single service, however, to make things a bit more complex we'll allow several clients to access the service at the same time.

As the example we'll implement time-providing service. Client will send 'GET' request to the broker, the broker will forward client's request to the service itself. Service will process the request (retrieve current time) and it will send it back to the broker. Finally, the broker will forward the time back to the client.

We assume that there is OpenAMQ broker (amq_server) already running on your network. If needed, have a look at OpenAMQ documentation to download, build and run the broker.

Client side

Firstly, we have to devise a way to wire the broker (i.e. to define exchanges, queues and bindings) to get client's requests. For this purpose, direct exchange named 'services' is created witin the broker. It will handle all incoming client requests. (Note that receiving the request doesn't neccessarily mean that the broker is able to route it to the proper destination.)

Secondly, client has to create its own queue to receive results from remote services (response queue). As we've mentioned above, several clients can access the service at the same time. Consequently, each client has to have its own private queue with unique name. (In the following example we will refer to it as queue '#XYZ' for one client and '#XYQ' for the other client.)

Further, second exchange named responses has to be defined to handle the service responses. Afterwards it is necessary to "connect" client's response queue to responses exchange by defining binding with a specific routing key (client's response queue name, for example). Then it is the time to subscribe to responses (cosume) and send (publish) requests.

Requests have to be published with correct routing key and reply-to path. Reply-to path is stored in reply_to message content property. Setting reply_to to client's response queue name allows response messages from the service to get back to the client. Text above is summarised in the figure 2. Following steps should be done by the client:

  • create new AMQP connection & session
  • if it does not exist, create direct exchange services (the exchange exists if there is already a client or a service running)
  • if it does not exist, create direct exchange responses (the exchange exists if there is already a client running)
  • create uniquely named queue - unique name will be assigned by broker when queue_name in amq_client_session_queue_declare function call is an empty string; broker returns assigned name in session property queue; in our example we will refer to this queue as queue #XYZ
  • bind the queue #XYZ to responses exchange with the routing key '#XYZ' (that way we have unique routing key for responses from the service)
  • start consuming from client's responses queue #XYZ
  • send GET request with proper routing key and reply-to property (in our example routing key for current time service is 'cur_time'; reply_to text can be set with amq_content_basic_set_reply_to function call
  • wait for a reply and process it
  • destroy AMQP session & connection
soa_time_client.png

Fig. 2 OpenAMQ broker configuration and message paths on the client side (rk - routing key).

On figure 2 there is no service connected to the broker and therefore no requests can be processed. In this case client should be noticed that current-time-providing service is not available and inform the user about the fact. To implement this feature we have to set parameters mandatory and immediate when calling amq_client_session_basic_publish. The undeliverable request will be then returned by the broker and you can retrieve them using amq_client_session_returned function.

Service side

At this point we have to configure the broker to be able to receive and route messages to the service's queue. To do so we have to define (if it does not exist) services direct exchange. (Note that services exchange is defined by client as well, so it may already exist even if this is the first instance of the service that is being run.) Afterwards, we have to create queue which will be used to hold all the current time requests. We'll name it time_pool queue. The last thing to do is to define a binding between services exchange and time_pool queue with predefined routing key. In our implementation the key is cur_time, which means that all current time requests from client have to be sent to services exchange with cur_time routing key.

Above we've mentioned reply_to message content property. Service instance has to take this property into account when sending response back to the client (well, to the responses exchange actually). It sohuld set the routing key of the response message to the value it retrieved from the reply_to property of the request. Proper routing of response messages and setup of unique client's queues are done by the client. Text above can be summarised into figure 3. Service instance should perform following steps:

  • create new AMQP connection & session
  • if it does not exist, create direct exchange services
  • create time_pool queue
  • bind the time_pool queue to services exchange with routing key cur_time
  • start consuming messages from time_pool queue
  • do following steps in a loop:
    • wait for client request message
    • analyse message text (command)
    • send message with current time to responses exchange (routing key has to be equal to the client's response queue name - it can be retrieved from request's reply_to property)
  • destroy AMQP session & connection
soa_time.png

Fig. 3 OpenAMQ broker configuration and message paths (rk - routing key).

Client, Server & OpenAMQ broker

Following self explanatory sequence diagram (Fig. 4) details the communication between the client, server and OpenAMQ broker in our RPC implementation:

soa_time1.png

Fig. 4 OpenAMQ RPC communcation (rk - routing key, xchg - exchange).

The code

Following code examples implement current time service (server_time.c) and simple client (client.c) working with OpenAMQ broker:

Compile the examples:

$ c -l client.c
Compiling client...
Linking client...
$ c -l server_time.c
Compiling server_time...
Linking server_time...

Start OpenAMQ broker:

$ amq_server

Start service instance:

$ ./server_time localhost timeserver
localhost: OpenAMQ Server/1.2c4 - Linux - Production release
My ID: timeserver
Waiting:

Start the client:

$ ./client localhost client
localhost: OpenAMQ Server/1.2c4 - Linux - Production release
My ID: client
Enter routing key for 'GET' message [Q for exit]:

At this point client program should be ready to accept request (GET) message's routing key . Enter 'cur_time' to address current time service instance. Shortly afterwards current time provided by the service should appear on the standard output.

Enter routing key for 'GET' message [Q for exit]: cur_time
Send : rk 'cur_time', text 'GET: It is me client.', reply_to '#2'
Reply: 'I'm timeserver, time: 14:54:18.'

AMQP load balancing

AMQP protocol specifies that store and forward queue holds messages and distributes these between consumers on a round-robin basis. This is ideal for load balancing where there are several service instances providing the same service.

In our example it would mean that there are several service instances consuming from 'time_pool' queue. To do so just run more than one current time service provider (server_time). Following picture (Fig. 5) is a sequence diagram showing how three current time service instances interact with the client.

soa_time_HA.png

Fig. 5 AMQP load balancing (rk - routing key, xchg - exchange).

To test load balancing start three service instances from three different terminals server_time. The executable is already compiled, so there is no need to compile anew. OpenAMQ broker has to be running:

$ ./server_time localhost timeserver0
localhost: OpenAMQ Server/1.2c4 - Linux - Production release
My ID: timeserver0
Waiting:
$ ./server_time localhost timeserver1
localhost: OpenAMQ Server/1.2c4 - Linux - Production release
My ID: timeserver1
Waiting:
$ ./server_time localhost timeserver2
localhost: OpenAMQ Server/1.2c4 - Linux - Production release
My ID: timeserver2
Waiting:

Start the client (client is already compiled so no need to recompile) and enter 'cur_time' routing key several times. Following text should appear on standard output:

$ ./client localhost client
localhost: OpenAMQ Server/1.2c4 - Linux - Production release
My ID: client
Enter routing key for 'GET' message [Q for exit]: cur_time
Send : rk 'cur_time', text 'GET: It is me client.', reply_to '#3'
Reply: 'I'm timeserver0, time: 15:52:13.'
Enter routing key for 'GET' message [Q for exit]: cur_time
Send : rk 'cur_time', text 'GET: It is me client.', reply_to '#3'
Reply: 'I'm timeserver1, time: 15:52:17.'
Enter routing key for 'GET' message [Q for exit]: cur_time
Send : rk 'cur_time', text 'GET: It is me client.', reply_to '#3'
Reply: 'I'm timeserver2, time: 15:52:20.'
Enter routing key for 'GET' message [Q for exit]: cur_time
Send : rk 'cur_time', text 'GET: It is me client.', reply_to '#3'
Reply: 'I'm timeserver0, time: 15:52:55.'
Enter routing key for 'GET' message [Q for exit]: q

ID of service instance that processed the request can be seen in front of time. Note that consecutive requests are served by service instances in round-robin fashion.

Two different services

It is a trivial task to add new service type to our enterprise service bus. Exchange topology of 'services' and 'responses' allows us to do so even without restarting the broker. The only thing needed is that new service provider creates service queue and binds it to 'services' exchange with predefined routing key.

In our example we will implement current date service. Thus, current date request messages should have their own queue to be routed to by 'services' exchange. In our example it is named 'date_pool' and it has to be created by the first current date service instance. Predefined routing key 'cur_date' is used to bind 'services' exchange to 'date_pool' queue.

Figure 6 shows message paths for two client accessing two different services.

soa_date_time.png

Fig. 6 OpenAMQ RPC communication between two clients & two service types (rk - routing key).

Following code implements current date service (server_date.c). You can use existing client executable to invoke the service:

To test current date service you should compile it first:

$ c -l server_date.c
Compiling server_date...
Linking server_date...

After compiling, start server_date. OpenAMQ broker has to be running in advance:

$ ./server_date localhost dateserver
localhost: OpenAMQ Server/1.2c4 - Linux - Production release
My ID: dateserver
Waiting:

Start client afterwards (if you are following this tutorial step-by-step, it is already compiled, so no need to recompile it) and enter 'cur_date' routing key. Following text should appear on the standard output:

$ ./client localhost client
localhost: OpenAMQ Server/1.2c4 - Linux - Production release
My ID: client
Enter routing key for 'GET' message [Q for exit]: cur_date
Send : rk 'cur_date', text 'GET: It is me client.', reply_to '#0'
Reply: 'I'm dateserver, date: 11.12.2007.'
Enter routing key for 'GET' message [Q for exit]:

When current time service instance is running (server_time) entering 'cur_time' routing key to the client will cause remote invokation of current time service - as expected:

$ ./client localhost client
localhost: OpenAMQ Server/1.2c4 - Linux - Production release
My ID: client
Enter routing key for 'GET' message [Q for exit]: cur_date
Send : rk 'cur_date', text 'GET: It is me client.', reply_to '#0'
Reply: 'I'm dateserver, date: 11.12.2007.'
Enter routing key for 'GET' message [Q for exit]: cur_time
Send : rk 'cur_time', text 'GET: It is me client.', reply_to '#0'
Reply: 'I'm timeserver, time: 14:26:20.'
Enter routing key for 'GET' message [Q for exit]: q

Note that is possible (and desirable in practice) to use several service providers for current time or current date even with several clients.

Servicing services

Current date and time are examples of simple reusable services. Usually several simpler services are called by a "super" service. In so, client is interfacing only with the "super" service instance and therefore handling much less programming and communication complexity.

Following figure (Fig. 7) shows sequence diagram of such a setup where current date/time service instance is invoking current date service and current time service.

soa_serving_service.png

Fig. 7 Service instance invoking other services (rk - routing key, xchg - exchange).

Note the fact that date/time service has to create response queue and bind it to 'responses' queue in the same way as regular client does.

Following code implements current date/time service (server_date_time.c):

To test current date/time service compile it first:

$ c -l server_date_time.c
Compiling server_date_time...
Linking server_date_time...

After compiling, start server_date_time, server_date, server_time binaries from different terminals. OpenAMQ broker has to be running:

$ ./server_date_time localhost datetimeserver
localhost: OpenAMQ Server/1.2c4 - Linux - Production release
My ID: datetimeserver
Waiting:
$ ./server_date localhost dateserver
localhost: OpenAMQ Server/1.2c4 - Linux - Production release
My ID: dateserver
Waiting:
$ ./server_time localhost timeserver
localhost: OpenAMQ Server/1.2c4 - Linux - Production release
My ID: timeserver
Waititng:

Start client afterwards and enter 'cur_date_time' routing key. Following text should appear on the standard output:

$ ./client localhost client
localhost: OpenAMQ Server/1.2c4 - Linux - Production release
My ID: client
Enter routing key for 'GET' message [Q for exit]: cur_date_time
Send : rk 'cur_date_time', text 'GET: It is me client.', reply_to '#1'
Reply: 'I'm dateserver, date: 11.12.2007.I'm timeserver, time: 17:47:14.'
Enter routing key for 'GET' message [Q for exit]:

As you would expect using 'cur_time' or 'cur_date' routing keys still invokes current date and current time services.

Also note that load balancing works as expected even in case of the services invoking each other no matter how many different services you use, how many instances of each of them is running.

Comments

Add a New Comment

Edit | Files | Tags | Source | Print

rating: +1+x

Author

Pavol Malosek <moc.qmtsaf|kesolam#moc.qmtsaf|kesolam>

All tutorials

Performance Tests: This tutorial explains how to do OpenAMQ performance tests using 0MQ performance testing framework.

Broker federation: How to setup a geographically distributed federation of OpenAMQ brokers and how to define dataflows between individual geographical locations.

The ESB cookbook: How to implement your own Enterprise Service Bus, step by step, using OpenAMQ.

Publishing contents: How to publish contents at high speed without bizarre memory issues.

Tuning OpenAMQ for low latency: How to tune your operating system and hardware to get the lowest latency, i.e. the best response times, from your OpenAMQ broker.

The mandatory and immediate flags: How to use the 'mandatory' and 'immediate' flags on the Basic.Publish method.

Load balancing: How to use OpenAMQ to distribute work between multiple applications, a technique called "load balancing".

Content based routing: How to route messages based on their header properties. It is a fast and simple way to do 'content based routing' without needing to inspect entire messages. We explain the principles and provide worked examples.

Transient or durable subscriptions: How to make subscriptions that are transient (come and go with their consuming applications) or durable (stay around).

The AMQ model: How the AMQ model works: this is a basic backgrounder for anyone starting to use OpenAMQ in their applications.

Developing on Windows: How to build OpenAMQ client applications using MSVC on Windows

Handling Ctrl-C in applications: How to properly detect an interrupt (Ctrl-C) and shut-down gracefully in C/C++ WireAPI applications.