Content based routing

Problem

Content based routing means that the routing is done on the whole message body rather than on single property like routing key.

AMQP specification doesn't define full-blown content based routing. There is a good reason for that. Examining complete message - given it can be several megabytes long - would not be computationally viable. The broker would spend just too much time analysing messages and that would create a bottleneck in the network.

AMQP thus separates message into two parts called header and body. Body is the long part of the message and it will never be examined in the process of routing. Header is a short part of the message and it can contain arbitrary properties that can be examined by routing algorithm.

Explanation

You can think of message headers as of a simple table containing name-value pairs. Imagine that individual messages are describing news items. Then you can have message header filled like this:

name value
type article
category investigative
region France
author Jean Valjean
title Toulon galleys

With such a header attached to each message, we would expect clients to be able to subscribe for all the investigative articles from France or all the sport-related news from Russia or all the articles written by Jean Valjean etc.

To implement such a routing mechanism headers exchange type should be used. Each message should be accompanied by corresponding header, while each binding between the exchange and subscriber's queue (subscription) should contain a list of name-value pairs describing the messages client would like to receive.

So, for example, binding to get all the investigative articles would look like this:

name value
type article
category investigative

Binding to get all news items written by Jean Valjean would look like this:

name value
author Jean Valjean

To get all the sport digests from Russia, you would create following binding:

name value
type digest
category sport
region Russia

The routing of a message is shown on the following picture. The article matches author=Jean Valjean binding and is delivered to the queue B. It matches type=article & category=investigative binding as well and is delivered to the queue C. However, it doesn't match region=Spain binding and so it isn't delivered to the queue A:

cbr1.png

Have a look at the news publishing application implementation here:

Compile the examples:

$ c -l news_publisher.c
Compiling news_publisher...
Linking news_publisher...
$ c -l news_terminal.c
Compiling news_terminal...
Linking news_terminal...
$ c -l subscribe_to_news.c
Compiling subscribe_to_news...
Linking subscribe_to_news...

Start OpenAMQ broker:

$ amq_server

Start the news publishing application:

$ ./news_publisher localhost

Start the news terminal:

$ ./news_terminal localhost johnm

Now, subscribe for a topic you are interested in. The criteria are entered as name-value pairs, i.e. if you want to receive all the messages where type=article and category=science do it like this:

$ ./subscribe_to_news localhost johnm type article category science

At this point the news items should start appearing in your news terminal window.

Comments

Add a New Comment

Edit | Files | Tags | Source | Print

rating: +2+x

Author

Martin Sustrik <moc.xitami|kirtsus#moc.xitami|kirtsus>

All tutorials

Performance Tests: This tutorial explains how to do OpenAMQ performance tests using 0MQ performance testing framework.

Broker federation: How to setup a geographically distributed federation of OpenAMQ brokers and how to define dataflows between individual geographical locations.

The ESB cookbook: How to implement your own Enterprise Service Bus, step by step, using OpenAMQ.

Publishing contents: How to publish contents at high speed without bizarre memory issues.

Tuning OpenAMQ for low latency: How to tune your operating system and hardware to get the lowest latency, i.e. the best response times, from your OpenAMQ broker.

The mandatory and immediate flags: How to use the 'mandatory' and 'immediate' flags on the Basic.Publish method.

Load balancing: How to use OpenAMQ to distribute work between multiple applications, a technique called "load balancing".

Content based routing: How to route messages based on their header properties. It is a fast and simple way to do 'content based routing' without needing to inspect entire messages. We explain the principles and provide worked examples.

Transient or durable subscriptions: How to make subscriptions that are transient (come and go with their consuming applications) or durable (stay around).

The AMQ model: How the AMQ model works: this is a basic backgrounder for anyone starting to use OpenAMQ in their applications.

Developing on Windows: How to build OpenAMQ client applications using MSVC on Windows

Handling Ctrl-C in applications: How to properly detect an interrupt (Ctrl-C) and shut-down gracefully in C/C++ WireAPI applications.