Programming PAL

Writing OpenAMQ Test Scenarios

Overview

What is PAL?

We originally designed AMQP in 2004 using using the iMatix ASL framework for protocols. AMQP inherits its technical structure, up to and including connections and channels, and XML model, from ASL. ASL includes a tool for scripting protocols like AMQP, called PAL, the protocol automation language.

PAL is an XML language intended to make it simple to script AMQP scenarios, most importantly for writing test scripts. PAL can be used to develop certain types of AMQP application but this is not its intent.

PAL has two sets of commands:

  1. A set of commands that implement a simple scripting language.
  2. A set of commands that automate the protocol.

The scripting language functionality provides loops, conditional and repeated blocks, variables etc. The protocol method commands talk to the server and manipulate content.

ASL protocols have the useful property of being high-level. That is, the methods generally need little or no abstraction to be immediately obvious and useful to application developers. This makes it reasonable in PAL to simply expose the protocol methods directly to the scripting language. This strategy is helped by:

  • The use of clear and consistent names for methods and method properties.
  • The use of intelligent defaults for optional properties.

ASL protocols share the same connection and channel initiation and tear-down architecture. The methods used to do this - such as Connection.Tune - are hidden from the PAL developer. Specifically, we hide:

  • All Connection class methods.
  • The Channel.Open and Close methods.
  • All methods sent by the server and received by the client. Since PAL is for client-side automation, methods sent by the server cannot be scripted.

Architecture of PAL

PAL is a "codebot" written using the iMatix GSL language. The PAL codebot is an interpreter and code generator that understands the PAL language and AMQP, and generates code (ANSI C) for the script. The PAL codebot for AMQP is called amq_pal_gen.gsl and sits on the $IBASE/bin path. To run PAL you need to have GSL installed (this is built and installed when you build OpenAMQ).

To build a PAL program called example.pal you would issue these commands:

gsl example.pal
c -l example

The gsl command loads PAL program, which is an XML file, and looks for a property named 'script', that tells GSL which codebot to run. So the basic form of all PAL files is:

<?xml?>
<pal script = "amq_pal_gen">
    ...
</pal>

If the 'script' property is missing, you can specify it on the command line as follows (but this is usually extra work and we don't recommend it):

gsl -script:amq_pal_gen.gsl example.pal

The PAL code generation and compilation are encapsulated in a shell script called 'pal' which is provided by the OpenAMQ installation, for Unix and for Windows systems.

Sample PAL scripts

This section is in the form of a tutorial that explains each functionality with an example. Here is a sample PAL script:

<?xml?>
<pal script = "amq_pal_gen">
    <echo>
    Hello world!
    </echo>
</pal>

To compile the script use "pal" command. It will create an executable for you. You can run the executable straight away. Use "-h" option to get command line parameters for the executable.

$ pal hello.pal
2007/03/23 06:46:41: gsl/4 M: Generating hello.c...
Compiling hello...
Linking hello...
$ ./hello
06:46:56: Hello world!
$

Here is a script that demonstrates some of the standard PAL control commands:

<?xml?>
<pal script = "amq_pal_gen">
    <set name = "index" value = "0" />
    <repeat>
        <inc name = "index" />
        <if name = "index" value = "10">
            <break/>
        </if>
        <else>
            <echo>I can count up to $index</echo>
        </else>
    </repeat>
</pal>

And an equivalent, shorter version:

<?xml?>
<pal script = "amq_pal_gen">
    <repeat times = "10" counter = "index">
        <echo>I can count up to $index</echo>
    </repeat>
</pal>

To connect to a server and open a session we use the <session> control command. Here is a script that connects to a server and then echoes the connection properties:

<?xml?>
<pal script = "amq_pal_gen">
    <session>
        <echo>channel_max=$channel_max</echo>
        <echo>frame_max=$frame_max</echo>
        <echo>heartbeat=$heartbeat</echo>
        <echo>server_copyright=$server_copyright</echo>
        <echo>server_host=$server_host</echo>
        <echo>server_information=$server_information</echo>
        <echo>server_platform=$server_platform</echo>
        <echo>server_product=$server_product</echo>
        <echo>version_major=$version_major</echo>
        <echo>version_minor=$version_minor</echo>
    </session>
</pal>

Note that the script does not specify what server to talk to, nor the IP port. These and other options are passed on the command-line. For the standard C PAL implementation run the script executable with "-h" to get a list of all options.

Having established a session we can send methods to the server:

<?xml?>
<pal script = "amq_pal_gen">
    <session>
        <queue_declare queue = "myqueue" />
        <queue_bind queue = "myqueue" exchange = "myexchange" />
        <basic_content size = "64000" message_id = "id-0001" />
        <basic_publish exchange = "myexchange" routing_key = "myqueue" />
        <basic_get queue = "myqueue" />
        <basic_arrived>
            <echo>Message '$message_id' came back to us</echo>
        </basic_arrived>
        <empty>
            <echo>Message did not come back, this is bad!</echo>
        </empty>
    </session>
</pal>

PAL lets us define often-used method arguments at the 'session' level. These are then inherited to methods that don't explicity specify them. So we can rewrite the above script to make it shorter:

<?xml?>
<pal script = "amq_pal_gen">
    <session
        exchange = "myexchange"
        queue = "myqueue"
        >
        <exchange_declare class = "fanout" />
        <queue_declare />
        <queue_bind />
        <basic_content size = "64000" message_id = "id-0001" />
        <basic_publish routing_key = "myqueue" />
        <basic_get />
        <basic_arrived>
            <echo>Message '$message_id' came back to us</echo>
        </basic_arrived>
        <empty>
            <echo>Message did not come back, this is bad!</echo>
        </empty>
    </session>
</pal>

We can also create content bodies by reading data from test data files, or by running helper commands. See the 'read' and 'exec' options for the content commands. It's as simple as (for instance):

<basic_content exec = "perl -S myprog.pl" />

Scripts can be made flexible by passing arguments on the command line. Here is a simple example:

<?xml?>
<pal script = "amq_pal_gen">
    <set name = "number" value = "1234" cmdline = "N" />
    <set name = "string" value = "abcd" cmdline = "S" />
    <echo>Number=$number, string=$string</echo>
</pal>

Which we can run with the options -N and -S:

cmdline -N 9999 -S XXXX

Lastly let's look at macros, which are ways of collecting repetitive commands into groups to save time:

<?xml?>
<pal script = "amq_pal_gen">
    <macro name = "queue new">
        <exchange_declare exchange = "stdqueue" class = "fanout" />
        <queue_declare queue = "$queue" />
        <queue_bind queue = "$queue" exchange = "stdqueue" />
    </macro>
    <macro name = "send message">
        <basic_content size = "$size" message_id = "id-$random" />
        <basic_publish exchange = "stdqueue" routing_key = "$queue" />
    </macro>
    <session>
        <set name = "queue" value = "myqueue" />
        <invoke macro = "queue new" />
        <invoke macro = "send message">
            <set name = "size" value = "64000" />
        </invoke>
        <basic_get queue = "myqueue" />
        <basic_arrived>
            <echo>Message '$message_id' came back to us</echo>
        </basic_arrived>
        <empty>
            <echo>Message did not come back, this is bad!</echo>
        </empty>
    </session>
</pal>

If you use macros to any extent you'll want to look at the <include> command, described in the next section.

Scripting commands

Summary

These are the basic scripting commands, which can be nested to form scripts of any complexity:

invoke  - invoke a macro
server  - start a protocol server
timer   - start or reset performance timer
set     - define or modify a variable
inc     - increment a counter variable
dec     - decrement a counter variable
random  - set variable to randomised value
read    - get input from console, assign to variable
echo    - echo text to the console
abort   - echo text to the console and abort the script
assert  - assert some condition is true
repeat  - repeat a loop some number of times
while   - repeat a loop while some condition is true
break   - exit a loop
if      - execute commands if a condition is true
else    - execute commands if the previous if condition was false
elsif   - combined if and else
wait    - wait for the server to return data

Overall PAL script structure

The overall structure of the PAL script is:

<pal script = "amq_pal_gen">
    [ <include filename = "filename" /> ]...
    [ <macro name = "macroname">
        [ script command ]...
      </macro> ]...
    [ <session>
        [ script command ]...
      </session> ]...
</pal>

The include command

The <include> command copies the contents of another PAL file into the current script. It has this syntax:

<include filename = "scriptfile" />
  • The filename must include the file extension (usually .pal). The included file should not have a <pal> level but may contain macros or script commands.

The macro command

The <macro> command defines a block of commands that can be reused in as a single command in the script. It has this syntax:

<macro
    name = "macroname">
    [ script command ]...
</macro>
  • Macros have no effect until they are used through the 'invoke' command.

The session command

The <session> command defines a session:

<session
  [ restart = "0 | 1" ]
  [ server = "servername" ]
  [ failover = "seconds" ]
  [ direct = "0 | 1" ]
    >
    [ script command ]...
</session>
  • PAL may in future allow multiple sessions to be started in parallel, but for now sessions are executed serially.
  • If the restart option is 1, the session will restart in a new connection, whatever the state of previous sessions. If 0, the session will restart in the previous connection, if any.
  • The servername can be used to test multiple servers in a single script. This option is not used for general-purpose scripts.
  • If failover is set to an integer greater than zero, on a broken connection the script will pause for the specified number of seconds, and then try to reconnect to the same or alternate server. To use alternate servers, specify multiple server names in the 'server' attribute, seperated by spaces.
  • If direct is set to 1, the PAL script will use Direct Mode to publish and receive messages. See the OpenAMQ user guide for details on Direct Mode.

The invoke command

The <invoke> command expands a macro:

<invoke
    macro = "macroname"
    />
  • If the macro uses variables in commands, you can set these variables either before the <invoke> command, or inside it, using <set> commands.

The server command

The <server> commands starts or restarts a protocol server:

<server
    name = "servername"
  [ stdout = "filename" ]
  [ stderr = "filename" ]
  [ where = "directory" ]
    />
  • Do not specify a file extension (.exe) or your scripts will not be portable.
  • If a protocol server was already started, this command stops the server and then restarts it.
  • Only one protocol server can be started at a time.
  • The name value can include arbitrary server arguments but not shell redirection commands.
  • To redirect the server's output, use the stdout and stderr options.

The timer command

The <timer> commands shows or resets the script timer.

<timer
  [ action = "show | reset" ]
    />
  • The action is optional and defaults to "reset".

The set command

The <set> command defines a variable. Variables can be strings or integers. You can use variables in repeat, while, and if blocks, and as symbols for templating arguments and strings. Untyped variables are typed according to their value.

<set
    name = "variablename"
  [ value = "newvalue" ]
  [ type = "string | integer" ]
    cmdline = "char"
    />
  • The value is optional, and defaults to "".
  • If the value is purely numeric, the type will default to "integer", and if not the type will default to "string".
  • The cmdline option specifies a single character. Do not use one of the command-line options already used by the PAL implementation (see section at the end of this document).

The inc command

The <inc> command increments an integer variable:

<inc
    name = "variablename"
    />

The dec command

The <dec> command decrements an integer variable:

<dec
    name = "variablename"
    />
  • Decrementing a variable below zero is illegal and raises a fatal error. This is done to catch script errors - negative values are normally not meaningful in test scripts.

The random command

The <random> command sets a variable to a random value within a specified range:

<random
    name = "variablename"
  [ min = "minvalue" ]
    max = "maxvalue"
    />
  • The minimum is optional, and defaults to zero.

The read command

The <read> command accepts a line of input from the console and assigns this to a variable:

<read
    name = "variablename"
  [ prompt = "promptstring" ]
    />
  • The prompt is optional; if defined, this will be shown to the user (with no newline) before the console waits for input.

The echo command

The <echo> command echoes a line of text:

<echo [trace = "1|2|3"]>line of text</echo>
  • The text can use variables with the syntax: $variablename.
  • The trace level set using a command-line switch. Use the help option (-h) on the test program for details.

The assert command

The <assert> command tests a condition and aborts the script if the condition is false.

<assert
    name = "variablename"
  [ test = "eq | ne | gt | lt | ge | le" ]
  [ value = "testvalue" ]
    >[line of text]</assert>
  • The variablename is a script variable, or a connection or session property, or a standard PAL variable.
  • If the test and value are not specified, they default to "ne" and "0" or "" depending on the type of variable.
  • If just the test is not specified, it defaults to "eq".
  • If the assert statement includes a message, this is printed before an assertion failure.

The repeat command

The <repeat> command defines an iterative loop, which can run forever or for a specified number of times. The counter is global (do not use the same counter for two nested loops). To access the counter within the repeat loop, use $variablename.

<repeat
  [ counter = "variablename" ]
  [ times = "integer" ]
  [ progress = "integer" ]
    >
    [ script command ]...
</repeat>
  • If the times attribute is not specified, the loop will run forever or until the script does a <break/>.
  • The counter does not need to be previously defined. If no counter is specified, the repeat loop will create its own internal counter which cannot then be used as a symbolic value.
  • If the progress option is set to an integer N, then after every N passes through the loop, the test script will print a dot to the standard error output.

The while command

The <while> command defines a conditional loop, which runs so long as a specified condition is true:

<while
    name = "variablename"
  [ test = "eq | ne | gt | lt | ge | le" ]
  [ value = "testvalue" ]
  [ counter = "variablename" ]
  [ progress = "integer" ]
    >
    [ script command ]...
</while>
  • See the <assert> command for an explanation of the test and value properties.
  • If a counter is specified, this variable is automatically set to zero when the while loop starts and incremented each time the loop runs. You can access the counter variable after the while loop.
  • If the progress option is set to an integer N, then after every N passes through the loop, the test script will print a dot to the standard error output.

The break command

The <break> command exits the enveloping repeat or while loop and has this syntax:

<break/>

The if command

The <if> command defines a block that is executed if a specific condition is true:

<if
    name = "variablename"
  [ test = "eq | ne | gt | lt | ge | le" ]
  [ value = "testvalue" ]
    >
    [ script command ]...
</if>
  • See the <assert> command for an explanation of the test and value properties.

The else command

The <else> command defines a block that is executed if the previous <if> condition was false:

<else>
    [ script command ]...
</else>

The elsif command

The <elsif> command defines a block that is executed if the previous <if> condition was false and some further condition is true:

<elsif
    name = "variablename"
  [ test = "eq | ne | gt | lt | ge | le" ]
  [ value = "testvalue" ]
    >
    [ script command ]...
</elsif>
  • See the <assert> command for an explanation of the test and value properties.

The wait command

The <wait> command pauses the script for a number of milliseconds, or until content is received from the server:

<wait
  [ timeout = "milliseconds" ]
    />
  • Inside a session the default timeout is 'forever'. Outside a session, the default timeout is 'zero'.

Here is an example of using the <wait> command:

<?xml?>
<pal script = "amq_pal_gen">
    <set name = "index" value = "0" />
    <echo>Waiting without an active connection...</echo>
    <wait timeout = "1000" />
    <session>
        <echo>Waiting inside an active connection...</echo>
        <wait timeout = "1000" />
    </session>
    <echo>OK</echo>
</pal>

The abort command

The <abort> command echoes a line of text and halts the script.

<abort>line of text</abort>
  • The text can use variables with the syntax: $variablename.

The exit command

The <exit> command halts the script.

<exit [status = "value"] >
  • The default status value is 0.

PAL variables

PAL uses the convention '$name' to allow variable substitution. This is allowed in:

  • The body of <echo> and <abort> commands.
  • All attributes except variablenames.

PAL defines all connection and session properties as variables. These are the connection properties:

  • channel_max - proposed maximum channels.
  • class_id - failing method class.
  • frame_max - proposed maximum frame size.
  • heartbeat - desired heartbeat delay.
  • host - server to connect to.
  • known_hosts - list of known hosts.
  • method_id - failing method ID.
  • reply_code - reply code from server.
  • reply_text - localised reply text.
  • server_copyright - server copyright.
  • server_host - server hostname and port.
  • server_information - other server information.
  • server_platform - operating platform.
  • server_product - name of server implementation.
  • server_version - version of server.
  • version_major - protocol major version.
  • version_minor - protocol major version.

These are the session properties returned by various AMQP response methods:

  • active - start/stop content frames.
  • class_id - failing method class.
  • cluster_id - Cluster id.
  • consumer_count - number of consumers.
  • consumer_tag - consumer tag.
  • content_size - message content size.
  • delivery_tag - server-assigned delivery tag.
  • exchange - exchange name.
  • identifier - staging identifier.
  • message_count - number of messages in queue.
  • method_id - failing method ID.
  • queue - queue name.
  • redelivered - message is being redelivered.
  • reply_code - reply code from server.
  • reply_text - localised reply text.
  • routing_key - Message routing key.
  • staged_size - already staged amount.
  • ticket - access ticket granted by server.

Note that the standard ASL technique for returning values from protocol methods is via the session properties. Thus the variable 'message_count' holds the number of messages after a queue.browse request and a queue.browse-ok response.

PAL also defines these built-in variables:

  • $script - name of current PAL script.
  • $connection - 1 if the connection is alive, else 0.
  • $session - 1 if the session is alive, else 0.
  • $random - a random integer in the range 0..32767, when used as an insertion value, produces a 4-digit hex string.
  • $body_size - the body size of the last content to be created, arrived, or returned.

You must not name your own variables using any of the connection, session, or built-in names. PAL resolves a variable reference in this order:

  1. First, in-built variables.
  2. Content properties, inside an arrived/returned loop.
  3. Session properties.
  4. Connection properties.
  5. Script variables and counters.

If you want to use a dollar sign '$' in a string, but not do a variable replacement, put two backslashes after it, like this:

    <basic_publish routing_key = "$\\0.00" />

Here is a sample script that demonstrates various ways of using variables:

<?xml?>
<pal script = "amq_pal_gen">
    <set name = "expect_major"   value = "9" />
    <set name = "exchange_class" value = "fanout" />
    <set name = "queue"          value = "test-queue" />
    <set name = "exchange"       value = "test-exchange" />
    <set name = "times"          value = "100" />
    <set name = "size"           value = "64000" />
    <session
        queue    = "$queue"
        exchange = "$exchange"
        >
        <echo>Connected to $server_product/$server_version - $server_platform</echo>
        <assert name = "version_major" value = "$expect_major" />
        <exchange_declare class = "$exchange_class" />
        <queue_declare />
        <queue_bind />
        <repeat times = "$times" counter = "id">
          <basic_content size = "$size" content_type = "text/html" message_id = "id-$id" />
          <basic_publish routing_key = "$queue" />
        </repeat>
        <repeat>
            <basic_get />
            <basic_returned>
                <echo>Returned: $message_id</echo>
            </basic_returned>
            <basic_arrived>
                <inc name = "count" />
                <echo>Arrived: $message_id</echo>
            </basic_arrived>
            <empty>
                <break/>
            </empty>
        </repeat>
        <echo>Total number of messages exchanged: $count</echo>
    </session>
</pal>

General syntax for protocol methods

A protocol method command sends a protocol method to the server. If the method is a synchronous method, the script waits for a response from the server. If the method is asynchronous, the script continues without waiting. The basic syntax for protocol method commands is:

<class_method [properties...]>
    <field_table_name>
      [ <field
            name = "fieldname"
          [ value = "fieldvalue" ]
          [ type = "string | integer"  ("string") ]
            /> ]...
    </field_table_name>
</class_method>

Properties that are not specified take a default value, which is zero for numeric properties, FALSE for Boolean properties, and NULL for strings and field tables.

See the specification of each protocol command in the WireAPI documentation above, or the AMQP specifications themselves, for details.

These are the protocol method commands you can use in a PAL script:

Exchange.Declare:

<exchange_declare
    [ exchange = "exchange name" ]
    [ type = "exchange type" ]
    [ passive = "do not create exchange" ]
    [ auto_delete = "auto-delete when unused" ]
/>

Exchange.Delete:

<exchange_delete
    [ exchange = "exchange name" ]
/>

Queue.Declare:

<queue_declare
    [ queue = "queue name" ]
    [ passive = "do not create queue" ]
    [ exclusive = "request an exclusive queue" ]
    [ auto_delete = "auto-delete queue when unused" ]
/>

Queue.Bind:

<queue_bind
    [ queue = "queue name" ]
    [ exchange = "exchange name" ]
    [ routing_key = "message routing key" ]
    >
    [ <arguments .../> ]
/>

Queue.Unbind:

<queue_unbind
    [ queue = "..." ]
    [ exchange = "..." ]
    [ routing_key = "..." ]
    >
    [ <arguments .../> ]
</queue_unbind>

Queue.Purge:

<queue_purge
    [ queue = "queue name" ]
/>

Queue.Delete:

<queue_delete
    [ queue = "queue name" ]
/>

Basic.Consume:

<basic_consume
    [ queue = "queue name" ]
    [ consumer_tag = "consumer tag" ]
    [ no_local = "do not deliver own messages" ]
    [ no_ack = "auto-acknowledge messages" ]
    [ exclusive = "request exclusive access" ]
/>

Basic.Cancel:

<basic_cancel
    [ consumer_tag = "consumer tag" ]
/>

Basic.Publish:

<basic_publish
    [ exchange = "exchange name" ]
    [ routing_key = "Message routing key" ]
    [ mandatory = "indicate mandatory routing" ]
    [ immediate = "request immediate delivery" ]
/>

Basic.Get:

<basic_get
    [ queue = "queue name" ]
/>

Content manipulation

Basic content commands

PAL provides a command to create a basic content and set its properties. Only one basic content is ever active. If your script creates a content, that causes any previously created content to be destroyed. The syntax for the <basic_content command is:

<basic_content
  [ size = "bodysize"  ("1024") ]
  [ fill = "random | null | repeat"  ("random") ]
  [ read = "..." ]
  [ exec = "..." ]
  [ headers = "0|(1)" ]
  [ content_type = "propertyvalue" ]
  [ content_encoding = "propertyvalue" ]
  [ message_id = "propertyvalue" ]
    >
    [ <headers>
      [ <field
            name = "fieldname"
          [ value = "fieldvalue" ]
          [ type = "string | integer"  ("string") ]
            /> ]...
    </headers> ]
  [ content text ]
</basic_content>
  • The size attribute specifies the size in octets of the content buffer. Its default value is "1024".
  • The fill attribute specifies the fill mode. It can be "random", which sets the body to random data, or "null", which sets it to binary zeroes, or "repeat", which repeats the content text up to the specified size.
  • The body of the content item optionally provides a content text. If this is set, it's reformatted as a single line of text, and used as message body. This overrides the default fill ('random').
  • The read attribute specifies a file from which to read the content body. This is useful when you want to send test messages with a specific format.
  • The exec attribute specifies a command to run, so that the stdout of the command can be used as the content body. The command must be the name of an executable program, on the path, with arguments as desired. The program receives the current content body as stdin, much like a web server CGI program.
  • If the headers field is set to zero, the output of the executed program is not reparsed. If one, the output is reparsed to collect message properties and headers as follows: each line specifies a header field name, followed by ":", followed by a space and a value. Field names starting with "x-table-" are stored as-is (minus the x-table- prefix) in a field table with that name. Other fields must match known content properties. Hyphens are allowed in field names, and field names are case-insensitive. The headers are ended with a blank line. Parsed headers create a CGI-like interface for calling programs.
  • For each content property defined in the protocol (except field tables) PAL defines an attribute for the content command.
  • For field tables, PAL defines a child entity with the same name, e.g. 'headers'. Field tables are then constructed from one or more <field> definitions.
  • After a content command, the script can access the content body size as a variables ($body_size in expressions, or body_size in assertions and conditions).

Processing arrived contents

The <basic_arrived> command lets your PAL program process incoming messages delivered by the server. This command acts as a loop, and repeats for each arrived content at the moment it is invoked.

<basic_arrived
  [ counter = "variablename" ]
    >
    [ script command ]...
</basic_arrived>
<empty>
    [ script command ]...
</empty>
  • If a counter is specified, this variable is automatically set to zero when the loop starts and incremented each time the loop runs. You can access the counter variable after the loop.
  • If there was no arrived content, the script executes the following <empty> command, if any.

You can use these variables within an arrived loop:

  • $body_text - content body as printable text.
  • $body_size - the body size of the content.
  • $exchange - the exchange to which the content was published.
  • $routing_key - the original routing_key specified by the publisher.
  • $producer_id - the producer id of the publishing connection.
  • $content_type - MIME content type.
  • $content_encoding - MIME content encoding.
  • $delivery_mode - non-persistent or persistent.
  • $priority - message priority, 0 to 9.
  • $correlation_id - application correlation identifier
  • $reply_to - the destination to reply to.
  • $expiration - expiration specification.
  • $message_id - the application message identifier.
  • $timestamp - message timestamp.
  • $type - message type name.
  • $user_id - creating user id.
  • $app_id - creating application id.
  • $sender_id - the sending application id.

Additionally, if the content contains header fields (in the 'headers' table), these are made available as:

  • $header-fieldname - where 'fieldname' is the name of each field.

Processing returned content

We process returned content in a similar way to arrived content:

<basic_returned
  [ counter = "variablename" ]
    >
    [ script command ]...
</basic_returned>
<empty>
    [ script command ]...
</empty>
  • If a counter is specified, this variable is automatically set to zero when the loop starts and incremented each time the loop runs. You can access the counter variable after the loop.
  • If there was no returned content, the script executes the following <empty> command, if any.

Synchronous content processing

PAL does not provide any asynchronous content processing. The script runs as a single-threaded procedure from start to end. Content will arrive when the script is busy, i.e. during any command that talks to the server. To process content after such commands, use the 'arrived' commands. To process content while not doing such commands, use <wait/> and then use the arrived command.

Nowait methods

AMQP/0.9 has a "nowait" option on some methods that turn normally synchronous methods into asynchronous ones. This is important for applications that do large volumes of wiring (queue creation, etc.) since it can dramatically cut the start-up time.

WireAPI implements nowait as a second set of methods:

  • <exchange_declare_nowait … />
  • <exchange_delete_nowait … />
  • <queue_declare_nowait … />
  • <queue_bind_nowait … />
  • <queue_purge_nowait … />
  • <queue_delete_nowait … />
  • <queue_unbind_nowait … />
  • <basic_consume_nowait … />
  • <basic_cancel_nowait … />

Each of these methods takes the same arguments as their synchronous versions.

To use the nowait functionality on automatically-named queues (private exclusive queues), you need to use the default queue functionality, i.e:

  • <queue_declare/> with no name (creates a name automatically)
  • <queue_bind/> with no name (uses the last created queue name)
  • <queue_consume/> with no name (uses the last used queue name)

Error handling

Most PAL errors cause the script to stop. The one exception is a connection failure (meaning the server disappered unexpectedly): if the session failover property is set, the script will try to failover to a backup server.

PAL implementations

ANSI C implementation

The standard C implementation creates a command-line program that accepts these arguments:

Syntax: program [options...]
Options:
  -s server        Server:port to connect to (localhost)
  -t level         Set trace level (default = 0)
                   0=none, 1=low, 2=medium, 3=high
  -r count         Repeat script count times, 0=forever (1)
  -c               Clock the script (0)
  -a               Animate: show script commands
  -e               Execute in single-step mode (0)
  -q               Quiet mode: no messages
  -v               Show version information
  -h               Show summary of command-line options
The order of arguments is not important. Switches and filenames
are case sensitive. See documentation for detailed information.

Performance measurements

The -c option clocks the script and produces performance measurement output. Here is an example of a simple stress test script:

<?xml?>
<pal
    name = "stress"
    script = "amq_pal_gen"
    >
    This script sends a large number of messages to a queue and then
    reads them back.  The number of messages can be specified on the
    command-line.
    <session exchange = "myexchange" queue = "myqueue" >
        <set name = "number-of-messages" cmdline = "N" value = "1000" />
        <set name = "message-size" cmdline = "S" value = "1000" />
        <exchange_declare type = "fanout" />
        <queue_declare />
        <queue_bind />
        <repeat times = "$number-of-messages" counter = "index" progress = "100">
            <basic_content size = "$message-size" message_id = "id-$index" />
            <basic_publish routing_key = "myqueue" />
        </repeat>
        <while name = "arrived" test = "lt" value = "$number-of-messages" progress = "100">
            <basic_get />
            <basic_arrived>
                <inc name = "arrived" />
            </basic_arrived>
        </while>
    </session>
</pal>

Which produces this output (the figures are obviously just an example):

. . .
16:41:26: I: elapsed time:781 msecs
16:41:26: I: outgoing messages:1000 (976 Kbytes)
16:41:26: I: incoming messages:1000 (976 Kbytes)
16:41:26: I: total messages:2000 (1952 Kbytes) average:2560/sec (2499 Kbytes/sec)
16:41:26: I: message latency min=280 max=410 mean=331 dev=37 msecs

Other implementations

There are no other implementations of PAL at present.

PAL Internals

Terminology

ASL
Abstract Server Language - an iMatix technology for defining protocols and building clients and servers that implement such protocols. ASL is a language, a set of protocol standards, and a set of tools. ASL is itself constructed using XNF.
XNF
XML Normal Form - an iMatix technology for defining XML grammars and building code generators that implement such grammars. XNF is a language, and a set of tools. XNF is constructed using XNF. The key point of XNF is to generate a full validating and denormalising parser and to couple this with hand-written back-end code generators (aka "targets").
GSL
Generator Scripting Lanuage - an iMatix technology used to build code generators. GSL is a scripting language designed to work with tree-structured data (usually XML) and inject this data into a template-driven code generation process. All iMatix code generators are written in GSL, and most are complex enough to benefit from being constructed using XNF.
PAL
Protocol Automation Language - an iMatix technology for writing scripts to automate the client half of a discussion carried out in a protocol built with ASL. PAL is a add-on to ASL.

ASL refresher

iMatix ASL - the Abstract Server Language - is a protocol and software construction toolkit we built in order to help develop OpenAMQ. The concept of ASL is loosely based on the concept of formal grammars such as ASN.1, but in typical iMatix fashion is rather more aggressive in aiming to generate high-quality code and documentation.

ASL is a formal language, a grammar. It does not define the low-level wire-protocol but rather the higher-level language that a protocol carries. We describe this language in terms of "methods", grouped into "classes". The specific technology used to carry these methods depends on the specific implementation. Thus, ASL is abstract.

ASL is an extensible framework - a single ASL grammar can be fed to any number of different back-end "targets", each generating some different output. Today we have targets for:

  • A C implementation of client and server layers.
  • A Java implementation of client layers.
  • Documentation in iMatix gurudoc format.

This diagram shows the overall ASL code generation process:

 .----------------.
 |  ASL grammar   |         ASL grammar written as XML files, using
 | for protocol P |         inheritence and other XNF techniques.
 `----------------'
         :
+------------------+
|  ASL front-end   |        GSL parser, generated from asl.xnf.
|      parser      |        using the iMatix XNF toolkit.
+------------------+
         :
 .----------------.
 |  Denormalised, |         Data structure held in memory.
 | validated tree |
 `----------------'
         :
+------------------+
|     Specific     |        GSL code generator, written by hand.
|  back-end target |
+------------------+
         :
 .----------------.
 | Generated text |         Documentation or code as desired.
 |    outputs     |
 `----------------'

We can summarise the approach that drives ASL:

  1. Define an abstract and high-level syntax capable of supporting the full functionality required by all possible protocols in our domain.
  2. Implement the code generators for this abstract syntax.
  3. Use the syntax to define our full protocol.

Note the major benefit of using ASL: writing a large and rich protocol is cheap, as 100% of the protocol support both at the client and server side is generated, leaving only the functional implementation of the protocol methods as work for the programmer.

Large-scale Testing

By making it cheap to design and implement functionality on top of a standard technical base, ASL also encourages large and explicit protocols. While a hand-built protocol might use a single method for several purposes, ASL would encourage the definition of several individual methods. This clarity is pleasant for the application developer, but it means that testing must also happen on a large scale.

This - cheap large-scale testing - is the problem that PAL solves.

The PAL Architecture

These diagrams show how PAL fits into the toolchain. First, the process of constructing a PAL generator for a particular protocol, 'P':

      .----------------.
      |  ASL grammar   |         ASL grammar written as XML files, using
      |    "p.asl"     |         inheritence and other XNF techniques.
      `----------------'
              :
     +------------------+
     |  ASL front-end   |        GSL parser, generated from asl.xnf.
     |      parser      |        using the iMatix XNF toolkit.
     |  "asl_gen.gsl"   |
     +------------------+
              :
      .----------------.
      |  Denormalised, |         Data structure held in memory.
      | validated tree |
      `----------------'
              :
     +------------------+
     |       PAL        |        GSL code generator, hand-made.
     |  back-end target |
     |   "asl_pal.gsl"  |
     +------------------+
        :           :
+-----------+   +-----------+
|  PAL XNF  |   |  PAL stdc |    GSL code generators, hand-made
|  driver   |   |   driver  |
+-----------+   +-----------+
      :               :
.-----------.         :
|   PAL/P   |         :          XNF grammar of PAL language
|  grammar  |         :          specifically for protocol 'P'
`-----------'         :
      :               :
.-----------.   .-----------.
| front-end |   |   stdc    |    Components of a PAL generator
| for PAL/P |   |  target   |    specifically for protocol 'P'
`-----------'   `-----------'

To illustrate, these are the commands that would be used to perform the above chain, for a protocol named 'P':

gsl -target:pal p.asl
gsl p_pal.xnf

Second, the process of turning a PAL script into executable code:

 .----------------.
 |  PAL/P script  |         PAL script written as XML files, using
 |  "example.pal" |         include and other XNF techniques.
 `----------------'
         :
+------------------+
|  PAL/P front-end |        GSL parser, generated from p.asl
|      parser      |        (see previous diagram).
| "p_pal_gen.gsl"  |
+------------------+
         :
 .----------------.
 |  Denormalised, |         Data structure held in memory.
 | validated tree |
 `----------------'
         :
+------------------+
|       stdc       |        GSL code generator, generated from
|  back-end target |        p.asl (see previous diagram).
| "p_pal_stdc.gsl" |
+------------------+
         :
 .----------------.
 |  Source code   |         Generated C test program.
 |  "example.c"   |
 `----------------'

To illustrate, these are the commands that would be used to perform the above chain, for a script called "example":

gsl example.pal

The resulting test program is compiled and linked as usual.

Comments

Add a New Comment

Edit | Files | Tags | Source | Print

rating: +2+x

Author

iMatix Corporation

Installing and using OpenAMQ

Introduction to OpenAMQ: This document is an introduction to the concept of business messaging in general, and to OpenAMQ in particular. It is intended for new OpenAMQ users who wish to understand the problems that OpenAMQ solves, and how OpenAMQ can be useful in software applications.

Basic use of OpenAMQ: This document explains how to get OpenAMQ running on your system. It explains how to download the software, how to unpack and build it (if you are using a source package), and how to run basic tests on the resulting software.

Advanced use of OpenAMQ: This guide is for people who need to configure and manage OpenAMQ servers. We explain how to configure and tune an OpenAMQ server, covering these topics: logging, monitoring, high-availability failover, and joining OpenAMQ servers into wide-area federations.

Writing applications

Programming WireAPI: This is the main guide for developers who wish to use OpenAMQ in their applications. We describe WireAPI, the C/C++ API that OpenAMQ provides for accessing AMQP. Expert WireAPI users may wish to read the iMatix iCL guide, but this document is otherwise self-complete.

Programming PAL: This guide is for OpenAMQ developers who need a quick way to write test cases and simple scenarios. We explain the PAL language, an XML scripting tool that gives you a fast way to construct AMQP applications to test routing models, performance and stability tests, and other test cases.

Programming the Console: This document explains how to write applications that automate management of OpenAMQ servers via console automation. The OpenAMQ console automation architecture offers developers different ways of accessing the functionality of the console API and integrating it with their own preferred tools and management facilities.

Technical library

Developer's Guide to ASL: This is a technical guide for protocol developers who wish to use the iMatix ASL framework for the development of connected client-server protocols. ASL is a generic framework that uses a protocol modeling language to construct the whole infrastructure for a given protocol. ASL was built primarily to support AMQP.

Developer's Guide to iCL: This is a technical guide for developers who wish to understand how the iMatix iCL framework works. iCL is a class-oriented modelling language for C applications and is one of the basic frameworks used in iMatix applications such as OpenAMQ.

Developer's Guide to MOP: This is a technical guide for developers who wish to understand how the iMatix code generation frameworks are constructed. We explain the principles of model oriented programming, and the basics of code generation using the iMatix GSL language. This provides essential basic knowledge for anyone intending to modify the OpenAMQ software.

Developer's Guide to SMT: This is a technical guide for developers who wish to understand how the iMatix SMT framework works. To use this guide the reader should be familiar with the iMatix iCL framework, and the iMatix Model Oriented Programming principles.

RFCs

The CML Request for Comments: We describe a generic technique for managing AMQP servers from a remote client application. This technique consists of a standard transport mechanism built over AMQP, and a standard XML language used to exchange information between a management component built-in to the server, and a management application. This is a request for comments, it is not a standard.