The Aust Gate

A recent work project has been working with messaging queues and exploring how they work. One of the issues has been how to get systems running different systems to talk to each other. One system uses ActiveMQ and another runs RabbitMQ which has led to some interest in how to effectively bridge the systems into some sort of federation.

I’ve been using messaging to link distributed databases together through some homegrown middle-ware and services. In many cases the messages tends to be a subscription to a published message, a simple work queue in essence.

One issue is that Rabbit does not deal with messages in Java Messaging  System (JMS) and ActiveMQ does not deal with Advanced Message Queuing Protocol (AMQP). Fortunately Rabbit does have a Streaming Text Oriented Messaging Protocol (STOMP) connector which can be used in bridging systems together in conjunction with the AMQP Client plugin. (At one point, adding plugins to Rabbit was a pain but with the 2.70 release, it is a question of running a simple command: rabbitmq-plugins enable <plugin-name>. However I fear I may be digressing.) ActiveMQ allows STOMP as one of the enabled protocols on startup if defined as a transport connector in the XML.

As an exercise (a conversation that I had during drafting this post has meant that the  RabbitMQ servers may be taken out of commission on the current project but are better for another part of the project), I have set up a RabbitMQ service listening to and serving STOMP on port 61613 (the default) and with ActiveMQ listening to and serving STOMP from port 61618. Since I am using the same box I cannot get the two services listening to the same port. I have tried on a virtual machines to get Rabbit on one instance of Ubuntu to listen to ActiveMQ on a different box serving on port 61618 but got a badaddress error – which given this afternoon’s noodling around with Rabbit and ActiveMQ makes sense given that the RabbitMQ plugin appears to be producing and consuming. I found this frustrating since I could get the PHP STOMP library which Fusesource produce to listen to the port in the initial experiments.

My original idea was to see if I could get a message sent to Rabbit from Active. Having seen the direct connection fail, I had a look at ActiveMQ’s networking section to see if I could get alter the XML configuration to include a RabbitMQ endpoint as it suggests that this should be possible. A quick test this afternoon was not exactly successful since the broker would not start if I added the networkconnection section to the default XML. The documents would suggest that it would listen but when I got the default listening to port 61613, it appears to load but the message is not passed through from RabbitMQ to ActiveMQ. When I looked at the ActiveMQ log, I saw the error Wire format negotiation timeout: peer did not send his wire format which is a fatal error. That would suggest that the broker will do this for JMS but not STOMP but I could be wrong on that.

Taking a queue from a brief discussion of JMS to JMS bridging, a STOMP server in Python bridging a STOM consumer to a database, I began writing a bridge which listens to the RabbitMQ endpoint and then publishes to a queue on ActiveMQ (or vice versa if the configuration is switched around) and the JMS to JMS bridge ActiveMQ documentation, I wrote a light weight bridge in PHP which connects the STOMP endpoints and publishes to each queue. Creating a simple consumer/producer, since I would expect any required passing between such producers to be a direct message rather than a fan-out message or topic, I managed to get the message passed across in under a second but I need to explore the amount of microseconds. The quick test that I ran suggests that there are 0.0087219 microseconds between producing a RabbitMQ message and consuming in ActiveMQ. I need to run this several times and admittedly the performance may not be desired in financial or similar high performance systems but then would that use be trying to bridge systems rather than federating like systems instead.

The bridge could also be used to transform the messages between protocols. As I had at one point, I had one system using RabbitMQ and AMQP which ActiveMQ does not support yet and one in ActiveMQ using JMS. To reduce the need for multiple consumers to take in the same message but in different messaging formats, then the bridge could be used to transform the message into the recipient protocol which could be useful if the message being sent is a standard one.

It might also allow a flexibility in having message brokers placed in different networks in that a consumer connection does not have to be maintained across the firewalls. Rather the connection is made on production so any connection does not have to have a keep-alive call being made.

I still need to run more tests for speed and to test the inevitable speed reduction in swapping protocols but this approach appears to make solve some messaging network problems using PHP. Looking at the Camel documentation, this piece of code begins to look like doing the same sort of thing in PHP but right now I am not thinking about that (though the possible uses do come to mind).