In my last post about monitoring ActiveMQ, I looked at various advisory queues. I also mused on the possibility of reading the stats from the queues and topics and parsing them so that if issues occur, they can be dealt with quickly rather than waiting for an issue to be reported.
Although this information is in the web console, that relies on operations an administration looking at it constantly which is not the best use of time. Rather we want to be able to get statistics quickly and automatically. There are XML feeds for queues and topics.
Partially from curiosity, as I was writing this in my spare time, I used Node.js which has always struck me as a really powerful tool in this sort of area. Rather than having endless cron jobs (which might be quicker in one sense as I am more used to parsing in PHP), why not use a tool which is designed in some sense for this sort of use?
Two posts really got me started with this. Firstly this post by Chad Lung about his experiences and Charlie Robbins’ post on the Nodejitsu blog about 6 essential Node.js modules. Since XML2JS was recommended by both that became the obvious module to use and parse the result into JSON. In the short term, I have used the http client to poll the queue every few seconds to get its statistics.
The initial output for each queue is:
<queue name=”example.A”><stats size=”0″ consumerCount=”1″ enqueueCount=”0″ dequeueCount=”0″/><feed><atom>queueBrowse/example.A?view=rss&feedType=atom_1.0</atom><rss>queueBrowse/example.A?view=rss&feedType=rss_2.0</rss></feed></queue>
This shows the queue name, its counts for enqueue and dequeue and feeds for the individual queues, so you could subscribe to the individual queues which provide information about the messages on the queue, such as publish date and id.
From there I have set up some ranges for the number of allowed pending messages before the system logs errors and a quick check to see if there are more incoming messages with a margin of error than dequeued messages and if so, to log a warning. XML2JS allows you to load the parsed message into JSON using JSON.stringify which I then parsed to extract the values that I wanted.
Small but quick to set up, it means that I have a daemon which runs in the background making some checks against the queues and, in part, could also be used to effectively ping the server on to see if it responding. A minor wrinkle in the scheme of things.
I had been toying around with Node looking for something to really use it on and this seemed like the ideal project. I am half tempted to look at it for writing a small monitoring tool to pull together various strands which I have been looking at in the last couple of blog posts and explore their feasibility and practicality, sort of like Monit for Active.
A final word. This brief series of posts has been sitting in my mind for sometime as has using/monitoring Message Oriented Middleware. It started when I read R I Pienaar’s 5 part series on using STOMP and common messaging patterns which I really recommend if you have not read the series. Also several conversations with our sys admin kicked me into writing it and some code.
Thanks for the link back 🙂
ActiveMQ also has an internal statistics broker that you can communicate with using the request-reply pattern, I wrote some checks @ https://github.com/ripienaar/monitoring-scripts/tree/master/activemq that uses that and might be worth looking into
You’re welcome. Thanks for writing the original series. 🙂 I’ll look up the scripts. I was scratching an itch for these posts but think that there is something more that can be done for any production system.
I’m confused, where is your actual code?
Originally it did not have the code which was a quick hack one morning. I’ve found the original code that I hacked up to explore the idea that I had.
For production, I worked on something similar in Perl (as the company use(s|d) Nagios).