As part of the synchronisation part of a project, I’ve been trying to integrate systems together with a messaging system. This is not a trendy decision but one based on scalability and flexibility at the core.
I wrote a micro-messaging server for my previous job which was dealing with a few hundred messages per day by the time I left the role. (Post discussing part of it here.) That was my first encounter with the world of messaging and the power that it potentially gives the developer in building large scale projects.
Its main advantage is that you can concentrate on building (sub)systems that can do their own job. Rather than having to think about linking them to each other tightly and synchronously, they can sit apart and do their own job. Rather than building a bazaar, we are left with more cathedrals, to use Eric Raymond’s analogy. It also suggests that the tightly bound systems are not running to full potential because they are trying to connect synchronously.
Although you can link the services together using Enterprise Service Buses, SOAP and WSDL, the new generation of messaging services offer a different way of linking systems. Rather than being synchronously linked, it can mean that many systems can be linked asynchronously as well as synchronously. It does give a certain amount of resilience if a system fails, the message can held in a queue. It also means that service endpoints can be reworked or taken out of service.
But back to post in hand…
One of the things that I’ve been looking at is how to synchronise disparate systems. Since each system has its own speciality, I need to find a way of updating the changes and pushing them across the platform. I want to build a system that allows other people in the team to slot in or take out systems with the minimum of disruption and configuration to running services.
As such RabbitMQ offers a range of type of queues and messages which need the minimum of altering to run.
The direct message allows a system to push data to another one with out needing a reply, effectively point to point messaging. I’m currently looking at this with regards to logging systems. Once the logging message is created, whether it is information or a fatal error, it does not need to reply back to the system since the log is read only. Only the developers really need to read the logs. Once of the issues that I had with my earlier system was that two systems could try to write to the log at the same time and the second would fail due to the file of queue being temporarily locked. At least with Rabbit, I can write a consumer to and know that I can rely on Rabbit’s internals to help me avoid this. The fanout type, from what I’ve seen on Slideshare presentations, also allows for one message to be sent to a variety of consumer simultaneously so avoiding blocking whilst a process completes.
In addition to this though, the topics and RPC queues promise to allow me to be able to use Rabbit as a news and remote services point.
Using the topics type, items can be moved around to provide, I would hope, targeting information such as news services and providing information to front end developers.
Equally the rpc types offer a world of exploration into providing services to internal and external consumers rather than having to web services throughout. It strikes me as allowing the separation of public and private services as well as internal failover systems in the case of blocking or failure. The end user never needs to see the failure, only the requested results, or that these will be supplied when the system has recovered. Banish the 500 internal server error page!
Perhaps that is a brief skate over the excitement that I’m feeling about Rabbit as I’ve come across the latter types relatively recently. I’m really excited and lurking on the mailing lists seeing what else can be done.
I’m also enjoying playing with differing libraries and languages such as PHP and node.js to consume, publish and connect. I’d even consider using Java in some cases to write interconnecting systems and to create a stable back end. Clearly messaging is not the whole story but it is a major tool in the back end development toolkit.
Having learned about some of the internals of messaging (and, looking back, failed fast in a few of them), I’m happy to get back into this world and to investigate its potential in greater depth. Tying this into the worlds of NoSQL and node.js will be intriguing. This is, I hope, the first of many posts exploring the area in full.
No Comments