Resilent Messaging with Node.js – RabbitMQ or ZeroMQ

There are a lot of postings around Node.js offering code snippets pushing stock data and more. Unfortunately, few of these postings offer thoughts around resilence, and deployment within an enterprise DMZ supporting resilence.

Lets start by consider the simple scenario of a pricing engine that provides a continues stream of prices, which we want to deliver to a HTML client via a Node.js server. For this posting, lets ignore the issue of the pricing engine failing, and concentrate on failure of the Node.js server, and disconnection of the HTML client. If in the simple scenario we had the Node.js server connect to the pricing engine, on a HTML client connecting to the Node.js server, if the Node.js server died, we would loose any data the pricing engine sent between the time the Node.js crashed, and the time the client reconnected to a new Node.js server, or the original re-started Node.js server. We’ll ignore expired prices for this posting as well 🙂

Essentially we need a queue between the pricing engine and the Node.js server. From a open source perspective there are a few possibilities that are of interest: RabbitMQ and 0MQ. 0MQ performance is somewhat impressive if this posting is to be believed. Firehose provides some details of using Node.js, coffescript and 0MQ for real-time data streaming.

RabbitMQ with Node.js also features prominently in posting, and offers an extensive list of “How To” documentations. Building a realtime Feed with RabbitMQ Node.js AMQP and mySQL offers thoughts on a similar problem:

Node.js will send back any data that is waiting for the client, or is sent to the client during the connection lifetime, which recycles every 50 seconds. Node source of data is from RabbitMQ.

Interestingly, “RabbitMQ, Node.js and Java Goodness” discusses stock price delivery using node.js, with code available on github.

Further reading:

  • rabbitmq + node.js = rabbit.js
  • Using the RabbitMQ service on Cloud Foundry with Node.JS
  • Building a Node.js Events App Using RabbitMQ, Websockets, and Django
  • RabbitMQ and Node.js Websockets experiment
  • AMQP/RabbitMQ or Node.js for live web applications?

~ by mdavey on December 20, 2011.

3 Responses to “Resilent Messaging with Node.js – RabbitMQ or ZeroMQ”

  1. Why not redis? The messaging throughput of a redis server will surprise you.


  2. Queuing quotes is the worst thing you can do…. unless your queue size is 1. You’d rather skip some quotes. Sending old quotes is a business risk. There are “snipers” who just wait for such stale quotes.

  3. You’re a genius Roman. Let’s just skip quotes! Nobody would ever need to actually have their order book in a valid state. “Old Quotes” never actually become top-of-book after a trade execution or order cancellation!

    I’ve been wasting so much bandwidth and so much processing power!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: