Anyone using Storm in Capital Markets?


Storm was open sourced sometime ago thanks to Twitter. Storm, a scalable real-time computation system, so think CEP without the storage engine. The coolness is in the distributed and fault-tolerant features.

The important properties of Storm are:

  1. Simple programming model. Similar to how MapReduce lowers the complexity of doing parallel batch processing, Storm lowers the complexity for doing real-time processing.
  2. Runs any programming language. You can use any programming language on top of Storm. Clojure, Java, Ruby, Python are supported by default. Support for other languages can be added by implementing a simple Storm communication protocol.
  3. Fault-tolerant. Storm manages worker processes and node failures.
  4. Horizontally scalable. Computations are done in parallel using multiple threads, processes and servers.
  5. Guaranteed message processing. Storm guarantees that each message will be fully processed at least once. It takes care of replaying messages from the source when a task fails.
  6. Fast. The system is designed so that messages are processed quickly and uses ØMQ as the underlying message queue.
  7. Local mode. Storm has a “local mode” where it simulates a Storm cluster completely in-process. This lets you develop and unit test topologies quickly.

~ by mdavey on January 25, 2012.

One Response to “Anyone using Storm in Capital Markets?”

  1. […] M Davey asks if this has much use in Capital Markets here. […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

 
%d bloggers like this: