Pancake Stack: End-to-End, Real-time ML and AI Pipeline

InfoQ has an interesting read on the last-mile of theApache Spark machine learning pipeline, “Chris Fregly on the PANCAKE STACK Workshop and Data Pipelines”.

Its interesting to read that even Netflix and other struggled in this space:

The idea for PipelineIO dates back a few years to my time at Netflix where we were forced to build a custom ML prediction/serving layer. There wasn’t – and still isn’t, in my opinion – a production-ready, fault-tolerant, and low latency open source system to serve Netflix-scale predictions and recommendations in real-time

PipelineIO offers a good architecture overview on its home page:


~ by mdavey on September 12, 2016.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: