Midori: Fine-Grained Process Scaling

Finally, Joe Duffy has started blogging again, and with it, we get details of Midori (programming language, compilers, OS, its services, applications, and the overall programming models), the multi-year journey.

A key to achieving asynchronous everything was ultra-lightweight processes

I’d be interested to know if Midori influenced Orleans – given the actor model similarities.  Orleans code is available in GitHub.

Throughout the “Asynchronous Everything” article there are a number of great callouts:

  • Midori did not have demand paging which, in a classical system, means that touching a piece of memory may physically block to perform IO.
  • Midori was an entire OS written to use garbage collected memory – Unfortunately, we don’t see Joe offer a view if Midori was a viable production OS.
  • Avoid superfluous allocations like the plague.  Gen0 collections are NOT free.
  • One of the tricks to garbage collection working at the scale of Midori was precisely the fine-grained process model, where each process had a distinct heap that was independently collectible.
  • Processes were ultra-lightweight and single-threaded – event loop based.
  • Batches of messages could be stuffed into channels before they filled up. They were delivered in chunks at-a-time – Interesting, as similar concepts have been used in financial software for many years.
  • “zero-copy”, SharedData was a automatic ref-counted pointer to some immutable data in a heap shared between processes.  Interested in how this worked across machines
  • causality IDs that flowed with messages across processes
  • asynchronous everywhere

~ by mdavey on December 2, 2015.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: