Marten 8.12 with New Plumbing

Until today’s Marten 8.12 release, Marten’s Async Daemon and a great deal of Wolverine‘s internals were both built around the venerable TPL DataFlow library. I had long considered a move to the newer System.Threading.Channels library, but put that off for the previous round of major releases because there was just so much other work to do and Channels isn’t exactly a drop in replacement for the “block” model in TPL DataFlow that we use so heavily in the Critter Stack.

But of course, a handful of things happened to make me want to finally tackle that conversion:

1. A JasperFx Software client was able to produce behavior under load that proved that the TPL DataFlow ActionBlock wasn’t perfectly sequential even when it was configured with strict ordering
2. That same client commissioned work on what will be the “partitioned sequential messaging” feature in Wolverine 5.0 that enables Wolverine to group messages on user defined criteria to greatly eliminate concurrent access problems in Critter Stack applications under heavy load

Long story short, we rewired Marten’s Async Daemon and all of Wolverine’s internals to use Channels, but underneath a new set of (thin) abstractions and wrappers that mimics the TPL DataFlow “ITargetBlock” idea. Our new blocks allow us to compose producer/consumer chains in some places, while also enabling our new “partitioned sequential messaging” feature that will hit in Wolverine 5.0.

If you’re curious, or want to laugh at us, or steal them for your own TPL DataFlow conversion, our “Blocks” wrappers are on GitHub here.