Recent Marten & Wolverine Improvements and Roadmap Update

I’d love any feedback on any of this of course. And from something I wrote in a survey of sorts about the commercial product ideas down below yesterday (which is partially a response to a recent query wanting to know how Marten stacks up against AxonIQ from the JVM world):

There’s definitely an opportunity for a full blown Event Driven Architecture stack in the .NET ecosystem – and frankly, Jeremy really wants the Critter Stack to grow into the very best Event Driven Architecture toolset on the planet to the point where shops will purposely adopt .NET just because of the Critter Stack

I’m honestly just thinking out loud in this post, but a lot has been released for both Marten and Wolverine since the big Marten 7.0 release and the last time I published a roadmap update for the two big toolsets.

Here’s some recent highlights you might have missed from the past two months:

• Revisioned Documents in Marten 7
• Marten 7 makes “Write Model” Projections Super — blue/green deployment capabilities for Marten “write model” projections
• Recent Critter Stack Multi-Tenancy Improvements (w/ Wolverine 2.0 too!) — dynamically adding new tenant databases
• Resiliency and Low Level Improvements in Marten 7
• LINQ Query Improvements in Marten 7
• “Partial” Document Updates in Marten 7
• Testing Asynchronous Projections in Marten
• Wolverine’s New PostgreSQL Messaging Transport
• Strict Ordered Message Handling with Wolverine
• First Class Event Subscriptions in Marten
• Marten, Metrics, and Open Telemetry Support — long awaited Open Telemetry support for Marten. Wolverine has had Open Telemetry support since before 1.0
• Critter Stack Improvements for Event Driven Architecture — a robust set of recipes for subscribing to events from a Marten event store being processed by Wolverine handlers or published through Wolverine messaging

What’s Next?

Getting Marten 7.0 and the accompanying Wolverine 2.0 release across the finish line enabled a flurry of follow up features the past two month — largely driven by a couple JasperFx Software client projects (yeah!). Moving forward, I think these are the remaining strategic features that will hopefully go in soon:

• Marten will get the ability to use PostgreSQL read replicas for read-only queries very soon as a way to scale applications
• A new, alternative “Quick Append Event” workflow to Marten. The current internal mechanism in Marten for appending events is built for maximal support for “Inline” projections. This new model would simplify the runtime mechanics for appending events and hopefully make the Marten event store more robust in the face of concurrent requests than it is today. This model should also allow for faster performance if users opt into this mechanism.
• Some ability to efficiently raise or append events (or side effects of some sort) from running projections. This has been in the backlog for a long time. I’d certainly feel better about this if we had some concrete use cases that folks want to do here. The “Quick Append Event” workflow would be a prerequisite
• Using PostgreSQL partitioning on the Marten streams and events tables. This is the oldest item in the Marten backlog that’s been kicked down the road forever, but I think this is potentially huge for Marten scalability. This would probably be done in conjunction with some tactical improvements to the Marten projection model and the Wolverine aggregate handler workflow to make the archiving more accessible. The biggest issue has always been in how to handle the database migration model for this feature to convert brownfield applications
• Wolverine 3.0
  • Try to eliminate the hard dependency on Lamar as the IoC container for Wolverine. Most people don’t care, but the folks who do care really don’t like that. So far from my research it looks like the answer is going to be supporting the built in .NET DI container or Lamar with the current Wolverine model — and we can maybe think about supporting other IoC containers with a step back in the runtime optimizations that Wolverine can do today with Lamar. I think it’s quickly coming to the point where all other IoC libraries besides the built in ServiceProvider container from Microsoft die off — even though there are still plenty of areas where that container is lacking compared to alternatives. Alas.
  • Try to apply the Wolverine error handling policies that today only work for Wolverine message handlers to HTTP endpoints

Critter Stack Pro

The Marten & Wolverine community is helping Babu, Jeffry Gonzalez & I brainstorm ideas for the future “Critter Stack Pro” suite of commercial add on tools. The goal is to (make money) make the “Critter Stack” be much more manageable in production environments, help troubleshoot production support issues, heal the system from runtime problems, and understand resource utilization. We don’t have the exact roadmap or exact technical approach locked down yet.

Right now that looks like:

• A headless library to better distribute Marten projections and subscriptions across a running cluster of processes. This is “ready for testing” by a JasperFx customer
• A management console application that will be offered both as an ASP.Net Core add on library for a single application or distributed as a standalone Docker image for managing multiple systems from one console
  • Analyze system configuration
  • Manage Wolverine’s “dead letter queue” for messages, including the ability to replay messages
  • Some integration with Open Telemetry and metrics data emitted from Marten and/or Wolverine applications, probably at a summary level with navigation to the “real” observability platform (Prometheus? Grafana? Something totally different?)
  • Management for Marten Asynchronous Projections and Subscriptions
    • Performance information
    • Triggering rebuilds or replays
    • Pausing/restarting projections or subscriptions
  • Tenant Management
    • Dynamically add or remove tenant databases
    • Pause tenants
    • Understand resource utilization and performance on a tenant by tenant basis
  • Marten Event Store Explorer — and we’re collecting several ideas for this
  • Wolverine Message Processing Explorer — ditto
  • Wolverine Scheduled Message Dashboard

My fervent hope is that this tooling will be demonstrable for friendly early adopters at the end of the 2nd quarter, and looking good in the 4th quarter to try to make a serious push for sales in the all important 1st quarter of next year.

And Beyond!

I’m very interested in porting just the event store functionality from Marten to a new library targeting SQL Server as the backing store. The goal here would be to give it the same Wolverine support as the existing Marten functionality. This would be pending some of the Marten projection model stabilizing up above.

Maybe adding CosmosDb and/or DynamoDb support to Wolverine.

And who knows? It’s likely something I’m not even aware of now will be the highest priority in the 3rd and 4th quarters!