Critter Stack Futures

Starting this month, I think I’m going to blog openly about the ideas and directions that the “Critter Stack” team (Marten and Wolverine) and community is considering for where things go next. I think we’d love to hear any feedback or further suggestions about where this goes (and here’s the link to the Critter Stack Discord channel). I think we’re being mostly reactive to “what are our users struggling with” and “what are folks telling us about what’s stopping them from adopting Marten or Wolverine?”

For the immediate future, I’m trying to get my act together to actually have a real business structure and being ready to start offering support or consulting contracts. I’m personally catching up on open Marten bugs as I’ve been mostly busy with Wolverine lately and not helping the rest of the team much.

Strategic Things

Here are the big themes that we’ve identified that we need to push Marten and/or Wolverine into contention for world domination (or at least entice enough paying users to make a living off of our passion projects here). The first four bullets are happening this year and the rest is just a fanciful vision board.

1. 1st class subscriptions in Marten — This is the ability to register to listen to feeds of event data from Marten. Maybe you want to stream event data through to Kafka for additional processing. Maybe you want to update an ElasticSearch index for your data. Definitely you want this to work with all the reliability, monitoring, and error handling capabilities that you’d expect.
2. Linq improvements in Marten — Try to utilize JSONPath operators that are available in recent versions of PostgreSQL that would re-enable the usage of GIN/GIST indexes that was somewhat lost in V4. Try to greatly improve Marten’s Linq querying within document sub-collections. I hate working on the Linq parser in Marten, but I hate seeing Linq-related bugs filter in even more as folks try more and more things, so it’s time
3. Massive scalability of event projections — This is likely to be a new alternative to the current Marten async daemon that is able to load balance asynchronous projection processing across multiple application nodes. This improved daemon will be built with a combination of Marten and Wolverine as an add on product, likely with some kind of dual usage commercial license and official support license.
4. Zero downtime, blue/green deployment for event sourcing — Closely related to the previous bullet. Everything you need to be able to blue/green deploy your application using Marten event sourcing without any down time. So, you’ll have support for versioned projections and zero downtime projection rebuilds as well. This will most likely be part of the commercial add on package for the Critter Stack
5. User interface for monitoring or managing the Critter Stack — Just a place holder. Not sure what the exact functionality would be here. And this will absolutely be a dual license commercial product of some sort.
6. Sql Server backed event store — While the document database feature set in Marten is unlikely to ever be completely ported to Sql Server, it’s maybe perfectly possible to support Marten’s event sourcing on a Sql Server foundation
7. Marten for the JVM??? — Just stay tuned on this one down the line. In all likelihood this would mean running Marten’s event store in a separate process, then using some kind of language neutral proxy mechanism (gRPC?) to capture events. Tentatively the idea for projections is to let users use TypeScript/JavaScript to define projects that will run in WebAssembly.
8. AOT compilation / Serverless optimized recipe — There’s no chance in the world that any combination of Marten or Wolverine can work with AOT compilation without some significant changes on our side. I think it’s going to end up requiring some level of code generation to get there. I’m not clear about whether or not enough folks care about this right now to justify the effort.

Tactical Things

And also a list of hopefully quick wins that help spur Critter Stack adoption

1. Open Telemetry support in Marten. We have this in Wolverine already, but not yet for Marten activity
2. Ability to raise events from projections in Marten, or issue commands as aggregates are updates, or I don’t know yet. All I know is that right now this seems to be coming up a lot in user questions in Discord
3. Document versioning in Marten
4. Kafka transport in Wolverine
5. Amazon SNS support in Wolverine
6. Strong-typed identifiers. Folks have been asking for this periodically in Marten. When it exists in Marten, I’d also like to pursue being able to exploit strong typed identifiers in Wolverine middleware to “know” when to load entities from identifiers automatically
7. Expanding multi-tenancy support in Wolverine. Today Wolverine has a robust model for Marten-backed multi-tenancy in the message handling, but I’d like to see this extended to detecting tenant identifiers automatically in HTTP requests. I’d also like to extend the multi-tenancy support to EF Core backed persistence and SQL Server backed storage.
8. Lightweight partial updates in Marten. This is the ability to issue updates to part of a Marten document without first loading the entire document. We’ve had this functionality from the very beginning, but it depends on Javascript support in PostgreSQL through the PLv8 extension that’s in a tenuous state. The new model would use native PostgreSQL features in place of the older JavaScript model.

Waddaya think?

Anything above sound compelling to you? Have questions about how some of that would work? Wanna make suggestions about how it should be done? Have *gasp* completely different suggestions for what we should improve instead in Marten/Wolverine to make it more attractive to your shop? Fire away in comments here or the Critter Stack Discord channel.