Announcing Polecat: Event Sourcing with SQL Server

Polecat is now completely supported by JasperFx Software and automatically part of any existing and future support agreements through our existing plans.

Polecat was released as 1.0 this past week (with 1.1 & now 1.2 coming soon). Let’s call it what it is, Polecat is a port of (most of) Marten to target SQL Server 2025 and SQL Server’s new JSON data type. For folks not familiar with Marten, Polecat is in one library:

  1. A very full fledged Event Store library for SQL Server that includes event projection and subscriptions, Dynamic Consistency Boundary support, a large amount of functionality for Event Sourcing basics, rich event metadata tracking capabilities, and even rich multi-tenancy support.
  2. A feature rich set of Document Database capabilities backed by SQL Server including LINQ querying support

And while Polecat is brand spanking new, it comes out of the gate with the decade old Marten pedigree and its own Wolverine integration for CQRS usage. I’m confident in saying Polecat is now the best technical option for using Event Sourcing with SQL Server in the .NET ecosystem.

And of course, if you’re a shop with deep existing roots into EF Core usage, Polecat also comes with projection support to EF Core, so Polecat can happily coexist with EF Core in the same systems.

Alright, let’s just into a quick start. First, let’s say you’ve started a brand new .NET project through dotnet run webapi and you’ve added a reference to Polecat through Nuget (and you have a running SQL Server 2025 instance handy too of course!). Next, let’s start with the inevitable AddPolecat() usage in your Program file:

builder.Services.AddPolecat(options =>
{
// Connection string to your SQL Server 2025 database
options.Connection("Server=localhost;Database=myapp;User Id=sa;Password=YourStrong!Password;TrustServerCertificate=True");
// Optionally change the default schema (default is "dbo")
options.DatabaseSchemaName = "myschema";
});

Polecat can be used without IHost or IServiceCollection registrations by just directly building a DocumentStore object.

Next, let’s say you’ve got this simplistic document type (entity in Polecat parlance):

public class User
{
public Guid Id { get; set; }
public required string FirstName { get; set; }
public required string LastName { get; set; }
public bool Internal { get; set; }
}

And now, let’s use Polecat within some Minimal API endpoints to capture and query User documents:

// Store a document
app.MapPost("/user", async (CreateUserRequest create, IDocumentSession session) =>
{
var user = new User
{
FirstName = create.FirstName,
LastName = create.LastName,
Internal = create.Internal
};
session.Store(user);
await session.SaveChangesAsync();
});
// Query with LINQ
app.MapGet("/users", async (bool internalOnly, IDocumentSession session, CancellationToken ct) =>
{
return await session.Query<User>()
.Where(x => x.Internal == internalOnly)
.ToListAsync(ct);
});
// Load by ID
app.MapGet("/user/{id:guid}", async (Guid id, IQuerySession session, CancellationToken ct) =>
{
return await session.LoadAsync<User>(id, ct);
});

For folks used to EF Core, I should point out that Polecat has its own “it just works” database migration subsystem that in the default development mode will happily make sure that all necessary database tables, views, and functions are exactly as they should be at runtime so you don’t have to fiddle with database migrations when all you want to do is just get things done.

While I initially thought that we’d mainly focus on the event sourcing support, we were also able to recreate the mass majority of Marten’s document database capabilities (including the “partial update” model, LINQ support, soft deletes, multi-tenancy, and batch updates for starters) as well if you’d only be interested in that feature set by itself.

Moving over to event sourcing instead, let’s say you’re into fantasy books like I am and you want to build a system to model the journeys and adventures of a quest in your favorite fantasy series. You might model some of the events in that system like:

public record QuestStarted(string Name);
public record MembersJoined(string Location, string[] Members);
public record MembersDeparted(string Location, string[] Members);
public record QuestEnded(string Name);

And you model the current state of the quest party like this:

public class QuestParty
{
public Guid Id { get; set; }
public string Name { get; set; } = "";
public List<string> Members { get; set; } = new();
public void Apply(QuestStarted started)
{
Name = started.Name;
}
public void Apply(MembersJoined joined)
{
Members.AddRange(joined.Members);
}
public void Apply(MembersDeparted departed)
{
foreach (var member in departed.Members)
Members.Remove(member);
}
}

The step above isn’t strictly necessary for event sourcing, but you usually need a projection of some sort sooner or later.

And finally, we can add events by starting a new event stream:

var store = DocumentStore.For(opts =>
{
opts.Connection("Server=localhost,1433;Database=myapp;User Id=sa;Password=YourStrong!Password;TrustServerCertificate=True");
});
await using var session = store.LightweightSession();
// Start a new stream with initial events
var questId = session.Events.StartStream<QuestParty>(
new QuestStarted("Destroy the Ring"),
new MembersJoined("Rivendell", ["Frodo", "Sam", "Aragorn", "Gandalf"])
);
await session.SaveChangesAsync();

And even append some new ones to the same stream later:

await using var session = store.LightweightSession();
session.Events.Append(questId,
new MembersJoined("Moria", ["Gimli", "Legolas"]),
new MembersDeparted("Moria", ["Gandalf"])
);
await session.SaveChangesAsync();

And derive the current state of our quest:

var party = await session.Events.AggregateStreamAsync<QuestParty>(questId);
// party.Name == "Destroy the Ring"
// party.Members == ["Frodo", "Sam", "Aragorn", "Gimli", "Legolas"]

And there’s much, much more of course, including everything you’d need to build real systems based on our 10 years and counting supporting Marten with PostgreSQL.

How is Polecat Different than Marten?

There are of course some differences besides just the database engine:

  • Polecat is using source generators instead of the runtime code generation that Marten does today
  • Polecat will only support System.Text.Json for now as a serialization engine
  • Polecat only supports the “Quick Append” option from Marten
  • There is no automatic dirty checking
  • No “duplicate fields” support so far, we’re going to reevaluate that though
  • Plenty of other technical baggage features I flat out didn’t want to support in Marten didn’t make the cut, but I can’t imagine anyone will miss any of that!

Summary

For over a decade people have been telling me that Marten would be more successful and adopted by more .NET shops if it only supported SQL Server in addition to or instead of PostgreSQL. While I’ve never really disagreed with that idea — and it’s impossible to really prove the counter factual anyway — there have always been real blockers in both SQL Server’s JSON support lagging far behind PostgreSQL and frankly the time commitment on my part to be able to attempt that work in the first place.

So what changed to enable this?

  1. SQL Server 2025 added much better JSON support rivaling PostgreSQL’s JSONB type
  2. We had already invested in pulling the basic event abstractions and projection support out of Marten and into a common library called JasperFx.Events as part of the Marten 8.0 release cycle and that work was always meant to be an enabler for what is now Polecat
  3. Claude & Opus 4.5/4.6 turned out to be very, very good at grunt work

That second item had to this point been a near disaster in my mind because of how much work and time that took compared to the benefits and was the single most time consuming part of Polecat development. Let’s just say that I’m very relieved that that effort didn’t turn out to be a very expensive sunk cost for JasperFx!

I have no earthly idea how much traction Polecat will really get, but we’ve already had some interest from folks who have wanted to use Marten, but couldn’t get their .NET shop to adopt PostgreSQL. I’m hopeful!

Critter Stack Roadmap for March 2026

It’s only a month since I’ve written an update on the Critter Stack roadmap, but it’s maybe worth some time on my part to update what I think the roadmap is now. The biggest change is the utter dominance of AI in the software development discourse and the fact that Claude usage has allowed us to chew through a shocking amount of backlog in the past 6 weeks. That’s probably also changed my own thinking about what should be next throughout this year.

First, some updates on what’s been added to the Critter Stack in just the last month:

By the time you read this, we may very well have Polecat 1.0 out as well.

Short Term

The short term priority for myself and JasperFx Software is to deliver the CritterWatch MVP in a usable form by the end of March.

Marten, Wolverine, and even Polecat have no major new features planned for the short term and I think they will only get tactical releases for bug fixes and JasperFx client requests for a little while. And let me tell you, it feels *weird* to say that, but we’ve blown through a tremendous amount of the backlog so far in 2026.

Medium Term

  • Enhance CritterWatch until it’s the best in class monitoring tool for asynchronous messaging and event sourcing. Part of that will probably be adding quite a bit more functionality for development time as well.
  • For a JasperFx Software client, we’re doing PoC work on scaling Marten to be able to handle having several hundred billion events in a single system. I’m going to assume that this PoC will probably lead to enhancements in both Marten and Wolverine!
  • We’ll finally add some direct support to Marten for the PostGIS PostgreSQL extension
  • I’m a little curious to try to use the hstore extension with Marten as a possible way to optimize our new DCB support
  • Play with Pgvector and TimescaleDb in combination with Marten as some kind of vague “how can we say that Marten is even more awesome for AI?”
  • There’s going to be a new wave of releases later this year for Marten 9.0, Wolverine 6.0, and Polecat 2.0 that will mostly about performance optimizations and especially finding ways to optimize the cold start time of applications using these tools.
  • Babu and I (really all Babu so far) are going to be building a set of AI skills for using the Critter Stack tools that will be curated in a GitHub repository and available to JasperFx Software clients. I do not know what the full impact of AI tools are really going to be on software development, but I personally want to plan for the worst case that AI tools plus LLM-friendly documentation drastically reduces the demand for consulting and try to belatedly pivot JasperFx Software to being at least partially a product company.
  • Build tooling for spec driven development using the Critter Stack. I don’t have any details beyond “hey, wouldn’t that be cool?”. My initial thought is to play with Gherkin specifications that generates “best practices” Critter Stack code with the accompanying automated tests to boot.
  • One way or another, we’ll be building MCP support into the Critter Stack, but again, I don’t know anything more than “hey, wouldn’t that be cool?”

Long Term

Profit?

I’m playing with the idea of completely rebooting Storyteller as a new spec driven development tool. I have the Nuget rights to the “Storyteller” name and graphics from Khalid (a necessary requirement for any successful effort on my part), and I’ve always wanted to go back to it some day.

Natural Keys in the Critter Stack

Just to level set everyone, there are two general categories of identifiers we use in software:

  • “Surrogate” keys are data elements like Guid values, database auto numbering or sequences, or snowflake generated identifiers that have no real business meaning and just try to be unique values.
  • “Natural” keys have some kind of business meaning and usually utilize some piece of existing information like email addresses or phone numbers. A natural key could also be an external supplied identifier from your clients. In fact, it’s quite common to have your own tracking identifier (usually a surrogate key) while also having to track a client or user’s own identification for the same business entity.

That very last sentence is where this post takes off. You see Marten can happily track event streams with either Guid identifiers (surrogate key) or string identifiers — or strong typed identifiers that wrap an inner Guid or string, but in this case that’s really the same thing, just with more style I guess. Likewise, in combination with Wolverine for our recommended “aggregate handler workflow” approach to building command handlers, we’ve only supported the stream id or key. Until now!

With the Marten 8.23 and Wolverine 5.18 releases last week (we’ve been very busy and there are newer releases now), you are now able to “tag” Marten (or Polecat!) event streams with a natural key in addition to its surrogate stream id and use that natural key in conjunction with Wolverine’s aggregate handler workflow.

Of course, if you use strings as the stream identifier you could already use natural keys, but let’s just focus on the case of Guid identified streams that are also tagged with some kind of natural key that will be supplied by users in the commands sent to the system.

First, to tag streams with natural keys in Marten, you have to have a strong typed identifier type for the natural key. Next, there’s a little bit of attribute decoration in the targeted document type of a single stream projection, i.e., the “write model” for an event stream. Here’s an example from the Marten documentation:

public record OrderNumber(string Value);
public record InvoiceNumber(string Value);
public class OrderAggregate
{
public Guid Id { get; set; }
[NaturalKey]
public OrderNumber OrderNum { get; set; }
public decimal TotalAmount { get; set; }
public string CustomerName { get; set; }
public bool IsComplete { get; set; }
[NaturalKeySource]
public void Apply(OrderCreated e)
{
OrderNum = e.OrderNumber;
CustomerName = e.CustomerName;
}
public void Apply(OrderItemAdded e)
{
TotalAmount += e.Price;
}
[NaturalKeySource]
public void Apply(OrderNumberChanged e)
{
OrderNum = e.NewOrderNumber;
}
public void Apply(OrderCompleted e)
{
IsComplete = true;
}
}

In particular, see the usage of [NaturalKey] which should be self-explanatory. Also see the [NaturalKeySource] attribute that we’re using to mark when a natural key value might change. Marten is starting to use source generators for some projection internals (in place of some nasty, not entirely as efficient as it should have been, Expression-compiled-to-Lambda functions).

And that’s that, really. You’re now able to use the designated natural keys as the input to an “aggregate handler workflow” command handler with Wolverine. See Natural Keys from the Wolverine documentation for more information.

For a little more information:

  • The natural keys are stored in a separate table, and when using FetchForWriting(), Marten is doing an inner join from the tag table for that natural key type to the mt_streams table in the Marten database
  • You can change the natural key against the surrogate key
  • We expect this to be most useful when you want to use the Guid surrogate keys for uniqueness in your own system, but you frequently receive a natural key from API users of your system — or at least this has been encountered by a couple different JasperFx Software customers.
  • The natural key storage does have a unique value constraint on the “natural key” part of the storage
  • Really only a curiosity, but this was done in the same wave of development as Marten’s new DCB support

New Improvements to Marten’s LINQ Support

I hold these two facts to be simultaneously true:

  1. Language Integrated Query (LINQ) is the singular best feature in .NET that developers would miss out working in other development platforms
  2. Developing and supporting a LINQ provider is a nastily hard and laborious task for an OSS author and probably my least favorite area of Marten to work in

Alright, on that note, let’s talk about a couple potentially important recent improvements to Marten’s LINQ support. First, we’ve received the message loud and clear that Marten was sometimes hard when what you really need is to fetch data from more than one document type at a time. We’d also got some feedback about the difficultly overall in making denormalized views projected from events with a mix of different streams and potentially different reference documents.

For projections in the event sourcing space, we added the Composite Projection capability. For straight up document database work, Marten 8.23 introduced support for the LINQ GroupJoin operator as shown in this test from the Marten codebase:

public class JoinCustomer
{
public Guid Id { get; set; }
public string Name { get; set; } = "";
public string City { get; set; } = "";
}
public class JoinOrder
{
public Guid Id { get; set; }
public Guid CustomerId { get; set; }
public string Status { get; set; } = "";
public decimal Amount { get; set; }
}
[Fact]
public async Task GroupJoin_select_outer_entity_properties()
{
// It's just setting up some Customer and Order data in the database
await SetupData();
var results = await _session.Query<JoinCustomer>()
.GroupJoin(
_session.Query<JoinOrder>(),
c => c.Id,
o => o.CustomerId,
(c, orders) => new { c, orders })
.SelectMany(
x => x.orders,
(x, o) => new { x.c.Name, x.c.City })
.ToListAsync();
results.Count.ShouldBe(3);
results.Count(r => r.City == "Seattle").ShouldBe(2); // Alice's 2 orders
results.Count(r => r.City == "Portland").ShouldBe(1); // Bob's 1 order
}

This is of course brand new, which means there are probably “unknown unknown” bugs in there, but just give us a reproduction in a GitHub issue and we’ll address whatever it is.

Select/Where Hoisting

Without getting into too many details, the giant “hey, let’s rewrite our LINQ support almost from scratch!” effort in Marten V7 a couple years ago made some massive strides in our LINQ provider, but unintentionally “broke” our support for chaining Where clauses *after* Select transforms. To be honest, that’s nothing I even realized you could or would do with LINQ, so I was caught off guard when we got a couple bug reports about that later. No worries now, because you can now do that with Marten as this new test shows:

    [Fact]
    public async Task select_before_where_with_different_type()
    {
        var doc1 = new DocWithInner { Id = Guid.NewGuid(), Name = "one", Inner = new InnerDoc { Value = 10, Text = "low" } };
        var doc2 = new DocWithInner { Id = Guid.NewGuid(), Name = "two", Inner = new InnerDoc { Value = 50, Text = "mid" } };
        var doc3 = new DocWithInner { Id = Guid.NewGuid(), Name = "three", Inner = new InnerDoc { Value = 90, Text = "high" } };

        theSession.Store(doc1, doc2, doc3);
        await theSession.SaveChangesAsync();

        // Select().Where() - the problematic ordering from GH-3009
        var results = await theSession.Query<DocWithInner>()
            .Select(x => x.Inner)
            .Where(x => x.Value > 40)
            .ToListAsync();

        results.Count.ShouldBe(2);
        results.ShouldContain(x => x.Value == 50);
        results.ShouldContain(x => x.Value == 90);
    }

Summary

So you might ask, how did we suddenly get to a point where there are literally no open GitHub issues related to LINQ in the Marten codebase? It turns out that the LINQ provider support is an absolutely perfect place to just let Claude go fix it — but know that that is backed up by about a 1,000 regression tests for LINQ to chew through at the same time.

My limited experience suggests that the AI assisted development really works when you have very well defined and executable acceptance requirements and better yet tests for your AI agent to develop to. Duh.

Fun Five Year Retrospective on Marten Adoption

In the midst of some let’s call it “market” research to better understand how Marten stacks up to a newer competitor, I stumbled back over a GitHub discussion I initiated in 2021 called “What would it take for you to adopt Marten?” long before I was able to found JasperFx Software. I seeded this original discussion with my thoughts about the then forthcoming giant Marten 4.0 release that was meant to permanently put Marten on a solid technical foundation for all time and address all known significant technical shortcomings of Marten.

Narrators’s voice: the V4 release was indeed a major step forward and still shapes a great deal of Marten’s internals, but it was not even remotely the end all, be all of technical releases and V5 came out less than 6 months later to address shortcomings of V4 and to add first class multi-tenancy through separate databases. And arguably, V7 just three years later was nearly as big a change to Marten’s internals.

So now that it’s five years later and Marten’s usage numbers are vastly greater than that moment in time in 2021, let me run through the things we thought needed to change to garner more usage, whether or not and how those ideas took fruit, and whether or not I think those ideas made any difference in the end.

Enterprise Level Support

People frequently told me that they could not seriously consider Marten or later Wolverine without there being commercial support for those tools or at least a company behind them. As of now, JasperFx Software (my company) provides support agreements for any tool under the JasperFx GitHub organization. I would say though that the JasperFx support agreement ends up being more like an ongoing consulting engagement rather than the “here’s an email for support, we’ll response within 72 hours” licensing agreement that you’d be getting from other Event Driven Architecture tools and companies in the .NET space.

Read more about that in Little Diary of How JasperFx Helps Our Clients.

And no, we’re not a big company at all, but we’re getting there and at least “we” isn’t just the royal “we” now:)

I’m hoping that JasperFx is able to expand when we are able to start selling the CritterWatch commercial add on soon.

More Professional Documentation

Long story short, a good, a modern looking website for your project is an absolute must. Today, all of the Critter Stack / JasperFx projects use VitePress and MarkdownSnippets for our documentation websites. Plus we have real project logo images that I really like myself created by Khalid Abuhakmeh. Babu Annamalai did a fantastic job on setting up our documentation infrastructure.

People do still complain about the documentation from time to time, but after I was mercilessly flogged online for the StructureMap documentation being so far behind in the late 00’s and FubuMVC never really having had any, I’ve been paranoid about OSS documentation ever since and we as a community try really hard to curate and expand our documentation. Anne Erdtsieck especially has added quite a bit of explanatory detail to the Marten documentation in the last six months.

It’s only anecdotal evidence, but the availability of the LLMS-friendly docs plus the most recent advances in AI LLM tools seem to have dramatically reduced the amount of questions we’re fielding in our Discord chat rooms while our usage numbers are still accelerating.

Oh, and I cannot emphasize more how important and valuable it is to be able to both quickly publish documentation updates and to enable users to quickly make suggestions to the documentation through pull requests.

Moar YouTube Videos

I dragged my feet on this one for a long time and probably still don’t do well enough, but we have the JasperFx Software Channel now with some videos and plenty of live streams. I’ve had mostly positive feedback on the live streams, so it’s just up to me to get back in a groove on this one.

SQL Server or CosmosDb Support in Addition to PostgreSQL

The most common complaint or concern about Marten in its first 5-7 years was that it only supported PostgreSQL as a backing data store. The most common advice we got from the outside was that we absolutely had to have SQL Server support in order to be viable inside the .NET ecosystem where shops do tend to be conservative in technology adoption and also tend to favor Microsoft offerings.

While I’ve always seen the obvious wisdom in supporting SQL Server, I never believed that it was practical to replicate Marten’s functionality with SQL Server. Partially because SQL Server lagged far behind PostgreSQL in its JSON capabilities for a long time and partially just out of sheer bandwidth limitations. I think it’s telling that nobody built a truly robust and widely used event store on top of SQL Server in the mean time.

But it’s 2026 and the math feels very different in many ways:

  1. PostgreSQL has grown in stature and at least in my experience, far more .NET shops are happy to take the plunge into PostgreSQL. It absolutely helps that the PostgreSQL ecosystem has absolutely exploded with innovation and that PostgreSQL has first class managed hosting or even serverless support on every cloud provider of any stature.
  2. SQL Server 2025 introduced a new native JSON type that brought SQL Server at least into the same neighborhood as PostgreSQL’s JSONB type. Using that, the JasperFx Software is getting close to releasing a full fledged port of most of Marten (you won’t miss the parts that were left out, I know I won’t!) called “Polecat” that will be backed by SQL Server 2025. We’ll see how much traction that tool gets, but early feedback has been positive.
  3. While we’re not there yet on Event Sourcing, at least Wolverine does have CosmosDb backed transactional inbox and outbox support as well as other integration into Wolverine handlers. I don’t have any immediate plans for Event Sourcing with CosmosDb other than “wouldn’t that be nice?” kind of thoughts. I don’t hear that many requests for this. I get even less feedback about DynamoDb, but I’ve always assumed we’d get around to that some day too.

Better Support for Cross Document Views or Queries

So, yeah. Document database approaches are awesome when your problem domain is well described by self-contained entities, but maybe not so much if you really need to model a lot of relationships between different first class entities in your system. Marten already had the Include() operator in our LINQ support, but folks aren’t super enthusiastic about it all the time. As Marten really became mostly about Event Sourcing over time, some of this issue went away for folks who could focus on using projections to just write documents out exactly as your use cases needed — which can sometimes happily eliminate the need for fancy JOIN queries and AutoMapper type translation in memory. However, I’ve worked with several JasperFx clients and other users in the past couple years who had real struggles with creating denormalized views with Marten projections, so that needed work too.

While that complaint was made in 2021, we now have or are just about to get in the next wave of releases:

  • The new “composite projection” model that was designed for easier creation of denormalized event projection views that has already met with some early success (and actionable feedback). This feature was also designed with some performance and scalability tuning in mind as well.
  • The next big release of Marten (8.23) will include support for the GroupJoin LINQ provider. Finally.
  • And let’s face it, EF Core will always have better LINQ support than Marten over all and a straight up relational table is probably always going to be more appropriate for reporting. To that end, Marten 8.23 will also have an extension library that adds first class event projections that write to EF Core.

Polecat 1.0 will include all of these new Marten features as well.

Improving LINQ Support

LINQ support was somewhat improved for that V4 release I was already selling in 2021, but much more so for V7 in early 2024 that moved us toward using much more PostgreSQL specific optimizations in JSONB searching as we were able to utilize JSONPath searching or back to the PostgreSQL containment operator.

At this point, it has turned out that recent versions of Claude are very effective at enhancing or fixing issues in the LINQ provider and at this point we have zero open issues related to LINQ for the first time since Marten’s founding back in 2015!

There’s one issue open as I write this that has an existing fix that hasn’t been committed to master yet, so if you go check up on me, I’m not technically lying:)

Open Telemetry Support and other Improved Metadata

Open Telemetry support is table stakes for .NET application framework tools and especially for any kind of Event Driven Architecture or Event Sourcing tool like Marten. We’ve had all that since Marten V7, with occasional enhancements or adjustments since in reaction to JasperFx client needs.

More Sample Applications

Yeah, we could still do a lot better on this front. Sigh.

One thing I want to try doing soon is developing some Claude skills for the Critter Stack in general, and a particular focus on creating instructions for best practices converting codebases to the Critter Stack. As part of that, I’ve identified about a dozen open source sample applications out there that would be good targets for this work. It’s a lazy way to create new samples applications while building an AI offering for JasperFx, but I’m all about being lazy sometimes.

We’ll see how this goes.

Scalability Improvements including Sharding

We’ve done a lot here since that 2021 discussion. Some of the Event Sourcing scalability options are explained here. This isn’t an exhaustive list, but since 2021 we have:

  • Much better load distribution of asynchronous projection and subscription work within clusters
  • Support for PostgreSQL read replicas
  • First class support for managing PostgreSQL native partitioning with Marten
  • A ton of internal improvements including work to utilize the latest, greatest low level support in Npgsql for query batching

And for that matter, I’ve finishing up a proposal this weekend for a new JasperFx client looking to scale a single Marten system to the neighborhood of 200-300 billion events, so there’s still more work ahead.

Event Streaming Support or Transactional Outbox Integration

This was frequently called out as a big missing feature in Marten in 2021, but with the integration into the full Critter Stack with Wolverine, we have first class event streaming support that was introduced in 2024 for every messaging technology that Wolverine supports today, which is just about everything you could possibly think to use!

Management User Interface

Sigh. Still in flight, but now very heavily in flight with a CritterWatch MVP promised to a JasperFx client by the end of this month. Learn more about that here:

Cloud Hosting Models and Recipes

People have brought this up a bit over the years, but we don’t have much other than some best practices with using Marten inside of Docker containers. I think it helps us that PostgreSQL is almost ubiquitous now and that otherwise a Marten application is just a .NET application.

SignalR + the Critter Stack

It’s early so I should be too cocky, but JasperFx Software is having success in integrating SignalR with both Wolverine and Marten in our forthcoming CritterWatch product. In this post I’ll show you how we’re doing that from the server side C# code all the way down to the client side TypeScript.

Last week I did a live stream talking about many of the details and a way too early demonstration of CritterWatch, JasperFx Software‘s long planned management console for the “Critter Stack” tools (Marten, Wolverine, and soon to be Polecat).

A big technical wrinkle in the CritterWatch approach so far is our utilization of the SignalR messaging support built into Wolverine. Just like with external messaging brokers like Rabbit MQ or Azure Service Bus, Wolverine does a lot of work to remove the technical details of SignalR and let’s you focus on just writing your application code.

In some ways, CritterWatch is kind of a man in the middle between the intended CritterWatch user interface (Vue.js) and the Wolverine enabled applications in your system:

Note that Wolverine will be required for CritterWatch, but if you only today use Marten and want to use CritterWatch to manage just the event sourcing, know that you will be able to use a very minimalistic Wolverine setup just for communication to CritterWatch without having to migrate your entire messaging infrastructure to Wolverine. And for that matter, Wolverine now has a pretty robust HTTP transport for asynchronous messaging that would work fine for CritterWatch integration.

As I said earlier, CritterWatch is going to depend very heavily on two way WebSockets communication between the user interface and the CritterWatch server, and we’re utilizing Wolverine’s SignalR messaging transport (which was purposefully built for CritterWatch in the first place) to get that done. In the CritterWatch codebase, we have this little bit of Wolverine configuration:

    public static void AddCritterWatchServices(this WolverineOptions opts, NpgsqlDataSource postgresSource)
    {
        // Much more of course...
        opts.Services.AddWolverineHttp();
        
        opts.UseSignalR();
        
        // The publishing rule to route any message type that implements
        // a marker interface to the connected SignalR Hub
        opts.Publish(x =>
        {
            x.MessagesImplementing<ICritterStackWebSocketMessage>();
            x.ToSignalR();
        });


        // Really need this so we can handle messages in order for 
        // a particular service
        opts.MessagePartitioning.UseInferredMessageGrouping();
        opts.Policies.AllListeners(x => x.PartitionProcessingByGroupId(PartitionSlots.Five));
    }

And at the bottom of the ASP.Net Core application hosting CritterWatch, we’ll have this to configure the request pipeline:

builder.Services.AddWolverineHttp();
var app = builder.Build();
// Little bit more in the real code of course...
app.MapWolverineSignalRHub("/api/messages");
return await app.RunJasperFxCommands(args);

As you can infer from the Wolverine publishing rule above, we’re using a marker interface to let Wolverine “know” what messages should always be sent to SignalR:

/// <summary>
/// Marker interface for all messages that are sent to the CritterWatch web client
/// via web sockets
/// </summary>
public interface ICritterStackWebSocketMessage : ICritterWatchMessage, WebSocketMessage;

We also use that marker interface in a homegrown command line integration to generate TypeScript versions of all those messages with NJsonSchema as well as message types that go from the user interface to the CritterWatch server. Wolverine’s SignalR integration assumes that all messages sent to SignalR or received from SignalR are wrapped in a Cloud Events compliant JSON wrapper, but the only required members are type that should identify what type of message it is and data that holds the actual message body as JSON. To make this easier, when we generate the TypeScript code we also insert a little method like this that we can use to identify the message type sent from the client to the Wolverine powered back end:

export class CompactStreamResult implements WebsocketMessage {
serviceName!: string;
streamId!: string;
success!: boolean;
error!: string | undefined;
queryId!: string | undefined;
// THIS method is injected by our custom codegen
// and helps us communicate with the server as
// this matches Wolverine's internal identification of
// this message
get messageTypeName() : string{
return "compact_stream_result";
}
// other stuff...
init(_data?: any) {
if (_data) {
this.serviceName = _data["serviceName"];
this.streamId = _data["streamId"];
this.success = _data["success"];
this.error = _data["error"];
this.queryId = _data["queryId"];
}
}
static fromJS(data: any): CompactStreamResult {
data = typeof data === 'object' ? data : {};
let result = new CompactStreamResult();
result.init(data);
return result;
}
toJSON(data?: any) {
data = typeof data === 'object' ? data : {};
data["serviceName"] = this.serviceName;
data["streamId"] = this.streamId;
data["success"] = this.success;
data["error"] = this.error;
data["queryId"] = this.queryId;
return data;
}
}

Most of the code above is generated by NJsonSchema, but our custom codegen inserts in the get messageTypeName() method, that we use in the client side code below to wrap up messages to send back up to our server:

  async function sendMessage(msg: WebsocketMessage) {
    if (conn.state === HubConnectionState.Connected) {
      const payload = 'toJSON' in msg ? (msg as any).toJSON() : msg
      const cloudEvent = JSON.stringify({
        id: crypto.randomUUID(),
        specversion: '1.0',
        type: msg.messageTypeName,
        source: 'Client',
        datacontenttype: 'application/json; charset=utf-8',
        time: new Date().toISOString(),
        data: payload,
      })
      await conn.invoke('ReceiveMessage', cloudEvent)
    }
  }

In the reverse direction, we receive the raw message from a connected WebSocket with the SignalR client, interrogate the expected CloudEvents wrapper, figure out what the message type is from there, deserialize the raw JSON to the right TypeScript type, and generally just relay that to a Pinia store where all the normal Vue.js + Pinia reactive user interface magic happens.

export function relayToStore(data: any){
const servicesStore = useServicesStore();
const dlqStore = useDlqStore();
const metricsStore = useMetricsStore();
const projectionsStore = useProjectionsStore();
const durabilityStore = useDurabilityStore();
const eventsStore = useEventsStore();
const scheduledMessagesStore = useScheduledMessagesStore();
const envelope = typeof data === 'string' ? JSON.parse(data) : data;
switch (envelope.type){
case "dead_letter_details":
dlqStore.handleDeadLetterDetails(DeadLetterDetails.fromJS(envelope.data));
break;
case "dead_letter_queue_summary_results":
dlqStore.handleDeadLetterQueueSummaryResults(DeadLetterQueueSummaryResults.fromJS(envelope.data));
break;
case "all_service_summaries":
const allSummaries = AllServiceSummaries.fromJS(envelope.data);
servicesStore.handleAllServiceSummaries(allSummaries);
if (allSummaries.persistenceCounts) {
for (const pc of allSummaries.persistenceCounts) {
durabilityStore.handlePersistenceCountsChanged(pc);
}
}
if (allSummaries.metricsRollups) {
metricsStore.handleAllMetricsRollups(allSummaries.metricsRollups);
}
break;
case "summary_updated":
servicesStore.handleSummaryUpdated(SummaryUpdated.fromJS(envelope.data));
break;
case "agent_and_node_state_changed":
servicesStore.handleAgentAndNodeStateChanged(AgentAndNodeStateChanged.fromJS(envelope.data));
break;
case "service_summary_changed":
servicesStore.handleServiceSummaryChanged(ServiceSummaryChanged.fromJS(envelope.data));
break;
case "metrics_rollup":
metricsStore.handleMetricsRollup(MetricsRollup.fromJS(envelope.data));
break;
case "all_metrics_rollups":
metricsStore.handleAllMetricsRollups(AllMetricsRollups.fromJS(envelope.data));
break;
case "shard_states_changed":
projectionsStore.handleShardStatesChanged(ShardStatesChanged.fromJS(envelope.data));
break;
case "persistence_counts_changed":
durabilityStore.handlePersistenceCountsChanged(PersistenceCountsChanged.fromJS(envelope.data));
break;
case "stream_details":
eventsStore.handleStreamDetails(StreamDetails.fromJS(envelope.data));
break;
case "event_query_results":
eventsStore.handleEventQueryResults(EventQueryResults.fromJS(envelope.data));
break;
case "compact_stream_result":
eventsStore.handleCompactStreamResult(CompactStreamResult.fromJS(envelope.data));
break;
// *CASE ABOVE* -- do not remove this comment for the codegen please!
}
}

And that’s really it. I omitted some of our custom codegen code (because it’s hokey), but it doesn’t do much more than find the message types in the .NET code that are marked as going to or coming from the Vue.js client and writes them as matching TypeScript types.

But wait, Marten gets into the act too!

With the Marten + Wolverine integration through this:

        opts.Services.AddMarten(m =>
        {
            // Other stuff...
            m.Projections.Add<ServiceSummaryProjection>(ProjectionLifecycle.Async);
        }).IntegrateWithWolverine(w =>
        {
            w.UseWolverineManagedEventSubscriptionDistribution = true;
        });

Marten can also get into the SignalR act through its support for “Side Effects” in projections. As a certain projection for ServiceSummary is updated with new events in CritterWatch, we can raise messages reflecting the new changes in state to notify our clients with code like this from a SingleStreamProjection:

    public override ValueTask RaiseSideEffects(IDocumentOperations operations, IEventSlice<ServiceSummary> slice)
    {
        var hasShardStates = slice.Events().Any(x => x.Data is ShardStatesUpdated);

        if (hasShardStates)
        {
            var shardEvent = slice.Events().Last(x => x.Data is ShardStatesUpdated).Data as ShardStatesUpdated;
            slice.PublishMessage(new ShardStatesChanged(slice.Snapshot.Id, shardEvent!.States));
        }

        if (slice.Events().All(x => x.Data is IImpactsAgentOrNodes || x.Data is ShardStatesUpdated))
        {
            if (!hasShardStates)
            {
                slice.PublishMessage(new AgentAndNodeStateChanged(slice.Snapshot.Id, slice.Snapshot.Nodes, slice.Snapshot.Agents));
            }
        }
        else
        {
            slice.PublishMessage(new ServiceSummaryChanged(slice.Snapshot));
        }

        return new ValueTask();
    }

The Marten projection itself knows absolutely nothing about where those messages will go or how, but Wolverine kicks in to help its Critter Stack sibling and it deals with all the message delivery. The message types above all implement the ICritterStackWebSocketMessage interface, so they will get routed by Wolverine to SignalR. To rewind, the workflow here is:

  1. CritterWatch constantly receives messages from Wolverine applications with changes in state like new messaging endpoints being used, agents being reassigned, or nodes being started or shut down
  2. CritterWatch saves any changes in state as events to Marten (or later to SQL Server backed Polecat)
  3. The Marten async daemon processes those events to update CritterWatch’s ServiceSummary projection
  4. As pages of events are applied to individual services, Marten calls that RaiseSideEffects() method to relay some state changes to Wolverine, which will..
  5. Send those messages to SignalR based on Wolverine’s routing rules and on to the client side code which…
  6. Relays the incoming messages to the proper Pinia store

Summary

I won’t say that using Wolverine for processing and sending messages via SignalR is justified in every application, but it more than pays off if you have a highly interactive application that sends any number of messages between the user interface and the server.

Sometime last week I said online that no project is truly a failure if you learned something valuable from that effort that could help a later project succeed. When I wrote that I was absolutely thinking about the work shown above and relating that to a failed effort of mine called Storyteller (Redux + early React.js + roll your own WebSockets support on the server) that went nowhere in the end, but taught me a lot of valuable lessons about using WebSockets in a highly interactive application that has directly informed my work on CritterWatch.

Big Critter Stack Releases

The Critter Stack had a big day today with releases for both Marten and Wolverine.

First up, we have Marten 8.22 that included:

  • Lots of bug fixes, including several old LINQ related bugs and issues related to full text search that finally got addressed
  • Some improvements for the newer Composite Projections feature as users start to use it in real project work. Hat tip to Anne Erdtsieck on this one (and a JasperFx client needing an addition to it as well)
  • Some optimizations, including a potentially big one as Marten can now use a source generator to build some of the projection code that before depended on not perfectly efficient Expression compilation. This will impact “self aggregating” snapshot projections that use the Apply / Create / ShouldDelete conventions

Next, a giant Wolverine 5.16 release that brings:

  • Many, many bug fixes
  • Several small feature requests for our HTTP support
  • Improved resiliency for Kafka especially but also for any usage of external message brokers with Wolverine. See Sending Error Handling. Plus better error handling for durable listener endpoints when the transactional inbox database is unavailable
  • Wait, what? Wolverine has experimental support for CosmosDb as a transactional inbox/outbox and all of Wolverine’s declarative persistence helpers?
  • The ability to mark some message handlers or HTTP endpoints as opting out of automatic transactional middleware (for a JasperFx client). See this, but it applies to all persistence options.
  • Modular monolith usage improvements for a pair of JasperFx clients who are helping us stretch Wolverine to yet more use cases.
  • More to come on this, but we’ve recently slipped in Sqlite and Oracle support for Wolverine

Building a Greenfield System with the Critter Stack

JasperFx Software works hand in hand with our clients to improve our client’s outcomes on software projects using the “Critter Stack” (Marten and Wolverine). Based on our engagements with client projects as well as the greater Critter Stack user base, we’ve built up quite a few optional usages and settings in the two frameworks to solve specific technical challenges.

The unfortunate reality of managing a long lived application framework such as Wolverine or a complicated library like Marten is the need to both continuously improve the tools as well as trying really hard not to introduce regression errors to our clients when they upgrade tools. To that end, we’ve had to make several potentially helpful features be “opt in” in the tools, meaning that users have to explicitly turn on feature flag type settings for these features. A common cause of this is any change that introduces database schema changes as we try really hard to only do that in major version releases (Wolverine 5.0 added some new tables to SQL Server or PostgreSQL storage for example).

And yes, we’ve still introduced regression bugs in Marten or Wolverine far more times than I’d like, even with trying to be careful. In the end, I think the only guaranteed way to constantly and safely improve tools like the Critter Stack is to just be responsive to whatever problems slip through your quality gates and try to fix those problems quickly to regain trust.

With all that being said, let’s pretend we’re starting a greenfield project with the Critter Stack and we want to build in the best performing system possible with some added options for improved resiliency as well. To jump to the end state, this is what I’m proposing for a new optimized greenfield setup for users:

 var builder = Host.CreateApplicationBuilder();

builder.Services.AddMarten(m =>
{
    // Much more coming...
    m.Connection(builder.Configuration.GetConnectionString("marten"));

    // 50% improvement in throughput, less "event skipping"
    m.Events.AppendMode = EventAppendMode.Quick;
    // or if you care about the timestamps -->
    m.Events.AppendMode = EventAppendMode.QuickWithServerTimestamps;

    // 100% do this, but be aggressive about taking advantage of it
    m.Events.UseArchivedStreamPartitioning = true;

    // These cause some database changes, so can't be defaults,
    // but these might help "heal" systems that have problems
    // later
    m.Events.EnableAdvancedAsyncTracking = true;

    // Enables you to mark events as just plain bad so they are skipped
    // in projections from here on out.
    m.Events.EnableEventSkippingInProjectionsOrSubscriptions = true;

    // If you do this, just now you pretty well have to use FetchForWriting
    // in your commands
    // But also, you should use FetchForWriting() for command handlers 
    // any way
    // This will optimize the usage of Inline projections, but will force
    // you to treat your aggregate projection "write models" as being 
    // immutable in your command handler code
    // You'll want to use the "Decider Pattern" / "Aggregate Handler Workflow"
    // style for your commands rather than a self-mutating "AggregateRoot"
    m.Events.UseIdentityMapForAggregates = true;

    // Future proofing a bit. Will help with some future optimizations
    // for rebuild optimizations
    m.Events.UseMandatoryStreamTypeDeclaration = true;

    // This is just annoying anyway
    m.DisableNpgsqlLogging = true;
})
// This will remove some runtime overhead from Marten
.UseLightweightSessions()

.IntegrateWithWolverine(x =>
{
    // Let Wolverine do the load distribution better than
    // what Marten by itself can do
    x.UseWolverineManagedEventSubscriptionDistribution = true;
});

builder.Services.AddWolverine(opts =>
{
    // This *should* have some performance improvements, but would
    // require downtime to enable in existing systems
    opts.Durability.EnableInboxPartitioning = true;

    // Extra resiliency for unexpected problems, but can't be
    // defaults because this causes database changes
    opts.Durability.InboxStaleTime = 10.Minutes();
    opts.Durability.OutboxStaleTime = 10.Minutes();

    // Just annoying
    opts.EnableAutomaticFailureAcks = false;

    // Relatively new behavior that will store "unknown" messages
    // in the dead letter queue for possible recovery later
    opts.UnknownMessageBehavior = UnknownMessageBehavior.DeadLetterQueue;
});

using var host = builder.Build();

return await host.RunJasperFxCommands(args);

Now, let’s talk more about some of these settings…

Lightweight Sessions with Marten

The first option we’re going to explicitly add is to use “lightweight” sessions in Marten:

var builder = Host.CreateApplicationBuilder();

builder.Services.AddMarten(m =>
{
    // Elided configuration...
})
// This will remove some runtime overhead from Marten
.UseLightweightSessions()

By default, Marten will use a heavier version of IDocumentSession that incorporates an Identity Map internally to track documents (entities) already loaded by that session. Likewise, when you request to load an entity by its identity, Marten’s session will happily check if it has already loaded that entity and gives you the same object back to you without making the database call.

The identity map usage is mostly helpful when you have unclear or deeply nested call stacks where different elements of the code might try to load the same data as part of the same HTTP request or command handling. If you follow “Critter Stack” and what we call the best practices especially for Wolverine usage, you’ll know that we very strongly recommend against deep call stacks and excessive layering.

Moreover, I would argue that you should never need the identity map behavior if you were building a system with an idiomatic Critter Stack approach, so the default session type is actually harmful in that it adds extra runtime overhead. The “lightweight” sessions run leaner by completely eliminating all the dictionary storage and lookups.

Why you ask is the identity map behavior the default?

  1. We were originally designing Marten as a near drop in replacement for RavenDb in a big system, so we had to mimic that behavior right off the bat to be able to make the replacement in a timely fashion
  2. If we changed the default behavior, it can easily break code in existing systems that upgrade in ways that are very hard to predict and unfortunately hard to diagnose. And of course, this is most likely a problem in the exact kind of codebases that are hard to reason about. How do I know this and why am I so very certain this is so you ask? Scar tissue.

Marten’s Aggregation Projection Subsystem

Marten has very rich support for projecting events into read, write, or query models. While there are other capabilities as well, the most common usage is probably to aggregate related events into a singular view. Marten projections can be executed Live, meaning that Marten does the creation of the view by loading the target events into memory and building the view on the fly. Projections can also be executed Inline, meaning that the projected views are persisted as part of the same transaction that captures the events that apply to that projection. For this post though, I’m mostly talking about projections running asynchronously in the background as events are captured into the database (think eventual consistency).

Aggregate Projections in Marten combine some sort of grouping of events and process them to create a single aggregated document representing the state of those events. These projections come in two flavors:

Single Stream Projections create a rolled up view of all or a segment of the events within a single event stream. These projections are done either by using the SingleStreamProjection<TDoc, TId> base type or by creating a “self aggregating” Snapshot approach with conventional Create/Apply/ShouldDelete methods that mutate or evolve the snapshot based on new events.

Multi Stream Projections create a rolled up view of a user-defined grouping of events across streams. These projections are done by sub-classing the MultiStreamProjection<TDoc, TId> class and is further described in Multi-Stream Projections. An example of a multi-stream projection might be a “query model” within an accounting system of some sort that rolls up the value of all unpaid invoices by active client.

You can also use a MultiStreamProjection to create views that are a segment of a single stream over time or version. Imagine that you have a system that models the activity of a bank account with event sourcing. You could use a MultiStreamProjection to create a view that summarizes the activity of a single bank account within a calendar month.

The ability to use explicit code to define projections was hugely improved in the Marten 8.0 release.

Within your aggregation projection, you can express the logic about how Marten combines events into a view through either conventional methods (original, old school Marten) or through completely explicit code.

Within an aggregation, you have advanced options to:

Simple Example

The most common usage is to create a “write model” that projects the current state for a single stream, so on that note, let’s jump into a simple example.

I’m huge into epic fantasy book series, hence the silly original problem domain in the very oldest code samples. Hilariously, Marten has fielded and accepted pull requests that corrected our modeling of the timeline of the Lord of the Rings in sample code.

Martens on a Quest

Let’s say that we’re building a system to track the progress of a traveling party on a quest within an epic fantasy series like “The Lord of the Rings” or the “Wheel of Time” and we’re using event sourcing to capture state changes when the “quest party” adds or subtracts members. We might very well need a “write model” for the current state of the quest for our command handlers like this one:

public sealed record QuestParty(Guid Id, List<string> Members)
{
// These methods take in events and update the QuestParty
public static QuestParty Create(QuestStarted started) => new(started.QuestId, []);
public static QuestParty Apply(MembersJoined joined, QuestParty party) =>
party with
{
Members = party.Members.Union(joined.Members).ToList()
};
public static QuestParty Apply(MembersDeparted departed, QuestParty party) =>
party with
{
Members = party.Members.Where(x => !departed.Members.Contains(x)).ToList()
};
public static QuestParty Apply(MembersEscaped escaped, QuestParty party) =>
party with
{
Members = party.Members.Where(x => !escaped.Members.Contains(x)).ToList()
};
}

For a little more context, the QuestParty above might be consumed in a command handler like this:

public record AddMembers(Guid Id, int Day, string Location, string[] Members);
public static class AddMembersHandler
{
public static async Task HandleAsync(AddMembers command, IDocumentSession session)
{
// Fetch the current state of the quest
var quest = await session.Events.FetchForWriting<QuestParty>(command.Id);
if (quest.Aggregate == null)
{
// Bad quest id, do nothing in this sample case
}
var newMembers = command.Members.Where(x => !quest.Aggregate.Members.Contains(x)).ToArray();
if (!newMembers.Any())
{
return;
}
quest.AppendOne(new MembersJoined(command.Id, command.Day, command.Location, newMembers));
await session.SaveChangesAsync();
}
}

How Aggregation Works

Just to understand a little bit more about the capabilities of Marten’s aggregation projections, let’s look at the diagram below that tries to visualize the runtime workflow of aggregation projections inside of the Async Daemon background process:

How Aggregation Works
  1. The Daemon is constantly pushing a range of events at a time to an aggregation projection. For example, Events 1,000 to 2,000 by sequence number
  2. The aggregation “slices” the incoming range of events into a group of EventSlice objects that establishes a relationship between the identity of an aggregated document and the events that should be applied during this batch of updates for that identity. To be more concrete, a single stream projection for QuestParty would be creating an EventSlice for each quest id it sees in the current range of events. Multi-stream projections will have some kind of custom “slicing” or grouping. For example, maybe in our Quest tracking system we have a multi-stream projection that tries to track how many monsters of each type are defeated. That projection might “slice” by looking for all MonsterDefeated events across all streams and group or slice incoming events by the type of monster. The “slicing” logic is automatic for single stream projections, but will require explicit configuration or explicitly written logic for multi stream projections.
  3. Once the projection has a known list of all the aggregate documents that will be updated by the current range of events, the projection will fetch each persisted document, first from any active aggregate cache in memory, then by making a single batched request to the Marten document storage for any missing documents and adding these to any active cache (see Optimizing Performance for more information about the potential caching).
  4. The projection will execute any event enrichment against the now known group of EventSlice. This process gives you a hook to efficiently “enrich” the raw event data with extra data lookups from Marten document storage or even other sources.
  5. Most of the work as a developer is in the application or “Evolve” step of the diagram above. After the “slicing”, the aggregation has turned the range of raw event data into EventSlice objects that contain the current snapshot of a projected document by its identity (if one exists), the identity itself, and the events from within that original range that should be applied on top of the current snapshot to “evolve” it to reflect those events. This can be coded either with the conventional Apply/Create/ShouldDelete methods or using explicit code — which is almost inevitably means a switch statement. Using the QuestParty example again, the aggregation projection would get an EventSlice that contains the identity of an active quest, the snapshot of the current QuestParty document that is persisted by Marten, and the new MembersJoined et al events that should be applied to the existing QuestParty object to derive the new version of QuestParty.
  6. Just before Marten persists all the changes from the application / evolve step, you have the RaiseSideEffects() hook to potentially raise “side effects” like appending additional events based on the now updated state of the projected aggregates or publishing the new state of an aggregate through messaging (Wolverine has first class support for Marten projection side effects through its Marten integration into the full “Critter Stack”)
  7. For the current event range and event slices, Marten will send all aggregate document updates or deletions, new event appending operations, and even outboxed, outgoing messages sent via side effects (if you’re using the Wolverine integration) in batches to the underlying PostgreSQL database. I’m calling this out because we’ve constantly found in Marten development that command batching to PostgreSQL is a huge factor in system performance and the async daemon has been designed to try to minimize the number of network round trips between your application and PostgreSQL at every turn.
  8. Assuming the transaction succeeds for the current event range and the operation batch in the previous step, Marten will call “after commit” observers. This notification for example will release any messages raised as a side effect and actually send those messages via whatever is doing the actual publishing (probably Wolverine).

Marten happily supports immutable data types for the aggregate documents produced by projections, but also happily supports mutable types as well. The usage of the application code is a little different though.

Starting with Marten 8.0, we’ve tried somewhat to conform to the terminology used by the Functional Event Sourcing Decider paper by Jeremie Chassaing. To that end, the API now refers to a “snapshot” that really just means a version of the projection and “evolve” as the step of applying new events to an existing “snapshot” to calculate a new “snapshot.”

Easier Query Models with Marten

The Marten community made our first big release of the new year with 8.18 this morning. I’m particularly happy with a couple significant things in this release:

  1. We had 8 different contributors in just the last month of work this release represents
  2. Anne Erdtsieck did a lot to improve our documentation for using our multi-stream projections for advanced query model projections
  3. The entire documentation section on projections got a much needed revamp and now includes a lot more information about capabilities from our big V8 release last year. I’m hopeful that the new structure and content makes this crucial feature set more usable.
  4. We improved Marten’s event enrichment ability within projections to more easily and efficiently incorporate information from outside of the raw event data
  5. The “Composite or Chained Projections” feature has been something we’ve talked about as a community for years, and now we have it

The one consistent theme in those points is that Marten just got a lot better for our users for creating “query models” in systems.

Let’s Build a TeleHealth System!

I got to be a part of a project like this for a start up during the pandemic. Fantastic project with lots of great people. Even though I wasn’t able to use Marten on the project at that time (we used a hand rolled Event Sourcing solution with Node.JS + TypeScript), that project has informed several capabilities added to Marten in the years since including the features shown in this post.

Just to have a problem domain for the sample code, let’s pretend that we’re building a new only TeleHealth system that allows patients to register for an appointment online and get matched up with a healthcare provider for an appointment that day. The system will do all the work of coordinating these appointments as well as tracking how the healthcare providers spend their time.

That domain might have some plain Marten document storage for reference data including:

  • Provider — representing a medical provider (Nurse? Physician? PA?) who fields appointments
  • Specialty — models a medical specialty
  • Patient — personal information about patients who are requesting appointments in our system

Switching to event streams, we may be capturing events for:

  • Board – events modeling a single, closely related group of appointments during a single day. Think of “Pediatrics in Austin, Texas for January 19th”
  • ProviderShift – events modeling the activity of a single provider working in a single Board during a single day
  • Appointment – events recording the progress of an appointment including requesting an appointment through the appointment being cancelled or completed

Better Query Models

The easiest and most common form of a projection in Marten is a simple “write model” that projects the information from a single event stream to a projected document. From our TeleHealth domain, here’s the “self-aggregating” Board:

public class Board
{
private Board()
{
}
public Board(BoardOpened opened)
{
Name = opened.Name;
Activated = opened.Opened;
Date = opened.Date;
}
public void Apply(BoardFinished finished)
{
Finished = finished.Timestamp;
}
public void Apply(BoardClosed closed)
{
Closed = closed.Timestamp;
CloseReason = closed.Reason;
}
public Guid Id { get; set; }
public string Name { get; private set; }
public DateTimeOffset Activated { get; set; }
public DateTimeOffset? Finished { get; set; }
public DateOnly Date { get; set; }
public DateTimeOffset? Closed { get; set; }
public string CloseReason { get; private set; }
}

Easy money. All the projection has to do is apply the raw event data for that one stream and nothing else. Marten is even doing the event grouping for you, so there’s just not much to think about at all.

Now let’s move on to more complicated usages. One of the things that makes Marten such a great platform for Event Sourcing is that it also has its dedicated document database feature set on top of the PostgreSQL engine. All that means that you can happily keep some relatively static reference data back in just plain ol’ documents or even raw database tables.

To that end, let’s say in our TeleHealth system that we want to just embed all the information for a Provider (think a nurse or a physician) directly into our ProviderShift for easier usage:

public class ProviderShift(Guid boardId, Provider provider)
{
public Guid Id { get; set; }
public int Version { get; set; }
public Guid BoardId { get; private set; } = boardId;
public Guid ProviderId => Provider.Id;
public ProviderStatus Status { get; set; } = ProviderStatus.Paused;
public string Name { get; init; }
public Guid? AppointmentId { get; set; }
// I was admittedly lazy in the testing, so I just
// completely embedded the Provider document directly
// in the ProviderShift for easier querying later
public Provider Provider { get; set; } = provider;
}

When mixing and matching document storage and events, Marten has always given you the ability to utilize document data during projections by brute force lookups in your projection code like this:

    public async Task<ProviderShift> Create(
        // The event data
        ProviderJoined joined, 
        IQuerySession session)
    {
        var provider = await session
            .LoadAsync<Provider>(joined.ProviderId);

        return new ProviderShift(joined.BoardId, provider);
    }

The code above is easy to write and conceptually easy to understand, but when the projection is being executed in our async daemon where the projection is processing a large batch of events at one time, the code above potentially sets you up for an N+1 query anti-pattern where Marten has to make lots of small database round trips to get each referenced Provider every time there’s a separate ProviderJoined event.

Instead, let’s use Marten’s recent hook for event enrichment and the new declarative syntax we just introduced in 8.18 today to get all the related Provider information in one batched query for maximum efficiency:

    public override async Task EnrichEventsAsync(SliceGroup<ProviderShift, Guid> group, IQuerySession querySession, CancellationToken cancellation)
    {
        await group

            // First, let's declare what document type we're going to look up
            .EnrichWith<Provider>()

            // What event type or marker interface type or common abstract type
            // we could look for within each EventSlice that might reference
            // providers
            .ForEvent<ProviderJoined>()

            // Tell Marten how to find an identity to look up
            .ForEntityId(x => x.ProviderId)

            // And finally, execute the look up in one batched round trip,
            // and apply the matching data to each combination of EventSlice, event within that slice
            // that had a reference to a ProviderId, and the Provider
            .EnrichAsync((slice, e, provider) =>
            {
                // In this case we're swapping out the persisted event with the
                // enhanced event type before each event slice is then passed
                // in for updating the ProviderShift aggregates
                slice.ReplaceEvent(e, new EnhancedProviderJoined(e.Data.BoardId, provider));
            });
    }

Now, inside the actual projection for ProviderShift, we can use the EnhancedProviderJoined event from above like this:

    // This is a recipe introduced in Marten 8 to just write explicit code
    // to "evolve" aggregate documents based on event data
    public override ProviderShift Evolve(ProviderShift snapshot, Guid id, IEvent e)
    {
        switch (e.Data)
        {
            case EnhancedProviderJoined joined:
                snapshot = new ProviderShift(joined.BoardId, joined.Provider)
                {
                    Provider = joined.Provider, Status = ProviderStatus.Ready
                };
                break;

            case ProviderReady:
                snapshot.Status = ProviderStatus.Ready;
                break;

            case AppointmentAssigned assigned:
                snapshot.Status = ProviderStatus.Assigned;
                snapshot.AppointmentId = assigned.AppointmentId;
                break;

            case ProviderPaused:
                snapshot.Status = ProviderStatus.Paused;
                snapshot.AppointmentId = null;
                break;

            case ChartingStarted charting:
                snapshot.Status = ProviderStatus.Charting;
                break;
        }

        return snapshot;
    }

In the sample above, I replaced the ProviderJoined event being sent to our projection with the richer EnhancedProviderJoined event, but there are other ways to send data to projections with a new References<T> event type that’s demonstrated in our documentation on this feature.

Sequential or Composite Projections

This feature was introduced in Marten 8.18 in response to feedback from several JasperFx Software clients who needed to efficiently create projections that effectively made de-normalized views across multiple stream types and used reference data outside of the events. Expect this feature to grow in capability as we get more feedback about its usage.

Here are a handful of scenarios that Marten users have hit over the years:

  • Wanting to use the build products of Projection 1 as an input to Projection 2. You can do that today by running Projection 1 as Inline and Projection 2 as Async, but that’s imperfect and sensitive to timing. Plus, you might not have wanted to run the first projection Inline.
  • Needing to create a de-normalized projection view that incorporates data from several other projections and completely different types of event streams, but that previously required quite a bit of duplicated logic between projections
  • Looking for ways to improve the throughput of asynchronous projections by doing more batching of event fetching and projection updates by trying to run multiple projections together

To meet these somewhat common needs more easily, Marten has introduced the concept of a “composite” projection where Marten is able to run multiple projections together and possibly divided into multiple, sequential stages. This provides some potential benefits by enabling you to safely use the build products of one projection as inputs to a second projection. Also, if you have multiple projections using much of the same event data, you can wring out more runtime efficiency by building the projections together so your system is doing less work fetching events and able to make updates to the database with fewer network round trips through bigger batches.

In our TeleHealth system, we need to have single stream “write model” projections for each of the three stream types. We also need to have a rich view of each Board that combines all the common state of the active Appointment and ProviderShift streams in that Board including the more static Patient and Provider information that can be used by the system to automate the assignment of providers to open patients (a real telehealth system would need to be able to match up the requirements of an appointment with the licensing, specialty, and location of the providers as well as “knowing” what providers are available or estimated to be available). We probably also need to build a denormalized “query model” about all appointments that can be efficiently queried by our user interface on any of the elements of BoardAppointmentPatient, or Provider.

What we really want is some way to efficiently utilize the upstream products and updates of the BoardAppointment, and ProviderShift “write model” projections as inputs to what we’ll call the BoardSummary and AppointmentDetails projections. We’ll use the new “composite projection” feature to run these projections together in two stages like this:

Before we dive into each child projection, this is how we can set up the composite projection using the StoreOptions model in Marten:

opts.Projections.CompositeProjectionFor("TeleHealth", projection =>
{
projection.Add<ProviderShiftProjection>();
projection.Add<AppointmentProjection>();
projection.Snapshot<Board>();
// 2nd stage projections
projection.Add<AppointmentDetailsProjection>(2);
projection.Add<BoardSummaryProjection>(2);
});

First, let’s just look at the simple ProviderShiftProjection:

public class ProviderShiftProjection: SingleStreamProjection<ProviderShift, Guid>
{
public ProviderShiftProjection()
{
// Make sure this is turned on!
Options.CacheLimitPerTenant = 1000;
}
public override async Task EnrichEventsAsync(SliceGroup<ProviderShift, Guid> group, IQuerySession querySession, CancellationToken cancellation)
{
await group
// First, let's declare what document type we're going to look up
.EnrichWith<Provider>()
// What event type or marker interface type or common abstract type
// we could look for within each EventSlice that might reference
// providers
.ForEvent<ProviderJoined>()
// Tell Marten how to find an identity to look up
.ForEntityId(x => x.ProviderId)
// And finally, execute the look up in one batched round trip,
// and apply the matching data to each combination of EventSlice, event within that slice
// that had a reference to a ProviderId, and the Provider
.EnrichAsync((slice, e, provider) =>
{
// In this case we're swapping out the persisted event with the
// enhanced event type before each event slice is then passed
// in for updating the ProviderShift aggregates
slice.ReplaceEvent(e, new EnhancedProviderJoined(e.Data.BoardId, provider));
});
}
public override ProviderShift Evolve(ProviderShift snapshot, Guid id, IEvent e)
{
switch (e.Data)
{
case EnhancedProviderJoined joined:
snapshot = new ProviderShift(joined.BoardId, joined.Provider)
{
Provider = joined.Provider, Status = ProviderStatus.Ready
};
break;
case ProviderReady:
snapshot.Status = ProviderStatus.Ready;
break;
case AppointmentAssigned assigned:
snapshot.Status = ProviderStatus.Assigned;
snapshot.AppointmentId = assigned.AppointmentId;
break;
case ProviderPaused:
snapshot.Status = ProviderStatus.Paused;
snapshot.AppointmentId = null;
break;
case ChartingStarted charting:
snapshot.Status = ProviderStatus.Charting;
break;
}
return snapshot;
}
}

Now, let’s go downstream and look at the AppointmentDetailsProjection that will ultimately need to use the build products of all three upstream projections:

public class AppointmentDetailsProjection : MultiStreamProjection<AppointmentDetails, Guid>
{
public AppointmentDetailsProjection()
{
Options.CacheLimitPerTenant = 1000;
Identity<Updated<Appointment>>(x => x.Entity.Id);
Identity<IEvent<ProviderAssigned>>(x => x.StreamId);
Identity<IEvent<AppointmentRouted>>(x => x.StreamId);
}
public override async Task EnrichEventsAsync(SliceGroup<AppointmentDetails, Guid> group, IQuerySession querySession, CancellationToken cancellation)
{
// Look up and apply specialty information from the document store
// Specialty is just reference data stored as a document in Marten
await group
.EnrichWith<Specialty>()
.ForEvent<Updated<Appointment>>()
.ForEntityId(x => x.Entity.Requirement.SpecialtyCode)
.AddReferences();
// Also reference data (for now)
await group
.EnrichWith<Patient>()
.ForEvent<Updated<Appointment>>()
.ForEntityId(x => x.Entity.PatientId)
.AddReferences();
// Look up and apply provider information
await group
.EnrichWith<Provider>()
.ForEvent<ProviderAssigned>()
.ForEntityId(x => x.ProviderId)
.AddReferences();
// Look up and apply Board information that matches the events being
// projected
await group
.EnrichWith<Board>()
.ForEvent<AppointmentRouted>()
.ForEntityId(x => x.BoardId)
.AddReferences();
}
public override AppointmentDetails Evolve(AppointmentDetails snapshot, Guid id, IEvent e)
{
switch (e.Data)
{
case AppointmentRequested requested:
snapshot ??= new AppointmentDetails(e.StreamId);
snapshot.SpecialtyCode = requested.SpecialtyCode;
snapshot.PatientId = requested.PatientId;
break;
// This is an upstream projection. Triggering off of a synthetic
// event that Marten publishes from the early stage
// to this projection running in a secondary stage
case Updated<Appointment> updated:
snapshot ??= new AppointmentDetails(updated.Entity.Id);
snapshot.Status = updated.Entity.Status;
snapshot.EstimatedTime = updated.Entity.EstimatedTime;
snapshot.SpecialtyCode = updated.Entity.SpecialtyCode;
break;
case References<Patient> patient:
snapshot.PatientFirstName = patient.Entity.FirstName;
snapshot.PatientLastName = patient.Entity.LastName;
break;
case References<Specialty> specialty:
snapshot.SpecialtyCode = specialty.Entity.Code;
snapshot.SpecialtyDescription = specialty.Entity.Description;
break;
case References<Provider> provider:
snapshot.ProviderId = provider.Entity.Id;
snapshot.ProviderFirstName = provider.Entity.FirstName;
snapshot.ProviderLastName = provider.Entity.LastName;
break;
case References<Board> board:
snapshot.BoardName = board.Entity.Name;
snapshot.BoardId = board.Entity.Id;
break;
}
return snapshot;
}
}

And also the definition for the downstream BoardSummary view:

public class BoardSummaryProjection: MultiStreamProjection<BoardSummary, Guid>
{
public BoardSummaryProjection()
{
Options.CacheLimitPerTenant = 100;
Identity<Updated<Appointment>>(x => x.Entity.BoardId ?? Guid.Empty);
Identity<Updated<Board>>(x => x.Entity.Id);
Identity<Updated<ProviderShift>>(x => x.Entity.BoardId);
}
public override Task EnrichEventsAsync(SliceGroup<BoardSummary, Guid> group, IQuerySession querySession, CancellationToken cancellation)
{
return group.ReferencePeerView<Board>();
}
public override (BoardSummary, ActionType) DetermineAction(BoardSummary snapshot, Guid identity, IReadOnlyList<IEvent> events)
{
snapshot ??= new BoardSummary { Id = identity };
if (events.TryFindReference<Board>(out var board))
{
snapshot.Board = board;
}
var shifts = events.AllReferenced<ProviderShift>().ToArray();
foreach (var providerShift in shifts)
{
snapshot.ActiveProviders[providerShift.ProviderId] = providerShift;
if (providerShift.AppointmentId.HasValue)
{
snapshot.Unassigned.Remove(providerShift.ProviderId);
}
}
foreach (var appointment in events.AllReferenced<Appointment>())
{
if (appointment.ProviderId == null)
{
snapshot.Unassigned[appointment.Id] = appointment;
snapshot.Assigned.Remove(appointment.Id);
}
else
{
snapshot.Unassigned.Remove(appointment.Id);
var shift = shifts.FirstOrDefault(x => x.Id == appointment.ProviderId.Value);
snapshot.Assigned[appointment.Id] = new AssignedAppointment(appointment, shift?.Provider);
}
}
return (snapshot, ActionType.Store);
}
}

Note the usage of the Updated<T> event types that the downstream projections are using in their Evolve or DetermineAction methods. That is a synthetic event added by Marten to communicate to the downstream projections what projected documents were updated for the current event range. These events are carrying the latest snapshot data for the current event range so the downstream projections can just use the build products without making any additional fetches. It also guarantees that the downstream projections are seeing the exact correct upstream projection data for that point of the event sequencing.

Moreover, the composite “telehealth” projection is reading the event range once for all five constituent projections, and also applying the updates for all five projections at one time to guarantee consistency.

Some the documentation on Composite Projections for more information about how this feature fits it with rebuilding, versioning, and non stale querying.

Summary

Marten has hopefully gotten much better at building “query model” projections that you’d use for bigger dashboard screens or search within your application. We’re hoping that this makes Marten a better tool for real life development.

The best way for an OSS project to grow healthily is having a lot of user feedback and engagement coupled with the maintainers reacting to that feedback with constant improvement. And while I’d sometimes like to have the fire hose of that “feedback” stop for a couple days, it helps drive the tools forward.

The advent of JasperFx Software has enabled me to spend much more time working with our users and seeing the real problems they face in their system development. The features I described in this post are a direct result of engagements with at least four different JasperFx clients in the past year and a half. Drop us a line anytime at sales@jasperfx.net and I’d be happy to talk to you about how we can help you be more successful with Event Sourcing using Marten.