CQRS Command Handlers with Marten

Hey, did you know that JasperFx Software offers both consulting services and support plans for the “Critter Stack” tools? One of the common areas where we’ve helped our clients is in using Marten or Wolverine when the usage involves quite a bit of potential concerns about concurrency. As I write this, I’m currently working with a JasperFx client to implement the FetchForWriting API shown in this post as a way of improving their system’s resiliency to concurrency problems.

You’ve decided to use event sourcing as your persistence strategy, so that your persisted state of record are the actual business events segregated by streams that represent changes in state to some kind of logical business entity (an invoice? an order? an incident? a project?). Of course there will have to be some way of resolving or “projecting” the raw events into a usable view of the system state, but we’ll get to that.

You’ve also decided to organize your system around a CQRS architectural style (Command Query Responsibility Segregation). With a CQRS approach, the backend code is mostly organized around the “verbs” of your system, meaning the “command” messages (this could be HTTP services, and I’m not implying that there automatically has to be any asynchronous messaging) that are handled to capture changes to the system (events in our case), and “query” endpoints or APIs that strictly serve up information about your system.

While it’s certainly possible to do either Event Sourcing or CQRS without the other, the two things do go together as Forrest Gump would say, like peas and carrots. Marten is certainly valuable as part of a CQRS with Event Sourcing approach within a range of .NET messaging or web frameworks, but there is quite a bit of synergy between Marten and its “Critter Stack” stable mate Wolverine (see the details about the integration here).

And lastly of course, you’ve quite logically decided to use Marten as the persistence mechanism for the events. Marten is also a strong fit because it comes with some important functionality that we’ll need for CQRS command handlers:

  • Marten’s event projection support can give us a representation of the current state of the raw event data in a usable way that we’ll need within our command handlers to both validate requested actions and to “decide” what additional events should be persisted to our system
  • The FetchForWriting API in Marten will not only give us access to the projected event data, but it provides an easy mechanism for both optimistic and pessimistic concurrency protections in our system
  • Marten allows for a couple different options of projection lifecycle that can be valuable for performance optimization with differing system needs

As a sample application problem domain, I got to be part of a successful effort during the worst of the pandemic to stand up a new “telehealth” web portal using event sourcing. One of the concepts in that system that we needed to track in that system was the activity of a health care provider (nurse, doctor, nurse practitioner) with events for when they were available and what they were doing at any particular time during the day for later decision making:

public record ProviderAssigned(Guid AppointmentId);

public record ProviderJoined(Guid BoardId, Guid ProviderId);

public record ProviderReady;

public record ProviderPaused;

public record ProviderSignedOff;

// "Charting" is basically just whatever
// paperwork they need to do after
// an appointment, and it was important
// for us to track that time as part
// of their availability and future
// planning
public record ChartingFinished;

public record ChartingStarted;

public enum ProviderStatus
{
    Ready,
    Assigned,
    Charting,
    Paused
}

But of course, at several points, you do actually need to know what the actual state of the provider’s current shift is to be able to make decisions within the command handlers, so we had a “write” model something like this:

// I'm sticking the Marten "projection" logic for updating
// state from the events directly into this "write" model,
// but you could separate that into a different class if you
// prefer
public class ProviderShift
{
    public Guid Id { get; set; }

    // This is important, this would be set by Marten to the 
    // current event number or revision of the ProviderShift
    // aggregate. This is going to be important later for
    // concurrency protections
    public int Version { get; set; }
    public Guid BoardId { get; private set; }
    public Guid ProviderId { get; init; }
    public ProviderStatus Status { get; private set; }
    public string Name { get; init; }
    public Guid? AppointmentId { get; set; }
    
    // The Create & Apply methods are conventional targets
    // for Marten's "projection" capabilities
    // But don't worry, you would never *have* to take a reference
    // to Marten itself like I did below jsut out of laziness
    public static ProviderShift Create(
        ProviderJoined joined)
    {
        return new ProviderShift
        {
            Status = ProviderStatus.Ready,
            ProviderId = joined.ProviderId,
            BoardId = joined.BoardId
        };
    }

    public void Apply(ProviderReady ready)
    {
        AppointmentId = null;
        Status = ProviderStatus.Ready;
    }

    public void Apply(ProviderAssigned assigned)
    {
        Status = ProviderStatus.Assigned;
        AppointmentId = assigned.AppointmentId;
    }

    public void Apply(ProviderPaused paused)
    {
        Status = ProviderStatus.Paused;
        AppointmentId = null;
    }

    // This is kind of a catch all for any paperwork the
    // provider has to do after an appointment has ended
    // for the just concluded appointment
    public void Apply(ChartingStarted charting)
    {
        Status = ProviderStatus.Charting;
    }
}

The whole purpose of the ProviderShift type above is to be a “write” model that contains enough information for the command handlers to “decide” what should be done — as opposed to a “read” model that might have richer information like the provider’s name that would be more suitable or usable for using within a user interface. “Write” or “read” in this case is just a role within the system, and at different times it might be valuable to have separate models for different consumers of the information and in other times be able to happily get by with a single model.

Alright, so let’s finally look at a very simple command handler related to providers that tries to mark the provider as being finished charting:

// Since we're focusing on Marten, I'm using an MVC Core
// controller just because it's commonly used and understood
public class CompleteChartingController : ControllerBase
{
    [HttpPost("/provider/charting/complete")]
    public async Task Post(
        [FromBody] CompleteCharting charting,
        [FromServices] IDocumentSession session)
    {
        // We're looking up the current state of the ProviderShift aggregate
        // for the designated provider
        var stream = await session
            .Events
            .FetchForWriting<ProviderShift>(charting.ProviderShiftId, HttpContext.RequestAborted);

        // The current state
        var shift = stream.Aggregate;
        
        if (shift.Status != ProviderStatus.Charting)
        {
            // Obviously do something smarter in your app, but you 
            // get the point
            throw new Exception("The shift is not currently charting");
        }
        
        // Append a single new event just to say 
        // "charting is finished". I'm relying on 
        // Marten's automatic metadata to capture
        // the timestamp of this event for me
        stream.AppendOne(new ChartingFinished());

        // Commit the transaction
        await session.SaveChangesAsync();
    }
}

I’m using the Marten FetchForWriting() API to get at the current state of the event stream for the designated provider shift (a provider’s activity during a single day). I’m also using this API to capture a new event marking the provider as being finished with charting. FetchForWriting() is doing two important things for us:

  1. Executes or finds the projected data for ProviderShift from the raw events. More on this a little later
  2. Provides a little bit of optimistic concurrency protection for our provider shift stream

Building on the theme of concurrency first, the command above will “remember” the current state of the ProviderShift at the point that FetchForWriting() is called. Upon SaveChangesAsync(), Marten will reject the transaction and throw a ConcurrencyException if some how, some way, some other request magically came through and completed against that same ProviderShift stream between the call to FetchForWriting() and SaveChangesAsync().

That level of concurrency is baked in, but we can do a little bit better. Remember that the ProviderShift has this property:

    // This is important, this would be set by Marten to the 
    // current event number or revision of the ProviderShift
    // aggregate. This is going to be important later for
    // concurrency protections
    public int Version { get; set; }

The projection capability of Marten makes it easy for us to “know” and track the current version of the ProviderShift stream so that we can feed it back to command handlers later. Here’s the full definition of the CompleteCharting command:

public record CompleteCharting(
    Guid ProviderShiftId,
    
    // This version is meant to mean "I was issued
    // assuming that the ProviderShift is currently
    // at this version in the server, and if the version
    // has shifted since, then this command is now invalid"
    int Version
);

Let’s tighten up the optimistic concurrency protection such that Marten will shut down the command handling faster before we waste system resources doing unnecessary work by passing the command version right into this overload of FetchForWriting():

// Since we're focusing on Marten, I'm using an MVC Core
// controller just because it's commonly used and understood
public class CompleteChartingController : ControllerBase
{
    [HttpPost("/provider/charting/complete")]
    public async Task Post(
        [FromBody] CompleteCharting charting,
        [FromServices] IDocumentSession session)
    {
        // We're looking up the current state of the ProviderShift aggregate
        // for the designated provider
        var stream = await session
            .Events
            .FetchForWriting<ProviderShift>(
                charting.ProviderShiftId, 
                
                // Passing the expected, starting version of ProviderShift
                // into Marten
                charting.Version,
                HttpContext.RequestAborted);

        // And the rest of the controller stays the same as
        // before....
    }
}

In the usage above, Marten will do a version check both at the point of FetchForWriting() using the version we passed in, and again during the call to SaveChangesAsync() to reject any changes made if there was a concurrent update to that same stream.

Lastly, Marten gives you the ability to opt into heavier, exclusive access to the ProviderShift with this option:

// We're looking up the current state of the ProviderShift aggregate
// for the designated provider
var stream = await session
    .Events
    .FetchForExclusiveWriting<ProviderShift>(
        charting.ProviderShiftId, 
        HttpContext.RequestAborted);

In that last usage, we’re relying on the underlying PostgreSQL database to get us an exclusive row lock on the ProviderShift event stream such that only our current operation is allowed to write to that event stream while we have the lock. This is heavier and comes with some risk of database locking problems, but solves the concurrency issue.

So that’s concurrency protection in FetchForWriting(), but I mostly skipped over when and how that API will execute the projection logic to go from the raw events like ProviderJoined, ProviderReady, or ChartingStarted to the projected ProviderShift.

Any projection in Marten can be calculated or executed with three different “projection lifecycles”:

  1. Live — in this case, a projection is calculated on the fly by loading the raw events in memory and calculating the current state right then and there. In the absence of any other configuration, this is the default lifecycle for the ProviderShift per stream aggregation.
  2. Inline — a projection is updated at the time any events are appended by Marten and persisted by Marten as a document in the PostgreSQL database.
  3. Async — a projection is updated in a background process as events are captured by Marten across the system. The projected state is persisted as a Marten document to the underlying PostgreSQL database

The first two options give you strong consistency models where the projection will always reflect the current state of the events captured to the database. Live is probably a little more optimal in the case where you have many writes, but few reads, and you want to optimize the “write” side. Inline is optimal for cases where you have few writes, but many reads, and you want to optimize the “read” side (or need to query against the projected data rather than just load by id). The Async model gives you the ability to take the work of projecting events into the aggregated state out of both the “write” and “read” side of things. This might easily be advantageous for performance and very frequently necessary for ordering or concurrency concerns.

In the case of the FetchForWriting() API, you will always have a strongly consistent view of the raw events because that API is happily wallpapering over the lifecycle for you. Live aggregation works as you’d expect, Inline aggregation works by just loading the expected document directly from the database, and Async aggregation is a hybrid model that starts from the last known persisted value for the aggregate and applies any missing events right on top of that (the async behavior was a big feature added in Marten 7).

By hiding the actual lifecycle behavior behind the FetchForWriting() signature, teams are able to experiment with different approaches to optimize their application without breaking existing code.

Summary

FetchForWriting() was built specifically to ease the usage of Marten within CQRS command handlers after seeing how much boilerplate code teams were having to use before with Marten. At this point, this is our strongly recommended approach for command handlers. Also note that this API is utilized within the Wolverine + Marten “aggregate handler workflow” usage that does even more to remove code ceremony from CQRS command handler code. To some degree, what is now Wolverine was purposely rebooted and saved from the scrap heap specifically because of that combination with Marten and the FetchForWriting API.

Personally, I’m opposed to any kind of IAggregateRepository or approach where the “write” model itself tracks the events that are applied or uncommitted. I’m trying hard to discourage folks using Marten away from this still somewhat popular old approach in favor of a more Functional Programming-ish approach.

FetchForWriting could be used as part of a homegrown “Decider” pattern usage if that’s something you prefer (I think the “decider” pattern in real life usage ends up devolving into brute force procedural code through massive switch statements personally).

The “telehealth” system I mentioned before was built in real life with a hand-rolled Node.js event sourcing implementation, but that experience has had plenty of influence over later Marten work including a feature that just went into Marten over the weekend for a JasperFx client to be able to emit “side effect” actions and messages during projection updates.

I was deeply unimpressed with the existing Node.js tooling for event sourcing at that time (~2020), but I would hope it’s much better now. Marten has absolutely grown in capability in the past couple years.

Wolverine’s New Message Batching

Hey, did you know that JasperFx Software offers both consulting services and support plans for the “Critter Stack” tools? The new feature shown in this post was done at the behest of a JasperFx support customer. And of course, we’re also more than happy to help you with any kind of .NET backend development:)

Wolverine‘s new 3.0.0-beta-1 release adds a long requested feature set for batching up message handling. What does that mean? Well, sometimes you might want to process a stream of incoming messages in batches rather than one at a time. This might be for performance reasons, or maybe there’s some kind of business logic that makes more sense to calculate for batches, or maybe you want a logical “debounce” in how your system responds to the incoming messages so you don’t update some resource on every single message received by your system.

To that end, Wolverine 3.0 introduces a new feature for batching up message handling within a Wolverine system. Let’s say that you have a message type in your system like this:

public record Item(string Name);

And for whatever reason, we need to process these messages in batches. To do that, we first need to have a message handler for an array of Item like so:

public static class ItemHandler
{
    public static void Handle(Item[] items)
    {
        // Handle this just like a normal message handler,
        // just that the message type is Item[]
    }
}

And yes, before you ask, so far Wolverine only understands an array of the batched message type as the input message for the batch handler.

With that in our system, now we need to tell Wolverine to group Item messages, and we do that with the following syntax:

theHost = await Host.CreateDefaultBuilder()
    .UseWolverine(opts =>
    {
        opts.BatchMessagesOf<Item>(batching =>
        {
            // Really the maximum batch size
            batching.BatchSize = 500;
            
            // You can alternatively override the local queue
            // for the batch publishing. 
            batching.LocalExecutionQueueName = "items";

            // We can tell Wolverine to wait longer for incoming
            // messages before kicking out a batch if there
            // are fewer waiting messages than the maximum
            // batch size
            batching.TriggerTime = 1.Seconds();
            
        })
            
            // The object returned here is the local queue configuration that
            // will handle the batched messages. This may be useful for fine
            // tuning the behavior of the batch processing
            .Sequential();
    }).StartAsync();

A particularly lazy and hopefully effective technique in OSS project documentation is to take code snippets directly out of test code, and that’s what you see above. Two birds with one stone. Sometimes that works out well.

And that’s that! Just to bring this a little more into focus, here’s an end to end test from the Wolverine codebase — which just for clarity, is using Wolverine’s built in test automation support for end to end testing:

[Fact]
public async Task send_end_to_end_with_batch()
{
    // Items to publish
    var item1 = new Item("one");
    var item2 = new Item("two");
    var item3 = new Item("three");
    var item4 = new Item("four");

    Func<IMessageContext, Task> publish = async c =>
    {
        // I'm publishing the 4 items in sequence
        await c.PublishAsync(item1);
        await c.PublishAsync(item2);
        await c.PublishAsync(item3);
        await c.PublishAsync(item4);
    };

    // This is the "act" part of the test
    var session = await theHost.TrackActivity()
        
        // Wolverine testing helper to "wait" until
        // the tracking receives a message of Item[]
        .WaitForMessageToBeReceivedAt<Item[]>(theHost)
        .ExecuteAndWaitAsync(publish);

    // The four Item messages should be processed as a single 
    // batch message
    var items = session.Executed.SingleMessage<Item[]>();

    items.Length.ShouldBe(4);
    items.ShouldContain(item1);
    items.ShouldContain(item2);
    items.ShouldContain(item3);
    items.ShouldContain(item4);
}

Alright, with all that being said, here’s a few more facts about the batch messaging support:

  1. There is absolutely no need to create a specific message handler for the Item message, and in fact, you should not do so
  2. The message batching is able to group the message batches by tenant id if your Wolverine system uses multi-tenancy
  3. If you are using a durable inbox in your system, Wolverine is not marking the incoming envelopes as handled until the messages are successfully handled inside a batch message
  4. Likewise, if a batch message fails to the point where it triggers a move to the dead letter queue, each individual message that was part of that original batch is moved to the dead letter queue separately

Summary

Hey, that’s actually all I had to say about that! Wolverine 3.0 will hopefully go RC later this week or next, with the official release *knock on wood* happening before I leave for Swetugg and a visit in Copenhagen with a JasperFx client in a couple weeks.

Time is a Flat Circle

This conceptual idea is apparently known to philosophers as “Eternal return” or “eternal recurrence.” Funny story for me, I worked with a business analyst one time who was a former philosopher. To purposely antagonize him, I gave a super amateurish explanation of Descartes writings and how I thought they applied to engineering in principle where I knew he could hear just to torture him while I butchered the subject. Coincidentally, this project had a bi-weekly mandatory team morale meeting. Go figure.

I can’t find the source of this quote today, but I’ve frequently heard and repeatedly used the following phrase to other folks (maybe from here?):

Software is never finished, only abandoned

I predominantly work with long lived codebases for software tools used by other developers. A very common source of frustration for me is making a bug fix release to burn down the open issue list, only to have all new issues get raised in the next couple hours. To some degree, any complicated software project is naturally going to feel like a Sisyphean task if you think of time as a straight line to being “finished” with the project for good.

Maybe this is a special problem for development tools (and I’ve worked on enough long lived business systems to say that nope, this is a pretty common issue for any long lived codebase), but no matter how hard I and the rest of the Critter Stack community try, folks will continuously come out of the wood work to:

  • Report previously unknown bugs
  • Stumble on unanticipated usages that aren’t well supported
  • Hit performance or concurrency problems that haven’t come up before
  • Simply make compelling suggestions about some new kind of use case for the tooling
  • Need to integrate your tool with some different sort of infrastructure or even a different runtime
  • Point out gaps in the documentation
  • Describe content in the documentation that isn’t clear enough, or flat out misstated
  • Express frustration about information they want to find in the documentation, but cannot — even if it is there, but just not in a way that made sense for the user to find

It’s admittedly exhausting sometimes trying to be “done” with long lived projects. I think my advice — not that I always live this myself — is to think of these projects as more of a cycle and a continuous process of slow, steady improvement rather than any kind of concrete project to be completed. For me especially, I need to constantly remind myself that technical documentation has to constantly pruned and improved in the face of user feedback.

My other scattered pieces of advice for dealing with the ownership of long lived codebases is to:

  • Remember to celebrate the good things you’d done once in a while instead of always being focused on what’s not good yet
  • Grant yourself some grace and not let it weigh on you just because there are some open bug reports or open questions at the moment
  • Give yourself permission to unplug from message boards, chat rooms, or social media when you just need some mental rest when off work or even when you just need to focus during a work day. I sometimes have to completely switch off Discord some days when I’m at my limit of incoming questions or problems
  • Have reasonable expectations for how fast you should be dealing with user issues, bugs, and feature requests — with the caveat for me that it’s a very different thing when those come in from paying clients or maybe even just from other contributors
  • And lastly, if nothing else, unplug immediately and walk away if you find yourself either being or wanting to be snappish or brusk or sarcastic or rude to people asking for help online. Again, be better than I am at this one:)

I’ve worked over the past year with a couple clients who were building greenfield systems with “Critter Stack” tools, and let me tell you, that’s been a blessing for my mental health. Whenever you get to do greenfield work, appreciate that time.

What I use for interacting with Git

A friend of mine asked me at lunch this week what I was using to interact or manipulate Git, and I think he might have been disappointed with my lack of sophistication. At this point, I’m not willing to cast aspersions on whatever folks use as long as they’re being effective in their own environment, and there are definitely folks with way more Git-fu than I have.

Offhand though, I tend to use:

  • JetBrains Rider for making commits while I’m working just because it’s really easy because that’s just the window you’re already in. A quick “CMD-K” in my keyboard shortcut setup, type up the message, and hit return if nothing looks out of place. Honestly, I do far more amend commits just because of how easy that is with a user interface, and that might have actually changed my workflow a little bit from what it was 5 years ago.
  • With a codebase that has a fast build, I’m might instead do it at the command line with a [run build] && git commit -a -m "commit message"
  • If I really need to look closer at an ongoing commit or look closely at the recent change history, I’ll use the GitHub Desktop user interface.
  • I pretty much only use GitHub Desktop for cherry picking or squashing commits — which I rarely do, but I will just happen to need here in the next hour
  • For everything else, including pulls, pushes, and creating or deleting branches, I just do the 4-finger swipe over to an open terminal window and do it directly in the command line. Yes, you can certainly open a terminal window directly in Rider, but I just don’t have muscle memory for that
  • Just to add yet another tool, I really like using VS Core for merge conflicts. For whatever reason, that feels the easiest to me

There you go, what I do here and there. Not particularly advanced, but I don’t feel like I have to spend much time at all with Git.

Why and How Marten is a Great Document Database

Just a reminder, JasperFx Software offers support contracts and consulting services to help you get the most out of the “Critter Stack” tools (Marten and Wolverine). If you’re building server side applications on .NET, the Critter Stack is the most feature rich tool set for Event Sourcing and Event Driven Architectures around. And as I hope to prove to you in this post, Marten is a great option as a document database too!

Marten as a project started as an ultimately successful attempt to replace my then company’s usage of an early commercial “document database” with the open source PostgreSQL database — but with a small, nascent event store functionality bolted onto the side. With the exception of LINQ provider related issues, most of my attention these days is focused on the event sourcing side of things with the document database features in Marten just being a perfect complement for event projections.

This week and last though, I’ve had cause to work with a different document database option and it served to remind me that hey, Marten has a very strong technical story as a document database option. With that being said, let me get on with lionizing Marten by starting with a quick start.

Let’s say that you are building a server side .NET application with some kind of customer data and you at least start by modeling that data like so:

public class Customer
{
    public Guid Id { get; set; }

    // We'll use this later for some "logic" about how incidents
    // can be automatically prioritized
    public Dictionary<IncidentCategory, IncidentPriority> Priorities { get; set; }
        = new();

    public string? Region { get; set; }

    public ContractDuration Duration { get; set; }
}

public record ContractDuration(DateOnly Start, DateOnly End);

public enum IncidentCategory
{
    Software,
    Hardware,
    Network,
    Database
}

public enum IncidentPriority
{
    Critical,
    High,
    Medium,
    Low
}

And once you have those types, you’d like to have that customer data saved to a database in a way that makes it easy to persist, query, and load that data with minimal developmental cost while still being as robust as need be. Assuming that you have access to a running instance of PostgreSQL (it’s very Docker friendly and I tend to use that as a development default), bring in Marten by first adding a reference to the “Marten” Nuget. Next, write the following code in a simple console application that also contains the C# code from above:

using Marten;
using Newtonsoft.Json;

// Bootstrap Marten itself with default behaviors
await using var store = DocumentStore
    .For("Host=localhost;Port=5432;Database=marten_testing;Username=postgres;password=postgres");

// Build a Customer object to save
var customer = new Customer
{
    Duration = new ContractDuration(new DateOnly(2023, 12, 1), new DateOnly(2024, 12, 1)),
    Region = "West Coast",
    Priorities = new Dictionary<IncidentCategory, IncidentPriority>
    {
        { IncidentCategory.Database, IncidentPriority.High }
    }
};

// IDocumentSession is Marten's unit of work 
await using var session = store.LightweightSession();
session.Store(customer);
await session.SaveChangesAsync();

// Marten assigned an identity for us on Store(), so 
// we'll use that to load another copy of what was 
// just saved
var customer2 = await session.LoadAsync<Customer>(customer.Id);

// Just making a pretty JSON printout
Console.WriteLine(JsonConvert.SerializeObject(customer2, Formatting.Indented));

And that’s that, we’ve got a working usage of Marten to save, then load Customer data to the underlying PostgreSQL database. Right off the bat I’d like to point out a couple things about the code samples above:

  • We didn’t have to do any kind of mapping from our Customer type to a database structure. Marten is using JSON serialization to persist the data to the database, and as long as the Customer type can be bi-directionally serialized to and from JSON, Marten is going to be able to persist and load the type.
  • We didn’t specify or do anything about the actual database structure. In its default “just get things done” settings, Marten is able to happily detect that the necessary database objects for Customer are missing in the database, and build those out for us on demand

So that’s the easiest possible quick start, but what about integrating Marten into a real .NET application? Assuming you have a reference to the Marten nuget package, it’s just an IServiceCollection.AddMarten() call as shown below from a sample web application:

builder.Services.AddMarten(opts =>
    {
        // You always have to tell Marten what the connection string to the underlying
        // PostgreSQL database is, but this is the only mandatory piece of 
        // configuration
        var connectionString = builder.Configuration.GetConnectionString("postgres");
        opts.Connection(connectionString);
    })
    // This is a mild performance optimization
    .UseLightweightSessions();

At this point in the .NET ecosystem, it’s more or less idiomatic to use an Add[Tool]() method to integrate tools with your application’s IHost, and Marten tries to play within the typical .NET rules here.

I think this idiom and the generic host builder tooling has been a huge boon to OSS tool development in the .NET space compared to the old wild, wild west days. I do wish it would stop changing from .NET version to version though.

So that’s all a bunch of simple stuff, so let’s dive into something that shows off how Marten — really PostgreSQL — has a much stronger transactional model than many document databases that only support eventual consistency:

public static async Task manipulate_customer_data(IDocumentSession session)
{
    var customer = new Customer
    {
        Name = "Acme",
        Region = "North America",
        Class = "first"
    };
    
    // Marten has "upsert", insert, and update semantics
    session.Insert(customer);
    
    // Partial updates to a range of Customer documents
    // by a LINQ filter
    session.Patch<Customer>(x => x.Region == "EMEA")
        .Set(x => x.Class, "First");

    // Both the above operations happen in one 
    // ACID transaction
    await session.SaveChangesAsync();

    // Because Marten is ACID compliant, this query would
    // immediately work as expected even though we made that 
    // broad patch up above and inserted a new document.
    var customers = await session.Query<Customer>()
        .Where(x => x.Class == "First")
        .Take(100)
        .ToListAsync();
}

That’s a completely contrived example, but the point is, because Marten is completely ACID-compliant, you can make a range of operations within transactional boundaries and not have to worry about eventual consistency issues in immediate queries that other document databases suffer from.

So what else does Marten do? Here’s a bit of a rundown because Marten has a significantly richer built in feature set than many other low level document databases:

And quite a bit more than that, including some test automation support I really need to better document:/

And on top of everything else, because Marten is really just a fancy library on top of PostgreSQL — the most widely used database engine in the world — Marten instantly comes with a wide array of solid cloud hosting options as well as being deployable to local infrastructure on premise. PostgreSQL is also very Docker-friendly, making it a great technical choice for local development.

What’s a Document Database?

If you’re not familiar with the term “document database,” it refers to a type of NoSQL database where data is almost inevitably stored as JSON data, where the database allows you to quickly marshal objects in code to the database, then query that data later right back into the same object structures. The huge benefit of document databases at development time is being able to code much more productively because you just don’t have nearly as much friction as you do when dealing with any kind of object-relational mapping with either an ORM tool or by writing SQL and object mapping code by hand.

Low Ceremony Sagas with Wolverine

Wolverine puts a very high emphasis on reducing code ceremony and tries really hard to keep itself out of your application code. Wolverine is also built with testability in mind. If you’d be interested in learning more about how Wolverine could simplify your existing application code or set you up with a solid foundation for sustainable productive development for new systems, JasperFx Software is happy to work with you!

Before I get into the nuts and bolts of Wolverine sagas, let me come right out and say that I think that compared to other .NET frameworks, the Wolverine implementation of sagas requires much less code ceremony and therefore easier code to reason about. Wolverine also requires less configuration and explicit code to integrate your custom saga with Wolverine’s saga persistence. Lastly, Wolverine makes the development experience better by building in so much support for automatically configuring development environment resources like database schema objects or message broker objects. I do not believe that any other .NET tooling comes close to the developer experience that the Wolverine and its “Critter Stack” buddy Marten can provide.

Let’s say that you have some kind of multi-step process in your application that might have some mix of:

  • Callouts to 3rd party services
  • Some logical steps that can be parallelized
  • Possibly some conditional workflow based on the results of some of the steps
  • A need to enforce “timeout” conditions if the workflow is taking too long — think maybe of some kind of service level agreement for your workflow

This kind of workflow might be a great opportunity to use Wolverine’s version of Sagas. Conceptually speaking, a “saga” in Wolverine is just a special message handler that needs to inherit from Wolverine’s Saga class and modify itself to track state between messages that impact the saga.

Below is a simple version from the documentation called Order:

public record StartOrder(string OrderId);

public record CompleteOrder(string Id);

public class Order : Saga
{
    // You do need this for the identity
    public string? Id { get; set; }

    // This method would be called when a StartOrder message arrives
    // to start a new Order
    public static (Order, OrderTimeout) Start(StartOrder order, ILogger<Order> logger)
    {
        logger.LogInformation("Got a new order with id {Id}", order.OrderId);

        // creating a timeout message for the saga
        return (new Order{Id = order.OrderId}, new OrderTimeout(order.OrderId));
    }

    // Apply the CompleteOrder to the saga
    public void Handle(CompleteOrder complete, ILogger logger)
    {
        logger.LogInformation("Completing order {Id}", complete.Id);

        // That's it, we're done. Delete the saga state after the message is done.
        MarkCompleted();
    }

    // Delete this order if it has not already been deleted to enforce a "timeout"
    // condition
    public void Handle(OrderTimeout timeout, ILogger<Order> logger)
    {
        logger.LogInformation("Applying timeout to order {Id}", timeout.Id);

        // That's it, we're done. Delete the saga state after the message is done.
        MarkCompleted();
    }

    public static void NotFound(CompleteOrder complete, ILogger logger)
    {
        logger.LogInformation("Tried to complete order {Id}, but it cannot be found", complete.Id);
    }
}

Order is really meant to just be a state machine where it modifies its own state in response to incoming messages and returns cascading messages (you could also use IMessageBus directly as a method argument if you prefer, but my advice is to use simple pure functions) that tell Wolverine what to do next in the multi-step process.

A new Order saga can be created by any old message handler by simply returning a type that inherits from the Saga type in Wolverine. Wolverine is going to automatically discover any public types inheriting from Saga and utilize any public instance methods following certain naming conventions (or static Create() methods) as message handlers that are assumed to modify the state of the saga objects. Wolverine itself is handling everything to do with loading and persisting the Order saga object between commands around the call to the message handler methods on the saga types.

If you’ll notice the Handle(CompleteOrder) method above, the Order is calling MarkCompleted() on itself. That will tell Wolverine that the saga is now complete, and direct Wolverine to delete the current Order saga from the underlying persistence.

As for tracking the saga id between message calls, there are naming conventions about the messages that Wolverine can use to pluck the identity of the saga, but if you’re strictly exchanging messages between a Wolverine saga and other Wolverine message handlers, Wolverine will automatically track metadata about the active saga back and forth.

I’d also ask you to notice the OrderTimeout message that the Order saga returns as it starts. That message type is shown below:

// This message will always be scheduled to be delivered after
// a one minute delay because I guess we want our customers to be
// rushed? Goofy example code:)
public record OrderTimeout(string Id) : TimeoutMessage(1.Minutes());

Wolverine’s cascading message support allows you to return an outgoing message with a time delay — or a particular scheduled time or any other number of options — by just returning a message object. Admittedly this ties you into a little more of Wolverine, but the key takeaway I want you to notice here is that every handler method is a “pure function” with no service dependencies. Every bit of the state change and workflow logic can be tested with simple unit tests that merely work on the before and after state of the Order objects as well as the cascaded messages returned by the message handler functions. No mock objects, no fakes, no custom test harnesses, just simple unit tests. No other saga implementation in the .NET ecosystem can do that for you anywhere nearly as cleanly.

So far I’ve only focused on the logical state machine part of sagas, so let’s jump to persistence. Wolverine has long had a simplistic saga storage mechanism with its integration with Marten, and that’s still one of the easiest and most powerful options. You can also use EF Core for saga persistence, but ick, that means having to use EF Core.

Wolverine 3.0 added a new lightweight saga persistence option for either Sql Server or PostgreSQL (without Marten or EF Core) that just stands up a little table for just a single Saga type and uses JSON serialization to persist the saga. Here’s an example:

using var host = await Host.CreateDefaultBuilder()
    .UseWolverine(opts =>
    {
        // This isn't actually mandatory, but you'll
        // need to do it just to make Wolverine set up
        // the table storage as part of the resource setup
        // otherwise, Wolverine is quite capable of standing
        // up the tables as necessary at runtime if they
        // are missing in its default configuration
        opts.AddSagaType<RedSaga>("red");
        opts.AddSagaType(typeof(BlueSaga),"blue");
       
        
       // This part is absolutely necessary just to have the 
       // normal transactional inbox/outbox support and the new
       // default, lightweight saga persistence
opts.PersistMessagesWithSqlServer(Servers.SqlServerConnectionString, "color_sagas");
        opts.Services.AddResourceSetupOnStartup();
    }).StartAsync();

Just as with the integration with Marten, Wolverine’s lightweight saga implementation is able to build the necessary database table storage on the fly at runtime if it’s missing. The “critter stack” philosophy is to optimize the all important “time to first pull request” metric — meaning that you can get a Wolverine application up fast on your local development box because it’s able to take care of quite a bit of environment setup for you.

Lastly, Wolverine 3.0 is adding optimistic concurrency checks for the Marten saga storage and the new lightweight saga persistence. That’s been an important missing piece of the Wolverine saga story.

Just for some comparison, check out some other saga implementations in .NET:

Wolverine is taking a leap forward with the first 3.0 Alpha Release

Why should you care about this Wolverine tool anyway? I’d say that if you’re building just about any kind of server side .NET application, that Wolverine will do more than any other existing server side .NET framework in the mediator, background processing, HTTP, or asynchronous messaging space to simplify your code, maximize the testability of your system code, and do so while still helping you write robust, well performing systems.

The first Wolverine 3.0-alpha-1 release just landed on Nuget for anybody who is either waiting on 3.0 features or willing to try out the new bits. Just to rewind, here was the theoretic plans for Wolverine 3.0, with the Aspire integration having fallen off a bit.

See the Wolverine migration guide for more details.

There are some additive changes to address some previous limitations of the Wolverine tooling I’ll get to below, but the two big ticket items in 3.0 are:

  • Wolverine is now completely decoupled from Lamar and happily able to run with the built in ServiceProvider now. Before, Wolverine was quietly replacing your IoC container with Lamar because it heavily relied on Lamar internal behavior for its runtime code generation. 3.0 ended that particular limitation. Not everyone cared, but the folks who did care, were particularly loud about their unhappiness about that and Lamar is probably heading into the subset anyway in the future. I felt like this was a very important limitation of Wolverine to address in this release. It’s also a precursor to further usage of .NET Aspire and enabling Wolverine to play nicely with just about any common recipe for bootstrapping .NET applications (Blazor, WPF, Orleans, you name it).
  • The leader election subsystem in Wolverine was pretty close to 100% rewritten to a much simpler, and so far as the internal testing shows, far more reliable and performant mechanism. This subsystem has been way too problematic in real usage, and I’m beyond relieved that there’s some serious improvements coming for this

As for smaller things so far, some other highlights are:

  • Being able to utilize multiple handlers for the same message type in the same application, and even determine which handlers execute for particular external listeners
  • The stateful saga support in Wolverine got some necessary optimistic concurrency protection at the behest of a JasperFx Software client
  • New “lightweight” saga options to utilize either PostgreSQL or Sql Server as JSON storage mechanisms so you don’t have to suffer the pain of EF Core mapping just to persist sagas if you aren’t using Marten
  • The Rabbit MQ integration is using the new version of the Rabbit MQ client that is finally async all the way through to prevent deadlock issues. There is also some significant improvement to the Rabbit MQ transport for header exchanges and more control over messaging conventions
  • There is a new Nuget of compliance tests for Wolverine to hopefully speed up the construction of new saga persistence providers, messaging transports, or message storage options that I hope will unlock new functionality in the following months

I’m actually hopeful that the final 3.0 release goes out early next week. I’m not sure what the remaining work will be to make it in, but I’m wanting to tackle:

  • Message batching, because that comes up fairly often
  • A round of enhancements to the EF Core integration with Wolverine to try to increase Wolverine utilization for folks who don’t use Marten for some bizarre reason

Update on Wolverine 3.0 and Aspire

I had earlier said that “full” support for .NET Aspire would be a key part of the Wolverine 3.0 plans. After kicking the tires more on .NET Aspire and seeing where user priorities and our own road map is, I’m going to back off that statement quite a bit. Here’s what’s definitely in and actually ready to go for 3.0 as it pertains to Wolverine + Aspire:

  • Wolverine was decoupled from Lamar such that it can run with the built in ServiceProvider. We’ll add an add in adapter to still use Lamar as well so folks don’t have to switch out IoC tools for the new Wolverine (Lamar is much more forgiving and supports a lot of use cases that ServiceProvider does not, so you may not want to switch)
  • That last point was important because the changes to the internals also made it possible for Wolverine to be able to use any flavor of application bootstrapping like Host.CreateApplicationBuilder() whereas before Wolverine was limited to IHostBuilder (there were internal reasons for that). Some of the .NET Aspire client libraries depend on different versions of the application builders, so Wolverine needed to adapt. And folks wanted that anyway, so there we go.

Now, as to what else Wolverine will support, it’s perfectly possible to use Aspire to launch Wolverine systems and Wolverine (and Marten) can happily export their Open Telemetry tracing and metrics to the .NET Aspire dashboard at runtime. You can see an example of that in my earlier post Marten, Metrics, and Open Telemetry Support.

Now on to the trickier parts. One of the things that .NET Aspire does is act as a replacement for docker compose for infrastructural concerns like SQL Server, PostgreSQL, Rabbit MQ, or Kafka and acts as a global configuration element for other infrastructure things like Azure Service Bus or AWS SQS. Somebody might have to correct me, but more or less, Aspire is launching the various applications and poking through environment variables for the configuration data that Aspire itself is defining and controlling (like PostgreSQL connection strings for example). To make that information easier to consume, the Aspire team and community have built a bunch of client adapter libraries like Aspire.RabbitMQ.Client or Aspire.Npgsql that are meant to hook your application to the resources configured by Aspire by adding service registrations to the application’s underlying IoC container.

After some research earlier this week as I get to work toward the Wolverine 3.0 release, I think that:

  • Aspire.Npgsql can already be used as is with Marten at least, and with the Wolverine + Marten integration. A little more work could enable Aspire.Npgsql to be used with PostgreSQL storage within Wolverine for use by itself or with EF Core. There’s no need for us to take a direct dependency on this library though
  • Aspire.RabbitMQ.Client creates a version conflict with the RabbitMQ.Client library for us right now, so that’s out. I’m leery of taking on the potential diamond dependency issue anyway, so we’ll probably never take a dependency on it
  • Aspire.Microsoft.Data.SqlClient registers a scoped dependency for SqlConnection, but doesn’t expose the connection information any other way. This would require quite a few changes to the Wolverine internals that I don’t think would pay off. We won’t use this, again partially because of the fear of diamond dependencies
  • Aspire.Azure.Messaging.ServiceBus just isn’t usable. It precludes several options for authentication to Azure Service Bus, and using it would knock out Wolverine’s ability to set up or tear down resources on the fly — which I think is a competitive advantage of Wolverine over other alternatives, so I’m not enthusiastic about this one either
  • Aspire.Confluent.Kafka doesn’t fit Wolverine at all where we want to have the broker connection information upfront, and where Wolverine is completely responsible for setting up consumers and producers

All told though, I don’t think that any of the Aspire.* client libraries are usable out of the box. I guess I’m not sure if these libraries were even meant to be used or are just example code that folks like me should use to build in more specific support. In all cases, I’m voting to hold off for now on any new, direct Aspire support until someone — hopefully a contributor or JasperFx Software client — directly asks for it.

Critter Stack Workshop at DevUp Next Week

If you’re planning on coming to my workshop, you’ll want .NET 8, Git, and some kind of Docker Desktop on your box to run the sample code I’ll use in the workshop. If Docker doesn’t work for you, you maybe want a local install of PostgreSQL and Rabbit MQ.

Hey folks, I’ll be giving the first ever workshop on building an Event Driven Architecture with the full “Critter Stack” at DevUp 2024 in St. Louis next week on Wednesday the 14th bright and early at 8:30 AM.

We’ll be working through a sample backend web service that also communicates with other headless services using Event Sourcing within a general CQRS architectural approach with the whole “Critter Stack. We’ll use Marten (over PostgreSQL) for our persistence strategy using both its event sourcing support and as a document database. We’ll combine that with Wolverine as a server side framework for background processing, asynchronous messaging, and even as an alternative HTTP endpoint framework. Lastly, just for fun, there’ll be guest appearances from other JasperFx tools like Alba and Oakton for automated integration testing and command line execution respectively.

So why would you want to come to this and what might you get out of it? I’m hoping the takeaways — even if you don’t intend to use Marten or Wolverine — will be:

  • A good introduction to event sourcing as a technical approach and some of the real challenges you’ll face when building a system using event sourcing as a persistence strategy
  • An understanding of what goes into building a robust CQRS system including dealing with transient errors, observability, concurrency, and how to best segment message processing to achieve self-healing systems
  • Challenging the industry conventional wisdom about the efficacy of Hexagonal/Clean/Onion Architecture approaches really are when I show what a very low ceremony “vertical slice architecture” approach can be like with the Wolverine + Marten combination while still being robust, observable, highly testable, and still keeping infrastructure concerns out of the business logic
  • Some exposure to Open Telemetry and general observability tooling for distributed systems you absolutely want if you don’t already have that
  • Techniques for automating integration tests against an Event Driven Architecture

Because I’m absolutely in the business of promoting the “Critter Stack” tools, I’ll try to convince you that:

  • Marten is already the most robust and feature rich solution for event sourcing in the .NET ecosystem while also being arguably the easiest to get up and going with
  • How the Wolverine + Marten combination makes CQRS with Event Sourcing a much easier architectural pattern to use
  • Wolverine’s emphasis on low ceremony code approaches can help systems be more successfully maintained over time by simply having much less noise code and layering in your systems while still being robust
  • The “Critter Stack” has an excellent story for automated integration testing support that can do a lot to make your development efforts more successful
  • Both Marten & Wolverine can help your teams achieve a low “time to first pull request” by doing a lot to configure necessary infrastructure like databases or message brokers on the fly for a better development experience

I’m excited, because this is my first opportunity to do a workshop on the “Critter Stack” tools, and I think we’ve got a very compelling technical story to tell about the tools! And if nothing else, I’m looking forward to any feedback that might help us improve the tools down the line.

And for any *ahem* older folks from St. Louis in my talk, I personally at the time that Jorge Orta was out at first and the Cards should have won that game.

Critter Stack Roadmap for the Rest of 2024

It’s been a little bit since I’ve written any kind of update on the unofficial “Critter Stack” roadmap, with the last update in February. A ton of new, important strategic features have been added to especially Marten since then, with plenty of expansion of Wolverine to boot. Before jumping into what’s to come, let me indulge in a bit of retrospective about what new features or improvements have been delivered in 2024 so far before getting into the road map in the next section.

2024 just so far!

At this point I feel like we’ve crossed off the mass majority of the features I thought we needed to add to Marten this year to be able to stand Marten up against basically any other event store infrastructure tooling on the whole damn planet. What that also means is that I think that Marten development probably slows down to nothing but bug fixes and community contributions as folks run into things. There are still some features in the backlog that I might personally work on, but that will be in the course of some ongoing and potential JasperFx client work.

That being said, let’s talk about the rest of the year!

The Roadmap for the Back Half of 2024

Obviously, this roadmap is just a snapshot in time and client needs, community requests, and who knows what changes from Microsoft or other related tools could easily change priorities from any of this. All that being said, this is the Critter Stack core team & I’s current vision of the next big steps.

  1. Wolverine 3.0 is an ongoing effort. I’m hopeful it can be out in the next couple weeks
  2. RavenDb integration with Wolverine. This is some client sponsored work that I’m hoping will set Wolverine up for easier integration with other database engines in the near future
  3. “Critter Watch” — an ongoing effort to build out a management and monitoring console application for any combination of Marten, Wolverine, and future critters. This will be a paid product. We’ve already had a huge amount of feedback from Marten & Wolverine users, and I’m personally eager to get this moving in the 3rd quarter
  4. Marten 8.0 and Wolverine 4.0 — the goal here is mostly a rearrangement of dependencies underneath both Marten & Wolverine to eliminate duplication and spin out a lot of the functionality around projections and the async daemon. This will also be a significant effort to spin off some new helper libraries for the “Critter Stack” to enable the next bullet point
  5. “Ermine” — a port of Marten’s event store capabilities and a subset of its document database capabilities to SQL Server. My thought is that this will share a ton of guts with Marten. I’m voting that Ermine will have direct integration with Wolverine from the very beginning as well for subscriptions and middleware similar to the existing Wolverine.Marten integration
  6. If Ermine goes halfway well, I’d love to attempt a CosmosDb and maybe a DynamoDb backed event store in 2025

As usual, that list is a guess and unlikely to ever play out exactly that way. All the same though, there’s my hopes and dreams for the next 6 months or so.

Did I miss something you were hoping for? Does any of that concern you? Let me and the rest of the Critter Stack community know either here or anywhere in our Discord room!