Read Your Writes with the Critter Stack

The image isn’t completely related, but it’s more than a little funny that the Superfriends had a super computer with incredible AI capabilities in the 70’s but yet its interface was writing out ticker tape. And for any other Gen X kids out there who read this: “meanwhile, at the Hall of Justice…”

JasperFx Software is happy to do interactive, customized workshops on the Critter Stack or Event Driven Architecture in general for your folks. Our workshops tend to be very conversational and we’ll happily tailor the content to the problems that your shop faces. Learn more about that on the “Workshops” page on the JasperFx website.

I was giving an online workshop this morning for a JasperFx client and one of our topics today was data consistency inside of a system using Event Sourcing. One of the slides from today is this one:

Roughly speaking, this is a guarantee that after a client writes or updates data, you can issue queries to that same client will be able to read the prior updates without any eventual consistency delay.

If you’ve only ever developed against relational databases and a single representation of data, you’re saying to yourself “well, of course, because that’s the way databases work?” However, if you’re working with Event Sourcing there’s a good chance that you run into Eventual Consistency of some sort and this “Read your own writes” guarantee is something you can’t take for granted — unless your event store is Marten, Polecat, or Fisher!

All of the Critter Stack event stores have plenty of options for strong consistency that frankly cannot be matched by the commercial event store tools or most of the OSS event stores out there for that matter. But let’s say for now that you have a real need to use our Async lifecycle for a single stream projection (performance? throughput? our blue/green deployment support requires that?), but you really want to be able to return the exact state of a single stream from an HTTP endpoint or from a message handler being used from IMessageBus.InvokeAsync<Report>() (Wolverine as a “mediator”). It’s a use case that’s common enough (HTTP endpoints or GraphQL mutations being the most common usage) that we’ve built some first class support for that into Marten, but first let’s look at an older usage.

You have long been able to do this:

			
// Today's pattern: forced flush + re-read
session.Events.StartStream<Report>(id, new ReportCreated("Q1"));
await session.SaveChangesAsync(ct);  // forced flush
// This gives you the absolutely latest version of Report based
// on the events written to the event store, but this would
// *probably* make a network round trip to the database to 
// fetch the data again
var report = await session.Events.FetchLatest<Report>(id, ct);  // re-read
return report;

		

The FetchLatest API in Marten will happily fetch you the latest version of Report for a single stream that reflects the latest events no matter what the underlying Marten ProjectionLifecycle happens to be. Except in some particular circumstances (Inline plus having the identity map turned on), calling that makes a separate round trip to the database — and network round trips are the leading cause of poor performance in enterprise-y applications, so we ideally don’t want that!

So let’s cut out the extra network round trip and change the API usage to ProjectLatest:

			
// Better: project locally including pending events
session.Events.StartStream<Report>(id, new ReportCreated("Q1"));
var report = await session.Events.ProjectLatest<Report>(id, ct);
// SaveChangesAsync happens later (e.g., Wolverine AutoApplyTransactions)
return report;

		

ProjectLatest is slightly different in that it will use the in memory events and any initial fetch of an aggregate projection within the same session to build out the latest, greatest Report aggregate above without making any extra round trips to the database for the best possible performance in your system.

Now though, let’s add Wolverine to the mix with its integration with Marten and what we call the “aggregate handler workflow” (it’s the Decider pattern, but I think we named our implementation of that before any of us had seen the original Decider pattern paper). Wolverine has a little helper for Marten/Polecat/Fisher called UpdatedAggregate you can see in usage in this sample code where Order is a projected aggregate view for a single stream:

			
public static (UpdatedAggregate, Events) Handle(
  MarkItemReady command, 
  [WriteAggregate]
  Order order)
{
    var events = new Events();
    if (order.Items.TryGetValue(command.ItemName, out var item))
    {
        item.Ready = true;
        events.Add(new ItemReady(command.ItemName));
    }
    else
    {
        throw new InvalidOperationException($"Item {command.ItemName} does not exist in this order");
    }
    if (order.IsReadyToShip())
    {
        events.Add(new OrderReady());
    }
    return (new UpdatedAggregate(), events);
}

		

When Wolverine sees that UpdatedAggregate type in your handler signature, Wolverine is making the single [WriteAggregate] type be the response type for either an HTTP handler or a “cascading” message type for message handlers and calling the correct Marten APIs for you in the most efficient way it can. You just have to worry about the declarative signature.

And if you’re one of those many people who very much care about OpenAPI, Wolverine.HTTP creates the correct metadata describing an HTTP endpoint’s response if you use UpdatedAggregate.

For more information, see:

Summary

Marten and Wolverine are all heavily used by a very engaged community, plus JasperFx is always active supporting the platform. The result is a mature set of tools that have already been through a lot of optimizations and happily provide solutions for almost any common use case in the realm of server side .NET development.