Building a Critter Stack Application: Marten as Event Store

Hey, did you know that JasperFx Software is ready for formal support plans for Marten and Wolverine? Not only are we trying to make the “Critter Stack” tools be viable long term options for your shop, we’re also interested in hearing your opinions about the tools and how they should change.

Let’s build a small web service application using the whole “Critter Stack” and their friends, one small step at a time. For right now, the “finished” code is at CritterStackHelpDesk on GitHub.

The posts in this series are:

Event Sourcing

Event Sourcing is a style of persistence where the single source of truth, system state is a read only, append only sequence of all the events that resulted in a change in the system state. Using our HelpDesk incident tracking application we first started describing in the previous post on Event Storming, that results in a sequence like this:

Sequence	Incident Id	Event Type
1	1	IncidentLogged
2	1	IncidentCategorized
3	2	IncidentLogged
4	3	IncidentLogged
5	1	IncidentResolved

An event log

As you could probably guess already from the table above, the events will be stored in one single log in the sequential order they were appended. You can also see that events will be categorized by their relationship to a single logical incident. This grouping is typically called a “stream” in event sourcing.

As a first quick foray into event sourcing, let’s look at using the Marten library to create an event store for our help desk application built on top of a PostgreSQL database.

In case you’re wondering, Marten is merely a fancy library that helps you access and treat the rock solid PostgreSQL database engine as both a document database and as an event store. Marten was purposely built on PostgreSQL specifically because of the unique JSON capabilities of PostgreSQL. It’s possible that the event store portion of Marten eventually gets ported to other databases in the future (Sql Server), but it’s highly unlikely that the document database feature set would ever follow.

Using Marten as an Event Store

This code is all taken from the CritterStackHelpDesk repository, and specifically the EventSourcingDemo console project. The repository’s README file has instructions on running that project.

First off, let’s build us some events that we can later store in our new event store:

public record IncidentLogged(
    Guid CustomerId,
    Contact Contact,
    string Description,
    Guid LoggedBy
);

public class IncidentCategorised
{
    public IncidentCategory Category { get; set; }
    public Guid UserId { get; set; }
}

public record IncidentPrioritised(IncidentPriority Priority, Guid UserId);

public record AgentAssignedToIncident(Guid AgentId);

public record AgentRespondedToIncident(        
    Guid AgentId,
    string Content,
    bool VisibleToCustomer);

public record CustomerRespondedToIncident(
    Guid UserId,
    string Content
);

public record IncidentResolved(
    ResolutionType Resolution,
    Guid ResolvedBy,
    DateTimeOffset ResolvedAt
);

You’ll notice there’s a (hopefully) consistent naming convention. The event types are named in the past tense and should refer clearly to a logical event in the system’s workflow. You might also notice that these events are all built with C# records. This isn’t a requirement, but it makes the code pretty terse and there’s no reason for these events to ever be mutable anyway.

Next, I’ve created a small console application and added a reference to the Marten library like so from the command line:

dotnet new console
dotnet add package Marten

Before we even think about using Marten itself, let’s get ourselves a new, blank PostgreSQL database spun up for our little application. Assuming that you have Docker Desktop or some functional alternative on your development machine, there’s a docker compose file in the root of the finished product that we can use to stand up a new database with:

docker compose up -d

Note, and this is an important point, there is absolutely nothing else you need to do to make this new database perfectly usable for the code we’re going to write next. No manual database setup, no SQL scripts for you to run, no other command line scripts. Just write code and go.

Next, we’re going to configure Marten in code, then:

Start a new “Incident” stream with a couple events
Append additional events to our new stream

The code to do nothing but what I described is shown below:

// This matches the docker compose file configuration
var connectionString = "Host=localhost;Port=5433;Database=postgres;Username=postgres;password=postgres";

// This is spinning up Marten with its default settings
await using var store = DocumentStore.For(connectionString);

// Create a Marten unit of work
await using var session = store.LightweightSession();

var contact = new Contact(ContactChannel.Email, "Han", "Solo");
var userId = Guid.NewGuid();

// I'm telling the Marten session about the new stream, and then recording
// the newly assigned Guid for this stream
var customerId = Guid.NewGuid();
var incidentId = session.Events.StartStream(
    new IncidentLogged(customerId, contact, "Software is crashing",userId),
    new IncidentCategorised
    {
        Category = IncidentCategory.Database,
        UserId = userId
    }
    
).Id;

await session.SaveChangesAsync();

// And now let's append an additional event to the 
// new stream
session.Events.Append(incidentId, new IncidentPrioritised(IncidentPriority.High, userId));
await session.SaveChangesAsync();

Let’s talk about what I just did — and did not do — in the code above. The DocumentStore class in Marten establishes the storage configuration for a single, logical Marten-ized database. This is an expensive object to create, so there should only ever be one instance in your system.

The actual work is done with Marten’s IDocumentSession service that I created with the call to store.LightweightSession(). The IDocumentSession is Marten’s unit of work implementation and plays the same role as DbContext does inside of EF Core. When you use Marten, you queue up operations (start a new event stream, append events, etc.), then commit them in one single database transaction when you call that SaveChangesAsync() method.

For anybody old enough to have used NHibernate reading this, DocumentStore plays the same role as NHibernate’s ISessionFactory.

So now, let’s read back in the events we just persisted, and print out serialized JSON of the Marten data just to see what Marten is actually capturing:

var events = await session.Events.FetchStreamAsync(incidentId);
foreach (var e in events)
{
    // I elided a little bit of code that sets up prettier JSON
    // formatting
    Console.WriteLine(JsonConvert.SerializeObject(e, settings));
}

The raw JSON output is this:

{
  "Data": {
    "CustomerId": "314d8fa1-3cca-4984-89fc-04b24122cf84",
    "Contact": {
      "ContactChannel": "Email",
      "FirstName": "Han",
      "LastName": "Solo",
      "EmailAddress": null,
      "PhoneNumber": null
    },
    "Description": "Software is crashing",
    "LoggedBy": "8a842212-3511-4858-a3f3-dd572a4f608f"
  },
  "EventType": "Helpdesk.Api.IncidentLogged, Helpdesk.Api, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null",
  "EventTypeName": "incident_logged",
  "DotNetTypeName": "Helpdesk.Api.IncidentLogged, Helpdesk.Api",
  "IsArchived": false,
  "AggregateTypeName": null,
  "StreamId": "018c1c9b-5bd0-4273-947d-83d28c8e3210",
  "StreamKey": null,
  "Id": "018c1c9b-5f03-47f5-8c31-1d1ba70fd56a",
  "Version": 1,
  "Sequence": 1,
  "Timestamp": "2023-11-29T19:43:13.864064+00:00",
  "TenantId": "*DEFAULT*",
  "CausationId": null,
  "CorrelationId": null,
  "Headers": null
}
{
  "Data": {
    "Category": "Database",
    "UserId": "8a842212-3511-4858-a3f3-dd572a4f608f"
  },
  "EventType": "Helpdesk.Api.IncidentCategorised, Helpdesk.Api, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null",
  "EventTypeName": "incident_categorised",
  "DotNetTypeName": "Helpdesk.Api.IncidentCategorised, Helpdesk.Api",
  "IsArchived": false,
  "AggregateTypeName": null,
  "StreamId": "018c1c9b-5bd0-4273-947d-83d28c8e3210",
  "StreamKey": null,
  "Id": "018c1c9b-5f03-4a19-82ef-9c12a84a4384",
  "Version": 2,
  "Sequence": 2,
  "Timestamp": "2023-11-29T19:43:13.864064+00:00",
  "TenantId": "*DEFAULT*",
  "CausationId": null,
  "CorrelationId": null,
  "Headers": null
}
{
  "Data": {
    "Priority": "High",
    "UserId": "8a842212-3511-4858-a3f3-dd572a4f608f"
  },
  "EventType": "Helpdesk.Api.IncidentPrioritised, Helpdesk.Api, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null",
  "EventTypeName": "incident_prioritised",
  "DotNetTypeName": "Helpdesk.Api.IncidentPrioritised, Helpdesk.Api",
  "IsArchived": false,
  "AggregateTypeName": null,
  "StreamId": "018c1c9b-5bd0-4273-947d-83d28c8e3210",
  "StreamKey": null,
  "Id": "018c1c9b-5fef-4644-b213-56051088dc15",
  "Version": 3,
  "Sequence": 3,
  "Timestamp": "2023-11-29T19:43:13.909+00:00",
  "TenantId": "*DEFAULT*",
  "CausationId": null,
  "CorrelationId": null,
  "Headers": null
}

And that’s a lot of noise, so let me try to summarize the blob above:

Marten is storing each event as serialized JSON in one table, and that’s what you see as the Data leaf in each JSON document above
Marten is assigning a unique sequence number for each event
StreamId is the incident stream identity that groups the events
Each event is assigned a Version that reflects its position within its stream
Marten tracks the kind of metadata that you’d probably expect, like timestamps, optional header information, and optional causation/correlation information (we’ll use this much later in the series when I get around to discussing Open Telemetry)

Summary and What’s Next

In this post I introduced the core concepts of event sourcing, events, and event streams. I also introduced the bare bones usage of the Marten library as a way to create new event streams and append events to existing events. Lastly, we took a look at the important metadata that Marten tracks for you in addition to your raw event data. Along the way, we also previewed how the Critter Stack can reduce development time friction by very happily building out the necessary database schema objects for us as needed.

What you are probably thinking at this point is something to the effect of “So what?” After all, jamming little bits of JSON data into the database doesn’t necessarily help us build a user interface page showing a help desk technician what the current state of each open incident is. Heck, we don’t yet have any way to understand the actual current state of any incident!

Fear not though, because in the next post I’ll introduce Marten “Projections” capability that will help us create the “read side” view of the current system state out of the raw event data in whatever format happens to be most convenient for that data’s client or user.