Building a Critter Stack Application: Web Service Query Endpoints with Marten

Hey, did you know that JasperFx Software is ready for formal support plans for Marten and Wolverine? Not only are we trying to make the “Critter Stack” tools be viable long term options for your shop, we’re also interested in hearing your opinions about the tools and how they should change. We’re also certainly open to help you succeed with your software development projects on a consulting basis whether you’re using any part of the Critter Stack or any other .NET server side tooling.

Let’s build a small web service application using the whole “Critter Stack” and their friends, one small step at a time. For right now, the “finished” code is at CritterStackHelpDesk on GitHub.

The posts in this series are:

1. Event Storming
2. Marten as Event Store
3. Marten Projections
4. Integrating Marten into Our Application
5. Wolverine as Mediator
6. Web Service Query Endpoints with Marten (this post)
7. Dealing with Concurrency
8. Wolverine’s Aggregate Handler Workflow FTW!
9. Command Line Diagnostics with Oakton
10. Marten as Document Database
11. Asynchronous Processing with Wolverine
12. Durable Outbox Messaging and Why You Care!
13. Wolverine HTTP Endpoints
14. Easy Unit Testing with Pure Functions
15. Vertical Slice Architecture
16. Messaging with Rabbit MQ
17. The “Stateful Resource” Model
18. Resiliency

Last time up we introduced Wolverine as to help us build command handlers as the “C” in the CQRS architecture. This time out, I want to turn our attention back to Marten and building out some query endpoints to get the “Q” part of CQRS going by exposing projected event data in a read-only way through HTTP web services.

When we talked before about Marten projections (read-only “projected” view representations of the source events), I mentioned that these projected views could be create with three different lifecycles:

1. “Live” projections are built on demand based on the current event data
2. “Inline” projections are updated at the time new events are captured such that the “read side” model is always strongly consistent with the raw event data
3. “Async” projections are continuously built by a background process in Marten applications and give you an eventual consistency model.

Alright, so let’s talk about when you might use different lifecycles of projection creation, then we’ll move on to how that changes the mechanics of how we’ll deliver projection data through web services. Offhand, I’d recommend a decision tree something like:

• If you want to optimize the system’s “read” performance more than the “writes”, definitely use the Inline lifecycle
• If you want to optimize the “write” performance of event capture and also want a strongly consistent “read” model that exactly reflects the current state, choose the Live lifecycle. Know though that if you go that way, you will want to model your system in such a way that you can keep your event streams short. It’s of course not exactly simple, because the Live aggregation time can also negatively impact command processing time if you need to first derive the current state in order to “decide” what new events should be emitted.
• If you want to optimize both the “read” and “write” performance, but can be a little relaxed about the read side consistency, you can opt for Async projections

For starters, let’s build just a simple HTTP endpoint that returns the current state for a single Incident within our new help desk system. As a quick reminder, the IncidentDetails aggerated projection we’re about to work with is built out like this:

public record IncidentDetails(
    Guid Id,
    Guid CustomerId,
    IncidentStatus Status,
    IncidentNote[] Notes,
    IncidentCategory? Category = null,
    IncidentPriority? Priority = null,
    Guid? AgentId = null,

    // Marten is going to set this for us in
    // the projection work
    int Version = 1
);

public record IncidentNote(
    IncidentNoteType Type,
    Guid From,
    string Content,
    bool VisibleToCustomer
);

public enum IncidentNoteType
{
    FromAgent,
    FromCustomer
}

// This class contains the directions for Marten about how to create the
// IncidentDetails view from the raw event data
public class IncidentDetailsProjection: SingleStreamProjection<IncidentDetails>
{
    public static IncidentDetails Create(IEvent<IncidentLogged> logged) =>
        new(logged.StreamId, logged.Data.CustomerId, IncidentStatus.Pending, Array.Empty<IncidentNote>());

    public IncidentDetails Apply(IncidentCategorised categorised, IncidentDetails current) =>
        current with { Category = categorised.Category };

    public IncidentDetails Apply(IncidentPrioritised prioritised, IncidentDetails current) =>
        current with { Priority = prioritised.Priority };

    public IncidentDetails Apply(AgentAssignedToIncident prioritised, IncidentDetails current) =>
        current with { AgentId = prioritised.AgentId };

    public IncidentDetails Apply(IncidentResolved resolved, IncidentDetails current) =>
        current with { Status = IncidentStatus.Resolved };

    public IncidentDetails Apply(ResolutionAcknowledgedByCustomer acknowledged, IncidentDetails current) =>
        current with { Status = IncidentStatus.ResolutionAcknowledgedByCustomer };

    public IncidentDetails Apply(IncidentClosed closed, IncidentDetails current) =>
        current with { Status = IncidentStatus.Closed };
}

I want to make sure that I draw your attention to the Version property of the IncidentDetails projected document. Marten itself has a naming convention (it can be overridden with attributes too) where it will set this member to the current stream version number when Marten builds this single stream projection. That’s going to be vital in the next post when we start introducing concurrency projection models.

For right now, let’s say that we’re choosing to use the Live style. In this case, we’ll need to do the aggregation on the fly, then stream that down the HTTP body like so with MVC Core:

    [HttpGet("/api/incidents/{incidentId}")]
    public async Task<IResult> Get(Guid incidentId)
    {
        // In this case, the IncidentDetails are projected "live"
        var details = await _session.Events.AggregateStreamAsync<IncidentDetails>(incidentId);

        return details != null
            ? Results.Json(details)
            : Results.NotFound();
    }

If, however, we chose to produce the projected IncidentDetails data Inline such that the projected data is already persisted to the Marten database as a document, we’d first make this addition to the AddMarten() configuration in the application’s Program file:

builder.Services.AddMarten(opts =>
{
    // You always have to tell Marten what the connection string to the underlying
    // PostgreSQL database is, but this is the only mandatory piece of 
    // configuration
    var connectionString = builder.Configuration.GetConnectionString("marten");
    opts.Connection(connectionString);
    
    // We have to tell Marten about the projection we built in the previous post
    // so that Marten will "know" how to project events to the IncidentDetails
    // projected view
    opts.Projections.Add<IncidentDetailsProjection>(ProjectionLifecycle.Inline);
});

Lastly, we could now instead write that web service method in our MVC Core controller as:

    [HttpGet("/api/incidents/{incidentId}")]
    public async Task<IResult> Get(Guid incidentId)
    {
        // In this case, the IncidentDetails are projected "live"
        var details = await _session.LoadAsync<IncidentDetails>(incidentId);

        return details != null
            ? Results.Json(details)
            : Results.NotFound();
    }

One last trick for now, let’s make the web service above much faster! I’m going to add another library into the mix with this Nuget reference:

dotnet add package Marten.AspNetCore

And let’s revisit the previous web service endpoint and change it to this:

public class IncidentController : ControllerBase
{
    private readonly IDocumentSession _session;

    public IncidentController(IDocumentSession session)
    {
        _session = session;
    }
    
    [HttpGet("/api/incidents/{incidentId}")]
    public Task Get(Guid incidentId)
    {
        return _session
            .Json
            .WriteById<IncidentDetails>(incidentId, HttpContext);
    }
    
    // other methods....

The WriteById() usage up above is an extension method from the Marten.AspNetCore package that lets you stream raw, persisted JSON data from Marten directly to the HTTP response body in an ASP.Net Core endpoint in a very efficient way. At no point are you even bothering to instantiate an IncidentDetails object in memory just to immediately turn around and serialize it right back to the HTTP response. There’s basically no other faster way to build a web service for this information.

Summary and What’s Next

In this entry we talked a little bit about the consequences of the projection lifecycle decision for your web service. We also mentioned about how Marten can provide the stream version into projected documents that will be valuable soon when we talk about concurrency. Lastly, I introduced the Marten.AspNetCore library and its extension methods to directly “stream” JSON data stored into PostgreSQL directly to the HTTP response in a very efficient way.

In the next post we’re going to look at Marten’s concurrency protections and discuss why you care about these abilities.