JasperFx Software helps our customers be more successful with their usage of the “Critter Stack” tools (or any other server side .NET tooling you might be using). The work in this post was delivered for a JasperFx customer to help protect their customer’s private information. If you need or want any help with event sourcing, Event Driven Architecture, or automated testing, drop us a note and we’d be happy to talk with you about what JasperFx can do for you.
I defy you to say the title of this post out loud in rapid succession without stumbling over it.
According to the U.S. Department of Labor, “Personal Identifiable Information” (PII) is defined as:
Any representation of information that permits the identity of an individual to whom the information applies to be reasonably inferred by either direct or indirect means.
Increasingly, Marten users are running into requirements to be able to “forget” PII that is persisted within a Marten database. For the document storage, I think this is relatively easy to do with a host of existing functionality including the partial update functionality that Marten got (back) in V7. For the event store though, there wasn’t anything built in that would have made it easy to erase or “mask” protected information within the persisted event data — until now!
For a variety of reasons, you may wish to remove or mask sensitive data elements in a Marten database without necessarily deleting the information as a whole. Documents can be amended with Marten’s Patching API. With event data, you now have options to reach into the event data and rewrite selected members as well as to add custom headers. First, start by defining data masking rules by event type like so:
var builder = Host.CreateApplicationBuilder();
builder.Services.AddMarten(opts =>
{
opts.Connection(builder.Configuration.GetConnectionString("marten"));
// By a single, concrete type
opts.Events.AddMaskingRuleForProtectedInformation<AccountChanged>(x =>
{
// I'm only masking a single property here, but you could do as much as you want
x.Name = "****";
});
// Maybe you have an interface that multiple event types implement that would help
// make these rules easier by applying to any event type that implements this interface
opts.Events.AddMaskingRuleForProtectedInformation<IAccountEvent>(x => x.Name = "****");
// Little fancier
opts.Events.AddMaskingRuleForProtectedInformation<MembersJoined>(x =>
{
for (int i = 0; i < x.Members.Length; i++)
{
x.Members[i] = "*****";
}
});
});
That’s strictly a configuration time effort. Next, you can apply the masking on demand to any subset of events with the IDocumentStore.Advanced.ApplyEventDataMasking() API. First, you can apply the masking for a single stream:
public static Task apply_masking_to_streams(IDocumentStore store, Guid streamId, CancellationToken token)
{
return store
.Advanced
.ApplyEventDataMasking(x =>
{
x.IncludeStream(streamId);
// You can add or modify event metadata headers as well
// BUT, you'll of course need event header tracking to be enabled
x.AddHeader("masked", DateTimeOffset.UtcNow);
}, token);
}
As a finer grained operation, you can specify an event filter (Func<IEvent, bool>) within an event stream to be masked with this overload:
public static Task apply_masking_to_streams_and_filter(IDocumentStore store, Guid streamId, CancellationToken token)
{
return store
.Advanced
.ApplyEventDataMasking(x =>
{
// Mask selected events within a single stream by a user defined criteria
x.IncludeStream(streamId, e => e.EventTypesAre(typeof(MembersJoined), typeof(MembersDeparted)));
// You can add or modify event metadata headers as well
// BUT, you'll of course need event header tracking to be enabled
x.AddHeader("masked", DateTimeOffset.UtcNow);
}, token);
}
Note that regardless of what events you specify, only events that match a pre-registered masking rule will have the header changes applied.
To apply the event data masking across streams on an arbitrary grouping, you can use a LINQ expression as well:
Finally, if you are using multi-tenancy, you can specify the tenant id as part of the same fluent interface:
public static Task apply_masking_by_tenant(IDocumentStore store, string tenantId, Guid streamId)
{
return store
.Advanced
.ApplyEventDataMasking(x =>
{
x.IncludeStream(streamId);
// Specify the tenant id, and it doesn't matter
// in what order this appears in
x.ForTenant(tenantId);
});
}
Here’s a couple more facts you might need to know:
The masking rules can only be done at configuration time (as of right now)
You can apply multiple masking rules for certain event types, and all will be applied when you use the masking API
The masking has absolutely no impact on event archiving or projected data — unless you rebuild the projection data after applying the data masking of course
Summary
The Marten team is at least considering support for crypto-shredding in Marten 8.0, but no definite plans have been made yet. It might fit into the “Critter Stack 2025” release cycle that we’re just barely starting.
I’m working with a JasperFx Software client who is in the beginning stages of building a pretty complex, multi-step file import process that is going to involve several different services. For the sake of example code in this post, let’s say that we have the (much simplified from my client’s actual logical workflow) workflow from the diagram above:
External partners (or customers) are sending us an Excel sheet with records that our system will need to process and utilize within our downstream systems (invoices? payments? people? transactions?)
For the sake of improved throughput, the incoming file is broken into batches of records so the smaller batches can be processed in parallel
Each batch needs to be validated by the “Validation Service”
When each batch has been completely validated:
If there are any errors, send a rejection summary about the entire file to the original external partner
If there are no errors, try to send each record batch to “Downstream System #1”
When each batch has been completely accepted or rejected by “Downstream System #1”
If there are any rejections, send a rejection summary about the entire file to the original external partner
If all batches are accepted by “Downstream System #1”, try to send each record batch to “Downstream System #2”
When each batch has been completely accepted or rejected by “Downstream System #2”
If there are any rejections, send a rejection summary about the entire file to the original external partner and a message to “Downstream System #1” to reverse each previously accepted records in the file
If all batches are accepted by “Downstream System #2”, send a successful receipt message to the original external partner and archive the intermediate state
Right off the bat, I think we can identify a couple needs and challenges:
We need some way to track the current, in process state of an individual file and where all the various batches are in that process
At every point, make decisions about what to do next in the workflow based on the current state of the file based on incremental process. And to make this as clear as possible, I think it’s extremely valuable to be able to clearly write, read, unit test, and reason about this workflow code without any significant coupling to the surrounding infrastructure.
The whole system should be resilient in the face of the expected transient hiccups like a database getting overwhelmed or a downstream system being temporarily down and “work” should never get lost or hopefully even require human intervention at runtime
Especially for large files, we absolutely better be prepared for some challenging concurrency issues when lots of incoming messages attempt to update that central file import processing state
Make it all performance too of course!
Alright, so we’re definitely using both Marten for persistence and Wolverine for the workflow and messaging between services for all of this. The first basic approach for the state management is to use Wolverine’s stateful saga support with Marten. In that case we might have a saga type in Marten something like this:
// Again, express the stages in terms of your
// business domain instead of technical terms,
// but you'll do better than me on this front!
public enum FileImportStage
{
Validating,
Downstream1,
Downstream2,
Completed
}
// As long as it's JSON serialization friendly, you can happily
// tighten up the access here all you want, but I went for quick and simple
public class FileImportSaga :
// Only necessary marker type for Wolverine here
Saga,
// Opts into tracked version concurrency for Marten
// We probably want in this case
IRevisioned
{
// Identity for this saga within our system
public Guid Id { get; set; }
public string FileName { get; set; }
public string PartnerTrackingNumber { get; set; }
public DateTimeOffset Created { get; set; } = DateTimeOffset.UtcNow;
public List<RecordBatchTracker> RecordBatches { get; set; } = new();
public FileImportStage Stage { get; set; } = FileImportStage.Validating;
// Much more in just a bit...
}
Inside our system, we can start a new FileImportSaga and launch the first set of messages to validate each batch of records with this handler that reacts to a request to import a new file:
public record ImportFile(string fileName);
// This could have been done inside the FileImportSaga as well,
// but I think I'd rather keep that focused on the state machine
// and workflow logic
public static class FileImportHandler
{
public static async Task<(FileImportSaga, OutgoingMessages)> Handle(
ImportFile command,
IFileImporter importer,
CancellationToken token)
{
var saga = await importer.ReadAsync(command.fileName, token);
var messages = new OutgoingMessages();
messages.AddRange(saga.CreateValidationMessages());
return (saga, messages);
}
}
public interface IFileImporter
{
Task<FileImportSaga> ReadAsync(string fileName, CancellationToken token);
}
Let’s say that we’re receiving messages back from the Validation Message like this:
public record ValidationResult(Guid Id, Guid BatchId, ValidationMessage[] Messages);
public record ValidationMessage(int RecordNumber, string Message);
Quick note, if Wolverine is handling the messaging in the downstream systems, it’s helping make this easier by tracking the saga id in message metadata from upstream to downstream and back to the upstream through response messages. Otherwise you’d have to track the saga id on the incoming messages.
We could process the validation results in our saga one at a time like so:
// Use Wolverine's cascading message feature here for the next steps
public IEnumerable<object> Handle(ValidationResult validationResult)
{
var currentBatch = RecordBatches
.FirstOrDefault(x => x.Id == validationResult.BatchId);
// We'd probably rig up Wolverine error handling so that it either discards
// a message in this case or immediately moves it to the dead letter queue
// because there's no sense in trying to retry a message that can never be
// processed successfully
if (currentBatch == null) throw new UnknownBatchException(Id, validationResult.BatchId);
currentBatch.ReadValidationResult(validationResult);
var currentValidationStatus = determineValidationStatus();
switch (currentValidationStatus)
{
case RecordStatus.Pending:
yield break;
case RecordStatus.Accepted:
Stage = FileImportStage.Downstream1;
foreach (var batch in RecordBatches)
{
yield return new RequestDownstream1Processing(Id, batch.Id, batch.Records);
}
break;
case RecordStatus.Rejected:
// This saga is complete
MarkCompleted();
// Tell the original sender that this file is rejected
// I'm assuming that Wolverine will get the right information
// back to the original sender somehhow
yield return BuildRejectionMessage();
break;
}
}
private RecordStatus determineValidationStatus()
{
if (RecordBatches.Any(x => x.ValidationStatus == RecordStatus.Pending))
{
return RecordStatus.Pending;
}
if (RecordBatches.Any(x => x.ValidationStatus == RecordStatus.Rejected))
{
return RecordStatus.Rejected;
}
return RecordStatus.Accepted;
}
First off, I’m going to argue that the way that Wolverine supports its stateful sagas and its cascading message feature make the workflow logic pretty easy to unit test in isolation from all the infrastructure. That part is good, right? But what’s maybe not great is that we could easily be getting a bunch of those ValidationResult messages back for the same file at the same time because they’re handled in parallel, so we really need to be prepared for that.
We could rely on the Wolverine/Marten combination’s support for optimistic concurrency and just retry ValidationResult messages that fail because of caught ConcurrencyException, but that’s potentially thrashing the database and the application pretty hard. We could also solve this problem in a “sledgehammer to crack a nut” kind of way by using Wolverine’s strictly ordered listener approach that would force the file import status messages to be processed in order on a single running node:
builder.Host.UseWolverine(opts =>
{
opts.UseRabbitMq(builder.Configuration.GetConnectionString("rabbitmq"));
opts.ListenToRabbitQueue("file-import-updates")
// Single file, serialized access across the
// entire running application cluster!
.ListenWithStrictOrdering();
});
That solves the concurrency issue in a pretty hard core way, but it’s not going to terribly performant because you’ve eliminated all concurrency between different files and you’re making the system constantly load, then save the FileImportSaga data for intermediate steps. Let’s adjust this and incorporate Wolverine’s new message batching feature.
First off, let’s add a new validation batch message like so:
public record ValidationResultBatch(Guid Id, ValidationResult[] Results);
And a new message handler on our saga type for that new message type:
public IEnumerable<object> Handle(ValidationResultBatch batch)
{
var groups = batch.Results.GroupBy(x => x.BatchId);
foreach (var group in groups)
{
var currentBatch = RecordBatches
.FirstOrDefault(x => x.Id == group.Key);
foreach (var result in group)
{
currentBatch.ReadValidationResult(result);
}
}
return DetermineNextStepsAfterValidation();
}
// I pulled this out as a helper, but also, it's something
// you probably want to unit test in isolation on just the FileImportSaga
// class to nail down the workflow logic w/o having to do an integration
// test
public IEnumerable<object> DetermineNextStepsAfterValidation()
{
var currentValidationStatus = determineValidationStatus();
switch (currentValidationStatus)
{
case RecordStatus.Pending:
yield break;
case RecordStatus.Accepted:
Stage = FileImportStage.Downstream1;
foreach (var batch in RecordBatches)
{
yield return new RequestDownstream1Processing(Id, batch.Id, batch.Records);
}
break;
case RecordStatus.Rejected:
// This saga is complete
MarkCompleted();
// Tell the original sender that this file is rejected
// I'm assuming that Wolverine will get the right information
// back to the original sender somehhow
yield return BuildRejectionMessage();
break;
}
}
And lastly, we need to tell Wolverine how to do the message batching, which I’ll do first with this code:
public class ValidationResultBatcher : IMessageBatcher
{
public IEnumerable<Envelope> Group(IReadOnlyList<Envelope> envelopes)
{
var groups = envelopes
.GroupBy(x => x.Message.As<ValidationResult>().Id)
.ToArray();
foreach (var group in groups)
{
var message = new ValidationResultBatch(group.Key, group.OfType<ValidationResult>().ToArray());
// It's important here to pass along the group of envelopes that make up
// this batched message for Wolverine's transactional inbox/outbox
// tracking
yield return new Envelope(message, group);
}
}
public Type BatchMessageType => typeof(ValidationResultBatch);
}
Then lastly, in your Wolverine configuration in your Program file (or a helper method that’s called from Program), you’d tell Wolverine about the batching strategy like so:
builder.Host.UseWolverine(opts =>
{
// Other Wolverine configuration...
opts.BatchMessagesOf<ValidationResult>(x =>
{
x.Batcher = new ValidationResultBatcher();
x.BatchSize = 100;
});
});
With the message batching, you’re potentially putting less load on the database and improving performance by simply making fewer reads and writes over all. You might still have some concurrency concerns, so you have more options to control the parallelization of the ValidationResultBatch messages running locally like this in your UseWolverine() configuration:
opts.LocalQueueFor<ValidationResultBatch>()
// You *could* do this to completely prevent
// concurrency issues
.Sequential()
// Or depend on some level of retries on concurrency
// exceptions and let it parallelize work by file
.MaximumParallelMessages(5);
We could choose to accept some risk of concurrent access to an individual FileImportSaga (unlikely after the batching, but still), so let’s add some better optimistic concurrency checking with our friend Marten. For any given Saga type that’s persisted with Marten, just implement the IRevisioned interface to let Wolverine know to opt into Marten’s concurrency protection like so:
public class FileImportSaga :
// Only necessary marker type for Wolverine here
Saga,
// Opts into tracked version concurrency for Marten
// We probably want in this case
IRevisioned
That’s it, that’s all you need to do. What this does for you is create a check by Wolverine & Marten together that during the processing of any message on a FileImportSaga that no other message was successfully processed against that FileImportSaga between loading the initial copy of the saga at the time the transaction is committed. If Marten detects a concurrency violation upon the commit, it rejects the transaction and throws a ConcurrencyException. We can handle that with a series of retries to just have Wolverine retry the message from the new state with this error handling policy that I’m going to make specific to our FileImportSaga like so:
public class FileImportSaga :
// Only necessary marker type for Wolverine here
Saga,
// Opts into tracked version concurrency for Marten
// We probably want in this case
IRevisioned
{
public static void Configure(HandlerChain chain)
{
// Retry the message over again at least 3 times
// with the specified wait times
chain.OnException<ConcurrencyException>()
.RetryWithCooldown(100.Milliseconds(), 250.Milliseconds(), 250.Milliseconds());
}
// ... the rest of FileImportSaga
So now we’ve got the beginnings of a multi-step process using Wolverine’s stateful saga support. We’ve also taken some care to protect our file import process against concurrency concerns. And we’ve done all of this in a way where we can quite handily test the workflow logic by just doing state-based tests against the FileImportSaga with no database or message broker infrastructure in sight before we waste any time trying to debug the whole shebang.
Summary
The key takeaway I hope you get from this is that the full Critter Stack has some significant tooling to help you build complex, multi-step workflows. Pair that with the easy getting started stories that both tools have, and I think you have a toolset that allows you to quickly start while also scaling up to more complex needs when you need that.
As so very often happens, this blog post was bigger than I thought it would be, and I’m breaking it up into a series of a follow ups. In the next version of this post, we’ll take the same logical FileImportSaga and do the logical workflow tracking with Marten event sourcing to track the state and use some cool new Marten functionality for the workflow logic inside of Marten projections.
This might take a bit to get to, but I’ll also revisit this original implementation and talk about some extra Marten functionality to further optimize performance by baking in archiving through Marten soft-deletes and its support for PostgreSQL table partitioning.
So historically I’m actually pretty persnickety about being precise about technical terms and design pattern names, but I’m admittedly sloppy about calling something a “Saga” when maybe it’s technically a “Process Manager” and I got jumped online about that by a celebrity programmer. Sorry, not sorry?
The feature set shown in this post was built earlier this year at the behest of a JasperFx Software client who has some unusually high data throughput and wanted to have some significant ability to scale up Marten and Wolverine‘s ability to handle a huge number of incoming events. We originally put this into what was meant to be a paid add on product, but after consultation with the rest of the Critter Stack core team and other big users, we’ve decided that it would be best for this functionality to be in the OSS core of Wolverine.
JasperFx Software is currently working with a client who has a system with around 75 million events in their database and the expectation that that database could double soon. At the same time, they need to be running around 15-20 different event projections continuously running asynchronously to build read side views. To put it mildly, they’re going to want some serious ability for Marten (with a possible helping hand from Wolverine) to handle that data in a performant manner.
Before Marten 7.0, Marten could only run projections with a “hot/cold” ownership mode that resulted in every possible projection running on a single application node within the cluster. So, not that awesome for scalability to say the least. With 7.0, Marten can do some load distribution of different projections, but it’s not terribly predictable and has no guarantee of spreading the load out.
opts.Services.AddMarten(m =>
{
m.DisableNpgsqlLogging = true;
m.Connection(Servers.PostgresConnectionString);
m.DatabaseSchemaName = "csp";
// This was taken from Wolverine test code
// Imagine there being far more projections and
// subscriptions
m.Projections.Add<TripProjection>(ProjectionLifecycle.Async);
m.Projections.Add<DayProjection>(ProjectionLifecycle.Async);
m.Projections.Add<DistanceProjection>(ProjectionLifecycle.Async);
})
.IntegrateWithWolverine(m =>
{
// This makes Wolverine distribute the registered projections
// and event subscriptions evenly across a running application
// cluster
m.UseWolverineManagedEventSubscriptionDistribution = true;
});
Using the UseWolverineManagedEventSubscriptionDistribution() option in place of Marten’s own async daemon management will give you a load distribution more like this:
Using this model, Wolverine can spread the asynchronous load to more running nodes so you can hopefully get a lot more throughput in your asynchronous projections without overloading any one node.
With this option, Wolverine is going to ensure that every single known asynchronous event projection and every event subscription is running on exactly one running node within your application cluster. Moreover, Wolverine will purposely stop and restart projections or subscriptions to purposely spread the running load across your entire cluster of running nodes.
In the case of using multi-tenancy through separate databases per tenant with Marten, this Wolverine “agent distribution” will assign the work by tenant databases, meaning that all the running projections and subscriptions for a single tenant database will always be running on a single application node. This was done with the theory that this affinity would hopefully reduce the number of used database connections over all.
If a node is taken offline, Wolverine will detect that the node is no longer accessible and try to move start the missing projection/subscription agents on another active node.
If you run your application on only a single server, Wolverine will of course run all projections and subscriptions on just that one server.
Some other facts about this integration:
Wolverine’s agent distribution does indeed work with per-tenant database multi-tenancy
Wolverine does automatic health checking at the running node level so that it can fail over assigned agents
Wolverine can detect when new nodes come online and redistribute work
Wolverine is able to support blue/green deployment and only run projections or subscriptions on active nodes where a capability is present. This just means that you can add all new projections or subscriptions, or even just new versions of a projection or subscription on some application nodes in order to do try “blue/green deployment.”
This capability does depend on Wolverine’s built-in leadership election — which fortunately got a lot better in Wolverine 3.0
Future Plans
While this functionality will be in the OSS core of Wolverine 3.0, we plan to add quite a bit of support to further monitor and control this feature with the planned “Critter Watch” management console tool we (JasperFx) are building. We’re planning to allow users to:
Visualize and monitor which projections and/or subscriptions are running on which application node
See a correlation to performance metrics being emitted to the Open Telemetry tool of your choice — with Prometheus PromQL compatible tools being supported first
Be able to create affinity groups between projections or subscriptions that might be using the same event data as a possible optimization
Allow individual projections or subscriptions to be paused or restarted
Trigger manual projection rebuilds at runtime
Trigger “rewinds” of subscriptions at runtime
We’re also early in planning to port the Marten event sourcing support to additional database engines. The above functionality will be available for those other database engines when we get there.
This functionality was originally conceived of something like 5-6 years ago, and it’s personally very exciting to me to finally see it out in the wild!
I realize the title sounds a little too similar to somebody else’s 2025 platform proposals, but let’s please just overlook that
This is a “vision board” document I wrote up and shared with our core team (Anne, JT, Babu, and Jeffry) as well as some friendly users and JasperFx Software customers. I dearly want to step foot into January 2025 with the “Critter Stack” as a very compelling choice for any shop about to embark on any kind of Event Driven Architecture — especially with the usage of Event Sourcing as part of a system’s persistence strategy. Moreover, I want to arrive at a point where the “Critter Stack” actually convinces organizations to choose .NET just to take advantage of our tooling.I’d be grateful for any feedback.
As of now, the forthcoming Wolverine 3.0 release is almost to the finish line, Marten 7 is probably just about done growing, and work on “Critter Watch” (JasperFx Software’s envisioned management console tooling for the “Critter Stack”) is ramping up. Now is a good time to detail a technical vision for the “Critter Stack” moving into 2025.
The big goals are:
Simplify the “getting started” story for using the “Critter Stack”. Not just in getting a new codebase up, but going all the way to how a Critter Stack app could be deployed and opting into all the best practices. My concern is that there are getting to be way too many knobs and switches scattered around that have to be addressed to really make performance and deployment robust.
Deliver a usable “Critter Watch” MVP
Expand the “Critter Stack” to more database options, with Sql Server and maybe CosmosDb being the leading contenders and DynamoDb or CockroachDb being later possibilities
Streamline the dependency tree. Find a way to reduce the number of GitHub repositories and Nugets if possible. Both for our maintenance overhead and also to try to simplify user setup
“Critter Watch” and CritterStackPro.Projections (actually scratch the second part, that’s going to roll into the Wolverine OSS core, coming soon)
Ermine 1.0 – the Sql Server port of the Marten event store functionality
Out of the box project templates for Wolverine/Marten/Ermine usages – following the work done already by Jeffry Gonzalez
Future CosmosDb backed event store and Wolverine integration — but I’m getting a lot of mixed feedback about whether Sql Server or CosmosDb should be a higher priority
Opportunities to grow the Critter Stack user base:
Folks who are concerned about DevOps issues. “Critter Watch” and maybe more templates that show how to apply monitoring, deployment steps, and Open Telemetry to existing Critter Stack systems. The key point here is a whole lot of focus on maintainability and sustainability of the event sourcing and messaging infrastructure
Get more interest from mainstream .NET developers. Improve the integration of Wolverine and maybe Marten/Ermine as well with EF Core. This could include reaching parity with Marten for middleware support, side effects, and multi-tenancy models using EF Core. Also, maybe, hear me out, take a heavy drink, there could be an official Marten/Ermine projection integration to write projection data to EF Core? I know of at least one Critter Stack user who would use that. At this point, I’m leaning heavily toward getting Wolverine 3.0 out and mostly tackle this in the Wolverine 4.0 timeframe this fall
Expand to Sql Server for more “pure” Microsoft shops. Adding databases to the general Wolverine / Event Sourcing support (the assumption here is that the document database support in Marten would be too much work to move)
Introduce Marten and Wolverine to more people, period. Moar “DevRel” type activity! More learning videos. I’ll keep trying to do more conferences and podcasts. More sample applications. Some ideas for new samples might be a sample application with variations using each transport, using Wolverine inside of a modular monolith with multiple Marten stores and/or EF DbContexts, HTTP services, background processing. Maybe actually invest in some SEO for the websites.
Ecosystem Realignment
With major releases coming up with both Marten 8.0 and Wolverine 4.0 and the forthcoming Ermine, there’s an “opportunity” to change the organization of the code to streamline the number of GitHub repositories and Nugets floating around while also centralizing more code. There’s also an opportunity to centralize a lot of infrastructure code that could help the Ermine effort go much faster. Lastly, there are some options like code generation settings and application assembly determination that are today independently configured for Marten and Wolverine which repeatedly trips up our users (and flat out annoys me when I build sample apps).
We’re actively working to streamline the configuration code, but in the meantime, the current thinking about some of this is in the GitHub issue for JasperFx Ecosystem Dependency Reorganization. The other half of that is the content in the next section.
Projection Model Reboot
This refers to the “Reboot Projection Model API” in the Marten GitHub issue list. The short tag line is to move toward enabling easier usage of folks just writing explicit code. I also want us to tackle the absurdly confusing API for “multi-stream projections” as well. This projection model will be shared across Marten, Ermine (Sql Server-backed event store), and any future CosmosDb/DynamoDb/CockroachDb event stores.
Wrapping up Marten 7.0
Marten 7 introduced a crazy amount of new functionality on top of the LINQ rewrite, the connection management rewrite, and introduction of Polly into the core. Besides some (important) ongoing work for JasperFx clients, the remainder of Marten 7 is hopefully just:
Mark all synchronous APIs that invoke database access as [Obsolete]
Make a pass over the projection model and see how close to the projection reboot you could get. Make anything that doesn’t conform to the new ideal be [Obsolete] with nudges
Introduce the new standard code generation / application assembly configuration in JasperFx.CodeGeneration today. Mark Marten’s version of that as [Obsolete] with a pointer to using the new standard – which is hopefully very close minus namespaces to where it will be in the end
Wrapping up Wolverine 3.0
Introduce the new standard code generation / application assembly configuration in JasperFx.CodeGeneration today. Mark Marten’s version of that as [Obsolete] with a pointer to using the new standard – which is hopefully very close minus namespaces to where it will be in the end
Put a little more error handling in for code generation problems just to make it easier to fix issues later
Maybe, reexamine what work could be done to make modular monoliths easier with Wolverine and/or Marten
Maybe, consider adding back into scope improvements for EF Core with Wolverine – but I’m personally tempted to let that slide to the Wolverine 4 work
Summary
The Critter Stack core & I plus the JasperFx Software folks have a pretty audaciously ambitious plan for next year. I’m excited for it, and I’ll be talking about it in public as much as y’all will let me get away with it!
I know, command line parsing libraries are about the least exciting tooling in the entire software universe, and there are dozens of perfectly competent ones out there. Oakton though, is heavily used throughout the entire “Critter Stack” (Marten, Weasel, and Wolverine plus other tools) to provide command line utilities directly to any old .NET Core application that happens to be bootstrapped with one of the many ways to arrive at an IHost. Oakton’s key advantage over other command line parsing tools is its ability to easily add extension commands to a .NET application in external assemblies. And of course, as part of the entire JasperFx / Critter Stack philosophy of developer tooling, Oakton’s very concept was originally created to enhance the testability of custom command line tooling. Unlike some other tools *cough* System.CommandLine *cough*.
Oakton also has some direct framework-ish elements for environment checks and the stateful resource model used very heavily all the way through Marten and Wolverine to provide the very best development time experience possible when using our tools.
Today the extended JasperFx / Critter Stack community released Oakton 6.2 with some new, hopefully important use cases. First off, the stateful resource model that we use to setup, teardown, or just check “configured stateful resources” in our system like database schemas or message broker queues just got the concept of dependencies between resources such that you can control which resources are setup first.
Next, Oakton finally got a couple easy to use recipes for utilizing IoC services in Oakton commands (it was possible, just maybe a little higher ceremony that some folks prefer). The first way, assuming that you’re running Oakton from one of the many flavors of IHostBuilder or IHost like so:
// This would be the last line in your Program.Main() method
// "app" in this case is a WebApplication object, but there
// are other extension methods for headless services
return await app.RunOaktonCommands(args);
You can build an Oakton command class that uses “setter injection” to get IoC services like so:
public class MyDbCommand : OaktonAsyncCommand<MyInput>
{
// Just assume maybe that this is an EF Core DbContext
[InjectService]
public MyDbContext DbContext { get; set; }
public override Task<bool> Execute(MyInput input)
{
// do stuff with DbContext from up above
return Task.FromResult(true);
}
}
Just know that when you do this and execute a command that has decorated properties for services, Oakton is:
Building your system’s IHost
Creating a new IServiceScope from your application’s DI container, or in other words, a scoped container
Building your command object and setting all the dependencies on your command object by resolving each dependency from the scoped container created in the previous step
Executing the command as normal
Disposing the scoped container and the IHost, effectively in a try/finally so that Oakton is always cleaning up after the application
In other words, Oakton is largely taking care of annoying issues like object disposal cleanup, scoping, and actually building the IHost if necessary.
Oakton’s Future
The Critter Stack Core team & I are charting a course for our entire ecosystem I’m calling “Critter Stack 2025” that’s hoping to greatly reduce the technical challenges in adopting our tool set. As part of that, what’s now Oakton is likely to move into a new shared library (I think it’s just going to be called “JasperFx”) between the various critters (and hopefully new critters for 2025!). Oakton itself will probably get a temporary life as a shim to the new location as a way to ease the transition for existing users. There’s a balance between actively improving your toolset for potential new users and not disturbing existing users too much. We’re still working on whatever that balance ends up being.
Building and maintaining a large, hosted system that requires multi-tenancy comes with a fair number of technical challenges. JasperFx Software has helped several of our clients achieve better results with their particular multi-tenancy challenges with Marten and Wolverine, and we’re available to do the same for your shop! Drop us a message on our Discord server or email us at sales@jasperfx.net to start a conversation.
This is continuing a series about multi-tenancy with Marten, Wolverine, and ASP.Net Core:
Using Partitioning for Better Performance with Multi-Tenancy and Marten (future)
Multi-Tenancy in Wolverine with EF Core & Sql Server (future, and honestly, future functionality as part of Wolverine 4.0)
Dynamic Tenant Creation and Retirement in Marten and Wolverine (definitely in the future)
Let’s say that you’re using the Marten + PostgreSQL combination for your system’s persistence needs in a web service application. Let’s also say that you want to keep the customer data within your system in completely different databases per customer company (or whatever makes sense in your system). Lastly, let’s say that you’re using Wolverine for asynchronous messaging and as a local “mediator” tool. Fortunately, Wolverine by itself has some important built in support for multi-tenancy with Marten that’s going to make your system a lot easier to build.
Let’s get started by just showing a way to opt into multi-tenancy with separate databases using Marten and its integration with Wolverine for middleware, saga support, and the all important transactional outbox support:
// Adding Marten for persistence
builder.Services.AddMarten(m =>
{
// With multi-tenancy through a database per tenant
m.MultiTenantedDatabases(tenancy =>
{
// You would probably be pulling the connection strings out of configuration,
// but it's late in the afternoon and I'm being lazy building out this sample!
tenancy.AddSingleTenantDatabase("Host=localhost;Port=5433;Database=tenant1;Username=postgres;password=postgres", "tenant1");
tenancy.AddSingleTenantDatabase("Host=localhost;Port=5433;Database=tenant2;Username=postgres;password=postgres", "tenant2");
tenancy.AddSingleTenantDatabase("Host=localhost;Port=5433;Database=tenant3;Username=postgres;password=postgres", "tenant3");
});
m.DatabaseSchemaName = "mttodo";
})
.IntegrateWithWolverine(masterDatabaseConnectionString:connectionString);
Just for the sake of completion, here’s some sample Wolverine configuration that pairs up with the above:
// Wolverine usage is required for WolverineFx.Http
builder.Host.UseWolverine(opts =>
{
// This middleware will apply to the HTTP
// endpoints as well
opts.Policies.AutoApplyTransactions();
// Setting up the outbox on all locally handled
// background tasks
opts.Policies.UseDurableLocalQueues();
});
Now that we’ve got that basic setup for Marten and Wolverine, let’s move on to the first issue, how the heck does Wolverine “know” which tenant should be used? In a later post I’ll show how Wolverine.HTTP has built in tenant id detection, but for now, let’s pretend that you’re already taking care of tenant id detection from incoming HTTP requests some how within your ASP.Net Core pipeline and you just need to pass that into a Wolverine message handler that is being executed from within an MVC Core controller (“Wolverine as Mediator”):
[HttpDelete("/todoitems/{tenant}/longhand")]
public async Task Delete(
string tenant,
DeleteTodo command,
IMessageBus bus)
{
// Invoke inline for the specified tenant
await bus.InvokeForTenantAsync(tenant, command);
}
By using the IMessageBus.InvokeForTenantAsync() method, we’re invoking a command inline, but telling Wolverine what the tenant id is. The command handler might look something like this:
// Keep in mind that we set up the automatic
// transactional middleware usage with Marten & Wolverine
// up above, so there's just not much to do here
public static class DeleteTodoHandler
{
public static void Handle(DeleteTodo command, IDocumentSession session)
{
session.Delete<Todo>(command.Id);
}
}
Not much going on there in our code, but Wolverine is helping us out here by:
Seeing the tenant id value that we passed in before that Wolverine is tracking in its own Envelope structure (Wolverine’s version of Envelope Wrapper from the venerable EIP book)
Creates the Marten IDocumentSession for that tenant id value, which will be reading and writing to the correct tenant database underneath Marten
Now, let’s make this a little more complex by also publishing an event message in that message handler for the DeleteTodo message:
public static class TodoCreatedHandler
{
public static TodoDeleted Handle(DeleteTodo command, IDocumentSession session)
{
session.Delete<Todo>(command.Id);
// This
return new TodoDeleted(command.Id);
}
}
public record TodoDeleted(int TodoId);
Assuming that the TodoDeleted message is being published to a “durable” endpoint, Wolverine is using its transactional outbox integration with Marten to persist the outgoing message in the same tenant database and same transaction as the deletion we’re doing in that command handler. In other words, Wolverine is able to use the tenant databases for its outbox support with no other configuration necessary than what we did up above in the calls to AddMarten() and UseWolverine().
Moreover, Wolverine is even able to use its “durability agent” against all the tenant databases to ensure that any work that is somehow stranded by crashed processes.
Lastly, the TodoDeleted event message cascaded above from our message handler would be tracked throughout Wolverine with the tenant id of the original DeleteToDo command message so that you can do multi-part workflows through Wolverine while tracks the tenant id and utilizes the correct tenant database through Marten all along the way.
Summary
Building solutions with multi-tenancy can be complicated, but the Wolverine + Marten combination can make it a lot easier.
Hey, did you know that JasperFx Software offers both consulting services and support plans for the “Critter Stack” tools? Or for architectural or test automation help with any old server side .NET application. One of the other things we do is to build out custom features that our customers need in the “Critter Stack” — like the Marten-managed table partitioning for improved scaling and performance in this release!
A fairly sizable Marten 7.28 release just went live — or will at least be available on Nuget by the time you read this with a mix of new features and usability improvements. The biggest new feature is “Marten-Managed Table Partitioning by Tenant.” Lots of words! Consider this scenario:
You have a system with a huge number of events
You also need to use Marten’s support for multi-tenancy
For historical reasons and for the easy of deployment and management, you are using Marten’s “conjoined” multi-tenancy model and keeping all of your tenant data in the same database (this might have some very large cloud hosting cost saving benefits as well)
You want to be able to scale the database performance for all the normal reasons
PostgreSQL table partitioning to the rescue! In recent Marten releases, we’ve added support to take advantage of postgres table sharding as a way to improve performance in many operations — with one of the obvious first usages using table sharding per tenant id for Marten’s “conjoined” tenancy model. Great! Just tell Marten exactly what the tenant ids are and the matching partition configuration and go!
But wait, what if you have a very large number of tenants and might need to even add new tenants at runtime and without incurring any kind of system downtime? Marten now has a partitioning feature for multi-tenancy that can dynamically create per-tenant shards at runtime and manage the list of tenants in its own database storage like so:
var builder = Host.CreateApplicationBuilder();
builder.Services.AddMarten(opts =>
{
opts.Connection(builder.Configuration.GetConnectionString("marten"));
// Make all document types use "conjoined" multi-tenancy -- unless explicitly marked with
// [SingleTenanted] or explicitly configured via the fluent interfce
// to be single-tenanted
opts.Policies.AllDocumentsAreMultiTenanted();
// It's required to explicitly tell Marten which database schema to put
// the mt_tenant_partitions table
opts.Policies.PartitionMultiTenantedDocumentsUsingMartenManagement("tenants");
});
With some management helpers of course:
await theStore
.Advanced
// This is ensuring that there are tenant id partitions for all multi-tenanted documents
// with the named tenant ids
.AddMartenManagedTenantsAsync(CancellationToken.None,"a1", "a2", "a3");
If you’re familiar with the pg_partman tool, this was absolutely meant to fulfill a similar role within Marten for per-tenant table partitioning.
Aggregation Projections with Explicit Code
This is probably long overdue, but the other highlight that’s probably much more globally applicable is the ability to write more Marten event aggregation projections with strictly explicit code for folks who don’t care for the Marten conventional method approaches — or just want a more complicated workflow than what the conventional approaches can do.
You still need to use the CustomProjection<TDoc, TId> base class for your logic, but now there are simpler methods that can be overloaded to express explicit “left fold over events to create an aggregated document” logic as shown below:
public class ExplicitCounter: CustomProjection<SimpleAggregate, Guid>
{
public override SimpleAggregate Apply(SimpleAggregate snapshot, IReadOnlyList<IEvent> events)
{
snapshot ??= new SimpleAggregate();
foreach (var e in events.Select(x => x.Data))
{
if (e is AEvent) snapshot.ACount++;
if (e is BEvent) snapshot.BCount++;
if (e is CEvent) snapshot.CCount++;
if (e is DEvent) snapshot.DCount++;
}
// You have to explicitly return the new value
// of the aggregated document no matter what!
return snapshot;
}
}
The explicitly coded projections can also be used for live aggregations (AggregateStreamAsync()) and within FetchForWriting() as well. This has been a longstanding request, and will receive even stronger support in Marten 8.
LINQ Improvements
Supporting a LINQ provider is the gift that never stops giving. There’s some small improvements this time around for some minor things:
// string.Trim()
session.Query<SomeDoc>().Where(x => x.Description.Trim() = "something");
// Select to TimeSpan out of a document
session.Query<SomeDoc>().Select(x => x.Duration).ToListAsync();
// Query the raw event data by event types
var raw = await theSession.Events.QueryAllRawEvents()
.Where(x => x.EventTypesAre(typeof(CEvent), typeof(DEvent)))
.ToListAsync();
Hey, did you know that JasperFx Software offers both consulting services and support plans for the “Critter Stack” tools? One of the common areas where we’ve helped our clients is in using Marten or Wolverine when the usage involves quite a bit of potential concerns about concurrency. As I write this, I’m currently working with a JasperFx client to implement the FetchForWriting API shown in this post as a way of improving their system’s resiliency to concurrency problems.
You’ve decided to use event sourcing as your persistence strategy, so that your persisted state of record are the actual business events segregated by streams that represent changes in state to some kind of logical business entity (an invoice? an order? an incident? a project?). Of course there will have to be some way of resolving or “projecting” the raw events into a usable view of the system state, but we’ll get to that.
You’ve also decided to organize your system around a CQRS architectural style (Command Query Responsibility Segregation). With a CQRS approach, the backend code is mostly organized around the “verbs” of your system, meaning the “command” messages (this could be HTTP services, and I’m not implying that there automatically has to be any asynchronous messaging) that are handled to capture changes to the system (events in our case), and “query” endpoints or APIs that strictly serve up information about your system.
While it’s certainly possible to do either Event Sourcing or CQRS without the other, the two things do go together as Forrest Gump would say, like peas and carrots.Marten is certainly valuable as part of a CQRS with Event Sourcing approach within a range of .NET messaging or web frameworks, but there is quite a bit of synergy between Marten and its “Critter Stack” stable mate Wolverine (see the details about the integration here).
And lastly of course, you’ve quite logically decided to use Marten as the persistence mechanism for the events. Marten is also a strong fit because it comes with some important functionality that we’ll need for CQRS command handlers:
Marten’s event projection support can give us a representation of the current state of the raw event data in a usable way that we’ll need within our command handlers to both validate requested actions and to “decide” what additional events should be persisted to our system
The FetchForWriting API in Marten will not only give us access to the projected event data, but it provides an easy mechanism for both optimistic and pessimistic concurrency protections in our system
Marten allows for a couple different options of projection lifecycle that can be valuable for performance optimization with differing system needs
As a sample application problem domain, I got to be part of a successful effort during the worst of the pandemic to stand up a new “telehealth” web portal using event sourcing. One of the concepts in that system that we needed to track in that system was the activity of a health care provider (nurse, doctor, nurse practitioner) with events for when they were available and what they were doing at any particular time during the day for later decision making:
public record ProviderAssigned(Guid AppointmentId);
public record ProviderJoined(Guid BoardId, Guid ProviderId);
public record ProviderReady;
public record ProviderPaused;
public record ProviderSignedOff;
// "Charting" is basically just whatever
// paperwork they need to do after
// an appointment, and it was important
// for us to track that time as part
// of their availability and future
// planning
public record ChartingFinished;
public record ChartingStarted;
public enum ProviderStatus
{
Ready,
Assigned,
Charting,
Paused
}
But of course, at several points, you do actually need to know what the actual state of the provider’s current shift is to be able to make decisions within the command handlers, so we had a “write” model something like this:
// I'm sticking the Marten "projection" logic for updating
// state from the events directly into this "write" model,
// but you could separate that into a different class if you
// prefer
public class ProviderShift
{
public Guid Id { get; set; }
// This is important, this would be set by Marten to the
// current event number or revision of the ProviderShift
// aggregate. This is going to be important later for
// concurrency protections
public int Version { get; set; }
public Guid BoardId { get; private set; }
public Guid ProviderId { get; init; }
public ProviderStatus Status { get; private set; }
public string Name { get; init; }
public Guid? AppointmentId { get; set; }
// The Create & Apply methods are conventional targets
// for Marten's "projection" capabilities
// But don't worry, you would never *have* to take a reference
// to Marten itself like I did below jsut out of laziness
public static ProviderShift Create(
ProviderJoined joined)
{
return new ProviderShift
{
Status = ProviderStatus.Ready,
ProviderId = joined.ProviderId,
BoardId = joined.BoardId
};
}
public void Apply(ProviderReady ready)
{
AppointmentId = null;
Status = ProviderStatus.Ready;
}
public void Apply(ProviderAssigned assigned)
{
Status = ProviderStatus.Assigned;
AppointmentId = assigned.AppointmentId;
}
public void Apply(ProviderPaused paused)
{
Status = ProviderStatus.Paused;
AppointmentId = null;
}
// This is kind of a catch all for any paperwork the
// provider has to do after an appointment has ended
// for the just concluded appointment
public void Apply(ChartingStarted charting)
{
Status = ProviderStatus.Charting;
}
}
The whole purpose of the ProviderShift type above is to be a “write” model that contains enough information for the command handlers to “decide” what should be done — as opposed to a “read” model that might have richer information like the provider’s name that would be more suitable or usable for using within a user interface. “Write” or “read” in this case is just a role within the system, and at different times it might be valuable to have separate models for different consumers of the information and in other times be able to happily get by with a single model.
Alright, so let’s finally look at a very simple command handler related to providers that tries to mark the provider as being finished charting:
// Since we're focusing on Marten, I'm using an MVC Core
// controller just because it's commonly used and understood
public class CompleteChartingController : ControllerBase
{
[HttpPost("/provider/charting/complete")]
public async Task Post(
[FromBody] CompleteCharting charting,
[FromServices] IDocumentSession session)
{
// We're looking up the current state of the ProviderShift aggregate
// for the designated provider
var stream = await session
.Events
.FetchForWriting<ProviderShift>(charting.ProviderShiftId, HttpContext.RequestAborted);
// The current state
var shift = stream.Aggregate;
if (shift.Status != ProviderStatus.Charting)
{
// Obviously do something smarter in your app, but you
// get the point
throw new Exception("The shift is not currently charting");
}
// Append a single new event just to say
// "charting is finished". I'm relying on
// Marten's automatic metadata to capture
// the timestamp of this event for me
stream.AppendOne(new ChartingFinished());
// Commit the transaction
await session.SaveChangesAsync();
}
}
I’m using the Marten FetchForWriting() API to get at the current state of the event stream for the designated provider shift (a provider’s activity during a single day). I’m also using this API to capture a new event marking the provider as being finished with charting. FetchForWriting() is doing two important things for us:
Executes or finds the projected data for ProviderShift from the raw events. More on this a little later
Provides a little bit of optimistic concurrency protection for our provider shift stream
Building on the theme of concurrency first, the command above will “remember” the current state of the ProviderShift at the point that FetchForWriting() is called. Upon SaveChangesAsync(), Marten will reject the transaction and throw a ConcurrencyException if some how, some way, some other request magically came through and completed against that same ProviderShift stream between the call to FetchForWriting() and SaveChangesAsync().
That level of concurrency is baked in, but we can do a little bit better. Remember that the ProviderShift has this property:
// This is important, this would be set by Marten to the
// current event number or revision of the ProviderShift
// aggregate. This is going to be important later for
// concurrency protections
public int Version { get; set; }
The projection capability of Marten makes it easy for us to “know” and track the current version of the ProviderShift stream so that we can feed it back to command handlers later. Here’s the full definition of the CompleteCharting command:
public record CompleteCharting(
Guid ProviderShiftId,
// This version is meant to mean "I was issued
// assuming that the ProviderShift is currently
// at this version in the server, and if the version
// has shifted since, then this command is now invalid"
int Version
);
Let’s tighten up the optimistic concurrency protection such that Marten will shut down the command handling faster before we waste system resources doing unnecessary work by passing the command version right into this overload of FetchForWriting():
// Since we're focusing on Marten, I'm using an MVC Core
// controller just because it's commonly used and understood
public class CompleteChartingController : ControllerBase
{
[HttpPost("/provider/charting/complete")]
public async Task Post(
[FromBody] CompleteCharting charting,
[FromServices] IDocumentSession session)
{
// We're looking up the current state of the ProviderShift aggregate
// for the designated provider
var stream = await session
.Events
.FetchForWriting<ProviderShift>(
charting.ProviderShiftId,
// Passing the expected, starting version of ProviderShift
// into Marten
charting.Version,
HttpContext.RequestAborted);
// And the rest of the controller stays the same as
// before....
}
}
In the usage above, Marten will do a version check both at the point of FetchForWriting() using the version we passed in, and again during the call to SaveChangesAsync() to reject any changes made if there was a concurrent update to that same stream.
Lastly, Marten gives you the ability to opt into heavier, exclusive access to the ProviderShift with this option:
// We're looking up the current state of the ProviderShift aggregate
// for the designated provider
var stream = await session
.Events
.FetchForExclusiveWriting<ProviderShift>(
charting.ProviderShiftId,
HttpContext.RequestAborted);
In that last usage, we’re relying on the underlying PostgreSQL database to get us an exclusive row lock on the ProviderShift event stream such that only our current operation is allowed to write to that event stream while we have the lock. This is heavier and comes with some risk of database locking problems, but solves the concurrency issue.
So that’s concurrency protection in FetchForWriting(), but I mostly skipped over when and how that API will execute the projection logic to go from the raw events like ProviderJoined, ProviderReady, or ChartingStarted to the projected ProviderShift.
Any projection in Marten can be calculated or executed with three different “projection lifecycles”:
Live — in this case, a projection is calculated on the fly by loading the raw events in memory and calculating the current state right then and there. In the absence of any other configuration, this is the default lifecycle for the ProviderShift per stream aggregation.
Inline — a projection is updated at the time any events are appended by Marten and persisted by Marten as a document in the PostgreSQL database.
Async — a projection is updated in a background process as events are captured by Marten across the system. The projected state is persisted as a Marten document to the underlying PostgreSQL database
The first two options give you strong consistency models where the projection will always reflect the current state of the events captured to the database. Live is probably a little more optimal in the case where you have many writes, but few reads, and you want to optimize the “write” side. Inline is optimal for cases where you have few writes, but many reads, and you want to optimize the “read” side (or need to query against the projected data rather than just load by id). The Async model gives you the ability to take the work of projecting events into the aggregated state out of both the “write” and “read” side of things. This might easily be advantageous for performance and very frequently necessary for ordering or concurrency concerns.
In the case of the FetchForWriting() API, you will always have a strongly consistent view of the raw events because that API is happily wallpapering over the lifecycle for you. Live aggregation works as you’d expect, Inline aggregation works by just loading the expected document directly from the database, and Async aggregation is a hybrid model that starts from the last known persisted value for the aggregate and applies any missing events right on top of that (the async behavior was a big feature added in Marten 7).
By hiding the actual lifecycle behavior behind the FetchForWriting() signature, teams are able to experiment with different approaches to optimize their application without breaking existing code.
Summary
FetchForWriting() was built specifically to ease the usage of Marten within CQRS command handlers after seeing how much boilerplate code teams were having to use before with Marten. At this point, this is our strongly recommended approach for command handlers. Also note that this API is utilized within the Wolverine + Marten “aggregate handler workflow” usage that does even more to remove code ceremony from CQRS command handler code. To some degree, what is now Wolverine was purposely rebooted and saved from the scrap heap specifically because of that combination with Marten and the FetchForWriting API.
Personally, I’m opposed to any kind of IAggregateRepository or approach where the “write” model itself tracks the events that are applied or uncommitted. I’m trying hard to discourage folks using Marten away from this still somewhat popular old approach in favor of a more Functional Programming-ish approach.
FetchForWriting could be used as part of a homegrown “Decider” pattern usage if that’s something you prefer (I think the “decider” pattern in real life usage ends up devolving into brute force procedural code through massive switch statements personally).
The “telehealth” system I mentioned before was built in real life with a hand-rolled Node.js event sourcing implementation, but that experience has had plenty of influence over later Marten work including a feature that just went into Marten over the weekend for a JasperFx client to be able to emit “side effect” actions and messages during projection updates.
I was deeply unimpressed with the existing Node.js tooling for event sourcing at that time (~2020), but I would hope it’s much better now. Marten has absolutely grown in capability in the past couple years.
Just a reminder, JasperFx Software offers support contracts and consulting services to help you get the most out of the “Critter Stack” tools (Marten and Wolverine).If you’re building server side applications on .NET, the Critter Stack is the most feature rich tool set for Event Sourcing and Event Driven Architectures around. And as I hope to prove to you in this post, Marten is a great option as a document database too!
Marten as a project started as an ultimately successful attempt to replace my then company’s usage of an early commercial “document database” with the open source PostgreSQL database — but with a small, nascent event store functionality bolted onto the side. With the exception of LINQ provider related issues, most of my attention these days is focused on the event sourcing side of things with the document database features in Marten just being a perfect complement for event projections.
This week and last though, I’ve had cause to work with a different document database option and it served to remind me that hey, Marten has a very strong technical story as a document database option. With that being said, let me get on with lionizing Marten by starting with a quick start.
Let’s say that you are building a server side .NET application with some kind of customer data and you at least start by modeling that data like so:
public class Customer
{
public Guid Id { get; set; }
// We'll use this later for some "logic" about how incidents
// can be automatically prioritized
public Dictionary<IncidentCategory, IncidentPriority> Priorities { get; set; }
= new();
public string? Region { get; set; }
public ContractDuration Duration { get; set; }
}
public record ContractDuration(DateOnly Start, DateOnly End);
public enum IncidentCategory
{
Software,
Hardware,
Network,
Database
}
public enum IncidentPriority
{
Critical,
High,
Medium,
Low
}
And once you have those types, you’d like to have that customer data saved to a database in a way that makes it easy to persist, query, and load that data with minimal developmental cost while still being as robust as need be. Assuming that you have access to a running instance of PostgreSQL (it’s very Docker friendly and I tend to use that as a development default), bring in Marten by first adding a reference to the “Marten” Nuget. Next, write the following code in a simple console application that also contains the C# code from above:
using Marten;
using Newtonsoft.Json;
// Bootstrap Marten itself with default behaviors
await using var store = DocumentStore
.For("Host=localhost;Port=5432;Database=marten_testing;Username=postgres;password=postgres");
// Build a Customer object to save
var customer = new Customer
{
Duration = new ContractDuration(new DateOnly(2023, 12, 1), new DateOnly(2024, 12, 1)),
Region = "West Coast",
Priorities = new Dictionary<IncidentCategory, IncidentPriority>
{
{ IncidentCategory.Database, IncidentPriority.High }
}
};
// IDocumentSession is Marten's unit of work
await using var session = store.LightweightSession();
session.Store(customer);
await session.SaveChangesAsync();
// Marten assigned an identity for us on Store(), so
// we'll use that to load another copy of what was
// just saved
var customer2 = await session.LoadAsync<Customer>(customer.Id);
// Just making a pretty JSON printout
Console.WriteLine(JsonConvert.SerializeObject(customer2, Formatting.Indented));
And that’s that, we’ve got a working usage of Marten to save, then load Customer data to the underlying PostgreSQL database. Right off the bat I’d like to point out a couple things about the code samples above:
We didn’t have to do any kind of mapping from our Customer type to a database structure. Marten is using JSON serialization to persist the data to the database, and as long as the Customer type can be bi-directionally serialized to and from JSON, Marten is going to be able to persist and load the type.
We didn’t specify or do anything about the actual database structure. In its default “just get things done” settings, Marten is able to happily detect that the necessary database objects for Customer are missing in the database, and build those out for us on demand
So that’s the easiest possible quick start, but what about integrating Marten into a real .NET application? Assuming you have a reference to the Marten nuget package, it’s just an IServiceCollection.AddMarten() call as shown below from a sample web application:
builder.Services.AddMarten(opts =>
{
// You always have to tell Marten what the connection string to the underlying
// PostgreSQL database is, but this is the only mandatory piece of
// configuration
var connectionString = builder.Configuration.GetConnectionString("postgres");
opts.Connection(connectionString);
})
// This is a mild performance optimization
.UseLightweightSessions();
At this point in the .NET ecosystem, it’s more or less idiomatic to use an Add[Tool]() method to integrate tools with your application’s IHost, and Marten tries to play within the typical .NET rules here.
I think this idiom and the generic host builder tooling has been a huge boon to OSS tool development in the .NET space compared to the old wild, wild west days. I do wish it would stop changing from .NET version to version though.
So that’s all a bunch of simple stuff, so let’s dive into something that shows off how Marten — really PostgreSQL — has a much stronger transactional model than many document databases that only support eventual consistency:
public static async Task manipulate_customer_data(IDocumentSession session)
{
var customer = new Customer
{
Name = "Acme",
Region = "North America",
Class = "first"
};
// Marten has "upsert", insert, and update semantics
session.Insert(customer);
// Partial updates to a range of Customer documents
// by a LINQ filter
session.Patch<Customer>(x => x.Region == "EMEA")
.Set(x => x.Class, "First");
// Both the above operations happen in one
// ACID transaction
await session.SaveChangesAsync();
// Because Marten is ACID compliant, this query would
// immediately work as expected even though we made that
// broad patch up above and inserted a new document.
var customers = await session.Query<Customer>()
.Where(x => x.Class == "First")
.Take(100)
.ToListAsync();
}
That’s a completely contrived example, but the point is, because Marten is completely ACID-compliant, you can make a range of operations within transactional boundaries and not have to worry about eventual consistency issues in immediate queries that other document databases suffer from.
So what else does Marten do? Here’s a bit of a rundown because Marten has a significantly richer built in feature set than many other low level document databases:
Fine-grained control over identity map or automatic dirty checking to run lighter for better performance — or to opt into the heavier automatic dirty checking for convenience. The point here is that you have control
And quite a bit more than that, including some test automation support I really need to better document:/
And on top of everything else, because Marten is really just a fancy library on top of PostgreSQL — the most widely used database engine in the world — Marten instantly comes with a wide array of solid cloud hosting options as well as being deployable to local infrastructure on premise. PostgreSQL is also very Docker-friendly, making it a great technical choice for local development.
What’s a Document Database?
If you’re not familiar with the term “document database,” it refers to a type of NoSQL database where data is almost inevitably stored as JSON data, where the database allows you to quickly marshal objects in code to the database, then query that data later right back into the same object structures. The huge benefit of document databases at development time is being able to code much more productively because you just don’t have nearly as much friction as you do when dealing with any kind of object-relational mapping with either an ORM tool or by writing SQL and object mapping code by hand.
Wolverine puts a very high emphasis on reducing code ceremony and tries really hard to keep itself out of your application code. Wolverine is also built with testability in mind. If you’d be interested in learning more about how Wolverine could simplify your existing application code or set you up with a solid foundation for sustainable productive development for new systems, JasperFx Software is happy to work with you!
Before I get into the nuts and bolts of Wolverine sagas, let me come right out and say that I think that compared to other .NET frameworks, the Wolverine implementation of sagas requires much less code ceremony and therefore easier code to reason about. Wolverine also requires less configuration and explicit code to integrate your custom saga with Wolverine’s saga persistence. Lastly, Wolverine makes the development experience better by building in so much support for automatically configuring development environment resources like database schema objects or message broker objects. I do not believe that any other .NET tooling comes close to the developer experience that the Wolverine and its “Critter Stack” buddy Marten can provide.
Let’s say that you have some kind of multi-step process in your application that might have some mix of:
Callouts to 3rd party services
Some logical steps that can be parallelized
Possibly some conditional workflow based on the results of some of the steps
A need to enforce “timeout” conditions if the workflow is taking too long — think maybe of some kind of service level agreement for your workflow
This kind of workflow might be a great opportunity to use Wolverine’s version of Sagas. Conceptually speaking, a “saga” in Wolverine is just a special message handler that needs to inherit from Wolverine’s Saga class and modify itself to track state between messages that impact the saga.
Below is a simple version from the documentation called Order:
public record StartOrder(string OrderId);
public record CompleteOrder(string Id);
public class Order : Saga
{
// You do need this for the identity
public string? Id { get; set; }
// This method would be called when a StartOrder message arrives
// to start a new Order
public static (Order, OrderTimeout) Start(StartOrder order, ILogger<Order> logger)
{
logger.LogInformation("Got a new order with id {Id}", order.OrderId);
// creating a timeout message for the saga
return (new Order{Id = order.OrderId}, new OrderTimeout(order.OrderId));
}
// Apply the CompleteOrder to the saga
public void Handle(CompleteOrder complete, ILogger logger)
{
logger.LogInformation("Completing order {Id}", complete.Id);
// That's it, we're done. Delete the saga state after the message is done.
MarkCompleted();
}
// Delete this order if it has not already been deleted to enforce a "timeout"
// condition
public void Handle(OrderTimeout timeout, ILogger<Order> logger)
{
logger.LogInformation("Applying timeout to order {Id}", timeout.Id);
// That's it, we're done. Delete the saga state after the message is done.
MarkCompleted();
}
public static void NotFound(CompleteOrder complete, ILogger logger)
{
logger.LogInformation("Tried to complete order {Id}, but it cannot be found", complete.Id);
}
}
Order is really meant to just be a state machine where it modifies its own state in response to incoming messages and returns cascading messages (you could also use IMessageBus directly as a method argument if you prefer, but my advice is to use simple pure functions) that tell Wolverine what to do next in the multi-step process.
A new Order saga can be created by any old message handler by simply returning a type that inherits from the Saga type in Wolverine. Wolverine is going to automatically discover any public types inheriting from Saga and utilize any public instance methods following certain naming conventions (or static Create() methods) as message handlers that are assumed to modify the state of the saga objects. Wolverine itself is handling everything to do with loading and persisting the Order saga object between commands around the call to the message handler methods on the saga types.
If you’ll notice the Handle(CompleteOrder) method above, the Order is calling MarkCompleted() on itself. That will tell Wolverine that the saga is now complete, and direct Wolverine to delete the current Order saga from the underlying persistence.
As for tracking the saga id between message calls, there are naming conventions about the messages that Wolverine can use to pluck the identity of the saga, but if you’re strictly exchanging messages between a Wolverine saga and other Wolverine message handlers, Wolverine will automatically track metadata about the active saga back and forth.
I’d also ask you to notice the OrderTimeout message that the Order saga returns as it starts. That message type is shown below:
// This message will always be scheduled to be delivered after
// a one minute delay because I guess we want our customers to be
// rushed? Goofy example code:)
public record OrderTimeout(string Id) : TimeoutMessage(1.Minutes());
Wolverine’s cascading message support allows you to return an outgoing message with a time delay — or a particular scheduled time or any other number of options — by just returning a message object. Admittedly this ties you into a little more of Wolverine, but the key takeaway I want you to notice here is that every handler method is a “pure function” with no service dependencies. Every bit of the state change and workflow logic can be tested with simple unit tests that merely work on the before and after state of the Order objects as well as the cascaded messages returned by the message handler functions. No mock objects, no fakes, no custom test harnesses, just simple unit tests. No other saga implementation in the .NET ecosystem can do that for you anywhere nearly as cleanly.
So far I’ve only focused on the logical state machine part of sagas, so let’s jump to persistence. Wolverine has long had a simplistic saga storage mechanism with its integration with Marten, and that’s still one of the easiest and most powerful options. You can also use EF Core for saga persistence, but ick, that means having to use EF Core.
Wolverine 3.0 added a new lightweight saga persistence option for either Sql Server or PostgreSQL (without Marten or EF Core) that just stands up a little table for just a single Saga type and uses JSON serialization to persist the saga. Here’s an example:
using var host = await Host.CreateDefaultBuilder()
.UseWolverine(opts =>
{
// This isn't actually mandatory, but you'll
// need to do it just to make Wolverine set up
// the table storage as part of the resource setup
// otherwise, Wolverine is quite capable of standing
// up the tables as necessary at runtime if they
// are missing in its default configuration
opts.AddSagaType<RedSaga>("red");
opts.AddSagaType(typeof(BlueSaga),"blue");
// This part is absolutely necessary just to have the
// normal transactional inbox/outbox support and the new
// default, lightweight saga persistence
opts.PersistMessagesWithSqlServer(Servers.SqlServerConnectionString, "color_sagas");
opts.Services.AddResourceSetupOnStartup();
}).StartAsync();
Just as with the integration with Marten, Wolverine’s lightweight saga implementation is able to build the necessary database table storage on the fly at runtime if it’s missing. The “critter stack” philosophy is to optimize the all important “time to first pull request” metric — meaning that you can get a Wolverine application up fast on your local development box because it’s able to take care of quite a bit of environment setup for you.
Lastly, Wolverine 3.0 is adding optimistic concurrency checks for the Marten saga storage and the new lightweight saga persistence. That’s been an important missing piece of the Wolverine saga story.
Just for some comparison, check out some other saga implementations in .NET:
MassTransit’s saga support which somewhat inspired the Wolverine implementation and accounts for a big chunk of Marten usage through MassTransit’s usage of Marten for saga storage
NServiceBus is to my knowledge, the oldest tool in this space and they support sagas
I couldn’t find any support for sagas in Brighter, but feel free to correct that
The Shade Tree Developer 