Wolverine 6.30: Optimized at least once delivery with “Native Acks”

Wolverine 6.30 is out. It is a big release with one clear headline: a new endpoint modeEndpointMode.NativeAck, plus the supporting cast it needed to be trustworthy — lease renewal, an additional idempotency guard, partitioning, and a five-node chaos harness to address a reported issue. This release came with feature requests from JasperFx Software clients, and just to remind folks, being a JasperFx support client means that like Homer below, you have the secret handshake to get your Critter Stack needs and requests to the top of our priority list:

There is also a crop of transport fixes, two long-standing multi-tenancy gaps closed, and a couple of new HTTP and event sourcing conveniences. And inevitably, we also smuggled in some support for our soon forthcoming “Event Modeling” visualization support across the Critter Stack and more for CritterWatch visualization.

You’ve obviously detected that this post was largely drafted by AI of course, but the other side effect of AI usage is that we actually find many more little issues as we work and this 6.30 release includes quite a few bug fixes that spilled out of AI assisted efforts to improve our continuous integration safety nets and an embarrassing number of combinatorial bugs in the existing code that were discovered during the “Native Acks” effort.

Let’s take the tour.


🚀 The Fourth Endpoint Mode

This mode came about from interactions with a JasperFx client who needed to optimize the throughput of listening endpoints but still needed an “at least once” delivery guarantee. The existing Durable mode gives you the delivery guarantee and parallelization, but the extra database overhead was problematic for a flood of small messages that needed to be handled quickly. The Buffered mode gives you the maximal throughput, but a lesser delivery guarantee and is mostly suitable for “fire and forget” messages. Our Inline mode gives you the “at least once” guarantee without the database overhead of Durable, but limited parallelism and throughput. Our client used a different configuration change in Wolverine that alleviated their message pressure, but this new mode was going to be the next thing we tried for them if they needed it.

Wolverine has had three listener modes for a long time, and if you have configured a busy endpoint you have probably felt the shape of the tradeoff. You want parallelism. You want group-ordered processing. You do not want to lose messages when a pod gets rescheduled. And you would very much like to not stand up a database to get all three.

Until 6.30 you could have any two.

Broker ack timingLoss windowParallelismGroup partitioningDB cost
Inlineafter handler successnoneListenerCount onlynonenone
NativeAckafter handler successnoneMaximumParallelMessages✔️none
BufferedInMemoryat receipt, before the handlercrash loses buffered messagesMaximumParallelMessages✔️none
Durableafter the inbox insertnoneMaximumParallelMessages✔️inbox insert + mark-handled

NativeAck fills the empty cell: Buffered’s throughput and partitioning with Inline’s no-loss guarantee, and no database involvement.

opts.ListenToRabbitQueue("webhooks")
    .ProcessInParallelWithNativeAcks()
    .PartitionProcessingByGroupId(PartitionSlots.Five)
    .MaximumParallelMessages(10);

The mechanism is exactly what the name says. The broker delivery is held unacknowledged while the message flows through an in-memory, optionally group-partitioned execution block, and is settled natively from the completion continuation — acked on handler success, nacked or dead-lettered on terminal failure. Nothing is written to a database. Nothing is acknowledged ahead of its handler.

Three consequences follow, and all three are the point rather than side effects:

  • Back pressure is the broker’s prefetch window, not BufferingLimits. The broker stops delivering once its unacked ceiling is reached, so there is no BackPressureAgent at all.
  • A dying node loses nothing. Anything queued but not yet completed is still unacknowledged, so closing the channel or crashing hands every one of those deliveries back to the broker.
  • Shutdown redelivers, but redelivery is not duplicate execution. More on that below, because it is the part everyone gets wrong — including, until this release, our own documentation.

The guarantee, stated exactly

I want to be precise here rather than reassuring, because this is the sort of claim people build on:

Protection against intra-group concurrency is the hard guarantee. Strict sequential processing in original delivery order is not.

The sequential lane per group slot structurally guarantees that no two messages sharing a group id execute concurrently on the owning node. Original-order processing is not guaranteed under failure, requeue, or broker redelivery — a failed or redelivered message re-enters its lane later, never concurrently. That is the honest contract for native-ack retry semantics on every broker. If you need strict order under failure, keep the durable inbox.


📊 What a Redeployment Actually Costs

This was from the community.

Earlier versions of the docs said a rolling deploy produced “duplicate deliveries bounded by the prefetch depth.” That was honest reasoning. It was also never measured — and when we finally measured it, the figure turned out to be the wrong one to quote. It is the bound on redeliveries, and it overstates handler-visible duplicates by more than an order of magnitude.

So #3713 built the harness: a five node cluster, group-partitioned across five RabbitMQ slots, under a sustained flood deep enough that the broker was holding a full unacked window at every disruption, reading the broker’s own messages_unacknowledged at that instant. Ranges are across four consecutive runs.

ScenarioUnacked at disruptionDuplicate executionsRate
Steady state, no disruption000%
Rolling deploy, all 5 nodes drained and replaced18000%
One node killed outright mid-flood1803–4~0.1%
Two hard kills plus two rolling replacements1807–9~0.05%

Two findings matter more than the headline percentage:

A graceful rolling deploy costs zero duplicate executions. Draining settles the handlers that were already running, so nothing runs twice. The prefetch window is redelivered — those messages simply had not executed yet, so they run for the first time, and your handler cannot tell them from any other first delivery.

A hard kill costs about one duplicate per busy lane, not one per unacked message. Only handlers that were mid-flight when the connection died can run twice, and that population is the partition slot count, not the prefetch depth. Three or four per killed node against an unacked window of 180 — a factor of roughly 45, and it barely moved between runs.

Handlers still need to be idempotent; at-least-once is the contract and 0.1% of a flood is not a small number of messages. But size that work against in-flight lanes rather than against prefetch. The full writeup is in What a redeployment actually costs.


🔌 Transport Support: Opt-In and Default-Closed

Wolverine actually supports more messaging technology options than any other messaging tool in .NET, and man, that’s not always a blessing.

A transport must settle each delivery individually and tolerate settling out of order, because the execution block completes messages in handler-completion order rather than delivery order. Not every broker can express that, so support is opt-in and default-closed — calling ProcessInParallelWithNativeAcks() on a transport that has not claimed the mode throws at configuration time rather than degrading silently.

Seven transports qualified and shipped in this release:

TransportDocsIssue
RabbitMQNative Ack Endpoints#3708
Amazon SQSNative Ack Processing#4050
Azure Service BusNative ack endpoints#4051
NATS JetStreamNative Acks with Parallel Processing#4053
Redis StreamsNative Acks with Parallel Processing#4046
PulsarNative Ack Processing#4047
GCP Pub/SubConcurrency and flow control#4052

Kafka cannot and is out of scope. A cumulative offset commit has no way to express a gap. This is not an oversight to be fixed later — it is what the storage model means.

Two transports also refuse the mode for particular endpoints whose own settings contradict it, again at bootstrap rather than at runtime. Pulsar’s AcknowledgeCumulative() reintroduces exactly the gap-less commit problem that disqualifies Kafka. And an SQS FIFO queue exists to guarantee ordering within a message group, which native-ack lanes deliberately do not preserve — and which partitioning by group id does not rescue, because SQS blocks a message group behind its own in-flight head. Both combinations are rejected by name.

Brokers that put a clock on an unsettled delivery

This is about protecting you from problems that arise from long executing messages when using any kind of native broker acknowledgement. This isn’t an issue at all in our Durable mode, but becomes an issue using any other endpoint mode. We had to get more serious about this with the advent or our new NativeAck mode in 6.30.

On SQS, Azure Service Bus, JetStream and Pub/Sub, an unsettled delivery is on a timer — visibility timeout, lock duration, AckWait. Wolverine renews that clock for every delivery still sitting in a lane, for as long as it sits there (#4048).

This is unconditional under this mode and not something you opt into. Lane queue time is unbounded by design, so an un-renewed native-ack endpoint would be a duplicate-delivery generator by construction rather than merely at risk under a slow handler. A transport that declares such a clock but does not implement renewal is refused at startup.

SQS is a good illustration of how far the per-transport tuning goes — MaxNumberOfMessages defaults down under this mode:

opts.ListenToSqsQueue("webhooks")
.ProcessInParallelWithNativeAcks()
.PartitionProcessingByGroupId(PartitionSlots.Five)
.MaximumParallelMessages(10);

Instead of the usual 10, a native-ack endpoint receives twice the number of lanes that can be busy at once, clamped to the SQS maximum. Under every other mode the surplus messages in a batch are deleted before their handlers run, so a full batch is free and saves API calls. Here each one sits in a lane holding an unsettled delivery to renew and to redeliver on a crash. Setting the property explicitly always wins.


🛡️ The In-Memory Idempotency Guard

The durable inbox deduplicates on the primary key of its incoming table. NativeAck has no such table and is at-least-once by design, so 6.30 adds the non-durable analogue (#3710): an opt-in, bounded, in-memory set of the message ids this process has already handled on this endpoint.

opts.ListenToRabbitQueue("webhooks")
    .ProcessInParallelWithNativeAcks()
    .PartitionProcessingByGroupId(PartitionSlots.Five)

    // Opt in. Both arguments are optional; these are the defaults.
    .WithInMemoryIdempotency(window: 5.Minutes(), maxTracked: 100_000);

Or everywhere at once:

opts.Policies.AllListeners(x => x.WithInMemoryIdempotency());

Read the limits before you rely on it. The guard is per process and in memory, and three consequences follow — none of which is a bug:

  • A restart forgets everything. The very deploy that produces the redelivery burst also empties the guard on the node that starts up. It protects a running process against a redelivery it saw itself.
  • A second node never knew. With competing consumers, a redelivery can land on a different node than the original.
  • Eviction is generational, not exact. An id is remembered for at least half the window and at most the whole window — less if a flood of unique ids hits maxTracked first.

The promise is at-least-once delivery with best-effort deduplication, not exactly-once. If you need hard deduplication across restarts and nodes, that is what the durable inbox is for. Memory is bounded by construction — two rotating hash sets, no per-entry timestamps, no LRU bookkeeping, roughly single-digit megabytes at the 100,000 default. Details in In-Memory Idempotency Guard.


⚠️ The Fix You Might Actually Be Affected By

Buried in the supporting work is #3712, and it is worth pulling out because it may already apply to you.

Wolverine now validates listener configuration coherence at bootstrap instead of silently ignoring settings that the chosen mode cannot honor. In writing those checks we discovered that RabbitMQ queues default to Inline — and Inline supports neither parallelism nor group partitioning.

Which means: if you built a sharded topology with PublishToShardedRabbitQueues() and did not explicitly add ConfigureListening(x => x.BufferedInMemory())your partitioning was silently doing nothing. The configuration was accepted and ignored. Now it is rejected with an error that says so.

A companion fix (#4022) catches the related case where a local queue configured with ProcessInline() was accepted at configuration time and then threw a message-less NotSupportedException from deep inside agent startup.

The partitioning guide covers the topology options, including the NativeAck variant added in #3709 — partitioned clustering without the durable inbox, across all ten transports that support the sharded topology.


🏢 Multi-Tenancy: Conjoined EF Core Under a Marten-Owned Store

#4044 is a small feature with a genuinely sneaky root cause.

If Marten owns your message storage through IntegrateWithWolverine(), Wolverine’s message store is built from Marten’s NpgsqlDataSource and never sees a connection string. And NpgsqlDataSource.ConnectionString deliberately omits the password — so there is no string the conjoined DbContext could be configured with that the database would actually accept. The obvious fix (pull the connection string off the data source) produces an authentication failure at first use.

So there is a new DbDataSource overload that carries the credentials through intact:

var builder = Host.CreateApplicationBuilder();

var configuration = builder.Configuration;

builder.UseWolverine(opts =>
{
    // Marten owns the message storage here, so Wolverine's message store is built
    // from Marten's NpgsqlDataSource rather than from a connection string
    opts.Services.AddMarten(m =>
    {
        m.Connection(configuration.GetConnectionString("main")!);
    }).IntegrateWithWolverine();

    // ...which means the conjoined DbContext has to be configured from that same
    // DbDataSource. NpgsqlDataSource.ConnectionString deliberately omits the password,
    // so the connection string overload cannot authenticate in this setup
    opts.Services.AddDbContextWithWolverineManagedConjoinedTenancy<ConjoinedItemsDbContext>(
        (builder, dataSource) =>
        {
            builder.UseNpgsql((NpgsqlDataSource)dataSource);
        }, AutoCreate.CreateOrUpdate);
});

Registering the connection string overload in this setup now fails fast at startup with an error naming this overload, rather than surfacing later as an authentication failure.

A second defect on the same path is fixed too: IntegrateWithWolverine() never registered the tenant partitioning provider, so PartitionPerTenant() failed outright. Both are covered in With Marten Owning the Message Store.


🌐 HTTP and Event Sourcing

Marten concurrency conflicts as 409

This was a JasperFx client request.

An endpoint using [WriteAggregate] can lose an optimistic concurrency race — two clients posting to the same aggregate at once. Without a handler that escapes as an unhandled 500, even though nothing went wrong: optimistic concurrency did its job. 409 Conflict is the honest status.

6.30 ships a documented, tested recipe (#3764). The OnException middleware convention is all you need — but there are two exception types, and the second is easy to miss:

public static class MartenConcurrencyExceptionMiddleware
{
    // Marten's optimistic concurrency failures -- EventStreamUnexpectedMaxEventIdException from
    // the event store, and document level concurrency violations -- all derive from
    // JasperFx.ConcurrencyException, so one handler covers them
    public static ProblemDetails OnException(ConcurrencyException ex)
    {
        return new ProblemDetails
        {
            Status = 409,
            Title = "Conflict",
            Detail = ex.Message
        };
    }

    // StreamLockedException does NOT derive from ConcurrencyException -- it is a MartenException --
    // so the FetchForExclusiveWriting path needs its own handler. Catching only ConcurrencyException
    // silently misses it
    public static ProblemDetails OnException(StreamLockedException ex)
    {
        return new ProblemDetails
        {
            Status = 409,
            Title = "Conflict",
            Detail = ex.Message
        };
    }
}

That second handler is the whole reason this is a documented recipe rather than a one-liner in a FAQ. Marten.Exceptions.StreamLockedException — what FetchForExclusiveWriting throws on a contended stream — derives from MartenExceptionnot ConcurrencyException. A recipe that catches only the latter silently leaves the exclusive locking path returning 500s. See Recipe: Marten Concurrency Conflicts as 409.

[StreamState] and [StreamEvents]

New parameter attributes for handlers whose read is the raw stream rather than the folded aggregate — timeline views, audit endpoints, anything [ReadModel] cannot express (#3627).

They are store-agnostic across Marten, Polecat and Fisher, and on Marten both fetches are batched into a single round trip.

Event Model slices per route

HttpChainDescriptor and GrpcRpcDescriptor now carry the slice the route is, so a consumer walking endpoint by endpoint sees it next to the route rather than only through the assembled model (#4000).


🐛 Transport Fixes

The AI tools are sometimes good about finding combinatorial or lurking bugs while doing other work. It’s annoying, but I always ask Claude to file issues for any unrelated problems it finds while doing any work — then immediately turn around and try to address them.

A good crop this time, and several of them share a theme worth naming: silent failure. Each of these was doing the wrong thing without reporting anything.

Pulsar

  • Requeue, scheduled retry and dead-letter routing were simply unimplemented (#3797).
  • A global native-resiliency failure rule was swallowing every user-configured error policy in the entire application (#4079, also reported as #4075). A plain local queue in a UsePulsar() host got exactly one attempt. If you use Pulsar and have ever wondered why an OnException policy seemed inert, this is why.
  • Hot-tail listeners silently dropped deferred messages in every mode (#4060).

GCP Pub/Sub

Redis

  • DeleteStreamEntryOnAck silently never acked on Redis < 8.2, where XACKDEL is unsupported (#4058).

Ack reliability

  • A shared ack-attempt budget across stacked retry blocks, plus terminal-failure classification for Azure Service Bus and SQS so a permanent settle failure stops rather than burning the whole budget (#4012 — partially delivered; the remaining items are tracked there for 6.31).

⬆️ Upgrading

This release is additive. EndpointMode.NativeAck is opt-in per endpoint and default-closed per transport, and MaximumBrokerRedeliveries defaults to off.

Requires JasperFx 2.55.0.

The one thing to look at before upgrading is #3712, described above — the new listener coherence validation will reject at startup a configuration it previously accepted and ignored. That is a change in behavior, but the configuration it rejects was never doing what it looked like it was doing.


What’s Next

NativeAck is probably overdue and another option for Wolverine usage. We’ll be releasing a new version of our curated AI Skills this week that builds in decision making about endpoint usage in Wolverine applications.

Find us in the Critter Stack Discord or open an issue on GitHub.

The JasperFx / CritterStack AI and Event Modeling Strategy

Assuming you haven’t been living under a rock, you’ve probably noticed that topics related to Artificial Intelligence (AI) are pretty well dominating most conversations about software development right now.

I’d like to lay out the Critter Stack community and JasperFx‘s strategy is for AI assisted software development at this moment and talk about all the different spaghetti we’re trying to throw up against the wall so that the Critter Stack and JasperFx can continue to thrive in our new world order.

Vertical Slice Architecture

Switching to the Critter Stack’s impact on application code, Wolverine has been built with what we now call “Vertical Slice Architecture” (VSA) in mind from the very beginning. That emphasis on VSA code organization has turned out to have been accidentally prescient as Wolverine’s very terse approach to VSA is already very optimized for AI agentic development — especially when contrasted with more traditional server side .NET code organization that leans into layered architectures that force AI agents to burn more tokens traversing the code.

Here’s a tutorial on Vertical Slice Architecture with Wolverine.

The takeaway here is that the intended idiomatic usage of Wolverine in our recommendations is already well suited for AI assisted development. The available command line tools that we bake right into your application using the Critter Stack tools can help a lot too. However, there is the very real issue of helping users utilize the AI-friendly, terse, VSA style of code when folks are coming from Clean Architecture approaches or other styles that call for more projects, layers, and abstractions that we the Critter Stack community or your AI agent ideally wants to deal with. Likewise, the usage of all those small diagnostic tools aren’t terribly obvious for troubleshooting if you don’t know they’re there or aren’t a Linux guru who’s used to banging out obscure command line calls from memory.

Wolverine and the rest of the Critter Stack is very highly optimized for simple and terse code using the Vertical Slice Architecture approach, and that’s great for AI assisted development!

And it’s incumbent upon people like me to prove that out over time of course.

Event Sourcing is Great for AI!

Or at least that’s something that all of the folks like me that are heavily invested in Event Sourcing tooling are telling you!

Now that CritterWatch 1.0 is live, JasperFx Software has a commercial offering that gives you comprehensive MCP or command line integration with all the common APIs and capabilities of our Marten/Polecat/Fisher event stores.

Command Line Tooling

We’ve long invested in command line diagnostics up and down through the whole Critter Stack just because it’s a mechanically cheap way to do that, but now the AI agent usage has made those tools more valuable than ever.

Our “classic” dotnet run describe is the human friendly tool we’ve long used to dump system descriptions out for our human users and one of the first things I ask users to try to unwind many common issues with Marten or Wolverine usage, and that’s not going anywhere. We’ve also invested quite a bit recently in additional CLI tools in Wolverine 9.0 especially that have textual output optimized for AI agents to explain message routing, preview the generated source code around a message handler, troubleshoot why a message handler isn’t being found, and other common user questions. I expect we will continue to expand this support based on any trends we see with user confusion.

We’re also doubling down even farther on the CLI path with the shortly forthcoming CritterWatch tooling that will provide CLI access to every bit of of an application’s “Critter Stack” configuration, the dead letter queue storage, domain centric queries against Open Telemetry tracking tools you might be using such as Jaeger, and queries against any Marten or Polecat event storage including a new “projection step through” function to trouble shoot projections.

I recently wrote about how we improved and expanded our CLI tooling in the “Critter Stack 2026” wave or releases.

CritterWatch

CritterWatch and its exposed MCP server will give you access to messaging and event workflow both from a static configuration standpoint and runtime discovered view as well. Every single bit of information exposed through CritterWatch and every single action is also completely exposed through its MCP endpoints, including access to the event stores of your Critter Stack application.

CritterWatch also comes with command line tooling as well that gives AI agents access to cross-application information about systems, workflows, messaging, and event stores.

AI Skills

We’ve built, and will continue to curate and evolve a set of AI Skills files for the Critter Stack tools including Marten, Polecat, Wolverine, and even Alba to help users use our tools more effectively. For Wolverine, the AI Skills will help you write code with Wolverine idioms that lead to much tighter code — or they’ll let you write the code in the conceptual way you’re used to, then suggest how to adjust that code to a tighter or more performance Wolverine idiom. All of which should help you be more effective over time with AI assisted development.

The AI Skills also teach your AI agent how to utilize the built in CLI

You can purchase access to these AI Skills here on the JasperFx website. All of our support plans also include access to these AI Skills as well as the forthcoming CritterWatch packages that are hopefully available by the time you read this.

Spec Driven Development and Event Modeling with Bobcat

We’re working a little bit on our solution for “Spec Driven Development” with a tool (so far) called Bobcat. I’m not sure what exactly will land in Bobcat or be spun out, but so far it’s the spiritual successor to my old Storyteller project for effective automated integration testing.

As part of that, Bobcat has a Gherkin capability for expressing executable specifications (we’re not taking a direct dependency on Reqnroll yet, but I’m personally undecided about whether we do that in the long run). We’re mostly going down the Gherkin path just because of the existing support inside of VS Code and JetBrains Rider for editing Gherkin documents. We’ll definitely look at other models for expressing tests too (simply marking up C#/F# test code for visualization? A markdown input? Something completely different?).

Bobcat also has a “Supervisor” capability that uses the new Microsoft Testing Platform (MTP) to drive xUnit.Net (or any testing tool that supports MTP and dotnet test) test suites with a bit more supervised parallelism and selective test retries. The “Supervisor” is also built to “know” when and how to recycle docker containers or restart test processes based on observed test failures. That’s been a god send for me doing my own CritterWatch development where the automated test suites are huge and test health can degrade as docker containers or processes have been up too long. We’re also building out a little we application you can use to see ongoing test results inside of the very long running test suites — again, for my own sanity when an AI agent is running tests and you can’t always tell if it’s active.

This is still pretty mushy as far as details, but we’re making the test output of Bobcat specifications write out quite a bit of detail about what Wolverine messages occurred, events that were appended, and HTTP calls received during the test — including the timings to help optimize test runs over time. I’m hopeful that this will be beneficial as well for AI assisted development by trying to make the tests themselves help diagnose test failures for quicker correction.

Lastly — and for the moment — we’re landing an Event Modeling capability into the core of the Critter Stack, but making the future Bobcat user interface be the visualization of the models. The concept that I’m proposing so far is:

  • A new model and fluent interface API in our low level JasperFx.Events library that you can use to declare event types, read model types, and any other common elements of Event Storming or Event Modeling
  • A new user interface in Bobcat that can visualize the Event Modeling slices defined by that model
  • Integrate dotnet watch with that so you can interactively doodle with slice definitions and see the model change
  • Have the specified model overridden when real code is built in the system, which is making us introducing “Event Modeling” concepts directly into Wolverine and our Marten/Polecat/Fisher event stores
  • Integrate the Bobcat specifications directly into the Event Modeling visualization. Rather than have people waste time writing intermediate models for policies, constraints, or validation rules in a diagram, go straight to Behavior Driven Development specifications that become actionable specs
  • Add more command line options to export the event slice definitions for AI agent usage
  • Expand out AI Skills so an AI agent knows how to use Bobcat specifications and our models to build systems
  • Probably invest in extending our existing code generation support to build out the shell of a “slice” from the model as a first step

Philosophically, I’m coming from a background in code-and-TDD/BDD-centric Extreme Programming and I’ve long been very dubious about the efficacy of “low code” visual modeling approaches or application generators like JHipster. I’m also not enthusiastic about any of the intermediate DSL approaches I’m seeing for modeling event driven architectures using YAML, XML, or custom built textual DSLs. Because of the Critter Stack’s relentless focus on low ceremony code, I think that we’re better off just adding the visualization capabilities on top of the code rather than trying for a hugely time consuming user interface effort.

Obviously

LLM Callouts?

I don’t have any details yet, but JasperFx is going to work with at least one client to build in integrations to LLMs from Wolverine and Marten/Polecat/Fisher projections using the Microsoft.Extensions.AI abstractions.

I think that we will 100% add LLM integrations into CritterWatch in the near future.

Agent Orchestration

This might end up being no more than a “build your own lightsaber” project for me, but JasperFx is building out its own take on an agent orchestrator and durable agent memory on top of the Critter Stack. And conveniently enough, we now have Fisher for Sqlite backed event sourcing that will be very convenient for this effort.

To be honest, I’ve been impressed with KurrentDb’s Capacitor tool, and that’s low hanging fruit for a Critter Stack-backed option.

It’s somewhat likely that this tooling would either end up landing in CritterWatch or at least share the same license model as CritterWatch, but it’s going to be a commercial tool all the way.

Summary

As the imagery at the top was trying to suggest, in terms of our AI strategy, we’re trying to throw all the spaghetti up against the wall right now and see what ends up sticking. And that’s the best that I’ve got for now.

CritterWatch 1.0 is live!

CritterWatch 1.0 dropped yesterday, and you can read the official release on the JasperFx Software site.

This blog post is just me being thankful for all the folks who helped build CritterWatch or test it along the way:

  • Babu Annamalai was instrumental in CritterWatch, our company infrastructure, and the Critter Stack in general
  • Jeffry Gonzalez helped lay the foundation
  • Anne Erdstrieck made CritterWatch (and Marten/Wolverine) a lot better by testing early versions on a very complex and high volume system
  • Ben Virkler gave us a lot of feedback on usability and early problems (and we owe Ben some additional improvements for 1.1 that didn’t make the 1.0 wire)
  • Laurence Gillian was an early tester
  • Several other folks in Discord, and all of that feedback was helpful

And to Oskar Dudycz for all his contributions across the Critter Stack as much of CritterWatch are things that he and I were talking about years ago.

For my part, I founded JasperFx Software on the idea that we’d go down the “Open Core” model with business offerings of consulting, formal support agreements, and training around the Critter Stack tools, then supplement that with commercial add on tools for very advanced features, monitoring, and management. It took much longer than I’d hoped, but a couple months into JasperFx’s fourth year in business, we’re finally realizing the original vision.

And with all this out of the way, it’s time for me to get just a little break then start on CritterWatch 1.1!

CritterWatch RC.10 — the final one!

CritterWatch will have its 1.0 release tomorrow (Wednesday, August 19th) just in time for a live stream on YouTube to show just the user interface part of CritterWatch. Today though, we finally got one last RC.10 release out for some much delayed feedback.

We made a large amount of changes to optimize performance based on early customer feedback as we jumped right into the deep end of the pool and started by integrating CritterWatch into literally the single biggest Critter Stack system that we’re aware of.

This release candidate basically got us to what I expect the final product to be for 1.0, minus some user interface feedback and polishing at the last minute.

CritterWatch will require you to be running basically the latest of everything:

Stack: Wolverine 6.29.0 · JasperFx 2.52.0 · Marten 9.28.0 · Polecat 5.19.0 · Fisher 0.9.2 · Weasel 9.24.0

Three stores, one console

CritterWatch runs on Marten/PostgreSQLPolecat/SQL Server, and — new in this release —
Fisher/SQLite:

packagestoredatabase
CritterWatchMartenPostgreSQL
CritterWatch.SqlServerPolecatSQL Server
CritterWatch.SqliteFisherSQLite — a file, no server

The SQLite flavour needs no database service at all, which makes it the cheapest way to try the
console or to run it beside a small deployment.

Underneath, CritterWatch.Services is the store-agnostic core, compiled once and referenced by all
three.

Using the Wolverine “Side Effect” Model to Simplify Code

First off, let me peel some egg off my face because I had allowed Claude to write quite a bit of code without close enough examination until just now. Arguably, we’re all good because we do have test coverage for the code I just refactored, so it’s all good in the end, but maybe just remember that a human in the loop is a good idea. And also know that all CritterWatch code will be closely reviewed before we flip that to 1.0!

Here’s an HTTP endpoint from CritterWatch, our forthcoming monitoring console for the Critter Stack. It adds a tenant to a monitored service:

[WolverinePost("/api/critterwatch/tenants/{serviceName}/add")]
[Middleware(typeof(RequireMultiTenancyLicense))]
public static async Task AddTenant(
string serviceName,
AddTenantRequest request,
IDocumentSession session,
IMessageBus bus,
[FromServices] AuditLogService auditLog,
HttpContext httpContext)
{
// ... append an event, publish a command ...
await auditLog.LogAsync("AddTenant", serviceName, null,
$"Added tenant '{request.TenantId}' to {serviceName}",
new Dictionary<string, string> { ["tenantId"] = request.TenantId },
initiatedBy: AuditActor.From(httpContext.User));
}

Six parameters. Two of them are there for the audit log alone, and one of those — HttpContext — exists for a single expression: httpContext.User. We don’t read the request, the headers, the response, or anything else on it. We take the entire ASP.NET Core request context as a dependency to get at one ClaimsPrincipal.

That has a cost you feel in the test project. To test this method you need a store, a bus, an audit service and a request context. The audit behaviour — did we record the right action, against the right service, attributed to the right operator? — is only observable by standing all of that up and then querying the audit table afterwards.

Returning the intent instead of performing it

Wolverine has an interface called ISideEffect. It’s about as small as an interface gets:

public interface ISideEffect : IWolverineReturnType, INotToBeRouted;

There’s no method on it. The contract is a convention: return one of these from a handler or HTTP endpoint, and Wolverine will call any public Execute() or ExecuteAsync() method on it after your method returns. The interesting part is what happens to that method’s parameters — Wolverine registers each one as a dependency of the chain and resolves it for you.

So the audit log becomes a record:

public record AuditLog(
string Action,
string ServiceName,
string? TargetUri = null,
string? Details = null,
Dictionary<string, string>? Parameters = null,
string? InitiatedBy = null) : ISideEffect
{
public Task ExecuteAsync(AuditLogService auditLog, ClaimsPrincipal? user)
{
var actor = string.IsNullOrWhiteSpace(InitiatedBy) ? AuditActor.From(user) : InitiatedBy;
return auditLog.LogAsync(Action, ServiceName, TargetUri, Details, Parameters, actor);
}
}

And the endpoint stops mentioning either dependency:

[WolverinePost("/test/audited/{serviceName}")]
public static AuditLog Post(string serviceName)
=> new("TestAction", serviceName, Details: "smoke");

AuditLogService and ClaimsPrincipal are resolved onto the chain because ExecuteAsync asks for them. The endpoint declares neither. The HttpContext parameter didn’t move somewhere else — it stopped existing. The principal is resolved at execution time rather than threaded through a signature that never wanted it.

Pure Functions FTW!

If you want to peruse our published Wolverine Best Practices, we strongly recommend trying to make the behavioral methods of your message handlers or HTTP endpoints be “pure functions” whenever possible. We also recommend trying to simplify code by opting to remove asynchronous code from your handlers as well as a way to reducing noise code, and that was why I reached quickly for the new AuditLog side effect. Combine side effects with other Wolverine goodies like our cascading messages syntax for publishing messages and the aggregate handler workflow and you get the tools to really simplify any application code related to business logic.

And just to make this clear, using pure functions (when possible) is a great approach because that:

  • Isolates business or workflow logic from infrastructure concerns
  • Promotes testability through fast running unit tests
  • Reduces the code noise from asynchronous invocations

When not to reach for this

ISideEffect is for work you want to describe and let the framework perform. It’s a poor fit when the result of the work feeds the rest of your method — if you need the return value, you need the call, and a side effect only runs after you’ve returned.

It’s also not free indirection. A one-line await that nothing else depends on and nobody wants to test in isolation is fine as it is. What made the audit log worth converting was the ratio: two parameters and a request context, carried by five endpoints, to express one fact about an operation that had already happened.

That ratio is the tell. When a dependency exists only to record that something happened — audit, notification, telemetry, an outbound email — you are almost always better off returning a description of it and letting Wolverine make the call.

Summary

Hoo boy, let’s try to summarize this a bit:

  1. I think having a man in the loop for AI built code is still pretty important
  2. Pure functions are great for testability
  3. Code noise has a real cost for your ability to reason about code and how it works — and I think that is still true even with LLMs doing so much more of the grunt work now
  4. I think that our JasperFx curated AI Skills are valuable for developing with Wolverine and the rest of the Critter Stack because it’ll keep your LLM using more idiomatic features that can drastically shrink your code compared to typical .NET codebases, improve testability, and opt into Wolverine or Marten features that enhance performance. You can learn more about our AI Skills here.

Introducing Fisher: Sqlite Backed Document Db & Event Store Critter

I know, you were probably wandering around today and thinking to yourself, my life would be more complete if there was just a library out there that gave you the developer experience of the tried and true Marten library, but backed by Sqlite so you could just get things done on projects that don’t really need a database server.

To that end, let me introduce Fisher, our latest Critter Stack library that is officially our SQLite-backed Event Store and Document Database inside the Critter Stack. I pushed the first Nuget version today as 0.5.0 if you want to pull it down and play with an early version.

Fisher is a document database and event store for .NET, in the same family as Marten and Polecat — except that it runs on SQLite, which means it runs inside your process, and there is no database server anywhere in the picture.

dotnet add package Fisher
builder.Services.AddFisher(opts =>
{
opts.Connection("Data Source=app.db");
});

That’s the whole setup. No container, no connection to a host, no credentials, no waiting for a health check before your integration tests can run. Just go.

Why bother?

The Critter Stack already has two of these. Marten has been running on PostgreSQL for over a decade, and Polecat brought the same model to SQL Server 2025 earlier this year. So why a third?

Because I thought this would be a valuable persistence option for our commercial CritterWatch tool to help adoption, and also as a persistence option for an “AI-related commercial development tool to be named later” from JasperFx.

Because a meaningful number of .NET applications don’t want a database server, and up to now the answer from us was “well, use one anyway.” Think about:

  • Desktop and CLI applications
  • Edge and on-premises deployments where somebody else operates the box
  • Single-node services that will never scale out and shouldn’t pretend they might
  • Embedded reporting

It’s the same API

This is not a new library with a familiar accent. It implements the same JasperFx.Events abstractions the other two do, so a projection you wrote for Marten runs on Fisher unaltered:

// Documents
session.Store(new User { FirstName = "Jane", LastName = "Doe" });
await session.SaveChangesAsync();
var users = await session.Query<User>()
.Where(x => x.LastName == "Doe")
.ToListAsync();
// Eventsvar stream = session.Events.StartStream<Order>(new OrderPlaced("Acme", 199.95m));
await session.SaveChangesAsync();
var order = await session.Events.AggregateStreamAsync<Order>(stream.Id);

Fisher passes all 32 suites and 272 tests of JasperFx.Events.ComplianceTests, the shared cross-store suite Marten and Polecat also enroll in. That’s not me grading my own homework — it’s the same definition of correct that the other two are held to.

What’s in the box for 0.5.0: documents over all four identity types plus strong-typed wrappers, hierarchies, soft deletes, optimistic concurrency in both flavors, patching, bulk insert, duplicated fields, indexes, foreign keys, and a LINQ provider that does joins, grouping, aggregates and both paging styles. On the event side: every projection shape across every lifecycle, the async projection daemon, subscriptions, DCB tags, natural keys, event data masking, stream compacting, and both tenancy styles. Plus Fisher.AspNetCore and Fisher.EntityFrameworkCore.

Why “Fisher?”

This is a Fisher (sometimes called a “Fisher Cat”), yet another member of the Mustilidae family and essentially a “big marten”:

Some important details along the way…

Like I said earlier, Marten has been around for over a decade now and it’s been the most successful OSS project of my career (StructureMap has more downloads, but who cares, there’s a bazillion perfectly decent IoC containers out there). We added Polecat earlier this year to finally extend our event sourcing support to SQL Server using Marten’s API and usage as a pattern. Supporting Sqlite seemed like the obvious next step to have a true embedded database option for some of JasperFx’s work — plus Babu has been advocating for that for awhile!

Along the way as you might expect, we’ve made some intermediate steps to make this new multiple database engine support possible and hopefully sustainable over the long run:

  1. As part of Marten 8.0 last year, a great deal of the abstractions, projections support, and plenty of non-PostgreSQL dependent code for event sourcing in Marten was pulled out into a now shared JasperFx.Events library
  2. I purchased a Claude Max plan for JasperFx, just to be honest here
  3. Polecat 1/2/3 was built against JasperFx.Events in essentially a “just copy Marten” AI prompt
  4. As part of Marten 9.0, we pulled as much common code between Marten and Polecat into lower level, shared Weasel libraries
  5. For Polecat 5.0, we lifted a new shared Weasel.Storage library out of Marten, then shared that dependency with Polecat to standardize a lot more of the internal mechanics of the two libraries and eliminate some Polecat specific code. My hope was and is that that effort will make it easier for us to address problems or even enhancements in a generic way
  6. Recently, we also lifted quite a few automated tests in Marten as a new “event sourcing and document database” compliance test suite that we now share between Marten, Polecat, and Fisher. That effort flushed out some inconsistencies and a few bugs in Polecat, now fixed.
  7. Fisher was mostly built to the new compliance test suites

Again, just to be honest, I don’t think that either Polecat or Fisher would have been economically feasible without the heavy utilization of the AI assisted development. And also, as always, I think the AI assisted development goes a lot better when you can supply very clear acceptance criteria like the compliance tests.

TimescaleDB Support within Marten

This is a recent addition to Marten. We’ve started supporting many of the common PostgreSQL extensions that hare frequently supported by the major cloud providers. Outside of metrics collection or sensor data, I don’t have a great handle on what folks might use this for, so I’d love to hear from other folks what they’d want to do with TimescaleDB.

TimescaleDB support lets Marten turn its tables into TimescaleDB hypertables — automatically time-partitioned tables with columnar compression, retention policies, and continuous aggregates. It ships in the core Marten package under the MIT license, scoped behind its own Marten.TimescaleDB namespace, and is entirely opt-in at runtime via UseTimescaleDB() — stores that never call it pay nothing.

What it gives you today:

  • a one-line UseTimescaleDB() opt-in that registers the timescaledb extension on every database Marten manages
  • ProjectionAsHypertable<T>() to turn a time-bucketed flat table projection into a hypertable, with configurable chunk interval, compression, retention, and continuous aggregates
  • DocumentAsHypertable<T>() to turn an append-heavy document table (audit logs, metrics, activity records) into a hypertable partitioned by one of its own timestamp members
  • full participation in Marten’s schema migration model — the hypertable, its policies, and its continuous aggregates are created idempotently through the normal ApplyAllConfiguredChangesToDatabaseAsync path, and do not show up as drift on subsequent migrations

Requirements

The feature ships in core Marten, so there is no separate package to install — reach it with using Marten.TimescaleDB; and enable it with UseTimescaleDB().

TimescaleDB is a loadable module: unlike PostGIS or pgvector it must be listed in shared_preload_libraries before CREATE EXTENSION timescaledb will succeed. The official timescale/timescaledb images already do this. This repo ships docker-compose.timescaledb.yml (which runs on port 5433 so it can coexist with the main dev database) for local development, and a dedicated CI workflow using the timescale/timescaledb-ha image.

Enabling TimescaleDB on a store

using Marten;
using Marten.TimescaleDB;
var store = DocumentStore.For(opts =>
{
opts.Connection(connectionString);
// Registers CREATE EXTENSION IF NOT EXISTS timescaledb on every database
opts.UseTimescaleDB();
});

Flat table projections as hypertables

The cleanest, highest-value fit is a flat table projection that rolls up events into a time-bucketed table — per-minute/per-hour metrics, IoT rollups, activity counters, and the like. Because the projection’s table is written by the async daemon (or inline), TimescaleDB then gives you time-chunked storage, columnar compression of old chunks, continuous aggregates for dashboards, and automatic retention — all declaratively.

opts.Projections.Add(new MetricsProjection(), ProjectionLifecycle.Async);
opts.UseTimescaleDB(ts =>
{
ts.ProjectionAsHypertable<MetricsProjection>("captured_at", hyper =>
{
hyper.ChunkInterval = TimeSpan.FromHours(1);
hyper.CompressAfter = TimeSpan.FromDays(30);
hyper.RetainFor = TimeSpan.FromDays(365);
hyper.ContinuousAggregate("hourly_metrics", "1 hour",
"avg(value) as avg_val, max(value) as max_val");
});
});
public class MetricsProjection: FlatTableProjection
{
public MetricsProjection(): base("sensor_metrics", SchemaNameSource.EventSchema)
{
// The single primary key IS the time column — see the constraint below.
Table.AddColumn<DateTimeOffset>("captured_at").AsPrimaryKey();
Table.AddColumn<double>("value").NotNull();
Project<SensorReadingRecorded>(map =>
{
map.Map(x => x.Value, "value");
}, tablePrimaryKeySource: x => x.CapturedAt);
}
}

Configuration options

PropertyMaps toNotes
ChunkIntervalcreate_hypertable(..., chunk_time_interval => ...)Width of each time chunk. Defaults to TimescaleDB’s own default (7 days).
CompressAfterALTER TABLE ... SET (timescaledb.compress ...) + add_compression_policy(...)Enables columnar compression of chunks older than this age.
CompressSegmentBy / CompressOrderBycompression settingsOptional segment-by / order-by keys. Order-by defaults to the time column DESC.
RetainForadd_retention_policy(...)Drops chunks older than this age.
ContinuousAggregate(view, bucket, select, groupBy?)CREATE MATERIALIZED VIEW ... WITH (timescaledb.continuous)A self-refreshing rollup view. Marten creates the view WITH NO DATA; set the refresh policy (add_continuous_aggregate_policy) with your own operational tooling.

Compression / retention policies are applied on creation only

The compression and retention settings (CompressAfterRetainForCompressSegmentBy/CompressOrderBy) are emitted when the hypertable is first created. Later changes to those values are not diffed and re-applied on subsequent migrations — adjust an existing policy with TimescaleDB’s own add_/remove_compression_policy / add_/remove_retention_policy functions (or drop and recreate the hypertable). The hypertable, its policies, and its continuous aggregates are otherwise created idempotently and do not show up as drift.

The partition column must be the projection’s primary key

TimescaleDB requires the partitioning column to participate in every unique/primary key on a hypertable. A FlatTableProjection always has exactly one primary-key column and upserts ON CONFLICT against it, so the only shape that maps cleanly onto a hypertable is one where that single primary-key column is the time column (a time-bucketed rollup). If you configure ProjectionAsHypertable against a projection whose primary key is something else (e.g. the stream id), Marten fails fast at schema-application time with a descriptive error rather than letting TimescaleDB reject the create_hypertable call.

Document tables as hypertables

Append-heavy document types — audit logs, metrics, activity records — can be stored in a hypertable partitioned by one of their own timestamp members:

opts.UseTimescaleDB(ts =>
{
ts.DocumentAsHypertable<AuditEntry>(x => x.CreatedAt, hyper =>
{
hyper.ChunkInterval = TimeSpan.FromDays(1);
hyper.CompressAfter = TimeSpan.FromDays(30);
hyper.RetainFor = TimeSpan.FromDays(365);
});
});

Because TimescaleDB requires the partition column to be part of the primary key, DocumentAsHypertableduplicates the selected member into a NOT NULL column and adds it to the document table’s primary key, making it (id, created_at). Marten’s own schema model is updated to match, so there is no schema drift, and the generated upsert / update / delete SQL picks the composite key up automatically (the same machinery that backs list- and range-partitioned document tables).

The partition member must be immutable

Because the timestamp is now part of the primary key, it must not change for a given document id. Marten’s update path matches on the full primary key, so mutating the timestamp after the first Store would fail to find the existing row. DocumentAsHypertable is intended for append-heavy types whose timestamp is set once on creation and never modified. Loading and deleting by id still work (there is exactly one row per id), though a load by id alone cannot use chunk exclusion and will scan all chunks — query by the time column, or by id plus a time range, for time-partitioned performance.

Multi-tenancy

Hypertables work with Marten’s conjoined (single-table) tenancy — the tenant column is just another column on the chunked table. For database-per-tenant, each tenant database needs the timescaledb extension; UseTimescaleDB() registers it on every database Marten manages, so this is handled for you.

Declarative Testing Helper for Marten or Polecat Projections

We’ve had an undocumented until now API in Marten for years called EventProjectionScenario for declarative testing of Marten projections — somewhat based on the Scenario usage in our Alba library for ASP.Net Core testing. As part of some cleanup this week, I finally added some documentation and lifted that to where Polecat (Event Sourcing with SQL Server) can use it as well.

I’m more curious than anything to get some feedback here if anyone things this would be useful. After CritterWatch 1.0 lands, my attention is going to turn to the Critter Stack’s story for “Spec Driven Development,” and maybe this feature will be part of that.

Scripted Scenarios with EventProjectionScenario

For a more declarative way to test a projection end to end, Marten has a built-in scenario runner on IDocumentStore.Advanced that scripts a sequence of event appends and document assertions, then executes the whole sequence for you:

[Fact]
public async Task happy_path_test_with_inline_projection()
{
// This is from a shared testing context class we use
// to test Marten itself. This is just a short cut to say
// if I have a DocumentStore configured like this...
StoreOptions(opts =>
{
opts.Projections.Add(new UserProjection(), ProjectionLifecycle.Inline);
});
await theStore.Advanced.EventProjectionScenario(scenario =>
{
var id1 = Guid.NewGuid();
var id2 = Guid.NewGuid();
var id3 = Guid.NewGuid();
scenario.Append(Guid.NewGuid(), new CreateUser {UserId = id1, UserName = "Kareem"});
scenario.Append(Guid.NewGuid(), new CreateUser {UserId = id2, UserName = "Magic"});
scenario.Append(Guid.NewGuid(), new CreateUser {UserId = id3, UserName = "James"});
scenario.DocumentShouldExist<User>(id1);
scenario.DocumentShouldExist<User>(id2);
// In this usage you can make assertions against the
// expected state of the projected document
scenario.DocumentShouldExist<User>(id3, user => user.UserName.ShouldBe("James"));
scenario.Append(Guid.NewGuid(), new DeleteUser {UserId = id2});
scenario.DocumentShouldExist<User>(id1);
scenario.DocumentShouldNotExist<User>(id2);
scenario.DocumentShouldExist<User>(id3);
}, TestContext.Current.CancellationToken);
}

The scenario works with any projection lifecycle. If the store has any asynchronous projections registered, the scenario quietly spins up a projection daemon, waits for it to catch up after each batch of appended events, and shuts it down afterward — your test code looks identical either way.

A few things to know about how a scenario executes:

  • The Append() / StartStream() / AppendEvents() calls queue work — nothing touches the database until the scenario executes. The StartStream() overloads that generate their own stream id return that Guid so you can capture it for later assertions.
  • Consecutive appends are batched into a single commit; the pending work is saved whenever the next step is an assertion, and once more at the end of the scenario.
  • Assertion steps run against a query session after the projected data is up to date. DocumentShouldExist<T>() and DocumentShouldNotExist<T>() cover the common cases, and AssertAgainstProjectedData() is the general purpose hook for anything else.
  • If an action step fails, the scenario stops immediately — the remaining steps would be running against a state nobody intended. Failed assertions accumulate instead, and everything is reported at the end in a single ProjectionScenarioException that lists each step and what went wrong. Assertion failures inside the aggregate are typed as ProjectionScenarioAssertionException so tooling can tell them apart from infrastructure failures.

WARNING

By default the scenario deletes all event data plus the storage for every registered projection before it runs, so each scenario starts from a clean slate. Only use this feature against a test database! To run a scenario on top of existing data instead, set scenario.DeleteExistingData = false.

The scenario object exposes a few knobs:

await theStore.Advanced.EventProjectionScenario(scenario =>
{
// Keep any existing event/projection data (the default is to wipe it)
scenario.DeleteExistingData = false;
// Apply the whole scenario to one tenant when using multi-tenancy
scenario.TenantId = "tenant1";
// Maximum time to wait for async projections to catch up
// after each batch of events. The default is 30 seconds
scenario.Timeout = 5.Seconds();
// ... queue up appends and assertions
});

We’ll also have the “Projection Stepper” feature in CritterWatch that will allow you to step through a series of events to see how a projection creates and modifies its view event by event. That functionality is part of the CritterWatch user interface, but also exposed via an MCP endpoint on CritterWatch for easy access for AI agents building and troubleshooting Critter Stack applications using Event Sourcing.

Binary Event Serialization for Marten

This is a potentially big performance optimization you can opt into starting with Marten 9.0. Not coincidentally, we’re using this for CritterWatch to help optimize the responsiveness and database size for a JasperFx client this week.

Marten can serialize individual event types to a binary wire format (MemoryPackMessagePack, or anything else implementing IEventBinarySerializer) instead of the default JSON, trading a few of JSON’s ergonomic wins for a meaningful throughput and storage-size improvement on hot streams. See #4515 for the design discussion.

The opt-in is per event type — binary-serialized and JSON-serialized events coexist in the same mt_events table, so the feature can be rolled out on an existing store with no migration of existing data.

How it works

A second column, bdata bytea NULL, sits alongside the existing data jsonb NOT NULL on mt_events. The row-level discriminator is bdata IS NULL:

Whendatabdata
Event uses the JSON serializerfull JSON payloadNULL
Event uses an IEventBinarySerializerthe placeholder '{}'::jsonbthe serialized bytes

On read, Marten inspects bdata:

  • NULL → existing JSON deserialization path. Pre-feature rows continue to work without conversion.
  • non-null → IEventBinarySerializer.Deserialize(eventType, bytes).

Because the discriminator is on the row and the serializer is resolved per event type, the same stream can carry rows of either format with no special handling at the call site.

Quick start with Marten.MemoryPack

The companion Marten.MemoryPack NuGet package ships a ready-to-use IEventBinarySerializer over MemoryPack:

dotnet add package Marten.MemoryPack

Mark event types you want to serialize as binary with both [BinaryEvent] (so Marten picks them up) and [MemoryPackable] (so MemoryPack can serialize them):

using Marten.Events;
using MemoryPack;
[BinaryEvent]
[MemoryPackable]
public partial record TripStarted(Guid TripId, string DriverName, DateTimeOffset StartedAt);

Wire MemoryPack as the store-wide fallback for [BinaryEvent] types:

using Marten.MemoryPack;
var store = DocumentStore.For(opts =>
{
opts.Connection(connectionString);
// Wire MemoryPack as DefaultBinarySerializer. [BinaryEvent]-marked
// event types resolve to this serializer on registration. Works with
// every EventAppendMode (Rich / Quick / QuickWithServerTimestamps)
// and with BulkEventAppender — see the "Append modes" section.
opts.Events.UseMemoryPackSerializer();
});

Now TripStarted writes through MemoryPack to bdata; un-marked events continue to write JSON to data.

Registration ergonomics

Two equivalent ways to opt an event type in:

// 1. Attribute-driven — uses opts.Events.DefaultBinarySerializer as the resolver.
[BinaryEvent]
[MemoryPackable]
public partial record TripEnded(Guid TripId, DateTimeOffset EndedAt);
// 2. Fluent — wire an explicit per-type serializer (overrides any default).
opts.Events.UseBinarySerializer<TripEnded>(new MemoryPackEventSerializer());

Resolution order on EventMapping construction:

  1. Explicit opts.Events.UseBinarySerializer<TEvent>(...) for that type.
  2. [BinaryEvent] attribute + opts.Events.DefaultBinarySerializer.
  3. Otherwise, plain JSON (existing path).

If a type carries [BinaryEvent] but no per-type serializer was wired AND DefaultBinarySerializer is null, Marten throws at the first append with a remediation message naming both registration entry points.

Bring your own serializer

IEventBinarySerializer is small enough to implement directly against any binary format — MessagePack, protobuf, etc.:

public interface IEventBinarySerializer
{
byte[] Serialize(Type type, object data);
object Deserialize(Type type, byte[] data);
}

The serializer is a singleton — keep its state thread-safe.

On-disk shape

For binary events, data holds the literal {} placeholder so the existing data jsonb NOT NULL constraint stays intact (no schema relaxation):

-- binary-serialized event
select type, data::text, bdata is null
from mt_events where seq_id = 42;
-- type | data | bdata is null
-- --------------|------|---------------
-- trip_started | {} | false
-- JSON-serialized event in the same stream
select type, data::text, bdata is null
from mt_events where seq_id = 43;
-- type | data | bdata is null
-- --------------------- |---------------------------------|---------------
-- trip_comment_added | {"comment": "looking good", …} | true

Migration

Purely additive: the only schema change is bdata bytea NULL on mt_events. Existing rows have bdata = NULL (the column’s default for prior data) and read through the JSON path. Marten’s standard schema migration creates the column for existing installations — no event data conversion required.

Append modes

Binary event serialization works with every EventAppendMode Marten ships — RichQuick, and QuickWithServerTimestamps. The Quick modes route appends through the mt_quick_append_events PostgreSQL function, which carries a bdatas bytea[] parameter that’s inserted into mt_events.bdata in parallel with the existing bodies jsonb[]BulkEventAppender (the COPY-based bulk loader) also supports binary events — its COPY column list includes bdata, and each event row writes either the binary payload or NULL.

You don’t have to think about the append mode: binary opt-in is per event type and works identically across all of them.

Schema evolution — use versioned event types

Marten’s existing event upcasters operate on the JSON wire form and don’t generalize to a byte[] payload, so they don’t apply to binary events. The recommended pattern for evolving a binary event’s shape is introduce a new event type for each version rather than upcasting in place:

// Original
[BinaryEvent]
[MemoryPackable]
public partial record TripStarted(Guid TripId, string DriverName);
// Schema change — new fields. Don't edit TripStarted; add a new type.
[BinaryEvent]
[MemoryPackable]
public partial record TripStartedV2(Guid TripId, string DriverName, DateTimeOffset StartedAt);

When the projection / aggregate handles both versions explicitly, old streams keep replaying through the old type and new appends use the new type:

public class Trip
{
public Guid Id { get; set; }
public string DriverName { get; set; } = "";
public DateTimeOffset? StartedAt { get; set; }
public void Apply(TripStarted e) { Id = e.TripId; DriverName = e.DriverName; }
public void Apply(TripStartedV2 e) { Id = e.TripId; DriverName = e.DriverName; StartedAt = e.StartedAt; }
}

The coexistence design lets old rows (written as TripStarted) and new rows (written as TripStartedV2) live on the same stream without migration.

Why not in-place backward-compatible schema changes?

You can lean on MemoryPack’s backward-compatible field evolution ([MemoryPackOrder], nullable fields, the VersionTolerant mode) for additive-only changes to a single event type. That works as long as the serializer itself can deserialize old payloads into the new shape — but the moment a change goes beyond the serializer’s tolerance rules (renaming, type changes, splitting a field), there’s no JSON-style upcaster path to fall back on. Versioning the event type works for every shape of change and stays explicit about which version each row was written with.

Mixing binary + JSON

If you have an existing JSON-serialized event and want a future version to go binary, the same pattern applies: define a new [BinaryEvent]-marked type for the new version, leave the old (JSON) type and its upcasters alone, and have the aggregate handle both. The per-row dispatch already copes with mixed formats on the same stream.

See also