Like probably all software tool companies, JasperFx is working very hard to create a compelling story about the utilization of AI-assisted development with our tools. The MCP support — and command line tools too! — in CritterWatch is a major part of our AI strategy.

So far JasperFx Software has mostly been showing off CritterWatch as a user interface tool that you’ll use to peruse and explore what’s happening in your system.

Today, let’s shift to how CritterWatch empowers your AI tools to understand and even administer your running suite of Critter Stack applications through its MCP tools. I should note a few more things before we jump into the sample usages:

1. Everything exposed through the MCP tools in CritterWatch is also available through a command line package
2. The MCP tools are gated by your CritterWatch license
3. CritterWatch’s configurable RBAC support extends to the MCP functions
4. We have invested, and will continue to invest, in making JasperFx’s curated AI Skills know exactly how to take advantage of both the MCP and CLI tools — as well as all the other command line diagnostics already built into the Critter Stack!

Before we get into the real details, just know that everything below is a real recorded session from earlier today: Claude, pointed at the CritterWatch MCP endpoint on my dev machine, driving our full Aspire-orchestrated sample fleet that we use to develop and test CritterWatch itself — 25 monitored services, PostgreSQL and SQL Server event stores, RabbitMQ, Azure Service Bus emulator, and AWS SQS via LocalStack, all chattering away. The trouble in this session was manufactured on purpose with CritterWatch’s built-in chaos monkey tools (which are themselves MCP tools — more on that later), because a demo that waits around for production to genuinely catch fire makes for a long blog post. But every tool call, every JSON response, and every “wait, that’s not what I expected” moment is the real thing, lightly trimmed for length — and the blockquoted Agent replies you’ll see are how Claude actually wrote the raw JSON back up for me, tables and recommendations included. That’s not me grading my own homework — it’s the same MCP surface you’d point your own agent at.

Two minutes of setup

The MCP server rides along in the CritterWatch console host. If you’re already running CritterWatch, you mount it with two lines:

csharp

builder.Services.AddCritterWatchMcp();
// ...
app.MapCritterWatchMcp();  // mounts at /api/mcp

That registers 48 tools — 21 read tools and 27 action tools — over MCP’s streamable HTTP transport, deliberately configured stateless so every tool invocation sees the actual caller’s identity for authorization (a subtle thing that matters a lot once RBAC is in play — see the end of this post). If your host is already composing an MCP server from the per-tool packages, there’s a chaining overload that folds CritterWatch’s tools onto the same endpoint alongside them.

Pointing an agent at it is a config stanza, not a project. For Claude Desktop or Claude Code:

jsonc

{
  "mcpServers": {
    "critterwatch": {
      "url": "http://localhost:5173/api/mcp",
      "transport": "streamableHttp"
    }
  }
}

There’s a quick start for the consumer side that covers MCP Inspector, Claude Desktop, and rolling your own client. That’s the whole setup story. Now the fun part.

“What’s going on in production?”

I would like to point out that CritterWatch’s AI abilities should also be valuable at development time, and we’re investing quite a bit in our AI-assisted development story and expect CritterWatch to be a big part of that.

The session opened the way every on-call shift opens — with the vaguest possible question.

Me: What does my fleet look like right now?

Agent → summarize_cluster_health()

json

{
  "services": { "total": 25, "names": ["HelpDesk", "Inflow", "ItemService",
      "MTTripService", "PolecatTripService", "TripService", "..."] },
  "nodes": { "total": 29, "distinct": 4 },
  "endpointHealthByStatus": { "Active": 697, "Accepting": 352 },
  "brokerHealthByStatus": { "Healthy": 14 }
}

And here’s the part that raw JSON dashboards never give you — the agent turned that into an answer, not a data dump:

Agent: Your fleet looks healthy right now:

Monitored services	25
Wolverine nodes	29 (4 distinct hosts)
Endpoints	1,049 — all Active or Accepting, none stopped, no senders latched
Broker connections	14 / 14 healthy

Nothing looks off at the fleet level. Want me to drill into a specific service, check for active alerts, or look at projection lag anywhere?

Twenty-five services, over a thousand endpoints, fourteen healthy broker connections — read, interpreted, and handed back with the next three sensible questions already offered. Nothing was wrong — yet. So I gave it something to find.

Dead letter triage, the whole loop

The dead letter queue is where I think the MCP story goes from “neat” to “oh, I’d actually use this.” CritterWatch has a first-class Dead Letters page, but the MCP surface closes the full loop for an agent: two read tools to discover and explain what died, and two action tools to do something about it — without a human ever pasting envelope ids around.

For this session I used our internal chaos monkey tool to write 40 realistic dead letters into TripService’s message store — varied message types, varied exception types, and a configurable fraction seeded as genuinely recoverable, so “replay what will succeed” is a real decision rather than a scripted gesture. Then:

Me: What’s failing on TripService and why?

Agent → summarize_dead_letters("TripService")

json

{
  "serviceName": "TripService",
  "totalDeadLetters": 40,
  "groups": [
    { "database": "wolverinedb://postgresql/localhost/postgres/trips",
      "messageType": "ChaosSeededPayment",      "exceptionType": "System.TimeoutException",           "count": 6 },
    { "messageType": "ChaosSeededNotification", "exceptionType": "System.TimeoutException",           "count": 6 },
    { "messageType": "ChaosSeededReconciliation", "exceptionType": "System.InvalidOperationException", "count": 4 },
    "... grouped by message type × exception type, largest first ..."
  ],
  "databasesAnnounced": 1,
  "databasesAnswered": 1,
  "partial": false
}

This is the triage view: counts grouped by message type and exception type, per message database. Here’s how the agent presented it back to me — and notice that it doesn’t just reformat, it diagnoses:

Agent: TripService is carrying 40 dead letters, all in the trips PostgreSQL store. Every announced database answered (1 of 1), so these counts are the complete picture, not a floor.

Message type	Exceptions seen	Count
ChaosSeededPayment	Timeout, Arithmetic, InvalidOperation, Data, External	14
ChaosSeededNotification	Timeout, InvalidOperation, Data, Arithmetic	10
ChaosSeededReconciliation	Timeout, InvalidOperation, Data, External	10
ChaosSeededShipment	Data, Arithmetic, InvalidOperation, External	6

The shape here matters: many exception types spread across many message types reads like transient infrastructure trouble (timeouts, connection resets, contention) rather than a single poison message. Most of these should succeed on replay. Want me to pull the individual envelopes and try?

That “the shape here matters” reasoning isn’t something I prompted for — an agent (or a human) reads the pattern: lots of exception types across lots of message types smells like a dependency being down; one message type with one exception smells like a poison message. And do notice databasesAnnounced versus databasesAnswered — the reads fan out across every physical message database the service owns and tell you explicitly whether everyone reported. An empty result where only three of eight stores answered is not an empty queue, and the tool refuses to let an agent confuse the two. That distinction exists because of a real production failure mode where a console rendered “no dead letters found” over a queue quietly holding 42 of them.

From there the agent pulled individual envelopes — with ids, exception messages, and source URIs. Here’s one of them, verbatim:

json

{
  "id": "01a059a6-d30a-4906-ad13-5232ba7f466c",
  "database": "wolverinedb://postgresql/localhost/postgres/trips",
  "messageType": "Wolverine.CritterWatch.ChaosMonkey.ChaosSeededPayment",
  "exceptionType": "System.TimeoutException",
  "exceptionMessage": "Malformed payload rejected by the validator",
  "source": "TripService",
  "sentAt": "2026-08-31T17:21:20.362632+00:00"
}

And got to work:

Me: Replay whatever will succeed now, and get rid of the rest.

Agent → query_dead_letters("TripService", limit: 200) → replay_dead_letters("TripService", [40 envelope ids])

The replay is fanned out to the service’s own message store and re-handled there — CritterWatch relays the command over the same Wolverine messaging backbone it uses for everything else. A minute later the agent re-summarized:

json

{ "totalDeadLetters": 15,
  "groups": [ { "messageType": "ChaosSeededReconciliation",
                "exceptionType": "System.InvalidOperationException", "count": 5 }, "..." ] }

And then reported back with exactly the partition you’d want from a colleague:

Agent: Replay complete. Here’s how it shook out:

• 25 of 40 drained — every TimeoutException, DataException, ArithmeticException, and ExternalException cleared on retry, which confirms those were transient infrastructure failures.
• 15 bounced straight back to the queue, and every survivor is a System.InvalidOperationException — the handler rejects these messages deterministically. Replaying them again will produce the same result. These look like genuine poison: either the payload is malformed or the handler has a bug for this input shape.

I’d recommend discarding the 15 (or holding them for a developer to inspect first). There is no undo on a discard — say the word.

Genuinely poisonous messages don’t get better because you asked nicely, so I said the word:

Agent → discard_dead_letters("TripService", [15 envelope ids])

Agent: Done — 15 discarded. Final check: summarize_dead_letters reads 0 dead letters with all databases reporting ("partial": false), so that’s a true zero, not a zero with a shrug.

Forty dead letters to zero, with the agent narrating what it replayed, what it didn’t and why, asking before the irreversible step, and verifying its own cleanup. The ids flow from the read tools straight into the action tools; no console tab was harmed in the making of this triage.

“What happened with message 08df07a4-…?”

As part of the effort toward building CritterWatch, we purposely improved Wolverine’s OpenTelemetry output to enable the troubleshooting we’re showing in this section.

This one’s my favorite, because it’s the question you actually ask in an incident review. Somebody hands you a message id from a log line and wants to know: where did this thing come from, what did it do, and what happened because of it?

Wolverine has emitted rich OpenTelemetry spans for years — every send, receive, and handler execution, tagged with messaging.message_id, messaging.conversation_id, messaging.message_type, the destination URI, and the handler type. CritterWatch’s trace tools sit on top of whatever tracing backend you already run (Jaeger in my dev fleet; DataDog and Application Insights providers ship in the box) through one provider abstraction, so the agent never needs to know or care which backend answers.

Me: What happened with message 08df07a4-3ded-0488-baa4-8b734d770000?

Agent → query_recent_traces("TripPublisher", requiredTags: { "messaging.message_id": "08df07a4-3ded-0488-..." })

json

{
  "provider": { "name": "jaeger-dev", "providerType": "Jaeger" },
  "count": 1,
  "traces": [{
    "traceId": "a4221774888b043b0a41a4cbcbb22f72",
    "rootOperationName": "marten.trip.all.page.execution",
    "durationMs": 15.311,
    "spanCount": 9,
    "rootTags": { "database.uri": "postgresql://localhost/postgres/trips",
                  "event.floor": "2030", "event.ceiling": "2034" }
  }]
}

Look at that root span, because it’s telling you something genuinely interesting: this message wasn’t born in a controller or a handler. It was born inside a Marten async daemon page execution — the projection daemon chewing through events #2030–2034 on the trips database. The agent then pulled the full span tree with get_trace — spans come back flat with ParentSpanId edges, and every Wolverine span carries the semantic tags:

json

{ "spans": [
  { "serviceName": "TripService", "operationName": "send",
    "tags": { "messaging.message_id": "08df07a4-3ded-0488-baa4-8b734d770000",
              "messaging.message_type": "TripMessages.ContinueTrip",
              "messaging.message_payload_size_bytes": "49" } },
  { "serviceName": "TripPublisher", "operationName": "TripMessages.ContinueTrip",
    "tags": { "message.handler": "TripPublisher.ContinueTripHandler",
              "messaging.message_id": "08df07a4-3ded-0488-baa4-8b734d770000",
              "otel.status_code": "OK" } },
  "... 7 more spans: the daemon page, sibling sends, sibling handler executions ..."
] }

And here’s the answer the agent actually gave — the raw spans reconstructed into the story a human wants:

Agent: Here’s the full life of message 08df07a4-3ded-0488:

1. Born — inside a Marten async projection daemon page on postgresql://localhost/postgres/trips, while it processed events #2030–2034 (a 15.3ms page). This message is a consequence of event processing, not a user action.
2. Sent — TripService dispatched it as a TripMessages.ContinueTrip command over RabbitMQ (49-byte payload).
3. Handled — TripPublisher.ContinueTripHandler executed it on TripPublisher with status OK.
4. Siblings — the same parent operation emitted three more ContinueTrip messages in the same batch (…0474, …0514, …0100); all three were also handled successfully.

So: five events were processed, four commands came out of it, all four landed cleanly across two services. Nothing about this message needs your attention.

Events appended, the message they caused, the handler that ran, and the sibling messages that rode along — the entire causal neighborhood of one message id, reconstructed from spans your system was already emitting. And when I asked the structural version of the same question — “where does ContinueTrip fit in the overall workflow?” — the agent reached for describe_lifecycle, which returns the type’s complete lifecycle stitched across every monitored service: publisher → transport → handler → cascaded messages → appended events → projections, rendered as both structured JSON and a ready-to-paste Mermaid sequence diagram. The observed instance and the structural map, side by side.

Chasing a poisoned projection

At least in the early days of Marten, this was frequently one of the hardest things to troubleshoot, but CritterWatch is going to help a great deal.

Now for the event sourcing crowd. Async projections fail in a uniquely annoying way: some event, somewhere in a stream, makes your Apply method throw, and the failure surfaces a long way from the cause. So I armed CritterWatch’s deterministic projection poison against TripService — set_chaos_monkey_projection_poison(eventTypeName: "TripStarted") — which makes every apply of that event type throw. (Why deterministic instead of the random failure-rate knob? Because with a rate, the alert, the dead-letter drill-in, and the projection stepper each land on a different random event, which is exactly what a diagnosis story can’t use. The poison makes every surface name the same event.)

Within a couple of minutes the fleet got loud, and the agent’s list_active_alerts("TripService") came back looking like a proper incident:

json

{ "alerts": [
  { "severity": "Critical", "alertType": "AgentDown",
    "subject": "Distance:All",
    "message": "Projection agent for Distance:All had not sent a heartbeat for 4m 8s" },
  { "severity": "Warning", "alertType": "ProjectionStale",
    "subject": "Distance:All",
    "message": "Projection Distance:All has not advanced in 5m 3s" },
  { "severity": "Critical", "alertType": "MetricsDlqRate",
    "message": "Dead letter rate 110.0/hr (110 dead letters in the last hour)" },
  "..." ] }

And the dead letter summary told the sharper story — 2,100 dead letters, every last one of them the same shape:

json

{ "totalDeadLetters": 2100,
  "groups": [{ "messageType": "TripMessages.TripMaintenanceCheck",
               "exceptionType": "JasperFx.Events.Daemon.ApplyEventException",
               "count": 2100 }] }

One message type, one exception type. Poison. The agent queried an envelope and the exception message pointed straight at the crime scene:

json

{ "exceptionType": "JasperFx.Events.Daemon.ApplyEventException",
  "exceptionMessage": "Failure to apply event #5784 Id(95dca7ce-5330-4f1d-9f18-151f91670d59)" }

Which the agent rolled up into an incident assessment I’d happily paste into a ticket as-is:

Agent: This is a poison-event pattern, not an outage. Here’s the evidence chain:

• All 2,100 dead letters are one message type (TripMaintenanceCheck) failing with one exception type (ApplyEventException) — the opposite shape from this morning’s transient batch.
• The exception is thrown by the event store’s apply pipeline, not by handler logic — each of these handlers folds the Trip aggregate from its event stream, and the fold itself is what’s dying.
• The failure names the exact event: event #5784 on stream 95dca7ce-5330-4f1d-9f18-151f91670d59, a TripStarted apply.

Every incoming maintenance check is going to keep dead-lettering until the apply is fixed — replaying now would just bounce all 2,100 straight back. Recommended next step: open this stream in the projection stepper and watch the fold up to event #5784 to see exactly what state + event data the Apply method chokes on.

Read that middle bullet again: the agent distinguished “your handler threw” from “your aggregate fold threw” purely from the exception’s origin — which is exactly the distinction that decides where you go looking for the bug. And this is precisely the moment the projection stepper exists for: give it the projection and the stream, and it replays the events one at a time in the console, showing you the projected document’s state before and after each event — so you watch the state fold step by step right up until the apply that blows up, with the failing event’s data sitting in front of you. No more “add a Console.WriteLine to the Apply method and rebuild.”

The remediation ran back through MCP, in order: clear the poison (in real life: ship the fix to your Apply method), then page through and replay all 2,100 maintenance checks — and the agent narrated its own progress as it went, so I could watch the drain without asking:

Agent: Poison cleared. Replaying the backlog in pages of 200 and confirming each drains before continuing:

replayed 200 — ~1,900 remaining
replayed 200 — ~1,700 remaining
replayed 200 — ~1,500 remaining
   ...
replayed 200 — ~300 remaining
replayed 100 — queue empty ✓

All 2,100 replayed and handled successfully — the fix holds under the full backlog, not just a sample.

Then restart_projection("TripService", "Distance:All") to bounce the projection agent, whose restart command CritterWatch routes to whichever node currently owns that agent:

json

{ "accepted": true, "operation": "Restart",
  "serviceName": "TripService", "projectionShardName": "Distance:All",
  "agentUri": "event-subscription://tripservice/Distance:All" }

If the projected documents themselves had been corrupted, rebuild_projection is sitting right there in the same toolbox, per-tenant scoping included.

Interrogating the alert subsystem

The alert subsystem got exercised throughout, but it deserves its own beat. The fleet-wide rollup is a single call:

Agent → summarize_active_alerts()

json

{ "totalActive": 11, "criticalTotal": 4, "warningTotal": 7,
  "byService": [
    { "serviceName": "PolecatTripService", "total": 2, "critical": 2 },
    { "serviceName": "HelpDesk",           "total": 4, "critical": 1 },
    { "serviceName": "TripService",        "total": 2, "critical": 1 },
    "..." ] }

Which the agent handed back as a briefing rather than a JSON blob:

Agent: 11 active alerts across 6 of your 25 services — 4 critical, 7 warnings:

Service	Critical	Warning
PolecatTripService	2	—
HelpDesk	1	3
TripService	1	1
MTTripService	—	1
Trip3Service	—	1
IncidentService	—	1

The TripService critical is the dead-letter rate alert from this morning’s drill — that one’s explained and should decay on its own. PolecatTripService’s two criticals are the ones I’d look at next if you want to keep going.

That’s a real fleet with real background noise, not a sanitized screenshot — eleven active alerts across six services, most of them the ordinary grumbling of a dev fleet that’s been abused all afternoon. list_active_alerts filters by service or severity, get_alert drills into one, and the action side — acknowledge_alert, snooze_alert, clear_alert — flows through the same event-sourced alert streams the console UI writes to. When the agent ran acknowledge_alert("alert:TripService:MetricsDlqRate:*") on the dead-letter-rate alert (my fault, see above), that acknowledgment showed up in the console’s alert timeline like any operator action, attributed and auditable.

One alert in that list turned out to be my favorite kind of detail: a warning that TripService’s scheduled-job poller was sitting on a growing backlog of scheduled envelopes. That wasn’t chaos I’d injected — that was CritterWatch correctly flagging genuine congestion in the sample fleet’s trip lifecycle, which schedules a TripMaintenanceCheck fifteen seconds after every trip starts. The demo rig got caught by its own monitoring. I’ll take it.

Read models, tenants, and performance while we’re here

A few quick hits from the same session, because the read surface goes well past the headline acts:

• get_projection_lag("TripService") returned all seven projection shards with their sequence positions, assigned nodes, agent status, pause reasons, and any captured exceptions — the projection-monitoring view as data.
• query_documents("TripService", "trip") paged through the 6,356 live Trip documents in the service’s Marten store — the document store explorer as an MCP tool, which also happily queries the daemon’s DeadLetterEvent documents when you’re hunting skipped events.
• On the multi-tenant services, list_document_types enumerated each tenant-sharded store’s types, and the tenant action tools — add_tenant, enable_tenant, disable_tenant, and friends — drive the same runtime tenant lifecycle the console UI does, fanned out across every registered tenant source.
• list_backlog_hotspots and get_backlog_state roll up incoming/outgoing/scheduled/dead-letter depths per store, fleet-wide, for “is anything drowning?” at a glance.

That document query deserves one concrete look, because “the agent can read your read models” is easy to skim past. Asked for a sample of live trips, it came back with:

json

{ "documentType": "trip", "totalCount": 6356, "pageSize": 3,
  "documents": [
    { "Id": "01a0594f-8241-4049-a320-28d7bf42f04b", "State": "Oklahoma",
      "Active": true, "StartedOn": 40, "Traveled": 0 },
    { "Id": "01a0594f-8241-408c-a0b4-1ef964434335", "State": "Missouri",
      "Active": true, "StartedOn": 49, "Traveled": 0 }, "..." ] }

Agent: 6,356 Trip documents in the store. Something worth a second look, though: the trips I sampled all show "Traveled": 0 despite being active for a while — if that field is supposed to accumulate, the projection feeding it may not be receiving the travel events at all.

That last observation is the kind of thing you get for free when the entity reading your data can also reason about it — and yes, it was right: the sample fleet’s travel pipeline really was stalled behind that scheduled-message backlog the alerting had flagged. The agent connected two findings I hadn’t connected myself.

The other half: AI Skills

Here’s the thing I want to be really clear about, because I think it’s the actual strategy and not just a feature list. The MCP tools expose the live, queryable system. But a pile of tools doesn’t make an agent good at operations — an agent also needs to know the discipline. That’s what the JasperFx AI Skills are for, and the two are built as complementary halves of one story.

The skills are curated, versioned instruction sets your coding agent loads on demand — there are dozens covering Wolverine, Marten, Polecat, testing, and messaging patterns, and a set specifically for operating CritterWatch: dead-letter triage, lifecycle diagnostics, routing diagnostics, service actions, and setup. The DLQ triage skill, for instance, doesn’t just list the four tools — it teaches the loop (summarize → query → act, in that order, with the ids flowing through), teaches an agent to read grouped counts as symptoms (“many exception types across many message types reads as a dependency being down; one message type with one exception reads as a poison message”), and drills in the non-negotiable rule I mentioned earlier: an empty result with partial: true means some stores did not report — never answer “the queue is empty.” Every time the agent in this session checked databasesAnswered before declaring victory, that was the skill talking.

We hold ourselves to a standing rule internally: any time CritterWatch exposes new information through an MCP tool, the paired skill work ships with it. A tool with no skill coverage is an under-leveraged tool.

What else can the agent do?

The session above touched maybe half the catalog. The rest of the toolbox, quickly:

• Event stores and read models — the document explorer tools (list_document_types, query_documents, get_document) across every monitored service’s store, plus the Event Store Explorer and projection stepper in the console for stream-level spelunking.
• Alerts — list, summarize, drill in, acknowledge, snooze, clear; alert thresholds themselves are configurable in the console.
• Scheduled jobs — the console’s scheduled messages view tracks every service’s scheduled envelope backlog (and as you saw above, the alerting watches the poller’s drain rate for you).
• Projection monitoring and operations — get_projection_lag for the shard-by-shard truth, then pause_projection / restart_projection / rebuild_projection / eject_projection, all tenant-scopable.
• Performance — backlog state and hotspots fleet-wide, projection lag, plus the OpenTelemetry trace tools riding your existing Jaeger / DataDog / App Insights backend.
• Message routing — explain_message_routing answers “where would this message go and why?” from the actual runtime routing rules, which beats reasoning about conventions from memory every single time.
• Listeners and services — pause, restart, and drain listeners; evict a defunct service from the console.
• Chaos engineering — the same chaos monkey tools I used to rig this demo: failure rates, slow handlers, deterministic projection poison, and dead-letter seeding for game days against staging.

One important word about the gates

Everything in this post is a paid-tier capability — every MCP tool checks the CritterWatch license before doing anything, and an unlicensed host answers every tool call with a polite LicenseMissing envelope rather than your production data. And beyond licensing, the action tools run through CritterWatch’s opt-in RBAC: every mutating tool is gated on a named capability (dlq.replay, dlq.discard, chaos-monkey.configure, and so on) scoped to the target service as a resource — so “the agent may replay dead letters on TripService but touch nothing on the billing service” is a policy you can actually write. Even the dead-letter reads carry their own capability, because dead letters contain message bodies, and “may look at business data” deserves a separate grant from “may act on it.” RBAC ships off by default for the low-ceremony getting-started path, and turns on when you’re ready to hand an agent real credentials in a real environment. The stateless MCP transport exists precisely so those checks always see the current caller, not whoever happened to open the session.

Handing an AI agent a control surface for production is the kind of thing that should make you a little nervous. It makes me a little nervous, and we built it — which is exactly why the license gate, the capability model, and the per-service scoping aren’t bolted on after the fact.

Epilogue: the alerts that wouldn’t clear

Dogfooding is good, and this issue is already fixed for our upcoming 1.1 release.

One loose thread, because I promised you the “wait, that’s not what I expected” moments too. Sharp-eyed readers may have noticed something off in the poisoned-projection section: those AgentDown alerts claimed the projection agents hadn’t sent a heartbeat in over four minutes — when the poison had only been armed for two. And after the remediation, they lingered longer than they should have, over a daemon that a direct SQL query showed advancing every single second.

The symptoms were real. The mechanism was not what anyone in the room — human or agent — guessed at first. Chasing it down with these same tools against CritterWatch’s own telemetry pipeline turned into a genuinely fun piece of distributed-systems forensics, complete with a Postgres deadlock storm, a docker daemon picking the worst possible moment to restart, and a feedback loop that made a merely-slow pipeline read as a dead one. We caught it entirely in the course of putting this post and its samples together, and the fix is already in for the upcoming CritterWatch 1.1 release. The full diagnosis — and what it taught us about building telemetry lanes that shed load instead of aging it — is the next post.

Summary

• CritterWatch’s MCP server is two lines of host code and a config stanza in your agent, and it exposes 48 tools across dead letters, alerts, health, performance, traces, routing, documents, projections, tenants, listeners, and chaos.
• The dead-letter loop — summarize, query, replay, discard — closes end to end without a human ferrying envelope ids around.
• “What happened with message X” is answerable from the OpenTelemetry spans Wolverine already emits, through whatever tracing backend you already run.
• When a projection sours, the agent finds the exact poisoned event from the dead-letter evidence, and the projection stepper shows you the state folding right up to the failure.
• The AI Skills teach your agent the operational discipline the tools alone can’t; the two ship as halves of one story, and the skills are bundled with CritterWatch Professional and Enterprise.

If you want to try this against your own Wolverine or Marten system:

• Docs: critterwatch.jasperfx.net — start with the quickstart and the MCP quick start
• Plans and pricing: jasperfx.net/our-products/#critterwatch-plans
• AI Skills for the Critter Stack: ai-skills.jasperfx.net — also available bundled or standalone
• CritterWatch is included in or discounted with our support plans, which is also how you get a hotline to the people who wrote all of this

And if you just want to talk it through first — reach out. Happy to set you up with a trial license.

I read a post from Paul Stack last week I thought was interesting titled AI Broke the Assumptions Behind CI. I was maybe much more influenced by Extreme Programming (XP) back in the day, so I’d disagree a little bit with his characterization of CI being something tied to the pull request workflow in GitHub et al, but let’s talk more about this.

The original point of CI was really just to be constantly getting feedback on your code by building and running tests against it as you change code and adjust as needed if CI found problems. With the radically different part of XP at the time being that you actually wrote tests!

The general idea behind XP was to be able to work faster and more adaptively by providing your team with effective feedback loops to guide and correct the adaptation as you worked. I’d say after all these years that CI is still very important — but it’s time for a serious rethink in our new AI powered world order.

Back to what Paul was getting at in his post, here’s the CI process I admittedly use for Marten, Wolverine, or other Critter Stack tool development:

1. Write code locally and be constantly running the tests most closely related to whatever I’m changing
2. Create pull requests — which we do in no small part just for traceability more so that as a code review tool (GitHub release notes generation is tied to pull requests and that’s just very helpful)
3. Lazily allow GitHub Actions execute all kinds of test suites and smoke test harnesses against the pull request while I go off and worry about something else
4. Merge pull requests when CI goes green or fix broken tests

And that’s mostly worked out until just recently. However, with the extreme load that’s come from all of us yahoos using AI agents to code so much faster, GitHub Actions are very noticeably slower or flat out unreliable on the worst days. And of course, to make things worse, we’ve added a lot more tests and CI actions than we ever tried to do before.

Granted, the Critter Stack work I do is especially problematic in this regard because so much of what we do involves slow running tests that execute against databases and message brokers. We’re also having to constantly build up and tear down .NET applications in memory. Especially for us, but maybe for you too since so many of us depend on now overloaded CI servers, there’s a very real problem that depending on CI builds running on a remote box has become a sometimes unacceptable bottleneck.

To balance a “good enough” safety net and for getting things done, I’ll mirror Paul Stack’s post and say that in some cases I’m:

• Doing trunk based development like it’s 2007 and Subversion is the latest hotness!
• Strictly using local testing with a lighter weight set of suites for commits, and trying to be very selective of what subset of tests are executing based on the changes in flight.
• Using the full blown “HeavyGate” of test suites to do pushes to the remote main

In public projects where there’s value in using pull requests just for tracking, I think we’re going to have to get more creative about test run filtering to avoid running test suites that aren’t relevant to the changes in flight to get pull requests in without hours of delays.

On a side note, with CI builds being so slow, that’s forced us to be much more aggressive about stomping out flaky or otherwise unreliable tests so CI is far more consistent for us. I’ve also added a new critter named “Bobcat” (the project is actually old enough that I started it before we adopted the current Critter Stack naming scheme) that among other things, uses the Microsoft Testing Platform to supervise test runs and selectively do test retries, process restarts, and even hard Docker resets based on known test flakes. That’s been hugely helpful both locally where heavy development can break down when Docker containers have run too long in tests and also for CI where retrying a CI failure just in case it’s just a test flake is just too damn slow now.

Anyway, AI assisted development continues to take up more of my gray matter than I wished it did and I don’t think the adaptation is going to change any time soon.

Assuming you haven’t been living under a rock, you’ve probably noticed that topics related to Artificial Intelligence (AI) are pretty well dominating most conversations about software development right now.

I’d like to lay out the Critter Stack community and JasperFx‘s strategy is for AI assisted software development at this moment and talk about all the different spaghetti we’re trying to throw up against the wall so that the Critter Stack and JasperFx can continue to thrive in our new world order.

Vertical Slice Architecture

Switching to the Critter Stack’s impact on application code, Wolverine has been built with what we now call “Vertical Slice Architecture” (VSA) in mind from the very beginning. That emphasis on VSA code organization has turned out to have been accidentally prescient as Wolverine’s very terse approach to VSA is already very optimized for AI agentic development — especially when contrasted with more traditional server side .NET code organization that leans into layered architectures that force AI agents to burn more tokens traversing the code.

Here’s a tutorial on Vertical Slice Architecture with Wolverine.

The takeaway here is that the intended idiomatic usage of Wolverine in our recommendations is already well suited for AI assisted development. The available command line tools that we bake right into your application using the Critter Stack tools can help a lot too. However, there is the very real issue of helping users utilize the AI-friendly, terse, VSA style of code when folks are coming from Clean Architecture approaches or other styles that call for more projects, layers, and abstractions that we the Critter Stack community or your AI agent ideally wants to deal with. Likewise, the usage of all those small diagnostic tools aren’t terribly obvious for troubleshooting if you don’t know they’re there or aren’t a Linux guru who’s used to banging out obscure command line calls from memory.

Wolverine and the rest of the Critter Stack is very highly optimized for simple and terse code using the Vertical Slice Architecture approach, and that’s great for AI assisted development!

And it’s incumbent upon people like me to prove that out over time of course.

Event Sourcing is Great for AI!

Or at least that’s something that all of the folks like me that are heavily invested in Event Sourcing tooling are telling you!

Now that CritterWatch 1.0 is live, JasperFx Software has a commercial offering that gives you comprehensive MCP or command line integration with all the common APIs and capabilities of our Marten/Polecat/Fisher event stores.

Command Line Tooling

We’ve long invested in command line diagnostics up and down through the whole Critter Stack just because it’s a mechanically cheap way to do that, but now the AI agent usage has made those tools more valuable than ever.

Our “classic” dotnet run describe is the human friendly tool we’ve long used to dump system descriptions out for our human users and one of the first things I ask users to try to unwind many common issues with Marten or Wolverine usage, and that’s not going anywhere. We’ve also invested quite a bit recently in additional CLI tools in Wolverine 9.0 especially that have textual output optimized for AI agents to explain message routing, preview the generated source code around a message handler, troubleshoot why a message handler isn’t being found, and other common user questions. I expect we will continue to expand this support based on any trends we see with user confusion.

We’re also doubling down even farther on the CLI path with the shortly forthcoming CritterWatch tooling that will provide CLI access to every bit of of an application’s “Critter Stack” configuration, the dead letter queue storage, domain centric queries against Open Telemetry tracking tools you might be using such as Jaeger, and queries against any Marten or Polecat event storage including a new “projection step through” function to trouble shoot projections.

I recently wrote about how we improved and expanded our CLI tooling in the “Critter Stack 2026” wave or releases.

CritterWatch

CritterWatch and its exposed MCP server will give you access to messaging and event workflow both from a static configuration standpoint and runtime discovered view as well. Every single bit of information exposed through CritterWatch and every single action is also completely exposed through its MCP endpoints, including access to the event stores of your Critter Stack application.

CritterWatch also comes with command line tooling as well that gives AI agents access to cross-application information about systems, workflows, messaging, and event stores.

AI Skills

We’ve built, and will continue to curate and evolve a set of AI Skills files for the Critter Stack tools including Marten, Polecat, Wolverine, and even Alba to help users use our tools more effectively. For Wolverine, the AI Skills will help you write code with Wolverine idioms that lead to much tighter code — or they’ll let you write the code in the conceptual way you’re used to, then suggest how to adjust that code to a tighter or more performance Wolverine idiom. All of which should help you be more effective over time with AI assisted development.

The AI Skills also teach your AI agent how to utilize the built in CLI

You can purchase access to these AI Skills here on the JasperFx website. All of our support plans also include access to these AI Skills as well as the forthcoming CritterWatch packages that are hopefully available by the time you read this.

Spec Driven Development and Event Modeling with Bobcat

We’re working a little bit on our solution for “Spec Driven Development” with a tool (so far) called Bobcat. I’m not sure what exactly will land in Bobcat or be spun out, but so far it’s the spiritual successor to my old Storyteller project for effective automated integration testing.

As part of that, Bobcat has a Gherkin capability for expressing executable specifications (we’re not taking a direct dependency on Reqnroll yet, but I’m personally undecided about whether we do that in the long run). We’re mostly going down the Gherkin path just because of the existing support inside of VS Code and JetBrains Rider for editing Gherkin documents. We’ll definitely look at other models for expressing tests too (simply marking up C#/F# test code for visualization? A markdown input? Something completely different?).

Bobcat also has a “Supervisor” capability that uses the new Microsoft Testing Platform (MTP) to drive xUnit.Net (or any testing tool that supports MTP and dotnet test) test suites with a bit more supervised parallelism and selective test retries. The “Supervisor” is also built to “know” when and how to recycle docker containers or restart test processes based on observed test failures. That’s been a god send for me doing my own CritterWatch development where the automated test suites are huge and test health can degrade as docker containers or processes have been up too long. We’re also building out a little we application you can use to see ongoing test results inside of the very long running test suites — again, for my own sanity when an AI agent is running tests and you can’t always tell if it’s active.

This is still pretty mushy as far as details, but we’re making the test output of Bobcat specifications write out quite a bit of detail about what Wolverine messages occurred, events that were appended, and HTTP calls received during the test — including the timings to help optimize test runs over time. I’m hopeful that this will be beneficial as well for AI assisted development by trying to make the tests themselves help diagnose test failures for quicker correction.

Lastly — and for the moment — we’re landing an Event Modeling capability into the core of the Critter Stack, but making the future Bobcat user interface be the visualization of the models. The concept that I’m proposing so far is:

• A new model and fluent interface API in our low level JasperFx.Events library that you can use to declare event types, read model types, and any other common elements of Event Storming or Event Modeling
• A new user interface in Bobcat that can visualize the Event Modeling slices defined by that model
• Integrate dotnet watch with that so you can interactively doodle with slice definitions and see the model change
• Have the specified model overridden when real code is built in the system, which is making us introducing “Event Modeling” concepts directly into Wolverine and our Marten/Polecat/Fisher event stores
• Integrate the Bobcat specifications directly into the Event Modeling visualization. Rather than have people waste time writing intermediate models for policies, constraints, or validation rules in a diagram, go straight to Behavior Driven Development specifications that become actionable specs
• Add more command line options to export the event slice definitions for AI agent usage
• Expand out AI Skills so an AI agent knows how to use Bobcat specifications and our models to build systems
• Probably invest in extending our existing code generation support to build out the shell of a “slice” from the model as a first step

Philosophically, I’m coming from a background in code-and-TDD/BDD-centric Extreme Programming and I’ve long been very dubious about the efficacy of “low code” visual modeling approaches or application generators like JHipster. I’m also not enthusiastic about any of the intermediate DSL approaches I’m seeing for modeling event driven architectures using YAML, XML, or custom built textual DSLs. Because of the Critter Stack’s relentless focus on low ceremony code, I think that we’re better off just adding the visualization capabilities on top of the code rather than trying for a hugely time consuming user interface effort.

Obviously

LLM Callouts?

I don’t have any details yet, but JasperFx is going to work with at least one client to build in integrations to LLMs from Wolverine and Marten/Polecat/Fisher projections using the Microsoft.Extensions.AI abstractions.

I think that we will 100% add LLM integrations into CritterWatch in the near future.

Agent Orchestration

This might end up being no more than a “build your own lightsaber” project for me, but JasperFx is building out its own take on an agent orchestrator and durable agent memory on top of the Critter Stack. And conveniently enough, we now have Fisher for Sqlite backed event sourcing that will be very convenient for this effort.

To be honest, I’ve been impressed with KurrentDb’s Capacitor tool, and that’s low hanging fruit for a Critter Stack-backed option.

It’s somewhat likely that this tooling would either end up landing in CritterWatch or at least share the same license model as CritterWatch, but it’s going to be a commercial tool all the way.

Summary

As the imagery at the top was trying to suggest, in terms of our AI strategy, we’re trying to throw all the spaghetti up against the wall right now and see what ends up sticking. And that’s the best that I’ve got for now.

CritterWatch 1.0 dropped yesterday, and you can read the official release on the JasperFx Software site.

This blog post is just me being thankful for all the folks who helped build CritterWatch or test it along the way:

• Babu Annamalai was instrumental in CritterWatch, our company infrastructure, and the Critter Stack in general
• Jeffry Gonzalez helped lay the foundation
• Anne Erdstrieck made CritterWatch (and Marten/Wolverine) a lot better by testing early versions on a very complex and high volume system
• Ben Virkler gave us a lot of feedback on usability and early problems (and we owe Ben some additional improvements for 1.1 that didn’t make the 1.0 wire)
• Laurence Gillian was an early tester
• Several other folks in Discord, and all of that feedback was helpful

And to Oskar Dudycz for all his contributions across the Critter Stack as much of CritterWatch are things that he and I were talking about years ago.

For my part, I founded JasperFx Software on the idea that we’d go down the “Open Core” model with business offerings of consulting, formal support agreements, and training around the Critter Stack tools, then supplement that with commercial add on tools for very advanced features, monitoring, and management. It took much longer than I’d hoped, but a couple months into JasperFx’s fourth year in business, we’re finally realizing the original vision.

And with all this out of the way, it’s time for me to get just a little break then start on CritterWatch 1.1!

We’re doing a live stream today on the CritterWatch 1.0 release — but it’s maybe a little dicy whether the official release happens before or after the live stream:-)

CritterWatch will have its 1.0 release tomorrow (Wednesday, August 19th) just in time for a live stream on YouTube to show just the user interface part of CritterWatch. Today though, we finally got one last RC.10 release out for some much delayed feedback.

We made a large amount of changes to optimize performance based on early customer feedback as we jumped right into the deep end of the pool and started by integrating CritterWatch into literally the single biggest Critter Stack system that we’re aware of.

This release candidate basically got us to what I expect the final product to be for 1.0, minus some user interface feedback and polishing at the last minute.

CritterWatch will require you to be running basically the latest of everything:

Stack: Wolverine 6.29.0 · JasperFx 2.52.0 · Marten 9.28.0 · Polecat 5.19.0 · Fisher 0.9.2 · Weasel 9.24.0

Three stores, one console

CritterWatch runs on Marten/PostgreSQL, Polecat/SQL Server, and — new in this release —
Fisher/SQLite:

The SQLite flavour needs no database service at all, which makes it the cheapest way to try the
console or to run it beside a small deployment.

Underneath, CritterWatch.Services is the store-agnostic core, compiled once and referenced by all
three.

First off, let me peel some egg off my face because I had allowed Claude to write quite a bit of code without close enough examination until just now. Arguably, we’re all good because we do have test coverage for the code I just refactored, so it’s all good in the end, but maybe just remember that a human in the loop is a good idea. And also know that all CritterWatch code will be closely reviewed before we flip that to 1.0!

Here’s an HTTP endpoint from CritterWatch, our forthcoming monitoring console for the Critter Stack. It adds a tenant to a monitored service:

[WolverinePost("/api/critterwatch/tenants/{serviceName}/add")]
[Middleware(typeof(RequireMultiTenancyLicense))]
public static async Task AddTenant(
    string serviceName,
    AddTenantRequest request,
    IDocumentSession session,
    IMessageBus bus,
    [FromServices] AuditLogService auditLog,
    HttpContext httpContext)
{
    // ... append an event, publish a command ...
    await auditLog.LogAsync("AddTenant", serviceName, null,
        $"Added tenant '{request.TenantId}' to {serviceName}",
        new Dictionary<string, string> { ["tenantId"] = request.TenantId },
        initiatedBy: AuditActor.From(httpContext.User));
}

Six parameters. Two of them are there for the audit log alone, and one of those — HttpContext — exists for a single expression: httpContext.User. We don’t read the request, the headers, the response, or anything else on it. We take the entire ASP.NET Core request context as a dependency to get at one ClaimsPrincipal.

That has a cost you feel in the test project. To test this method you need a store, a bus, an audit service and a request context. The audit behaviour — did we record the right action, against the right service, attributed to the right operator? — is only observable by standing all of that up and then querying the audit table afterwards.

Returning the intent instead of performing it

Wolverine has an interface called ISideEffect. It’s about as small as an interface gets:

public interface ISideEffect : IWolverineReturnType, INotToBeRouted;

There’s no method on it. The contract is a convention: return one of these from a handler or HTTP endpoint, and Wolverine will call any public Execute() or ExecuteAsync() method on it after your method returns. The interesting part is what happens to that method’s parameters — Wolverine registers each one as a dependency of the chain and resolves it for you.

So the audit log becomes a record:

public record AuditLog(
    string Action,
    string ServiceName,
    string? TargetUri = null,
    string? Details = null,
    Dictionary<string, string>? Parameters = null,
    string? InitiatedBy = null) : ISideEffect
{
    public Task ExecuteAsync(AuditLogService auditLog, ClaimsPrincipal? user)
    {
        var actor = string.IsNullOrWhiteSpace(InitiatedBy) ? AuditActor.From(user) : InitiatedBy;
        return auditLog.LogAsync(Action, ServiceName, TargetUri, Details, Parameters, actor);
    }
}

And the endpoint stops mentioning either dependency:

[WolverinePost("/test/audited/{serviceName}")]
public static AuditLog Post(string serviceName)
    => new("TestAction", serviceName, Details: "smoke");

AuditLogService and ClaimsPrincipal are resolved onto the chain because ExecuteAsync asks for them. The endpoint declares neither. The HttpContext parameter didn’t move somewhere else — it stopped existing. The principal is resolved at execution time rather than threaded through a signature that never wanted it.

Pure Functions FTW!

If you want to peruse our published Wolverine Best Practices, we strongly recommend trying to make the behavioral methods of your message handlers or HTTP endpoints be “pure functions” whenever possible. We also recommend trying to simplify code by opting to remove asynchronous code from your handlers as well as a way to reducing noise code, and that was why I reached quickly for the new AuditLog side effect. Combine side effects with other Wolverine goodies like our cascading messages syntax for publishing messages and the aggregate handler workflow and you get the tools to really simplify any application code related to business logic.

And just to make this clear, using pure functions (when possible) is a great approach because that:

• Isolates business or workflow logic from infrastructure concerns
• Promotes testability through fast running unit tests
• Reduces the code noise from asynchronous invocations

When not to reach for this

ISideEffect is for work you want to describe and let the framework perform. It’s a poor fit when the result of the work feeds the rest of your method — if you need the return value, you need the call, and a side effect only runs after you’ve returned.

It’s also not free indirection. A one-line await that nothing else depends on and nobody wants to test in isolation is fine as it is. What made the audit log worth converting was the ratio: two parameters and a request context, carried by five endpoints, to express one fact about an operation that had already happened.

That ratio is the tell. When a dependency exists only to record that something happened — audit, notification, telemetry, an outbound email — you are almost always better off returning a description of it and letting Wolverine make the call.

Summary

Hoo boy, let’s try to summarize this a bit:

1. I think having a man in the loop for AI built code is still pretty important
2. Pure functions are great for testability
3. Code noise has a real cost for your ability to reason about code and how it works — and I think that is still true even with LLMs doing so much more of the grunt work now
4. I think that our JasperFx curated AI Skills are valuable for developing with Wolverine and the rest of the Critter Stack because it’ll keep your LLM using more idiomatic features that can drastically shrink your code compared to typical .NET codebases, improve testability, and opt into Wolverine or Marten features that enhance performance. You can learn more about our AI Skills here.