The feature was built for a current JasperFx Software client, and came with a wave of developments across both Marten and Wolverine to support a fairly complex, mission critical set of application integrations. The PostgreSQL transport new to Wolverine was part of this wave. Some time next week I’ll be blogging about the Marten event subscription capabilities that were built into Marten & Wolverine to support this client as well. The point being, JasperFx is wide open for business and we can help your shop succeed with challenging project work!
Wolverine now has the ability to support strict messaging order with its message listeners. Given any random listening endpoint in Wolverine, just add this directive below to make the message processing be strictly sequential (with the proviso that your error handling policies may impact the order on failures):
var host = await Host.CreateDefaultBuilder().UseWolverine(opts =>
{
opts.UseRabbitMq().EnableWolverineControlQueues();
opts.PersistMessagesWithPostgresql(Servers.PostgresConnectionString, "listeners");
opts.ListenToRabbitQueue("ordered")
// This option is available on all types of Wolverine
// endpoints that can be configured to be a listener
.ListenWithStrictOrdering();
}).StartAsync();
Some notes about the ListenWithStrictOrdering() directive you might have:
It’s supported with every external messaging broker that Wolverine supports, including Kafka, Azure Service Bus, AWS SQS, and Rabbit MQ. It is also supported with the two database backed transports (we have both kinds, Sql Server and PostgreSQL!)
When this directive is applied, Wolverine will only make the listener for each endpoint (in the case above, the Rabbit MQ named “ordered”) be active on a single node within your application. Today that distribution is just crudely spreading out the “exclusive listeners” evenly across the whole application cluster. Definitely note that the strict ordering comes at the cost of reduced throughput, so use this feature wisely! Did I mention that JasperFx Software is here and ready to work with your company on Critter Stack projects?
Every exclusive listener will quickly start up on a single node if WolverineOptions.Durability.Mode = DurabilityMode.Solo, and you may want to do that for local testing and development just to be a little quicker on cold starts
The ListenWithStrictOrdering will make the internal worker queue (Wolverine uses an internal TPL Dataflow ActionBlock in these cases) for “buffered” or “durable” endpoints be strictly sequential
You will have to have a durable message store configured for your application in order for Wolverine to perform the leadership election and “agent tracking” (what’s running where)
Summary
This is a powerful tool in the continually growing Wolverine tool belt. The strict ordering may also be used to alleviate some concurrency issues that some users have hit with event sourcing using Marten when a single stream may be receiving bursts of commands that impact the event stream. The leadership election and agent distribution in Wolverine, in conjunction with this “sticky” listener assignment, gives Wolverine a nascent ability for virtual actors that we will continue to exploit. More soon-ish!
JasperFx Software is open for business and offering consulting services (like helping you craft modular monolith strategies!) and support contracts for both Marten and Wolverine so you know you can feel secure taking a big technical bet on these tools and reap all the advantages they give for productive and maintainable server side .NET development.
I’ve been thinking, discussing, and writing a bit lately about the whole “modular monolith” idea, starting with Thoughts on “Modular Monoliths” and continuing onto Actually Talking about Modular Monoliths. This time out I think I’d like to just put out some demos and thoughts about where Marten and Wolverine fit well into the modular monolith idea — and also some areas where I think there’s room for improvement.
First off, let’s talk about…
Modular Configuration
Both tools use the idea of a “configuration model” (what Marten Fowler coined a Semantic Model years ago) that is compiled and built from a combination of attributes in the code, explicit configuration, user supplied policies, and built in policies in baseline Marten or Wolverine as shown below with an indication of the order of precedence:
In code, when you as a user configure Marten and Wolverine inside of the Program file for your system like so:
builder.Services.AddMarten(opts =>
{
var connectionString = builder.Configuration.GetConnectionString("marten");
opts.Connection(connectionString);
// This will create a btree index within the JSONB data
opts.Schema.For<Customer>().Index(x => x.Region);
})
// Adds Wolverine transactional middleware for Marten
// and the Wolverine transactional outbox support as well
.IntegrateWithWolverine();
builder.Host.UseWolverine(opts =>
{
opts.CodeGeneration.TypeLoadMode = TypeLoadMode.Static;
// Let's build in some durability for transient errors
opts.OnException<NpgsqlException>().Or<MartenCommandException>()
.RetryWithCooldown(50.Milliseconds(), 100.Milliseconds(), 250.Milliseconds());
// Shut down the listener for whatever queue experienced this exception
// for 5 minutes, and put the message back on the queue
opts.OnException<MakeBelieveSubsystemIsDownException>()
.PauseThenRequeue(5.Minutes());
// Log the bad message sure, but otherwise throw away this message because
// it can never be processed
opts.OnException<InvalidInputThatCouldNeverBeProcessedException>()
.Discard();
// Apply the validation middleware *and* discover and register
// Fluent Validation validators
opts.UseFluentValidation();
// Automatic transactional middleware
opts.Policies.AutoApplyTransactions();
// Opt into the transactional inbox for local
// queues
opts.Policies.UseDurableLocalQueues();
// Opt into the transactional inbox/outbox on all messaging
// endpoints
opts.Policies.UseDurableOutboxOnAllSendingEndpoints();
// Connecting to a local Rabbit MQ broker
// at the default port
opts.UseRabbitMq();
// Adding a single Rabbit MQ messaging rule
opts.PublishMessage<RingAllTheAlarms>()
.ToRabbitExchange("notifications");
opts.LocalQueueFor<TryAssignPriority>()
// By default, local queues allow for parallel processing with a maximum
// parallel count equal to the number of processors on the executing
// machine, but you can override the queue to be sequential and single file
.Sequential()
// Or add more to the maximum parallel count!
.MaximumParallelMessages(10)
// Pause processing on this local queue for 1 minute if there's
// more than 20% failures for a period of 2 minutes
.CircuitBreaker(cb =>
{
cb.PauseTime = 1.Minutes();
cb.SamplingPeriod = 2.Minutes();
cb.FailurePercentageThreshold = 20;
// Definitely worry about this type of exception
cb.Include<TimeoutException>();
// Don't worry about this type of exception
cb.Exclude<InvalidInputThatCouldNeverBeProcessedException>();
});
// Or if so desired, you can route specific messages to
// specific local queues when ordering is important
opts.Policies.DisableConventionalLocalRouting();
opts.Publish(x =>
{
x.Message<TryAssignPriority>();
x.Message<CategoriseIncident>();
x.ToLocalQueue("commands").Sequential();
});
});
The nested lambdas in AddMarten() and UseWolverine() are configuring the MartenOptions and WolverineOptions models respectively (the “configuration model” in that diagram above).
I’m not aware of any one commonly used .NET idiom for building modular configuration, but I do commonly see folks using extension methods for IServiceCollection or IHostBuilder to segregate configuration that’s specific to a single module, and that’s what I think I’d propose. Assuming that we have a module in our modular monolith system for handling workflow around “incidents”, there might be an extension method something like this:
public static class IncidentsConfigurationExtensions
{
public static WebApplicationBuilder AddIncidentModule(this WebApplicationBuilder builder)
{
// Whatever other configuration, services, et al
// we need for just the Incidents module
// Extra Marten configuration
builder.Services.ConfigureMarten(opts =>
{
// I'm just adding an index for a document type within
// this module
opts.Schema.For<IncidentDetails>()
.Index(x => x.Priority);
// Purposely segregating all document types in this module's assembly
// to a separate database schema
opts.Policies.ForAllDocuments(m =>
{
if (m.DocumentType.Assembly == typeof(IncidentsConfigurationExtensions).Assembly)
{
m.DatabaseSchemaName = "incidents";
}
});
});
return builder;
}
}
Which would be called from the overall system’s Program file like so:
var builder = WebApplication.CreateBuilder(args);
builder.AddIncidentModule();
// Much more below...
Between them, the two main Critter Stack tools have a lot of support for modularity through:
Wolverine’s extension and modules allow you to easily import message handlers or HTTP endpoints from other assemblies than the main application assembly
Both Marten and Wolverine allow you to register special extensions in your IoC container that allow you to add additional configuration to both Marten (IConfigureMarten) and Wolverine (IWolverineExtension)
All of the facilities I described above can be used to separate specific configuration for different modules within the module code itself.
Modular Monoliths and Backing Persistence
In every single experience report you’ll ever find about a team trying to break up and modernize a large monolithic application the authors will invariably say that breaking apart the database was the single most challenging task. If we are really doing things better this time with the modular monolith approach, we’d probably better take steps ahead of time to make it easier to extract services by attempting to keep the persistence for each module at least somewhat decoupled from the persistence of other modules.
Going even farther in terms of separation, it’s not unlikely that some modules have quite different persistence needs and might be better served by using a completely different style of persistence than the other modules. Just as an example, one of our current JasperFx Software clients has a large monolithic application where some workflow-centric modules would be a good fit for an event sourcing approach, while other modules are more CRUD centric or reporting-centric where a straight up RDBMS approach is probably much more appropriate.
So let’s finally bring Marten and Wolverine into the mix and talk about the Good, the Bad, and the (sigh) Ugly of how the Critter Stack fits into modular monoliths:
wah, wah, wah…
Let’s start with a positive. Marten sits on top of the very robust PostgreSQL database. So in addition to Marten’s ability to use PostgreSQL as a document database and as an event store, PostgreSQL out of the box is a rock solid relational database. Heck, PostgreSQL even has some ability to be used as either a graph database! The point is that using the Marten + PostgreSQL combination gives you a lot of flexibility in terms of persistence style between different modules in a modular monolith without introducing a lot more infrastructure. Moreover, Wolverine can happily utilize its PostgreSQL-backed transactional outbox with both Entity Framework Core and Marten targeting the same PostgreSQL database in the same application.
Continuing with another positive, let’s say that we want to create some logical separation between our modules in the database, and one way to do so would be to simply keep Marten documents in separate database schemas for each module. Repeating a code sample from above, you can see that configuration below:
public static WebApplicationBuilder AddIncidentModule(this WebApplicationBuilder builder)
{
// Whatever other configuration, services, et al
// we need for just the Incidents module
// Extra Marten configuration
builder.Services.ConfigureMarten(opts =>
{
// Purposely segregating all document types in this module's assembly
// to a separate database schema
opts.Policies.ForAllDocuments(m =>
{
if (m.DocumentType.Assembly == typeof(IncidentsConfigurationExtensions).Assembly)
{
m.DatabaseSchemaName = "incidents";
}
});
});
return builder;
}
So, great, the storage for Marten documents could easily be segregated by schema. Especially considering there’s little or no referential integrity relationships between Marten document tables, it should be relatively easy to move these document tables to completely different databases later!
And with that, let’s move more into “Bad” or hopefully not too “Ugly” territory.
The event store data in Marten is all in one single set of tables (mt_streams and mt_events). So every module utilizing Marten’s event sourcing could be intermingling their events in just these tables through the one single, AddMarten() store for the application’s IHost. You could depend on marking event streams by their aggregate type like so:
public static async Task start_stream(IDocumentSession session)
{
// the Incident type argument is strictly a marker for
// Marten
session.Events.StartStream<Incident>(new IncidentLogged());
await session.SaveChangesAsync();
}
I think we could ameliorate this situation with a couple future changes:
A new flag in Marten that would make it mandatory to mark every new event stream with an aggregate type specifically to make it easier to separate the events later to extract a service and its event storage
Some kind of helper to move event streams from one database to another. It’s just not something we have in our tool belt at the moment
Of course, it would also help immeasurably if we had a way to split the event store storage for different types of event streams, but somehow that idea has never gotten any traction within Marten and never rises to the level of a high priority. Most of our discussions about sharding or partitioning the event store data has been geared around scalability — which is certainly an issue here too of course.
Marten also has its concept of “separate stores” that was meant to allow an application to interact with multiple Marten-ized databases from a single .NET process. This could be used with modular monoliths to segregate the event store data, even if targeting the same physical database in the end. The very large downside to this approach is that Wolverine’s Marten integration does not today do anything with the separate store model. So no Wolverine transactional middleware, event forwarding, transactional inbox/outbox integration, and no aggregate handler workflow. So basically everything about the full “Critter Stack” integration that makes that tooling the single most productive event sourcing development experience in all of .NET (in my obviously biased opinion). Ugly.
Randomly, I heard an NPR interview with Eli Wallach very late in his life who was the actor who played the “Ugly” character in the famous western, and I could only describe him as effusively jolly. So basically a 180 degree difference from his character!
Module to Module Communication
I’ve now spent much more time on this post than I had allotted, so it’s time to go fast…
In my last post I used this diagram to illustrate the risk of coupling modules through direct usage of internals (the red arrows):
Instead of the red arrows everywhere above, I think I’m in favor of trying to limit the module to module communication to using some mix of a “mediator” tool or an in memory message bus between modules. That’s obviously going to come with some overhead, but I think (hope) that overhead is a net positive.
For a current client, I’m recommending they further utilize MediatR as they move a little more in the direction of modularity in their current monolith. For greenfield codebases, I’d recommend Wolverine instead because I think it does much, much more.
First, Wolverine has a full set of functionality to be “just a Mediator” to decouple modules from too much of the internals of another module. Secondly, Wolverine has a lot of support for background processing through local, in memory queues that could be very advantageous in modular monoliths where Wolverine can de facto be an in memory message bus. Moreover, Wolverine’s main entry point usage is identical for messages processed locally versus messages published through external messaging brokers to external processes:
public static async Task using_message_bus(IMessageBus bus)
{
// Use Wolverine as a "mediator"
// This is normally executed inline, in process, but *could*
// also be invoking this command in an external process
// and waiting for the success or failure ack
await bus.InvokeAsync(new CategoriseIncident());
// Use Wolverine for asynchronous messaging. This could
// start by publishing to a local, in process queue, or
// it could be routed to an external message broker -- but
// the calling code doesn't have to know that
await bus.PublishAsync(new CategoriseIncident());
}
The point here is that Wolverine can potentially set your modular monolith architecture up so that it’s possible to extract or move functionality out into separate services later.
All that being said about messaging or mediator tools, some of the ugliest systems I’ve ever seen utilized messaging or proto-Mediatr command handlers between logical modules. Those systems had code that was almost indecipherable by introducing too many layers and far too much internal messaging. I think I’d say that some of the root cause of the poor system code was from getting the bounded context boundaries wrong so that the messaging was too chatty. Using high ceremony anti-corruption layers also adds a lot of mental overhead to follow information flow through several mapping transformations. One of these systems was using the iDesign architectural approach that I think naturally leads to very poorly factored software architectures and too much harmful code ceremony. I do not recommend.
I guess my only point here is that no matter what well intentioned advice people like me try to give, or any theory of how to make code more maintainable any of us might have, if you find yourself saying to yourself about code that “this is way harder than it should be” you should challenge the approach and look for something different — even if that just leads you right back to where you are now if the alternatives don’t look any better.
One important point here about both modular monoliths or a micro service strategy or a mix of the two: if two or more services/modules are chatty between themselves and very frequently have to be modified at the same time, they’re best described as a single bounded context and should probably be combined into a single service or module.
Summary
Anyway, that’s enough from me on this subject for now, and this took way longer than I meant to spend on it. Time to get my nose back to the grindstone. I am certainly very open to any feedback about the Critter Stack tools limitations for modular monolith construction and any suggestions or requests to improve those tools.
Hey, did you know that JasperFx Software offers formal support plans for Marten and Wolverine? Not only are we making the “Critter Stack” tools be viable long term options for your shop, we’re also interested in hearing your opinions about the tools and how they should change.We’re also certainly open to help you succeed with your software development projects on a consulting basis whether you’re using any part of the Critter Stack or some completely different .NET server side tooling.
First off, let’s say that you have a simplistic document that can “self-aggregate” itself as a “Snapshot” in Marten like this:
public record InvoiceCreated(string Description, decimal Amount);
public record InvoiceApproved;
public record InvoiceCancelled;
public record InvoicePaid;
public record InvoiceRejected;
public class Invoice
{
public Invoice()
{
}
public static Invoice Create(IEvent<InvoiceCreated> created)
{
return new Invoice
{
Amount = created.Data.Amount,
Description = created.Data.Description,
// Capture the timestamp from the event
// metadata captured by Marten
Created = created.Timestamp,
Status = InvoiceStatus.Created
};
}
public int Version { get; set; }
public decimal Amount { get; set; }
public string Description { get; set; }
public Guid Id { get; set; }
public DateTimeOffset Created { get; set; }
public InvoiceStatus Status { get; set; }
public void Apply(InvoiceCancelled _) => Status = InvoiceStatus.Cancelled;
public void Apply(InvoiceRejected _) => Status = InvoiceStatus.Rejected;
public void Apply(InvoicePaid _) => Status = InvoiceStatus.Paid;
public void Apply(InvoiceApproved _) => Status = InvoiceStatus.Approved;
}
For asynchronous projections of any kind, we have a little bit of complication for testing. In a classic “Arrange, Act, Assert” test workflow, we’d like to exercise our projection — and mind you, I strongly recommend that testing happen within its integration with Marten rather than some kind of solitary unit tests with fakes — with a workflow like this:
Pump in some new events to Marten
Somehow magically wait for Marten’s asynchronous daemon running in a background thread progress to the point where it’s handled all of our newly appended events for all known, running projections
Load the expected documents that should have been persisted or updated from our new events by the projections running in the daemon, and run some assertions on the expected system state
For right now, I want to worry about the second bullet point and introduce a new (old, but it actually works correctly now) WaitForNonStaleProjectionDataAsync API introduced in Marten 7.5. You can see the new API used in this test from the new documentation on Testing Projections:
[Fact]
public async Task test_async_aggregation_with_wait_for()
{
// In your tests, you would most likely use the IHost for your
// application as it is normally built
using var host = await Host.CreateDefaultBuilder()
.ConfigureServices(services =>
{
services.AddMarten(opts =>
{
opts.Connection(
"Host=localhost;Port=5432;Database=marten_testing;Username=postgres;password=postgres;Command Timeout=5");
opts.DatabaseSchemaName = "incidents";
// Notice that the "snapshot" is running inline
opts.Projections.Snapshot<Invoice>(SnapshotLifecycle.Async);
})
// Using Solo in tests will help it start up a little quicker
.AddAsyncDaemon(DaemonMode.Solo);
}).StartAsync();
var store = host.Services.GetRequiredService<IDocumentStore>();
var invoiceId = Guid.NewGuid();
// Pump in events
using (var session = store.LightweightSession())
{
session.Events.StartStream<Invoice>(invoiceId, new InvoiceCreated("Blue Shoes", 112.24m));
await session.SaveChangesAsync();
session.Events.Append(invoiceId,new InvoiceApproved());
session.Events.Append(invoiceId,new InvoicePaid());
await session.SaveChangesAsync();
}
// Now, this is going to pause here in this thread until the async daemon
// running in our IHost is completely caught up to at least the point of the
// last event captured at the point this method was called
await store.WaitForNonStaleProjectionDataAsync(5.Seconds());
// NOW, we should expect reliable results by just loading the already
// persisted documents built by rebuilding the projection
await using var query = store.QuerySession();
// Load the document that was "projected" from the events above
// and immediately persisted to the document store
var invoice = await query.LoadAsync<Invoice>(invoiceId);
// Run assertions
invoice.Description.ShouldBe("Blue Shoes");
invoice.Status.ShouldBe(InvoiceStatus.Paid);
}
In the example projection, I’ve been capturing the timestamp in the Invoice document from the Marten event metadata:
public static Invoice Create(IEvent<InvoiceCreated> created)
{
return new Invoice
{
Amount = created.Data.Amount,
Description = created.Data.Description,
// Capture the timestamp from the event
// metadata captured by Marten
Created = created.Timestamp,
Status = InvoiceStatus.Created
};
}
But of course, if that timestamp has some meaning later on and you have any kind of business rules that may need to key off that time, it’s very helpful to be able to control the timestamps that Marten is assigning to create predictable automated tests. As of Marten 7.5, Marten uses the newer .NET TimeProvider behind the scenes, and you can replace it in testing like so:
[Fact]
public async Task test_async_aggregation_with_wait_for_and_fake_time_provider()
{
// Hang on to this for later!!!
var eventsTimeProvider = new FakeTimeProvider();
// In your tests, you would most likely use the IHost for your
// application as it is normally built
using var host = await Host.CreateDefaultBuilder()
.ConfigureServices(services =>
{
services.AddMarten(opts =>
{
opts.Connection(
"Host=localhost;Port=5432;Database=marten_testing;Username=postgres;password=postgres;Command Timeout=5");
opts.DatabaseSchemaName = "incidents";
// Notice that the "snapshot" is running inline
opts.Projections.Snapshot<Invoice>(SnapshotLifecycle.Async);
opts.Events.TimeProvider = eventsTimeProvider;
})
// Using Solo in tests will help it start up a little quicker
.AddAsyncDaemon(DaemonMode.Solo);
}).StartAsync();
var store = host.Services.GetRequiredService<IDocumentStore>();
var invoiceId = Guid.NewGuid();
// Pump in events
using (var session = store.LightweightSession())
{
session.Events.StartStream<Invoice>(invoiceId, new InvoiceCreated("Blue Shoes", 112.24m));
await session.SaveChangesAsync();
session.Events.Append(invoiceId,new InvoiceApproved());
session.Events.Append(invoiceId,new InvoicePaid());
await session.SaveChangesAsync();
}
// Now, this is going to pause here in this thread until the async daemon
// running in our IHost is completely caught up to at least the point of the
// last event captured at the point this method was called
await store.WaitForNonStaleProjectionDataAsync(5.Seconds());
// NOW, we should expect reliable results by just loading the already
// persisted documents built by rebuilding the projection
await using var query = store.QuerySession();
// Load the document that was "projected" from the events above
// and immediately persisted to the document store
var invoice = await query.LoadAsync<Invoice>(invoiceId);
// Run assertions, and we'll use the faked timestamp
// from our time provider
invoice.Created.ShouldBe(eventsTimeProvider.Start);
}
In the sample above, I used the FakeTimeProvider from the Microsoft.Extensions.TimeProvider.Testing Nuget package.
Summary
We take testability and automated testing very seriously throughout the entire “Critter Stack.” The testing of asynchronous projections has long been a soft spot that we hope is improved by the new capabilities in this post. As always, feel free to pop into the Critter Stack Discord for any questions.
As part of the big Marten 7 release a couple weeks ago, core team member Babu Annamalaimade a huge contribution with a brand new model for making “partial” document updates that’s backed with native PostgreSQL operations. See Babu’s post Marten native partial updates – patching for much more information and details about everything that’s supported now.
Let’s put this into perspective with a quick sample. Here’s a simplistic Wolverine handler that updates a single member on a stored document:
public static async Task Handle(ApproveInvoice command, IDocumentSession session)
{
// Load the invoice
var invoice = await session.LoadAsync<Invoice>(command.InvoiceId);
invoice.Approved = true;
// Tell Marten to persist the new version
session.Store(invoice);
// Commit the one pending change
await session.SaveChangesAsync();
}
In the code above, I’m loading the whole document, changing one property, and committing the changes back to the database. Under the covers we’re making two round trips to the database, deserializing the starting state of the Invoice document, then serializing the end state of the Invoice document.
With the new patching support, let’s rewrite that handler to this:
public static async Task Handle(ApproveInvoice command, IDocumentSession session)
{
// Tell Marten what to change
session.Patch<Invoice>(command.InvoiceId).Set(x => x.Approved, true);
// Commit the one pending change
await session.SaveChangesAsync();
}
In that second version, we’re doing a lot less work. There’s only one database call to overwrite the Approved value within the existing Invoice document. While it’s a nontrivial operation to reach inside the JSON in the database, we’re not having to do any serialization in memory.
Marten technically had this feature set already, but our older support depended on the PLv8 extension to PostgreSQL that’s more or less deprecated now. Babu’s work for Marten 7 brings this very important feature set back into play for the majority of users who don’t want to or can’t utilize the older PLv8 backed support.
Working inside of the LINQ provider code that’s ultimately interpreting C# code and trying to turn that into the most efficient Pgpsql code possible somehow makes me think about Alice in Wonderland where I’m definitely in the role of “Alice.”
A major set of emphasis for the Marten 7 release was to address the large accretion of LINQ related issues with a mostly new LINQ provider subsystem. In terms of improvements or just changes, we:
Completely removed the Remotion.Linq dependency (and the results of that are mixed so far)
Reworked the SQL generation for child collection querying
Greatly expanded the usage of .NET Dictionary types within LINQ queries
Expanded the reach of Select() transformations quite a bit — and that code hadn’t been touched or improved in any substantial way since Marten 1.0. You can see the list of patterns that Marten now supports in the acceptance test code.
Allowed for filtering of Include() queries, which has been a longstanding user “ask”
Child Collection Improvements
Alright, so back to the real problem. When Marten today encounters a LINQ query like this one:
Marten versions 4-6 had to generate a complicated SQL query using PostgreSQL Common Table Expressions to explode out the child collections into flat rows that can then be filtered to matching child rows, then finally uses a sub query filter on the original table to find the right rows. To translate, all that mumbo jumbo I said translates to “a big ass, slow query that doesn’t allow PostgreSQL to utilize its fancy GIN index support for faster JSONB querying.”
The Marten v7 support is smart enough to “know” when it can generate more efficient SQL for certain child collection filtering. In the case above, Marten v7 can use the PostgreSQL containment operator to utilize the GIN indexing support and just be simpler in general with SQL like this:
Another little micro-optimization we did for Marten V7 was to have as much SQL generation as possible use positional parameters (“&1”) as opposed to named parameters (“:p0”) that PostgreSQL itself expects to skip some SQL preprocessing in memory that we were forcing Npgsql to do for us earlier.
From early adopter feedback, some child collection queries in real systems have been a touch over an order of magnitude faster from V6 to V7 because of this change.
One more sample that I’m especially proud of. Let’s say you use this LINQ query:
var result = await theSession.Query<Root>()
.Where(r => r.ChildsLevel1.Count(c1 => c1.Name == "child-1.1") == 1)
.ToListAsync();
This one’s a little more complicated because you need to do a test of the number of matching child elements within a child collection. Again, Marten V6 will use a nasty and not terribly efficient common table expression approach to give you the right data. For Marten v7, we specifically asked the Marten user base if we could abandon support for any PostgreSQL versions lower than PostgreSQL 12 so we could use PostgreSQL’s JSONPath query support within our LINQ provider. That change got us to this SQL for the LINQ query from up above:
select d.id, d.data from public.mt_doc_root as d where jsonb_array_length(jsonb_path_query_array(d.data, '$.ChildsLevel1[*] ? (@.Name == $val1)', $1)) = $2
&1: {"val1":"child-1.1"}
&2: 1
That’s the child collection querying, which was absolutely a first class goal and time consuming task along the way.
Dictionary Queries
There was a large effort to improve Marten’s ability to query through Dictionary<TKey, TValue> members of a parent document type for V7 — which immediately led to plenty more user reported bugs we had to fix during the V7 pre-release cycle.
Here’a an example that’s now possible querying through Dictionary.ContainsKey():
var results = await theSession
.Query<Target>()
// This is querying through a dictionary
.Where(x => x.GuidDict.ContainsKey(guid))
.ToListAsync();
And some SelectMany() action:
var pairs = await theSession
.Query<Target>()
.SelectMany(x => x.StringDict.Keys)
.ToListAsync();
And selecting the dictionary:
var data = await theSession
.Query<SelectDict>()
.Select(x => x.Dict)
.ToListAsync();
And querying through dictionary values:
var values = await theSession.Query<Target>().SelectMany(x => x.StringDict.Values)
.Where(x => x == "value2")
.OrderBy(x => x)
.ToListAsync();
Filtered Include
Marten’s Include() functionality is an important way to improve system performance by fetching related documents in one database round trip. Many folks through the years have asked for a way to limit the number of “included” documents returned in these queries by specifying a Where() filter against the included document type. That very functionality landed in Marten V7 with this syntax shown below from our testing code:
var holders = await theSession.Query<TargetHolder>()
// Limit the fetched Target documents from the Include()
// by specifying a filter on Target documents
.Include<Target>(x => x.TargetId, x => list.Add(x), t => t.Color == Colors.Blue)
.ToListAsync();
Summary
While there are still some regression bugs coming in from the LINQ provider work (but they’re getting way more “edge case-y” and “WTH *would* you do that?”), I’m feeling good about the LINQ provider subsystem in Marten as a better foundation for us moving forward.
And for the appropriate coding soundtrack for me when I need to duck into the LINQ provider code in Marten to fix bugs:
Resiliency and Low Level Improvements in Marten 7 (this post)
I hate to tell you all this, but sometimes there’s going to be occasional hiccups with your system in production. Just sticking with using Marten and its connectivity to PostgreSQL under the covers, you could easily have:
Occasional network issues that cause transactions to fail
A database could be temporarily too busy and throw exceptions
Concurrency issues from trying to write to the same rows in the underlying database (Marten isn’t particularly prone to this, but I needed another example)
These typical transient errors happen, but that doesn’t mean that the operation that just failed couldn’t happily succeed if you just retried it in a little bit. To that end, Marten 7 replaced our previous homegrown resiliency approach (that didn’t really work anyway) with a reliance on the popular Polly library.
The default settings for Marten are shown below:
// Default Polly setup
var strategy = new ResiliencePipelineBuilder().AddRetry(new()
{
ShouldHandle = new PredicateBuilder().Handle<NpgsqlException>().Handle<MartenCommandException>().Handle<EventLoaderException>(),
MaxRetryAttempts = 3,
Delay = TimeSpan.FromMilliseconds(50),
BackoffType = DelayBackoffType.Exponential
}).Build();
The Marten docs here will tell you how to customize these policies at your discretion.
To put this into context, if you call IDocumentSession.SaveChangesAsync() to commit a unit of work, and there’s an exception that matches the criteria shown above, Polly will re-execute the queued operations from scratch with a brand new connection so the retry can succeed after the database or network has settled down.
Moreover, we followed the Polly recommendations on performance to utilize “static Lambdas” within the Marten internals to reduced the number of object allocations within Marten 7’s execution pipeline compared to the Marten 6 and earlier pipeline. It’s kind of ugly code, but you can see where and how we used Polly in QuerySession.Execution.cs.
Another big change for Marten 7 was how Marten is going to manage database connection lifecycles inside of Marten sessions. Prior to Marten 7, Marten sessions would open a database connection and keep it open until the session was ultimately disposed. This decision was originally made to optimize the integration between Marten and Dapper when Marten’s functionality was very limited.
Now though, Marten will only open a database connection within a session immediately before any operation that involves a database connection, and close that connection immediately after the operation is over (really just returning the underlying connection to the connection pool managed by Npgsql). With this change, it is now safe to run read-only queries through IQuerySession (or lightweight IDocumentSession) objects in multiple threads. That should make Marten be more effective within Hot Chocolate integrations:
Marten sessions can actually be thread safe so that you can let the default IoC registration behavior for Marten sessions happily work within Hot Chocolate requests where it wants to parallelize queries
Marten usage will be far, far less prone to connection leaks when developers create sessions without disposing them properly
And even if you’re not using Hot Chocolate at all, Marten 7 will be more parsimonious over all with how it uses database connections which should help with scalability of your system — and definitely will help with cloud hosting options that charge by the number of database connections used!
Final Thoughts
In the past month or so I’ve had more interaction with developers than usual who are highly suspicious of open source tooling and especially of alternative open source tooling in .NET that isn’t directly supported by Microsoft. It’s a fair point to some degree, because “Google/Bing-bility” is a highly underrated quality of a development tool. Some of that fear was thinking that the responsible developers behind a non mainstream tool are just going to get tired of it and abandon it on a whim when a shinier object comes around so it’s not even worth the time to care about those alternatives from the “cool kids.”
What I would like to point out with Marten in particular is that it’s nearly a decade old as a project (the document db features in Marten actually predate CosmosDb in .NET world) and still very active and growing. The work in this post is all about making fine grained refinements to the core project and I would hope demonstrate a dedication on the part of the whole community toward continuously improving Marten for serious work. In other words, I think development shops can absolutely feel confident in placing a technical bet on Marten.
Hey, did you know that JasperFx Software offers formal support plans for Marten and Wolverine? Not only are we making the “Critter Stack” tools be viable long term options for your shop, we’re also interested in hearing your opinions about the tools and how they should change.We’re also certainly open to help you succeed with your software development projects on a consulting basis whether you’re using any part of the Critter Stack or some completely different .NET server side tooling.
Marten 7.0 was released over the weekend, and Wolverine 2.0 followed yesterday mostly to catch up with the Marten dependency. One of the major improvements in this round of “Critter Stack” releases was to address a JasperFx client’s need for dynamically adding new tenant databases at runtime without having to do any kind of system deployment.
Doing that successfully meant adding a couple capabilities throughout the “Critter Stack.” First off, Marten needed a new multi-tenancy configuration strategy that allowed users to keep a list of valid tenant id and database connection strings for those tenants in a database table. Enter Marten 7’s new Master Table Tenancy Model. You can see the usage in this sample from the documentation:
using var host = await Host.CreateDefaultBuilder()
.ConfigureServices(services =>
{
services.AddMarten(sp =>
{
var configuration = sp.GetRequiredService<IConfiguration>();
var masterConnection = configuration.GetConnectionString("master");
var options = new StoreOptions();
// This is opting into a multi-tenancy model where a database table in the
// master database holds information about all the possible tenants and their database connection
// strings
options.MultiTenantedDatabasesWithMasterDatabaseTable(x =>
{
x.ConnectionString = masterConnection;
// You can optionally configure the schema name for where the mt_tenants
// table is stored
x.SchemaName = "tenants";
// If set, this will override the database schema rules for
// only the master tenant table from the parent StoreOptions
x.AutoCreate = AutoCreate.CreateOrUpdate;
// Optionally seed rows in the master table. This may be very helpful for
// testing or local development scenarios
// This operation is an "upsert" upon application startup
x.RegisterDatabase("tenant1", configuration.GetConnectionString("tenant1"));
x.RegisterDatabase("tenant2", configuration.GetConnectionString("tenant2"));
x.RegisterDatabase("tenant3", configuration.GetConnectionString("tenant3"));
// Tags the application name to all the used connection strings as a diagnostic
// Default is the name of the entry assembly for the application or "Marten" if
// .NET cannot determine the entry assembly for some reason
x.ApplicationName = "MyApplication";
});
// Other Marten configuration
return options;
})
// All detected changes will be applied to all
// the configured tenant databases on startup
.ApplyAllDatabaseChangesOnStartup();;
}).StartAsync();
With this tenancy model, Marten is able to discover newly added tenant databases at runtime as needed during normal transactions. And don’t fret about the extra activity, Marten is able to cache that information in memory to avoid making too many unnecessary database calls.
But what about if I need to decommission and remove a customer database? What if we need to move a tenant database at runtime? Can Marten create databases on the fly for us? How will I do that?!? And sorry, my reply is that will require at least an application restart for now, and any kind of fancier management for advanced multi-tenancy will likely go into the forthcoming “Critter Stack Pro” paid model later this year.
That’s part one. The next issue was that for users who also used Marten’s asynchronous projection feature, the “async daemon” subsystem in Marten needed to be able to discover new tenant databases in the background and ensure that all the asynchronous projections for these newly discovered databases are running in the background somewhere in the application. This led to a partial rewrite of the “async daemon” subsystem for Marten 7, but you can see the positive effect of that work in this test that “proves” that Marten is able to spin up projection building agents in the background at runtime:
[Fact]
public async Task add_tenant_database_and_verify_the_daemon_projections_are_running()
{
// In this code block, I'm adding new tenant databases to the system that I
// would expect Marten to discover and start up an asynchronous projection
// daemon for all three newly discovered databases
var tenancy = (MasterTableTenancy)theStore.Options.Tenancy;
await tenancy.AddDatabaseRecordAsync("tenant1", tenant1ConnectionString);
await tenancy.AddDatabaseRecordAsync("tenant2", tenant2ConnectionString);
await tenancy.AddDatabaseRecordAsync("tenant3", tenant3ConnectionString);
// This is a new service in Marten specifically to help you interrogate or
// manipulate the state of running asynchronous projections within the current process
var coordinator = _host.Services.GetRequiredService<IProjectionCoordinator>();
var daemon1 = await coordinator.DaemonForDatabase("tenant1");
var daemon2 = await coordinator.DaemonForDatabase("tenant2");
var daemon3 = await coordinator.DaemonForDatabase("tenant3");
// Just proving that the configured projections for the 3 new databases
// are indeed spun up and running after Marten's new daemon coordinator
// "finds" the new databases
await daemon1.WaitForShardToBeRunning("TripCustomName:All", 30.Seconds());
await daemon2.WaitForShardToBeRunning("TripCustomName:All", 30.Seconds());
await daemon3.WaitForShardToBeRunning("TripCustomName:All", 30.Seconds());
}
Now, switching over to Wolverine 2.0 and its contribution to the party, the third part to make this dynamic tenant database discovery work throughout the entire “Critter Stack” was for Wolverine to be able to also discover the new tenant databases at runtime and spin up its “durability agents” for background message scheduling and its transactional inbox/outbox support. It’s admittedly not well at all documented yet, but Wolverine has an internal system for leader election and “agent assignment” between running nodes in an application cluster to distribute work. Wolverine uses this subsystem to distribute the transactional inbox/outbox work for each tenant database across the application cluster. Look for more information on this capability as JasperFx Software will be exploiting this for a different customer engagement this year.
Marten 7.0 was released this week with a bevy of improvements and some important new features. One important area of concentration was a series of changes sponsored by a JasperFx Software client to improve Marten‘s scalability and to achieve more seamless deployments that involve changes to event store projections.
To show off this new work, I’d like to review CQRS command handler from my earlier Building a Critter Stack Application series. In that series, we had a command handler that attempted to apply changes to an existing event stream representing an Incident within our system:
public static async Task Handle(
CategoriseIncident command,
IDocumentSession session,
CancellationToken cancellationToken)
{
// Find the existing state of the referenced Incident
// but also set Marten up for optimistic version checking on
// the incident upon the call to SaveChangesAsync()
var stream = await session
.Events
.FetchForWriting<IncidentDetails>(command.Id, cancellationToken);
// Don't worry, we're going to clean this up later
if (stream.Aggregate == null)
{
throw new ArgumentOutOfRangeException(nameof(command), "Unknown incident id " + command.Id);
}
// We need to validate whether this command actually
// should do anything
if (stream.Aggregate.Category != command.Category)
{
var categorised = new IncidentCategorised
{
Category = command.Category,
UserId = SystemId
};
stream.AppendOne(categorised);
// This call may throw a ConcurrencyException!
await session.SaveChangesAsync(cancellationToken);
}
}
Mostly, I want to call your attention to this Marten API:
var stream = await session
.Events
.FetchForWriting<IncidentDetails>(command.Id, cancellationToken);
The FetchForWriting() API is an important building block for using Marten inside of a CQRS command handler where you need to quickly fetch the “write model” for an event stream that represents enough of the state of that stream to make decisions about next steps.
This API completely hides away how Marten is deriving that aggregated state as well as setting the command handler up to easily opt into Marten’s optimistic concurrency features. Before 7.0, this API can either be doing “Live” aggregation where Marten fetches the raw events and applies them in memory to create the aggregated state, or uses an “Inline” aggregation where Marten is always updating the projected aggregate at the time that new events are captured.
Both of those strategies give you the all important strong consistency between the raw event data and the aggregated state, but the “Live” mode can be expensive with bigger or longer event streams and the “Inline” mode adds some expense to every single transaction with Marten that involves capturing new events.
With an important assist from the new numeric revisioning strategy, Marten 7.0 now enables users to utilize the asynchronous projection feature in Marten as a potentially much more efficient way to achieve strong consistency between the raw event data and the aggregated state served up by FetchForWriting() API.
In the sample incident tracking, help desk application, we have this projection for calculating the IncidentDetails model:
public class IncidentDetailsProjection: SingleStreamProjection<IncidentDetails>
{
public static IncidentDetails Create(IEvent<IncidentLogged> logged) =>
new(logged.StreamId, logged.Data.CustomerId, IncidentStatus.Pending, Array.Empty<IncidentNote>());
public IncidentDetails Apply(IncidentCategorised categorised, IncidentDetails current) =>
current with { Category = categorised.Category };
public IncidentDetails Apply(IncidentPrioritised prioritised, IncidentDetails current) =>
current with { Priority = prioritised.Priority };
public IncidentDetails Apply(AgentAssignedToIncident prioritised, IncidentDetails current) =>
current with { AgentId = prioritised.AgentId };
public IncidentDetails Apply(IncidentResolved resolved, IncidentDetails current) =>
current with { Status = IncidentStatus.Resolved };
public IncidentDetails Apply(ResolutionAcknowledgedByCustomer acknowledged, IncidentDetails current) =>
current with { Status = IncidentStatus.ResolutionAcknowledgedByCustomer };
public IncidentDetails Apply(IncidentClosed closed, IncidentDetails current) =>
current with { Status = IncidentStatus.Closed };
}
Now, we have these three options for the projection registration that are all going to be supported by the FetchForWriting() API:
builder.Services.AddMarten(opts =>
{
// Do note that the three options below are all mutually
// exclusive
// Pre 7.0, our options were to either run the projection inline like this:
opts.Projections.Add<IncidentDetailsProjection>(ProjectionLifecycle.Inline);
// or rely on in memory calculation like:
opts.Projections.Add<IncidentDetailsProjection>(ProjectionLifecycle.Live);
// NOW though, with 7.0, we have this option:
opts.Projections.Add<IncidentDetailsProjection>(ProjectionLifecycle.Async);
// other configuration
})
Running the “write model” aggregation of the IncidentDetails projection gives us some immediate benefits:
There’s less work going on at the time of writing new event data to an incident stream if we don’t also have to do the work of updating the IncidentDetails model. That may make our system more responsive to users or outside clients of our application’s services
At the time that FetchForWriting() is called, Marten starts with whatever the current persisted state of the IncidentDetails aggregate document that is saved in the database, and apply any newer events on top of the persisted state for a strongly consistent model with the raw events
There’s some significant batching happening in the asynchronous daemon process that can be more efficient overall than the “Inline” projection calculation can be. That could be considerable for systems that need to capture events against multiple streams in the same transaction (that came up very recently from a user in our Discord room)
As I’m about to show next, this “Async” aggregation gives Marten the ability to allow for zero downtime deployments even if our IncidentDetails projection changes. It even allows for the ability to do blue/green deployments where the newer and older versions of IncidentDetails can coexist at runtime on the same database before the older version is completely retired!
I would like to point out that the mechanism for FetchForWriting() with asynchronous projections is batching up the necessary database queries in one round trip. I would like to argue that this is indicative of our attention to detail with Marten and a sign that Marten represents some serious minded engineering that you can feel good about as a technical dependency for important systems.
In the course of our system, someone up high decides we need to track whether issues are being duplicated. Maybe we add a new event like so:
public record IncidentDuplicated(Guid OtherId);
And update our projection to use this new event like so:
public IncidentDetails Apply(IncidentDuplicated duplicated, IncidentDetails current) =>
current with { DuplicateIds = current.DuplicateIds.Concat(new[] { duplicated.OtherId }).ToArray() };
To make this sample make more sense, let’s say that there were also some breaking changes to the IncidentDetails projection that made it incompatible with the old projected model.
Now, we’d like to utilize this new version of the projection immediately — but, wait! The existing system already has the old projection model persisted in database tables! If you used the “Inline” lifecycle, you would have to shut down the system, and rebuild the projection from scratch to replace the persisted data.
Now though, with Marten 7.0, we can instead make one more change to our projection and add this line to mark it as version 2:
public IncidentDetailsProjection()
{
ProjectionVersion = 2;
}
And finally, we’ll deploy our new version of the application — but because we’re being fancy, we’ll deploy in a blue/green way to only some of the application nodes while the older version of the application runs on other nodes.
When the application starts up with the new version of IncidentDetails, Marten will (this can vary based on your configuration, but let’s assume defaults for now):
Treat the V2 version of IncidentDetailsProjection as being completely separate from the original version. This means that Marten will create parallel tables for the V2 version of IncidentDetails and start calculating the IncidentDetailsProjection V2 from the starting position of the event store in the background (this is worth a much longer conversation later). Marten pulls this off by just adding an “_#” suffix to the tables and functions for the document where “#” is the version number if the version is greater than 1.
When FetchForWriting() is called on an individual stream, it might be effectively doing a “Live” aggregation until the projection catches up in the background with the “high water mark” of the event store, but at least you have zero required downtime for your IncidentDetails “write model.”
Marten’s async daemon subsystem got some spiffy improvements for V7, so now when you use the HotCold mode, the daemon just makes sure that every single projection for every individual database is running on exactly one node at a time, so the newly revisioned projection will be started up and actively run on one of the nodes that is running the latest revision of the system.
In the older version of the application still running on some nodes, Marten can happily just skip the “unknown” IncidentDuplicated events when it encounters them in the running asynchronous projections there rather than failing with repetitive exceptions.
Summary
Hey, this was a humongous set of improvements that I through out pretty quickly. All told, I’d summarize this by saying that Marten V7:
Provides potentially better system throughput and scalability
Allows for zero downtime deployments when a “write model” projection is revisioned (we’ve still got some work to do for multi-stream projections)
Even manages to support blue/green deployments for much more flexibility in promoting changes to your systems that use Marten
A new feature in the big Marten 7.0 release this weekend is an alternative to add numeric revisions to a document as a way of enforcing optimistic concurrency checks.
First off, from Martin Fowler’s seminal Patterns of Enterprise Application Architecture (which is just visible to me on my book case across the room as I write this even though it’s 20 years old now), an Optimistic Offline Lock is:
Prevents conflicts between concurrent business transactions by detecting a conflict and rolling back the transaction.
David Rice
In a simple usage, let’s say we’re building some kind of system to make reservations for restaurants. Logically, we’d have a document named Reservation, and we’ve decided that we want to use the numeric revisioning on this document. That document type could look something like this:
// By implementing the IRevisioned
// interface, we're telling Marten to
// use numeric revisioning with this
// document type and keep the version number
// on the Version property
public class Reservation: IRevisioned
{
public Guid Id { get; set; }
// other properties
public int Version { get; set; }
}
Now, let’s see this in action just a little bit:
public static async Task try_revisioning(IDocumentSession session, Reservation reservation)
{
// This will create a new document with Version = 1
session.Insert(reservation);
// "Store" is an upsert, but if the revisioned document
// is all new, the Version = 1 after changes are committed
session.Store(reservation);
// If Store() is called on an existing document
// this will just assign the next revision
session.Store(reservation);
// *This* operation will enforce the optimistic concurrency
// The supplied revision number should be the *new* revision number,
// but will be rejected with a ConcurrencyException when SaveChanges() is
// called if the version
// in the database is equal or greater than the supplied revision
session.UpdateRevision(reservation, 3);
// This operation will update the document if the supplied revision
// number is greater than the known database version when
// SaveChanges() is called, but will do nothing if the known database
// version is equal to or greater than the supplied revision
session.TryUpdateRevision(reservation, 3);
// Any checks happen only here
await session.SaveChangesAsync();
}
Summary
In the end, this is another alternative to the older Guid based version tracking that Marten has supported since 1.0. I don’t know about you, but I can certainly read and understand an integer much more easily than a random string of letters, numbers, and dashes.
In reality though, this feature was specifically built as a prerequisite to some serious improvements to the asynchronous projection support in Marten. Time and ambition permitting, the next Marten 7.0 blog post will show how Marten can support the strongly consistent “write model” projections you need for command processing while also being performant and allowing for zero downtime projection rebuilds.
Marten 7.0 is released to the wild as of right now! Before getting into the highlights of what’s improved in this release, let’s go right to thanking some of the folks who made big contributions to this release either through code, testing, or feedback:
Vedran Zakanj for sponsoring Marten and also contributing ideas around schema management
And a huge thanks to Lucas Wyland for specifically sponsoring the improved LINQ provider work with an equally huge apology from me on how long that took to finish
And to many more community members who helped improve Marten throughout this very long release cycle.
Highlights
This was a huge release, if not nearly as disruptive as Marten 4 was several years ago. I do not anticipate a lot of issues for users upgrading from Marten 6 to Marten 7, but see the migration guide for more details.
The highlights of Marten 7 are:
The LINQ query support was given a large overhaul that both expanded its supported use cases and led to significantly improved performance of many common sub collection queries — which has been a large complaint and request for improvement from the Marten community for several years
A “Partial” document update capability using native PostgreSQL functionality with no JavaScript in sight! That’s been a long requested capability.
The very basic database execution pipeline underneath Marten was largely rewritten to be far more parsimonious with how it uses database connections and to take advantage of more efficient Npgsql usage. We think these changes will make Marten both more efficient overall (these changes reduced the number of object allocations by quite a bit) and help system health through using fewer database connections
We introduced Polly for resiliency to transient errors like network hiccups or a temporarily overloaded database and actually made Marten able to properly execute retries of database writes and database reads
The “async daemon” subsystem was somewhat rewritten with substantial improvements for application scalability. The asynchronous projection support also has an all new scheme for resiliency that we think will be a big improvement for our users
An option to utilize Marten’s recommended FetchForWriting() API for “write model” aggregation with asynchronous projections. This may sound like a lot of mumbo jumbo, but it’s vital because this enables the next bullet point
The ability to do zero downtime deployments of some projection changes as well as to do blue/green deployments of revisioned projections. Much more on this later this week.
A new alternative for “revisioned” documents with a numeric version as an alternative to Marten’s existing GUID based versioning scheme for optimistic concurrency
We’ll see how big of a deal this turns out to be, but Marten 7 enables the usage of Project Aspire with Marten
As time permits, I will be writing deep dive blog posts on each of the individual bullet points above over the next couple weeks — partially as a way to force the completion of some not perfectly updated documentation!
You can Place a Technical Bet on Marten
There’s frequently an understandable hesitation on the part of software shops to take a bet on an open source tool as a critical piece of their technical infrastructure — and that’s sometimes worse in the .NET ecosystem where OSS adoption isn’t as widespread. All that aside, I’m here to tell you that you can feel safe making a large technical bet on Marten because:
Marten is already a very mature project that has been in production usage since its 1.0 release in 2016
While Marten doesn’t have every single issue around production support, deployments, and schema management fixed yet, we’ve got a detailed roadmap to shore up any remaining weaknesses of the tool and we’re in this for the long haul!
PostgreSQL itself is a very successful open source project that continuously innovates and provides a very solid technical foundation for Marten itself
We’ve invested a lot of time into refining Marten’s usability over the years and we think that attention to detail shines through
JasperFx Software offers support contracts and consulting work for Marten users
In conjunction with Wolverine’s integration with Marten, the full “Critter Stack” provides a very efficient and usable stack for Event Driven Architecture using Event Sourcing and a CQRS architecture
While Marten 7.0 made some significant improvements for scalability, the forthcoming “Critter Stack Pro” commercial add on tooling will take Marten to much larger data sets and transactional throughput
Because Marten does target .NET, it’s worth pointing out that at this point, Microsoft has no technical offerings for Event Sourcing and that will absolutely contribute to Marten’s viability
What’s Next and Summary
A lot of big, important, long requested, long planned features and improvements did not make the cut for V7. I blogged last week about the current roadmap for the entire Critter Stack. Moreover, some open bugs didn’t make it into 7.0 as well. And let’s be honest, there’s going to be a slew of bug reports streaming in this week when folks try out new 7.0 features and encounter usage permutations we didn’t anticipate. I’ve finally learned my lesson and made this release after having gotten some rest to be ready for whatever the issues turn out to be in the morning.
Wolverine 2.0 will also follow shortly, but the roadmap for that is pretty well just upgrading to Marten 7, dumping .NET 6, and fixing some low hanging fruit issues and requests before a release in the next couple days.
We’ll jump on whatever those Marten 7 issues turn out to be and all the questions about “what about *my* use case I don’t see on your list!” starting tomorrow, but for right now, this was a huge release filled with all kinds of substantial improvements that for the first time included significant client sponsored requests and please don’t steal my sunshine!
The Shade Tree Developer 