JasperFx Software frequently helps our customers wring better performance or scalability out of our customer’s systems. A somewhat frequent opportunity for improving the responsiveness and throughput of systems is merely identifying ways to batch up requests from middle tier, server side code to the backing database or databases. There’s a certain amount of overhead in making any network round trips between processes, and it often pays off in terms of performance to batch up queries or commands to reduce the number of network round trips.
Today I’m merely going to focus on Marten as a persistence tool and a bit on Wolverine as “Mediator” and show some ways that Marten reduces network round trips. Just know though that this general idea of reducing network round trips by batching up database queries or commands is certainly going to apply to improving performance with any other persistence tooling.
Batching Writes
First off, let’s just look at doing a mixed bag of “writes” with a Marten session to add, delete, or modify user data:
public static async Task modify_some_users(IDocumentSession session)
{
// Mixed bag of document operations
session.Insert(new User{FirstName = "Hans", LastName = "Gruber"});
session.Store(new User{FirstName = "John", LastName = "McClane"});
session.DeleteWhere<User>(x => x.LastName == "Miller");
session.Patch<User>(x => x.LastName == "May").Set(x => x.Nickname, "Mayday");
// Let's append some events too just for fun!
session.Events.StartStream<User>(new UserCreated("Harry", "Ellis"));
// Commit all the changes
await session.SaveChangesAsync();
}
What’s important to note in the code up above is that all the logical operations to insert, “upsert”, delete, patch, or start event streams is batched up into a single database round trip when session.SaveChangesAsync() is called. In the early days of Marten we tried a lot of different things to improve throughput in Marten, including alternative serializers, reducing string concatenation, code generation techniques, and alternative data structures internally. Our consistent finding was that the single biggest improvements always came from reducing network round trips, with alternative JSON serializers being a distant second, and every other factor far behind that.
If you’re curious about the technical underpinnings, Marten 7+ is creating a single NpgsqlBatch for all the commands and even using positional parameters because that’s a touch more efficient for the interaction with PostgreSQL.
Moving to another example, let’s say that you have workflow where you need to apply logical changes to a batch of Item entities using a mix of Marten and Wolverine. Here’s a first, naive cut at this handler:
public static class ApproveItemsHandler
{
// I'm passing in CancellationToken because:
// a. It's probably a good idea anyway
// b. That's how Wolverine "enforces" message timeouts
public static async Task HandleAsync(
ApproveItems message,
IDocumentSession session,
CancellationToken token)
{
foreach (var id in message.Ids)
{
var existing = await session.LoadAsync<Item>(id, token);
if (existing != null)
{
existing.Approved = true;
session.Store(existing);
}
}
await session.SaveChangesAsync(token);
}
}
Now, let’s assume that we could easily be getting 100-1000 different ids of Item entities to approve at any one time, which would make this operation chatty and potentially slow. Let’s make it a little worse though and add in Wolverine as a “mediator” to handle each individual Item inline:
public static class ApproveItemHandler
{
public static async Task HandleAsync(
ApproveItem message,
IDocumentSession session,
CancellationToken token)
{
var existing = await session.LoadAsync<Item>(message.Id, token);
if (existing == null) return;
existing.Approved = true;
await session.SaveChangesAsync(token);
}
}
public static class ApproveItemsHandler
{
// I'm passing in CancellationToken because:
// a. It's probably a good idea anyway
// b. That's how Wolverine "enforces" message timeouts
public static async Task HandleAsync(
ApproveItems message,
IMessageBus bus,
CancellationToken token)
{
foreach (var id in message.Ids)
{
await bus.InvokeAsync(new ApproveItem(id), token);
}
}
}
In terms of performance, the second version is even worse. We compounded the existing chattiness problem with looking up each Item individually by separating out the database “writes” to separate database calls and separate transactions within “Wolverine as Mediator” usage through that InvokeAsync()call. You should be aware that when you use any kind of in process “Mediator” tool like Wolverine, MediatR, Brighter, or MassTransit’s in process mediator functionality that each call to InvokeAsync() involves a certain amount of overhead and very likely means a nested transaction that gets committed independently from the parent message handling or HTTP request that triggered the InvokeAsync() call. I think I might go so far as to say that calling IMessageBus.InvokeAsync() from another message handler is a “guilty until proven innocent” type of approach.
I’d of course argue here that the performance may or may not end up being a big deal, but not having a transactional boundary around the original message processing can easily lead to inconsistent state in our system if any of the individual Item updates fail.
Let’s make one last version of this batch approve item handler with an eye toward reducing network round trips and keeping a strongly consistent transaction boundary around all the approvals (meaning they all succeed or all fail, no in between “who knows what really happened” state):
public static class ApproveItemsHandler
{
// I'm passing in CancellationToken because:
// a. It's probably a good idea anyway
// b. That's how Wolverine "enforces" message timeouts
public static async Task HandleAsync(
ApproveItems message,
IDocumentSession session,
CancellationToken token)
{
// Find all the related items in *one* network round trip
var items = await session.LoadManyAsync<Item>(token, message.Ids);
foreach (var item in items)
{
item.Approved = true;
session.Store(item);
}
await session.SaveChangesAsync().ConfigureAwait(false);
}
}
In the usage above, we’re making one database call to fetch the matching Item entities, and updating all of the impacted Item entities in a single batched database command within the IDocumentSession.SaveChangesAsync(). This version should almost always be much faster than the earlier versions where we issued individual queries for each Item, plus we have better transactional consistency in the case of system errors.
Lastly of course for the sake of completeness, we could just do this with one network round trip:
public static class ApproveItemsHandler
{
// Assuming here that Wolverine "auto-transaction"
// middleware is in place
public static void Handle(
ApproveItems message,
IDocumentSession session)
{
session
.Patch<Item>(x => x.Id.IsOneOf(message.Ids))
.Set(x => x.Approved, true);
}
}
That last version eliminates the usage of current state to validate the operation first or give us any indication of what exactly was changed, but hey, that’s the fastest possible way to code this with Marten and it might be suitable sometimes in your own system.
Batch Querying
Marten has strong support for batch querying where you can combine any number of disparate queries in a batch to the database, and read the results one at a time afterward. Here’s an example from the Marten documentation, but just know that session in this case is a Marten IQuerySession:
// Start a new IBatchQuery from an active session
var batch = session.CreateBatchQuery();
// Fetch a single document by its Id
var user1 = batch.Load<User>("username");
// Fetch multiple documents by their id's
var admins = batch.LoadMany<User>().ById("user2", "user3");
// User-supplied sql
var toms = batch.Query<User>("where first_name == ?", "Tom");
// Where with Linq
var jills = batch.Query<User>().Where(x => x.FirstName == "Jill").ToList();
// Any() queries
var anyBills = batch.Query<User>().Any(x => x.FirstName == "Bill");
// Count() queries
var countJims = batch.Query<User>().Count(x => x.FirstName == "Jim");
// The Batch querying supports First/FirstOrDefault/Single/SingleOrDefault() selectors:
var firstInternal = batch.Query<User>().OrderBy(x => x.LastName).First(x => x.Internal);
// Kick off the batch query
await batch.Execute();
// All of the query mechanisms of the BatchQuery return
// Task's that are completed by the Execute() method above
var internalUser = await firstInternal;
Debug.WriteLine($"The first internal user is {internalUser.FirstName} {internalUser.LastName}");
That’s a little more code and complexity than you might have otherwise if you just make the queries independently, but there’s some significant performance gains to be made from batching queries.
This is a much, much longer discussion than I have ambition for today, but the rampant usage of repository abstractions around raw persistence tooling like Marten has a tendency to knock out more powerful functionality like query batching. That’s especially compounded with “noun-centric” code organization where you may have IOrderRepository and IInvoiceRepository wrapping your raw persistence tooling, but yet frequently have logical operations that deal with both Order and Invoice data at the same time. With Wolverine especially, I’m pushing JasperFx clients and our users to try to get away with eschewing these kinds of abstractions and leaning hard into Wolverine’s “A-Frame Architecture” approach so you can utilize the full power of Marten (or EF Core or RavenDb or whatever else you actually use).
What I can tell you is that for a current JasperFx client, we’re looking in the long run to collapse and simplify and inline their current usage of Railway Programming and MediatR-calling-other-MediatR handlers as a way to enable us to utilize query batching to optimize some of their very complicated operations that today end up being very chatty between the server and database.
Including Related Entities when Querying
There are plenty of times you’ll have an operation in your system that needs information from multiple, related entity types. Marten provides its version of Include() in its LINQ provider as a way to batch query related documents in fewer network round trips, and hence better performance like this example from the tests:
[Fact]
public async Task simple_include_for_a_single_document()
{
var user = new User();
var issue = new Issue { AssigneeId = user.Id, Title = "Garage Door is busted" };
using var session = theStore.IdentitySession();
session.Store<object>(user, issue);
await session.SaveChangesAsync();
using var query = theStore.QuerySession();
// The following query will fetch both the Issue document
// and the related User document for the Issue in one
// network round trip
User included = null;
var issue2 = query
.Query<Issue>()
.Include<User>(x => included = x).On(x => x.AssigneeId)
.Single(x => x.Title == issue.Title);
included.ShouldNotBeNull();
included.Id.ShouldBe(user.Id);
issue2.ShouldNotBeNull();
}
I’ll refer you to the documentation for more alternative usages, but just know that Marten has this capability and it’s a valuable way to improve performance in your system by reducing the number of network roundtrips between your code and the backend.
Marten’s Include() functionality was originally inspired/copied from RavenDb. We’ve unfortunately had some confusion in the past from folks coming over from EF Core where its Include() means something very different. Oh, and just to pull aside the curtain, it’s not doing any kind of JOIN behind the scenes, but a temporary table + multiple SELECT() statements.
Summary
I just wanted to get a handful of things across in this post:
- Network round trips can easily be expensive and a contributing factor in poor system performance. Reducing the number of network round trips by batching queries can sometimes pay off overall even if that sometimes means more complex code
- Marten has several features specifically meant to improve system performance by batching database queries that you can utilize. Both Marten and Wolverine are absolutely built with this philosophy of reducing network round trips as much as possible
- Any coding or architectural strategy that results in excessive layering, long call stacks (A calls B that calls C that calls D that finally calls to a database), or really just obfuscates your understanding of how system operations lead to increased numbers of network round trips can easily be harmful to your system’s performance because you can’t easily “see” what your system is really doing
The Shade Tree Developer 