Optimistic Concurrency with Marten

The Marten community has been making substantial progress toward a potential 1.0-alpha release early next week. As part of that effort, I’m going to be blogging about the new changes and features.

Recent versions of Marten (>0.9.5) have a new feature that allows you to enforce offline optimistic concurrency checks against documents that you are attempting to persist. You would use this feature if you’re concerned about a document in your current session having been modified by another session since you originally loaded the document.

I first learned about this concept from Martin Fowler’s PEAA book. From his definition, offline optimistic concurrency:

Prevents conflicts between concurrent business transactions by detecting a conflict and rolling back the transaction.

In Marten’s case, you have to explicitly opt into optimistic versioning for each document type. You can do that with either an attribute on your document type like so:

    [UseOptimisticConcurrency]
    public class CoffeeShop : Shop
    {
        // Guess where I'm at as I code this?
        public string Name { get; set; } = "Starbucks";
    }

Or by using Marten’s configuration API to do it programmatically:

    var store = DocumentStore.For(_ =>
    {
        _.Connection(ConnectionSource.ConnectionString);

        // Configure optimistic concurrency checks
        _.Schema.For<CoffeeShop>().UseOptimisticConcurrency(true);
    });

Once optimistic concurrency is turned on for the CoffeeShop document type, a session will now only be able to update a document if the document has been unchanged in the database since it was initially loaded.

To demonstrate the failure case, consider the following  acceptance test from Marten’s codebase:

[Fact]
public void update_with_stale_version_standard()
{
    var doc1 = new CoffeeShop();
    using (var session = theStore.OpenSession())
    {
        session.Store(doc1);
        session.SaveChanges();
    }

    var session1 = theStore.DirtyTrackedSession();
    var session2 = theStore.DirtyTrackedSession();

    var session1Copy = session1.Load<CoffeeShop>(doc1.Id);
    var session2Copy = session2.Load<CoffeeShop>(doc1.Id);

    try
    {
        session1Copy.Name = "Mozart's";
        session2Copy.Name = "Dominican Joe's";

        // Should go through just fine
        session2.SaveChanges();

        // When session1 tries to save its changes, Marten will detect
        // that the doc1 document has been modified and Marten will
        // throw an AggregateException
        var ex = Exception<AggregateException>.ShouldBeThrownBy(() =>
        {
            session1.SaveChanges();
        });

        // Marten will throw a ConcurrencyException for each document
        // that failed its concurrency check
        var concurrency = ex.InnerExceptions.OfType<ConcurrencyException>().Single();
        concurrency.Id.ShouldBe(doc1.Id);
        concurrency.Message.ShouldBe($"Optimistic concurrency check failed for {typeof(CoffeeShop).FullName} #{doc1.Id}");
    }
    finally
    {
        session1.Dispose();
        session2.Dispose();
    }

    // Just proving that the document was not overwritten
    using (var query = theStore.QuerySession())
    {
        query.Load<CoffeeShop>(doc1.Id).Name.ShouldBe("Dominican Joe's");
    }

}

Marten is throwing an AggregateException for the entire batch of changes being persisted from SaveChanges()/SaveChangesAsync() after rolling back the current database transaction. The individual ConcurrencyException’s inside of the aggregated exception expose information about the actual document type and identity that failed.

7 thoughts on “Optimistic Concurrency with Marten

  1. This is awesome. A couple of questions:

    1) Does this only apply to dirty tracked sessions?

    2) Is there a way to get the version/timestamp of the current document as well as query for documents greater than a specific version (to get all changed documents)?

    1. 1.) No, I only used dirty tracked sessions in that test to cut out the explicit Store() calls. Optimistic concurrency works w/ every possible type of session IdentityMap behavior

      2.) Yeah, I didn’t show it out of laziness. `IDocumentStore.Advanced.MetadataFor(T doc)` will look up the version & last changed timestamp. Had to look, but we do need to add something to find the version according to the current IDocumentSession

  2. Jeremy, thanks for your article.

    I am using Marten DB and so far is working pretty well. The new requirement is to handle 100 TPS with Marten DB but, when I throw so many concurrent requests to Marten, when updating projections, I get hundreds of ConcurrencyException exceptions. I have exponential backoff to retry on that exception but, with that amount of concurrent requests, the system cannot keep up and the requirements are not met.

    I have scaled up the database server to the highest configuration you can have in AWS and that helps but that’s not enough.

    What do you recommend to use in this scenario?
    Have you Load tested Marten? Do you have the results somewhere?

    1. Hey, could you ask these kinds of questions on Discord rather than here? https://discord.gg/jWbV73zjSh

      I’d have to know a little more about your setup and configuration to answer anything definitive, but let me start by saying that I don’t think this is Marten’s fault per se. If you’re getting the Marten `ConcurrencyException`, it’s not because Marten is failing. I think you need a bit different design or opt into async projections at a guess.

Leave a comment