New Improvements to Marten’s LINQ Support

I hold these two facts to be simultaneously true:

1. Language Integrated Query (LINQ) is the singular best feature in .NET that developers would miss out working in other development platforms
2. Developing and supporting a LINQ provider is a nastily hard and laborious task for an OSS author and probably my least favorite area of Marten to work in

Alright, on that note, let’s talk about a couple potentially important recent improvements to Marten’s LINQ support. First, we’ve received the message loud and clear that Marten was sometimes hard when what you really need is to fetch data from more than one document type at a time. We’d also got some feedback about the difficultly overall in making denormalized views projected from events with a mix of different streams and potentially different reference documents.

For projections in the event sourcing space, we added the Composite Projection capability. For straight up document database work, Marten 8.23 introduced support for the LINQ GroupJoin operator as shown in this test from the Marten codebase:

public class JoinCustomer
{
    public Guid Id { get; set; }
    public string Name { get; set; } = "";
    public string City { get; set; } = "";
}
public class JoinOrder
{
    public Guid Id { get; set; }
    public Guid CustomerId { get; set; }
    public string Status { get; set; } = "";
    public decimal Amount { get; set; }
}
[Fact]
public async Task GroupJoin_select_outer_entity_properties()
{
    // It's just setting up some Customer and Order data in the database
    await SetupData();
    var results = await _session.Query<JoinCustomer>()
        .GroupJoin(
            _session.Query<JoinOrder>(),
            c => c.Id,
            o => o.CustomerId,
            (c, orders) => new { c, orders })
        .SelectMany(
            x => x.orders,
            (x, o) => new { x.c.Name, x.c.City })
        .ToListAsync();
    results.Count.ShouldBe(3);
    results.Count(r => r.City == "Seattle").ShouldBe(2); // Alice's 2 orders
    results.Count(r => r.City == "Portland").ShouldBe(1); // Bob's 1 order
}

This is of course brand new, which means there are probably “unknown unknown” bugs in there, but just give us a reproduction in a GitHub issue and we’ll address whatever it is.

Select/Where Hoisting

Without getting into too many details, the giant “hey, let’s rewrite our LINQ support almost from scratch!” effort in Marten V7 a couple years ago made some massive strides in our LINQ provider, but unintentionally “broke” our support for chaining Where clauses *after* Select transforms. To be honest, that’s nothing I even realized you could or would do with LINQ, so I was caught off guard when we got a couple bug reports about that later. No worries now, because you can now do that with Marten as this new test shows:

    [Fact]
    public async Task select_before_where_with_different_type()
    {
        var doc1 = new DocWithInner { Id = Guid.NewGuid(), Name = "one", Inner = new InnerDoc { Value = 10, Text = "low" } };
        var doc2 = new DocWithInner { Id = Guid.NewGuid(), Name = "two", Inner = new InnerDoc { Value = 50, Text = "mid" } };
        var doc3 = new DocWithInner { Id = Guid.NewGuid(), Name = "three", Inner = new InnerDoc { Value = 90, Text = "high" } };

        theSession.Store(doc1, doc2, doc3);
        await theSession.SaveChangesAsync();

        // Select().Where() - the problematic ordering from GH-3009
        var results = await theSession.Query<DocWithInner>()
            .Select(x => x.Inner)
            .Where(x => x.Value > 40)
            .ToListAsync();

        results.Count.ShouldBe(2);
        results.ShouldContain(x => x.Value == 50);
        results.ShouldContain(x => x.Value == 90);
    }

Summary

So you might ask, how did we suddenly get to a point where there are literally no open GitHub issues related to LINQ in the Marten codebase? It turns out that the LINQ provider support is an absolutely perfect place to just let Claude go fix it — but know that that is backed up by about a 1,000 regression tests for LINQ to chew through at the same time.

My limited experience suggests that the AI assisted development really works when you have very well defined and executable acceptance requirements and better yet tests for your AI agent to develop to. Duh.