Select Projections in Marten

When I did the DotNetRocks episode on Marten awhile back, they asked me what I thought the biggest holes in Marten functionality and where we would take it next. The first thing that came to my mind was the “read side.” By that I meant built in functionality to transform the raw documents stored with Marten into the shape needed for API’s, business functionality, and web pages. Fortunately, the very latest versions of Marten add some important functionality to support the Linq Select() keyword for fetching document data with transformations.

See CQRS from Martin Fowler for more context on where and how the “read side” fits into a software architecture.

To make this concrete, let’s say that you only really want one property or field of a stored document. That one field can now be fetched without incurring the cost of deserializing the raw JSON of the whole document. Instead, we’ll just fetch our one field:

        [Fact]
        public void use_select_in_query_for_one_field_and_first()
        {
            theSession.Store(new User { FirstName = "Hank" });
            theSession.Store(new User { FirstName = "Bill" });
            theSession.Store(new User { FirstName = "Sam" });
            theSession.Store(new User { FirstName = "Tom" });

            theSession.SaveChanges();

            theSession.Query<User>().OrderBy(x => x.FirstName).Select(x => x.FirstName)
                .First().ShouldBe("Bill");

        }

Maybe not *that* commonly useful, so what about if you want to select a subset of a large document? If you want to select into a smaller type, you can use code like this:

[Fact]
public void use_select_with_multiple_fields_to_other_type()
{
    theSession.Store(new User { FirstName = "Hank", LastName = "Aaron" });
    theSession.Store(new User { FirstName = "Bill", LastName = "Laimbeer" });
    theSession.Store(new User { FirstName = "Sam", LastName = "Mitchell" });
    theSession.Store(new User { FirstName = "Tom", LastName = "Chambers" });

    theSession.SaveChanges();

    var users = theSession.Query<User>().Select(x => new User2{ First = x.FirstName, Last = x.LastName }).ToList();

    users.Count.ShouldBe(4);

    users.Each(x =>
    {
        x.First.ShouldNotBeNull();
        x.Last.ShouldNotBeNull();
    });
}

In the case above, you are selecting the User document data into another type called “User2.” That’s great of course, but you can also bypass the need for a custom class and just select straight to an anonymous type like this:

[Fact]
public void use_select_with_multiple_fields_in_anonymous()
{
    theSession.Store(new User { FirstName = "Hank", LastName = "Aaron"});
    theSession.Store(new User { FirstName = "Bill", LastName = "Laimbeer"});
    theSession.Store(new User { FirstName = "Sam", LastName = "Mitchell"});
    theSession.Store(new User { FirstName = "Tom", LastName = "Chambers"});

    theSession.SaveChanges();

    var users = theSession.Query<User>().Select(x => new {First = x.FirstName, Last = x.LastName}).ToList();

    users.Count.ShouldBe(4);

    users.Each(x =>
    {
        x.First.ShouldNotBeNull();
        x.Last.ShouldNotBeNull();
    });
}

Finally, if all you need to do is stream the raw JSON data of your transformed documents straight to the HTTP response in a web request, you can skip the unnecessary JSON deserialization and serialization and probably achieve much better throughput in your web application by selecting straight to a JSON string:

[Fact]
public void use_select_to_another_type_and_to_json()
{
    theSession.Store(new User { FirstName = "Hank" });
    theSession.Store(new User { FirstName = "Bill" });
    theSession.Store(new User { FirstName = "Sam" });
    theSession.Store(new User { FirstName = "Tom" });

    theSession.SaveChanges();

    theSession.Query().OrderBy(x => x.FirstName).Select(x => new UserName { Name = x.FirstName })
        .ToListJson()
        .ShouldBe("[{\"Name\" : \"Bill\"},{\"Name\" : \"Hank\"},{\"Name\" : \"Sam\"},{\"Name\" : \"Tom\"}]");
}

In the code above, the JSON of the User document is transformed by Postgresql itself and returned to the caller as a single JSON string.

Not that you necessarily want to do a lot of this by hand (but you always could), the SQL above can be found and printed with this:

    var command = theSession
                .Query()
                .OrderBy(x => x.FirstName)
                .Select(x => new UserName {Name = x.FirstName})
                
                // ToCommand() is a Marten specific extension
                // we use as a diagnostic tool to understand
                // how Marten is treating any given Linq expression
                .ToCommand(FetchType.FetchMany);

    Console.WriteLine(command.CommandText);

That gives us this generated SQL:

select json_build_object('Name', d.data ->> 'FirstName') as json from public.mt_doc_user as d order by d.data ->> 'FirstName'

What’s left to do?

We’ve had a couple bugs with our Select() support from early adopters with permutations I hadn’t thought to test beforehand, so I’m pretty sure that there are more things to iron out. The best way to solve that problem is to just try to get more early users to beat on it and find what’s still missing.

The Select() support today doesn’t yet include any transformations of child collection data or transforming data within a JSON document. We’d probably also want to support selecting data straight out of child collections as well.

The big new thing in our “read side” repertoire is going to be calculated projections. You can follow that work on GitHub.