Planning the Next Couple Marten Releases

I think we’ve got things lined up for a Marten 2.9 release in the next week with these highlights:

• Fixes for netcoreapp2.1. A user found some issues with the Linq support so here we go.
• The upgrade to Npgsql 4.0
• Eliminating Netstandard 1.3 support. Marten will continue to target .Net 4.6 & Netstandard 2.0, and we’ll be running tests for net46, netcoreapp2.0, and netcoreapp2.1.
• Some improvements on exception messages in the event store support
• Upgrades to Newtonsoft.Json — but if the performance isn’t better than the baseline, I’ll leave it where it is

Brainstorming What’s Next

I’ve been admittedly putting  most of my OSS energy into other projects (Jasper/Lamar) for the past year, but the Marten community has been happily proceeding anyway. One of my goals for OSS work this year is to get the Marten issue list count (bugs, questions, and feature ideas) under 25 and keep it there so it’s only a single page in the GitHub website.

We’ve got a big backlog of issues and feature requests. For a potential Marten 2.9, I’d like to suggest:

• Making sure that users can specify database object names (but you’ll still have to use the “mt_” prefix)
• This is structural, but we need to do a bit of an overhaul in our Linq support and how it accesses information inside the JSONB data within different kinds of queries. As a user, you care because this should make the database queries for some common Linq operations faster
• Projected flat tables from documents for reporting support. Marten is never going to be a great solution for reporting and this is a cheap way to side step that issue in my opinion. This is low hanging fruit if someone wants to dip their toes into contributing to Marten.
• Support the Include() feature for fetching related collections of documents. This has been a big hole for a long time, and I think we stumbled on a good way of doing this within Postgresql finally.
• Allow users to do oddball queries and even indexes with Javascript functions. The Linq support is like a neverending arms race where you can’t possibly keep up with everything the user base is going to try to do. As an end around that, I’d like users to be able to express Javascript functions that can interrogate a document for matches and also enable those functions to be used as calculated indexes.
• A model where Marten query results can be piped directly to streams without ever having to go through any kind of “deserialization then serialization” two step process like you’d do if Marten was the backing data store for HTTP services that stream JSON data. This would be HUGE for performance in ASP.Net Core services. I also think it would make Marten a pretty good player in some kind of GraphQL backend. And don’t worry, this model would still allow you to any kind of supported document transform in the database so you wouldn’t be stuck serializing your raw document types to HTTP output.

That’s all for the document store, so now let’s talk about the event sourcing. I don’t have a huge amount of ideas because I don’t get to use it on real projects (yet), but here’s what I think just from responding to user problems:

• The async daemon is up for some love, and I think I’ve learned some things from Jasper work that will be beneficial back in Marten. First, I think we can finally get a model where Marten itself can handle distributing the ownership of the projections between multiple application nodes so you can more easily deploy the async daemon. After a tip from one of my new colleagues at Calavista, I also want to pursue using “tombstone” events as placeholders on event capture failures to make the async daemon potentially quite a bit more efficient at runtime.
• I think this would be an add on, but I have an idea for an alternative to ViewProjection that would let you write your projection as just methods on a class that take in any combination of the actual event, the IDocumentSession, the containing EventStream, or Event<T> metadata, then use Lamar to do its thing to generate the actual adapter to Marten’s projection model. I think this could end up being more efficient than ViewProjection is now at runtime, and certainly lead to cleaner code by eliminating the nested lambdas.
• Event Store partitioning?
• Event Store metadata extensions?

EDIT 6/8/2018: We had some discussions about this list on Gitter and I definitely forgot some big things here:

• The async daemon and projections in general needs to be multi-tenancy friendly
• Other users are interested in the multi-tenancy via database per tenant that got cut from 2.0 at the last minute

Any thoughts about any of this? Other requests? Comments are open, or head to Gitter to join the conversation about Marten.

Marten 3.0 with Sql Server Support????

I want to hold the line that there won’t be any kind of huge Marten 3.0 until there’s enough improvements to the JSON support in Sql Server vNext to justify an effort to finally make Marten database engine neutral. I’m also hoping that Marten 3.0 could completely switch all the Linq support to using queries based on SQL/Json too. See this blog post for why Marten does not yet support Sql Server.