Category Archives: Uncategorized

Marten 2.10.0 and 3.0.0-alpha-2 are released.

The latest set of changes to Marten were just published as Marten 2.10.0 and Marten 3.0.0-alpha-2, with the only real difference being that the 3.0 alpha supports Npgsql 4. The complete list of changes is here. The latest docs have also been published. I counted 15 users total (not including me) in this release as either contributors or just folks who took the time to write up an actionable bug report (and don’t minimize the importance of that because it helps a ton). Thank you to all the Marten contributors and all the folks who answer questions in the Gitter room.

Do note that I quietly and unilaterally exercised my rights as the project’s benevolent dictator to finally remove Marten’s targeting for Netstandard1.* as it was annoying the crap out of me and I didn’t think many folks would care. As of 2.10.0, Marten targets net46 and netstandard2.0.

I hope that more serious work on the 3.0 release starts up again soon, but we’ll see.

Advertisements

OSS Plans for 2018 at the Halfway Point

This might be just navel gazing, but I do have some announcements in here too. 

I’ve made some remarks in the past year that I wanted to drastically scale back my OSS work after I finished the big things that are in flight. I think I’ve changed my mind to just finding a sustainable pace and continuing on. What I’ve realized after getting over some creeping burnout is that:

  • It’s frequently more challenging than ordinary dev work and it helps to keep my skillset sharp now that I’m officially a “real architect” again
  • I get to interact with a lot of smart folks and learn about a lot of other kinds of development approaches and projects. That’s pretty key for me because I haven’t been able to do many development conferences or events the past several years
  • The OSS projects are the one professional thing I get to have control over
  • I just enjoy doing it

I wrote a post in January trying to lay out for myself what I hoped or planned to do this year in the various OSS projects I run. At the halfway point of the year, I just wanna do a little update and see for myself where and how plans have changed.

  • Jasper – The project doesn’t have any real adoption yet, but I think it’s rounding into shape with some compelling features and qualities for microservice development in .Net and reliable messaging. I should have a v0.9 release out shortly with quite a few improvements that went in this spring before I changed jobs in May. Jasper will hopefully hit 1.0 before the end of the year. In terms of development work remaining, I might play around with flushing out the FubuMVC-style HTTP service routing as an alternative to MVC. I also hope to add quite a bit of integration with Jasper to Azure technologies as a learning exercise for myself to finally get some cloud computing skills and join the modern world.
  • Lamar (was BlueMilk) – Lamar 1.0 was released a couple weeks ago and I’ve had plenty of early user feedback that’s already led to some fixes. I released Lamar 1.1 yesterday that specifically targets the “cold start” time for applications using Lamar. This release especially improves the start time for ASP.Net Core applications using Lamar
  • Marten – Of all the things I work on, Marten has the most adoption and the biggest community by far, so I’ve mostly just tried to keep my head above water taking in pull requests and doing occasional bug fixes. If this week goes well, I’ll be starting preliminary work on a Marten 3.0 release for later this year to try to clear out the backlog of feature requests. My goal for Marten this year is to somehow whittle the open GitHub issue list down to a single page and keep it there.
  • Storyteller – Storyteller is the one project of mine that I actually get to regularly use.  I’m hoping to use it extensively on a project at work for Behavior Driven Development type specifications with our client’s business logic. I released a v5 earlier this year that just streamlined the command line usage and getting started story for new projects that target the latest .Net SDK project system. If the year goes well, I’d like to do a rewrite of the specification editor user interface. This is partially to add some missing functionality Storyteller needs to be truly usable by non-developers, but mostly to give me a chance to level up in the latest Javascript tools.
  • AlbaAlba is a little library that helps you write HTTP contract tests against ASP.Net Core applications. The obvious comparison in .Net is to TestServer in ASP.Net Core, but I’d say it’s more like Pact from Ruby or PlaySpecification in Scala. I just released Alba.AspNetCore2 v1.4.1 that made Alba work with ASP.Net Core 2.1. I might get into Alba this year and put it on top of TestServer and call it a big bag of extensions for TestServer instead of being its own thing (even though they barely overlap in functionality).
  • Oakton – Andy Dote made a pull request for asynchronous commands that formed the basis for the Oakton 1.5.0 release. At this point, I think Oakton is essentially done, and you’ll find it underneath several of the projects in the rest of this list
  • StructureMap — All I’m doing at this point is trying to answer user questions as they come in, and that flow has slowed way down. I’m asking folks to consider moving to Lamar or another IoC container for newer development.

Planning the Next Couple Marten Releases

I think we’ve got things lined up for a Marten 2.9 release in the next week with these highlights:

  • Fixes for netcoreapp2.1. A user found some issues with the Linq support so here we go.
  • The upgrade to Npgsql 4.0
  • Eliminating Netstandard 1.3 support. Marten will continue to target .Net 4.6 & Netstandard 2.0, and we’ll be running tests for net46, netcoreapp2.0, and netcoreapp2.1.
  • Some improvements on exception messages in the event store support
  • Upgrades to Newtonsoft.Json — but if the performance isn’t better than the baseline, I’ll leave it where it is

Brainstorming What’s Next

I’ve been admittedly putting  most of my OSS energy into other projects (Jasper/Lamar) for the past year, but the Marten community has been happily proceeding anyway. One of my goals for OSS work this year is to get the Marten issue list count (bugs, questions, and feature ideas) under 25 and keep it there so it’s only a single page in the GitHub website.

We’ve got a big backlog of issues and feature requests. For a potential Marten 2.9, I’d like to suggest:

That’s all for the document store, so now let’s talk about the event sourcing. I don’t have a huge amount of ideas because I don’t get to use it on real projects (yet), but here’s what I think just from responding to user problems:

  • The async daemon is up for some love, and I think I’ve learned some things from Jasper work that will be beneficial back in Marten. First, I think we can finally get a model where Marten itself can handle distributing the ownership of the projections between multiple application nodes so you can more easily deploy the async daemon. After a tip from one of my new colleagues at Calavista, I also want to pursue using “tombstone” events as placeholders on event capture failures to make the async daemon potentially quite a bit more efficient at runtime.
  • I think this would be an add on, but I have an idea for an alternative to ViewProjection that would let you write your projection as just methods on a class that take in any combination of the actual event, the IDocumentSession, the containing EventStream, or Event<T> metadata, then use Lamar to do its thing to generate the actual adapter to Marten’s projection model. I think this could end up being more efficient than ViewProjection is now at runtime, and certainly lead to cleaner code by eliminating the nested lambdas.
  • Event Store partitioning?
  • Event Store metadata extensions?

EDIT 6/8/2018: We had some discussions about this list on Gitter and I definitely forgot some big things here:

  • The async daemon and projections in general needs to be multi-tenancy friendly
  • Other users are interested in the multi-tenancy via database per tenant that got cut from 2.0 at the last minute

Any thoughts about any of this? Other requests? Comments are open, or head to Gitter to join the conversation about Marten.

 

Marten 3.0 with Sql Server Support????

I want to hold the line that there won’t be any kind of huge Marten 3.0 until there’s enough improvements to the JSON support in Sql Server vNext to justify an effort to finally make Marten database engine neutral. I’m also hoping that Marten 3.0 could completely switch all the Linq support to using queries based on SQL/Json too. See this blog post for why Marten does not yet support Sql Server.

Compiling Code At Runtime with Lamar – Part 2

In my previous post, I introduced Lamar‘s ability and utilities for generating code in memory and compiling that code into usable, in memory assemblies with the Roslyn compiler. In this post I’m going to try to explain the more complicated “frames” model with Lamar that provides the backbone for the related Jasper framework’s middleware and runtime pipeline strategy.

The purpose of the “frames” model is to be able to generate dynamic methods by declaring a list of logical operations in generated code via Frame objects, then let Lamar handle:

  • Finding any missing dependencies of those frames
  • Determine what the necessary variable inputs are
  • Ordering all the frames based on dependency order just prior to generating the code

Before diving into an example, here’s a little class diagram of the main types:

LamarCodeGenClassDiagram

The various types represent:

  • Frame – Named after the StackFrame objects within stack traces in .Net. Models a logical action done in the generated code. Concrete examples in Lamar or Jasper would be calling a method on an object, calling a constructor function, or specific frame objects to create wrapped transaction boundaries or exception handling boundaries.
  • Variable – pretty well what it sounds like. This type models a variable within the generated method, but also includes information about what Frame created it to help order the dependencies
  • IVariableSource – mechanism to “find” or create variables. Examples in Lamar include resolving a service from an IoC container, passing along a method argument, or the example below that tells you the current time
  • IMethodVariables – interface that is used by Frame classes to go find their necessary Variable dependencies.

Alrighty then, let’s make this concrete. Let’s say that we want to generate and use dynamic instances of this interface:

public interface ISaySomething
{
    void Speak();
}

Moreover, I want a version of ISaySomething that will call the following method and write the current time to the console:

public static class NowSpeaker
{
    public static void Speak(DateTime now)
    {
        Console.WriteLine(now.ToString("o"));
    }
}

Our dynamic class for ISaySomething will need to pass the current time to the now parameter of that method. To help out here, there’s some built in helpers in Lamar specifically to write in the right code to get the current time to a variable of DateTime or DateTimeOffset that is named “now.”

To skip ahead a little bit, let’s generate a new class and object with the following code:

// Configures the code generation rules
// and policies
var rules = new GenerationRules("GeneratedNamespace");

// Adds the "now : DateTime" variable rule to 
// our generated code
rules.Sources.Add(new NowTimeVariableSource());

// Start the definition for a new generated assembly
var assembly = new GeneratedAssembly(rules);

// Add a new generated type called "WhatTimeIsIt" that will
// implement the 
var type = assembly.AddType("WhatTimeIsIt", typeof(ISaySomething));

// Getting the definition for the method named "Speak"
var method = type.MethodFor(nameof(ISaySomething.Speak));

// Adding a frame that calls the NowSpeaker.Speak() method and
// adding it to the generated method
var @call = new MethodCall(typeof(NowSpeaker), nameof(NowSpeaker.Speak));
method.Frames.Add(@call);

// Compile the new code!
assembly.CompileAll();

After all that, if we interrogate the source code for the generated type above (type.SourceCode), we’d see this ugly generated code:

    public class WhatTimeIsIt : Lamar.Testing.Samples.ISaySomething
    {


        public void Speak()
        {
            var now = System.DateTime.UtcNow;
            Lamar.Testing.Samples.NowSpeaker.Speak(now);
        }

    }

Some notes about the generated code:

  • Lamar was able to stick in some additional code to pass the current time into a new variable, and call the Speak(DateTime now) method with that value.
  • Lamar is smart enough to put that code before the call to the method (that kind of matters here)
  • The generated code uses full type names in almost all cases to avoid type collisions rather than trying to get smart with using statements in the generated code

So now let’s look at how Lamar was able to add the code to pass along DateTime.UtcNow. First off, let’s look at the code that just writes out the date variable:

public class NowFetchFrame : SyncFrame
{
    public NowFetchFrame(Type variableType)
    {
        // there's some sleight of hand here that's marking
        // this new Variable as created by this frame object
        // so that the dependency relationship is made
        Variable = new Variable(variableType, "now", this);
    }
    
    public Variable Variable { get; }
    
    public override void GenerateCode(
        GeneratedMethod method, 
        ISourceWriter writer)
    {
        writer.WriteLine($"var {Variable.Usage} 
            = {Variable.VariableType.FullName}.{nameof(DateTime.UtcNow)};");
        Next?.GenerateCode(method, writer);
    }
}

In the frame above, you’ll see that the GenerateCode() method writes its code into the source, then immediately turns around and tells the next Frame – if there is one – to generated its code. As the last step to write out the new source code, Lamar:

  1. Goes through an effort to find any missing frames and variables
  2. Sorts them with a topological sort (what frames depend on what other frames or variables, what variables are used or created by what frames)
  3. Organizes the frames into a single linked list
  4. Calls GenerateCode() on the first frame

In the generated method up above, the call to NowSpeaker.Speak(now) depends on the now variable which is in turn created by the NowFetchFrame, and that’s enough information for Lamar to order things and generate the final code.

Lastly, we had to use a custom IVariableSource to teach Lamar how to resolve the now variable. That code looks like this:

public class NowTimeVariableSource : IVariableSource
{
    public bool Matches(Type type)
    {
        return type == typeof(DateTime) || type == typeof(DateTimeOffset);
    }

    public Variable Create(Type type)
    {
        if (type == typeof(DateTime))
        {
            return new NowFetchFrame(typeof(DateTime)).Variable;
        }

        if (type == typeof(DateTimeOffset))
        {
            return new NowFetchFrame(typeof(DateTimeOffset)).Variable;
        }

        throw new ArgumentOutOfRangeException(nameof(type), "Only DateTime and DateTimeOffset are supported");
    }
}

Out of the box, the Lamar + Jasper combination uses variable sources for:

  • Services from the internal IoC container of the application
  • Method arguments
  • Variables that can be derived from a method argument like HttpContext.Request
  • The “now” convention shown here

Summary

I don’t know how popular this thing is going to be, but it’s powering the dynamic code generation of Jasper’s runtime pipeline and it’s the key to Jasper’s efficiency compared to other .Net tools with similar functionality. The early feedback I got was that this model was very abstract and hard to follow. I’m open to suggestions, but the very nature of needing to do the recursive dependency detection and ordering kind of necessitates a model like this in my opinion.

Compiling Code At Runtime with Lamar – Part 1

This code was originally written and proven out in the related Marten and described in a post titled Using Roslyn for Runtime Code Generation in Marten. This code was ripped out of Marten itself, but it’s happily running now in Lamar a couple years later.

As some of you know, I’ve been working on a new library called Lamar that I mean to be the next generation replacement for the venerable, but creaky StructureMap library. Besides the IoC Container support though, Lamar also provides some tooling and a model to generate and compile code at runtime, then ultimately load and use the newly generated types.

If all you want to do is take some C# code and compile that in memory to a new, in memory assembly, you can use
the Lamar.Compilation.AssemblyGenerator class.

Let’s say that you have a simple interface in your system like this:

    public interface IOperation
    {
        int Calculate(int one, int two);
    }

Next, let’s use AssemblyGenerator to compile code with a custom implementation of IOperation that we’ve generated
in code:

        [Fact]
        public void generate_code_on_the_fly()
        {
            var generator = new AssemblyGenerator();

            // This is necessary for the compilation to succeed
            // It's exactly the equivalent of adding references
            // to your project
            generator.ReferenceAssembly(typeof(Console).Assembly);
            generator.ReferenceAssembly(typeof(IOperation).Assembly);

            // Compile and generate a new .Net Assembly object
            // in memory
            var assembly = generator.Generate(@"
using Lamar.Testing.Samples;

namespace Generated
{
    public class AddOperator : IOperation
    {
        public int Calculate(int one, int two)
        {
            return one + two;
        }
    }
}
");

            // Find the new type we generated up above
            var type = assembly.GetExportedTypes().Single();
            
            // Use Activator.CreateInstance() to build an object
            // instance of our new class, and cast it to the 
            // IOperation interface
            var operation = (IOperation)Activator.CreateInstance(type);

            // Use our new type
            var result = operation.Calculate(1, 2);
         
            result.ShouldBe(3);
        }

There’s only a couple things going on in the code above:

  1. I added an assembly reference for the .Net assembly that holds the IOperation interface
  2. I passed a string to the ​GenerateCode() method, which successfully compiles my code and hands me back a .Net Assembly object
  3. Load the newly generated type from the new Assembly
  4. Use the new IOperation

If you’re not perfectly keen on doing brute force string manipulation to generate your code, you can
also use Lamar’s built in ISourceWriter to generate some of the code for you with
all its code generation utilities:

[Fact]
public void generate_code_on_the_fly_using_source_writer()
{
    var generator = new AssemblyGenerator();

    // This is necessary for the compilation to succeed
    // It's exactly the equivalent of adding references
    // to your project
    generator.ReferenceAssembly(typeof(Console).Assembly);
    generator.ReferenceAssembly(typeof(IOperation).Assembly);


    var assembly = generator.Generate(x =>
    {
        x.Namespace("Generated");
        x.StartClass("AddOperator", typeof(IOperation));
        
        x.Write("BLOCK:public int Calculate(int one, int two)");
        x.Write("return one + two;");
        x.FinishBlock();  // Finish the method
        
        x.FinishBlock();  // Finish the class
        x.FinishBlock();  // Finish the namespace
    });

    var type = assembly.GetExportedTypes().Single();
    var operation = (IOperation)Activator.CreateInstance(type);

    var result = operation.Calculate(1, 2);

    result.ShouldBe(3);
}

In Part 2, I’ll talk about the “frames” model that’s heavily used within Jasper (shown in this post).

Scheduled or Delayed Message Execution in Jasper

This is definitely not a replacement for something like Hangfire, but it’s very handy for what it does. 

Here’s a couple somewhat common scenarios in an event driven or messaging system:

  • Handling a message failed, but with some kind of problem that might be resolved if the message is retried later in a few seconds or maybe even minutes
  • A message is received that starts a long running process of some kind, and you may want to schedule a “timeout” process later that would send an email to users or do something to escalate the process if it has not finished by the scheduled time

For these kinds of use cases, Jasper supports the idea of scheduled execution, where messages can be sent with a logical “execute this message at this later time.”

Retry Later in Error Handling

The first usage of scheduled execution is in message handling error policies. Take this example below where I tell Jasper to retry handling an incoming message that fails with a TimeoutException again after a 5 second delay:

public class GlobalRetryApp : JasperRegistry
{
    public GlobalRetryApp()
    {
        Handlers
            .OnException<TimeoutException>()
            .RetryLater(5.Seconds());
    }
}

Behind the scenes, the usage of the RetryLater()method causes Jasper to schedule the incoming message for later execution if that error policy kicks in during processing.

Scheduling Messages Locally

To schedule a message to be processed by the current system at a later time, just use the IMessageContext.Schedule() method as shown below:

public async Task schedule_locally(IMessageContext context, Guid issueId)
{
    var timeout = new WarnIfIssueIsStale
    {
        IssueId = issueId
    };

    // Process the issue timeout logic 3 days from now
    // in *this* system
    await context.Schedule(timeout, 3.Days());
}

This method allows you to either express an exact time or to use a TimeSpan argument for delayed scheduling. Either way, Jasper ultimately stores the scheduled message against a UTC time. IMessageContext is the main service in Jasper for sending and executing messages or commands. It will be registered in your IoC container for any Jasper application.

Sending Scheduled Messages to Other Systems

To send a message to another system that should wait to execute the message on its end, use this syntax:

public async Task send_at_5_tomorrow_afternoon_yourself(IMessageContext context, Guid issueId)
{
    var timeout = new WarnIfIssueIsStale
    {
        IssueId = issueId
    };

    var time = DateTime.Today.AddDays(1).AddHours(17);


    // Process the issue timeout at 5PM tomorrow
    // Do note that Jasper quietly converts this
    // to universal time in storage
    await context.Send(timeout, e => e.ExecutionTime = time);
}

Do note that Jasper immediately sends the message. For right now, the thinking is that the receiving application will be responsible for handling the execution scheduling. We may choose later to make this be the responsibility of the sending application instead to be more usable when sending messages to other systems that aren’t using Jasper.

Persistent Job Scheduling

The default, in memory message execution scheduling is probably good enough merely for delayed message processing retries as shown above, However, if you’re using the scheduled execution as part of your business workflow, you probably want to be using the durable message persistence. Today your two options are to use either Postgresql with Marten, or the new Sql Server backed message persistence.

With durable messaging, the scheduled messages are persisted to a backing database so they aren’t lost if any particular node is shut down or the whole system somehow crashes. Behind the scenes, Jasper just uses polling to check the database for any scheduled messages that are ready to execute, and pulls these expired messages into the local worker queues of a running node for normal execution.

The scheduled messages can be processed by any of the running nodes within your system, but we take some steps to guarantee that only one node will execute specific scheduled messages. Rather than using any kind of leader election, Jasper just takes advantage of advisory locks in Postgresql or application locks in Sql Server as a lightweight, global lock between the running nodes within your application. It’s a much simpler (read, less effort on my time) mechanism than trying to do some kind of leader election between running nodes. It also allows Jasper to better spread the work across all of the nodes.

 

An Alternative Style of Routing for ASP.Net Core

Jasper started as an idea to build a far better, next generation version of an older framework called FubuMVC. FubuMVC was primarily a web framework meant to be an alternative to ASP.Net MVC that later got a service bus feature bolted on the side. My previous employer mostly wanted and needed the asynchronous messaging support, so that’s what most of the focus has been within Jasper so far. However, it does have some preliminary support for building HTTP APIs as well and that’s the subject of this post.

From the Carter (“Carter” is to NancyFx as Jasper is to FubuMVC) website:

Carter is a library that allows Nancy-esque routing for use with ASP.Net Core.

Cool, so in the ASP.Net Core space you at least have the main MVC Core framework and its style of implementing HTTP routes and the inevitable Sinatra inspired alternative. In the same vein, Jasper provides yet another alternative to implement routing on ASP.Net Core with a much improved version of what we did years ago with FubuMVC.

A Quick Example of Jasper Routing

Before I get into too many details, here’s a concrete example of a Jasper route handler from its testing library that allows you to post a JSON message to an endpoint and get a response back as JSON:

    public class NumbersEndpoint
    {
        // This action would respond to the url "POST: /sum"
        public static SumValue post_sum(SomeNumbers input)
        {
            return new SumValue{Sum = input.X + input.Y};
        }
    }

So right off the bat, you can probably see that Jasper still uses (by default) convention over configuration to derive the Url pattern attached to this method. I think most folks would react to that code and approach in one of a couple ways:

  1. OMG, that’s too much magic, and magic in code is evil! All code must be “simple” and explicit, no matter how much repetitive cruft and code ceremony I have to type and then later wade through to understand the code. You probably won’t care for Jasper, and probably don’t mind the repetitive base class declarations, attribute usage, and IActionResult code noise that bugs me when I use or have to read MVC Core code. There’s room in the world for both of us;-)
  2. That looks simple, clean, and I bet it’s easy to test. Tell me more!

So if you fell into the second group, or are at least open minded enough to learn a little more, let’s move on to some of the mechanics of defining routes and implementing handlers.

 

Discovering Routes within an Application

While Sinatra-inspired web frameworks like like Express.js tend to want you to explicitly register routes and their handlers, most .Net web frameworks I’m familiar with use some kind of type scanning to find candidate routes from the public types and methods exposed in your application’s main assembly. Jasper is no different in that it looks inside your application’s assembly (either the assembly containing your JasperRegistry or the main console application that bootstrapped Jasper) for concrete, public classes that are named with the suffix “Endpoint” or “Endpoints.” Within those types, Jasper looks for any public method on those types whose name begins with a supported HTTP method name (GET, POST, PUT, DELETE, or HEAD for the moment).

Do note that the endpoint methods can be either instance or static methods, as long as they are public and match the naming criteria.

Here’s an example:

    public class SomeEndpoints
    {
        // Responds to GET: /something
        public string get_something()
        {
            return "Something";
        }

        // Responds to POST: /something
        public string post_something()
        {
            return "You posted something";
        }
    }

Jasper does the type discovery using Lamar’s type scanning, which in turn is pretty well taken straight up from StructureMap 4.*’s type scanning, which in its turn was specifically designed to improve the cold start time in the last version of FubuMVC and provides the same benefit for Jasper (it does some pre-sorting of types and some parallelization that helps quite a bit when you have multiple type scanning conventions happening in the same application). This isn’t my first rodeo.

This discovery is somewhat customizable, but this time around I’m asking users to just use the minimal default conventions instead of making Jasper crazy configurable like FubuMVC was to its and its user’s detriment.

Defining Route Patterns

I didn’t realize this is missing until writing this post, but FubuMVC also supported attributes for explicitly defining or overriding route patterns. This just hasn’t quite made it to Jasper yet, but would have to for the inevitable exception cases.

First off, Jasper has a special naming convention for the root (“/”) url of your application. If an endpoint class is called HomeEndpoint or ServiceEndpoint (it’s up to your preference, but I’d advise you to only use one or the other), the route methods are just derived by the matching HTTP method names, like this:

    public class HomeEndpoint
    {
        // Responds to GET: /
        public string Index()
        {
            return "Hello, world";
        }

        // Responds to GET: /
        public string Get()
        {
            return "Hello, world";
        }

        // Responds to PUT: /
        public void Put()
        {
        }

        // Responds to DELETE: /
        public void Delete()
        {
        }
    }

The Index() method is a synonym for “GET” that was a convention in FubuMVC that I kept in Jasper.

For other endpoint action methods, the route is completely derived from the method name, where the method name would follow a pattern like this:

[http method name]_[segment1]_[segment2]_[segment3]

Roughly speaking, the underscore characters denote a “/” forward slash that separates segments in your route. The first segment is the HTTP method — and each action can only legally respond to a single HTTP method.

So a method with the signature post_api_v1_invoice() would respond to the route “POST: /api/v1/invoice.”

Cool, but now you’re probably asking, how do you pass in route arguments? That’s also a naming convention. Consider the method signature get_api_v1_invoice_id(Guid id). Jasper is looking for segments that match an argument to the method and it knows that those segments are really route parameters. By that logic, that method above responds to the route pattern “GET /api/v1/invoice/{id}.” At runtime when this route is matched, the last segment would be parsed to a Guid and the value passed to the method argument.

As of now, route arguments can be most of the primitive types you’d expect:

  • string
  • Guid
  • int/long/double
  • bool
  • enumerations
  • DateTime/DateTimeOffset

Jasper’s routing forces some limitations on you compared to ASP.Net Core’s Routing module. There’s no kind of route constraint other than full string segments and HTTP methods. I’m making a conscious tradeoff here in favor of performance versus the greater flexibility of the ASP.Net Core routing, with the additional upside that Jasper’s routing selection logic is far, far simpler than what the ASP.Net team did for MVC.

So what do I think are the advantages of this approach over some of the other alternatives in .Net land?

  • It’s really easy to navigate to the route handler from a Url. It’s not terribly difficult to go from a Url in some kind of exception report to using a keyboard shortcut in VS.Net or Rider that lets you navigate quickly to the actual method that handles that url.
  • Even though there’s some repetitiveness in defining all the segments of a route, I like that you can completely derive the route for a method by just the method name. From experience, it was a bit of cognitive load having to remember how to combine a controller type name with attributes and the method name to come up with the route pattern.
  • While using an attribute to define the route is mechanically easy, that’s a little extra code noise in my opinion, and you lose some of the code navigability via the method name when you do that.
  • The code is clean. By this I mean there doesn’t have to be any extraneous code noise from attributes or special return types or base classes getting in your way. Some people hate the magic, but I appreciate the terseness

Moreover, this approach is proven in large FubuMVC applications over a matter of years. I’m happily standing by this approach, even knowing that it’s not going to be for everyone.

Asynchronous Actions

All of the method signatures shown above are very simple and synchronous, but it’s a complicated world and many if not most HTTP endpoints will involve some kind of asynchronous behavior, so let’s look at more advanced usages.

For asynchronous behavior, just return Task or Task like this:

        public Task get_greetings(HttpResponse response)
        {
            response.ContentType = "text/plain";
            return response.WriteAsync("Greetings and salutations!");
        }

Injecting Services as Arguments

You’ll frequently need to get direct access to the HttpContext inside your action methods, and to do that, just take that in as a method argument like this:

        public Task post_values(HttpContext context)
        {
            // do stuff
        }

You can also take in arguments for any property of an HttpContext like HttpRequest or HttpResponse just to slim down your own code like this shown below:

        public Task post_values(HttpRequest request, HttpResponse response)
        {
            // do stuff
        }

Like MVC Core, Jasper supports “method injection” of registered service dependencies to the HTTP methods, but Jasper doesn’t require any kind of explicit attributes. Here’s an elided example from the load testing harness projects in Jasper:

        [MartenTransaction]
        public static async Task post_one(IMessageContext context, IDocumentSession session)
        {
            // Do stuff with the context and session
        }

In the case above, the IMessageContext and IDocumentSession are known service registrations, so they will be passed into the method by Jasper by resolving with Lamar either through generating “poor man’s DI” code if it can, or service location if it has to. See Jasper’s special sauce for a little more description of what Jasper is doing differently here.

Reading and Writing Content

To write content, Jasper keys off the return type of your endpoint action. If your method returns:

  • int or Task — the return value is written to the response status code
  • string or Task — the return value is written to the response with the content type “text/plain”
  • Any other T or Task — the return value is written to the response using Jasper’s support for content negotiation. Out of the box though, the only known content for a given type is JSON serialization using Newtonsoft.Json, so if you do nothing to customize or override that, it’s JSON in and JSON else. I feel like that’s a good default and useful in many cases, so it stays.

To post information Jasper let’s you work directly with strong typed objects while it handles the work of deserializing an HTTP body into your declared input body. Putting that altogether, this method used earlier:

    public class NumbersEndpoint
    {
        public static SumValue post_sum(SomeNumbers input)
        {
            return new SumValue{Sum = input.X + input.Y};
        }
    }

At runtime, Jasper will try to deserialize the posted body of the HTTP request into a SomeNumbers model object that will be passed into the post_sum method above. Likewise, the SumValue object coming out of the method will be serialized and written to the HTTP response as the content type “application/json.”

How Does Jasper Select and Execute the Route at Runtime?

I’m not going to get into too many details, but Jasper uses its own routing mechanism based on the Trie algorithm. I was too intimidated to try something like this in the FubuMVC days and we just used the old ASP.Net routing that’s effectively a table scan search. The new Trie algorithm searching is far more efficient for the way that Jasper uses routing.

Once Jasper’s routing matches a route to the Url of the incoming HTTP request, it can immediately call the corresponding RouteHandler method with this signature:

    public abstract class RouteHandler
    {
        public abstract Task Handle(HttpContext httpContext);

        // other methods
    }

As I explained in my previous post on the Roslyn-powered code weaving, Jasper generates a class at runtime that mediates between the incoming HttpContext and your HTTP endpoint, along with any necessary middleware, content negotiation, route arguments, or service resolution.

Potential Advantages of the Jasper Routing Style

I’m biased here, but I think that the Jasper style of routing and the runtime pipeline has some potentially significant advantages over MVC Core:

  • Cleaner code, and for me, “clean” means the absence of extraneous attributes, marker interfaces, base types, and other custom framework types
  • Less mechanical overhead in the runtime pipeline
  • Better performance
  • Less memory usage through fewer object allocations
  • Cleaner stack traces
  • The generated code will hopefully make Jasper’s behavior much more self-revealing when users start heavily using middleware

Most of this is probably a rehash of my previous post, Roslyn Powered Code Weaving Middleware.

 

 

What’s Already in the Box

Keep in mind this work is in flight, and I honestly haven’t worked on it much in quite awhile. So far Jasper’s HTTP support includes:

  • The routing engine (kind of necessary to make anything else go;))
  • Reverse Url lookup
  • Action discovery
  • Middleware support, both Jasper’s idiomatic version or ASP.Net Core middleware (Jasper’s routing is itself ASP.Net Core middleware)
  • The ability to apply middleware conventionally for cases like “put a transactional middleware on any HTTP action that is a POST, PUT, or DELETE”
  • Some basic content conventions like “an endpoint action that returns a string will be rendered as text/plain at runtime.
  • Basic content negotiation (which shares a lot of code with the messaging side of things)

What’s Missing or Where does This Go Next?

The first question is “whether this goes on?” I haven’t poured a ton of effort into the HTTP handling in Jasper. If there’s not much interest in this or my ambition level just isn’t there, it wouldn’t hurt that much to throw it all away and say Jasper is just an asynchronous messaging framework that can be used with or without ASP.Net Core.

Assuming the HTTP side of Jasper goes on, Jasper gets retagged as a “framework for accelerating microservice development in .Net” and then I think this stuff is next up:

  • Documentation. A conventional framework has a weird tendency of being useless if nobody knows what the conventions are.
  • A modicum of middleware extensions that integrate common tools into Jasper HTTP actions. Offhand, I’ve thought about integrations for IdentityServer and Fluent Validation that add some idiomatic Jasper middleware to be just a little bit more efficient than you’d get from ASP.Net Core middleware.
  • Optimization of the routing. There’s some open issues and known performance “fat” in the routing I’ve just never gotten around to doing
  • Some kind of idiomatic approach in Jasper for branching route handling, i.e. return this status code if the entity exists or redirect to this resource if some condition or return a 403 if the user doesn’t have permission. All of that is perfectly possible today with middleware, but it needs to be something easier for one-off cases.
  • Some level of integration of MVC Core elements within Jasper. I’m thinking you’d definitely want the ability to use IActionResult return types within Jasper in some cases. Even though idiomatic Jasper middleware would be more efficient at runtime, you’d probably want to reuse existing ActionFilters too. It might be valuable to even allow you to use MVC Core Controller classes within Jasper, but still use Jasper’s routing, middleware, and the more efficient IoC integration to achieve what I think will be better performance than by using MVC Core by itself.
  • OpenAPI (Swagger) support. I don’t think this would be too big, and I think I have some ideas about how to allow for general customization of the Swagger documents without forcing users to spray attributes all over the endpoint classes until you can barely see the real code.

What about Razor?

I have zero interest in ever supporting any kind of custom integration of Razor (the view engine support sucked to work on in FubuMVC), and I have it in mind that Razor is pretty tightly coupled to MVC Core’s internals anyway. My thought is that either the IActionResult support gives Jasper Razor support for free, or we just say that you use MVC Core for those pages — and it’s perfectly possible to use both Jasper and MVC Core in the same application. My focus with Jasper is on headless services anyway.