Long story short, several of my colleagues and I are building a new framework for managing asynchronous messaging between .Net services with the codename “Jasper” as a modernized replacement for our existing FubuMVC based messaging infrastructure. Once I get a good couple weeks of downtime during the holidays, I’ll be making the first big public alpha nuget of Jasper and start blogging up a storm about the project, but for right now, here’s this post about Jasper’s intended dynamic subscription model.

As part of our larger effort at work to move toward a microservices architecture, some of us did a big internal presentation last week about the progress so far in our new messaging infrastructure we intend to use next year as we transition to Netstandard2 and all the modern .Net stuff. I went over my time slot before we could talk about our proposal for how we plan to handle publish/subscribe messaging and service discovery. I promised this post in the hopes of getting some feedback.

The Basic Architecture

Inside of our applications, when we need to publish or send a message to other systems, we would use some code like this:

public Task SendMessage(IServiceBus bus)
{
    // In this case, we're sending an "InvoiceCreated"
    // message
    var @event = new InvoiceCreated
    {
        Time = DateTime.UtcNow,
        Purchaser = "Guy Fieri",
        Amount = 112.34,
        Item = "Cookbook"
    };

    // Mandatory that there is subscribers for this message
    return bus.Send(@event);

    // or send the message to any subscribers for this type
    // of message, but don't enforce the existence of any
    // subscribers
    return bus.Publish(@event);
    
}

The system publishing messages itself doesn’t need to know or even how to send that message to whatever the downstream system is. The infrastructure code underneath bus.Send() knows how to look up any registered subscribers for the “InvoiceCreated” event message in some kind of subscription storage and route the message accordingly.

The basic architecture is shown below:

Just to make this clear, there’s no technical reason why the “subscription storage” has to be a shared resource — and therefore a single point of failure — between the systems.

Now that we’ve got the barebones basics, here are the underlying problems we’re trying to solve to make that diagram above work in real life.

The Problem Statement

Consider a fairly large scale technical ecosystem composed of many services that need to work together by exchanging messages in an asynchronous manner. You’ve got a couple challenges to consider:

• How do the various applications “know” where and how to send or publish messages?
• How do you account for the potential change in message payloads or representations between systems?
• How could you create a living document that accurately describes how information flows between the systems?
• How can you add new systems that need to publish or receive messages without having to do intrusive work on the existing systems to accomodate the new system?
• How might you detect mismatches in capabilities between all the systems without having to just let things blow up in production?

Before I lay out our working theory about how we’ll do all of the above in our development next year, let’s talk about the downsides of doing all of that very badly. When I was still wet behind the ears as a developer, I worked at a very large manufacturing company (one of the “most admired” companies in the US at the time) that had an absolutely wretched “n-squared” problem where integration between lots of applications was done in an ad hoc manner with more or less hard coded, one off mechanisms between applications. Every time we needed a new integration, we had to effectively do all new work and break into all of the applications involved in the new data exchange.

I survived that experience with plenty of scars and I’m sure many of you have similar experiences. To solve this problem going forward, my shop has come up with the “dynamic subscription” model (the docs in that link are a bit out of date) in our forthcoming “Jasper” messaging framework.

Our Proposed Goals and Approach

A lot of this is already built out today, but it’s not in production yet and now’s a perfect time for feedback (hint, hint colleagues of mine). The primary goals in the Jasper approach is to:

• Eliminate any need for a central broker or any other kind of single point of failure
• Reduce direct coupling and between services
• Make it easy to update, browse, or delete subscriptions
• Provide tooling to validate the subscriptions across services
• Allow developers to quickly visualize how messages flow in our ecosystem between senders and receivers

Now, to put this into action. First, each application should declare the messages it needs to subscribe to with code like this:

    public class MyAppRegistry : JasperRegistry
    {
        public MyAppRegistry()
        {
            // Override where you want the incoming messages
            // to be sent.
            Subscribe.At("tcp://server1:2222");

            // Declare which messages this system wants to 
            // receive
            Subscribe.To();
            Subscribe.To();
        }
    }

Note: the “JasperRegistry” type fulfills the same role for configuring a Jasper application as the WebHostBuilder and Startup classes do in ASP.Net Core applications.

When the application configured by the “MyAppRegistry” shown above is bootstrapped, it calculates its subscription requirements that include:

• The logical name of the message type. By default this is derived from the .Net type, but doesn’t have to be. We use the logical name to avoid forcing you to share an assembly with the DTO message types
• All of the representations of the message type that this application understands and can accept. This is done to enable version messaging between our applications and to enable alternative serialization or even custom reading/writing strategies later
• The server address where the upstream applications should send the messages. We’re envisioning that this address will be the load balancer address when we’re hosting on premises in a single data center, or left blank for when we get to jump to cloud hosting when we’ll do some additional work to figure out the network address of the subscriber that’s most appropriate within the data center where the sender is hosted.

At deployment time, whenever a service is going to be updated, our theory is that you’ll have a step in your deployment process that will publish the subscription requirements to the proper subscription storage. If you’re using the built in command line support, and your Jasper application compiles to an executable called “MyApp.exe,” you could use this command line signature to publish subscriptions:

MyApp.exe subscriptions publish

At this point, we’re working under the assumption that we’ll be primarily using Consul as our subscription storage mechanism, but we also have a Postgresql-backed option as well. We think that Consul is a good fit here because of the way that it won’t create a single point of failure while also allowing us to replicate the subscription information between nodes through its own gossip protocol.

Validating Subscriptions

To take this capability farther, Jasper will allow you to declare what messages are published by a system like so:

    public class MyAppRegistry : JasperRegistry
    {
        public MyAppRegistry()
        {
            Publish.Message();
            Publish.Message();
            Publish.Message();
        }
    }

Note: You can also use an attribute directly on the published message types if you prefer, or use a convention for the braver.

When a Jasper application starts up, it combines the declaration of published message types with the known representations or message versions found in the system, and combines that information with the subscription requirements into what Jasper calls the “ServiceCapabilities” model (think of this as Swagger for Jasper messaging, or OpenAPI just in case Darrel Miller ever reads this;)).

Again, if your Jasper application compiles through dotnet publish to a file called “MyApp.exe”, you’ll get a couple more command line functions. First, to dump a JSON representation of the service to a file system folder, you can do this:

MyApp.exe subscriptions export --directory ~/subscriptions

My thinking here was that you could have a Git repository where all the services export their service capabilities at deployment time, because, that would enable you to use this command later:

MyApp.exe subscriptions validate --file subscription-report.json

The command above would read in all the service capability files in that directory, and analyze all the known message publishing and subscriptions to create a report with:

1. All the valid message routes from sender to receiver by message type, the address of the receiver, and the representation or version of the message
2. Any message types that are subscribed to, but not published by any service
3. Message types that are published by one or more services, but not subscribed to by any other system
4. Invalid message routing through mismatches in either accepted or published versions or transport mismatches (like if the sender can send messages only through TCP, but the receiver can only receive via HTTP)

Future Work

I thought we were basically done with this feature, but it’s not in production yet and we did come up with some additional items or changes before we go live:

• Build a subscription control panel that’ll be a web front end that allows you to analyze or even edit subscriptions in the subscription storage
• Publish the full service capabilities to the subscription storage (we only publish the subscriptions today)
• Get a bit more intelligent with Consul and how we would use message routing if running nodes of the named services are hosted in different data centers
• Create the idea of message “ownership,” as in “only this system should be processing this command message type”
• Some kind of cache busting in the running Jasper nodes to refresh the subscription information in memory whenever the subscription storage changes

I’ve been kicking around the idea for a possible resurrection of FubuMVC as a mostly new framework with the codename “Jasper”  for several years with some of my colleagues. This year myself and several members of our architecture team at work have started making that a reality as the centerpiece of our longer term microservices strategy.

In the end, Jasper will be a lightweight service bus for asynchronous messaging, a high performance alternative to MVC for HTTP API’s, and a substitute for MediatR inside of ASP.Net Core applications (those three usages share much more infrastructure code than you might imagine and the whole thing is still going to be much, much smaller than FubuMVC was at the end). For the moment, we’re almost entirely focused on the messaging functionality.

I haven’t kicked out an up to date Nuget yet, but there’s quite a bit of documentation and I’m just hoping to get some feedback out of that right now. If you’re at all interested in Jasper, feel free to raise GitHub issues or join our Gitter room.

The only thing I’m trying to accomplish in this post is to get a sanity check from other folks on whether or not the bootstrapping looks usable.

Getting Started

This is taken directly from the getting started documentation.

dotnet new console -n MyApp

public class MyAppRegistry : JasperRegistry
{
    public MyAppRegistry()
    {
        // Configure or select options in this constructor function
    }
}

using Jasper.CommandLine;

namespace MyApp
{
    class Program
    {
        static int Main(string[] args)
        {
            // This bootstraps and runs the Jasper
            // application as defined by MyAppRegistry
            // until the executable is stopped
            return JasperAgent.Run<MyAppRegistry>(args);
        }
    }
}

public class MyAppRegistry : JasperRegistry
{
    public MyAppRegistry()
    {
        Http.UseKestrel().UseUrls("http://localhost:3001");
        Transports.Lightweight.ListenOnPort(2222);
    }
}

Hosting environment: Production
Content root path: /Users/jeremill/code/jasper/src/MyApp/bin/Debug/netcoreapp1.1
Listening for messages at loopback://delayed/
Listening for messages at jasper://localhost:2333/replies
Listening for messages at jasper://localhost:2222/incoming
Now listening on: http://localhost:3001
Application started. Press Ctrl+C to shut down.

var host = new WebHostBuilder()
    .UseKestrel()
    .UseJasper<ServiceBusApp>()
    .Build();

host.Run();

public class HomeEndpoint
{
    public string Get()
    {
        return "Hello, world.";
    }
}

public class InvoiceCreated
{
    public Guid InvoiceId { get; set; }
}

public class InvoiceHandler
{
    public void Handle(InvoiceCreated created)
    {
        // do something here with the created variable...
    }
}

A couple months ago I blogged a little bit about a yet another OSS service bus project my shop is building out for messaging in .Net Core systems called Jasper that services as a wire compatible successor to the tooling we use in older .Net applications. While it’s already in production systems at work and doing fine, I have no clue if it’ll have any success as an OSS project. At the very least I’m going to squeeze some blog posts out of the process of building it and here we are.

Service bus frameworks are definitely an example of the Hollywood Principle where a framework handles much of the event handling and workflow while delegating to your application specific code through some kind of interface or idiom. In most of the cases I’ve seen over the years in .Net, you’ll see some kind of interface like the one below that allows you to plug your message handlers into your service bus infrastructure:

public interface IHandler
{
    Task Handle(T message);
}

I’ve certainly used this approach in a handful of cases, and there’s even some direct support for auto-registering this kind of service strategy inside of StructureMap if you want to roll your own framework. It’s easy to understand, adds some level of discoverability, and might help guide users. It’s also somewhat limiting in flexibility and the copious usage of generics can easily lead users into some bad places — and I say that partially based on a decade of helping folks with generics on the StructureMap user lists.

Jasper takes a different approach that relies much more on naming conventions and method signatures. To make that concrete, here’s the very simplest form of message handlers you can use in Jasper and if you don’t mind, let me leave how could this possibly work efficiently for a followup post (spoiler alert: Roslyn is awesome):

public class ExampleHandler
{
    public void Handle(Message1 message)
    {
        // Do work synchronously
    }

    public Task Handle(Message2 message)
    {
        // Do work asynchronously
        return Task.CompletedTask;
    }
}

Out of the box, Jasper finds and uses message handling methods by searching for concrete classes whose names are suffixed with either “Handler” or “Consumer” (there’s some historical reasons for having both) and then discovers message handling actions by analyzing the public methods on those classes for message handling candidates.

Right off the bat, you can see that Jasper allows you to write either synchronous or asynchronous methods to handle messages, so no more phantom “return Task.CompletedTask;” lines cluttering up your code.

Moreover, you can wring out a little more performance in your system by using static methods instead:

public static class StaticHandler
{
    public static Task Handle(Message3 message)
    {
        return Task.CompletedTask;
    }
}

It might be advantageous to use this approach to reduce memory allocations at run time and should give you slightly more efficient IL. Of course, any handler method, static or otherwise, isn’t terribly helpful unless you can get at the services within your application that you’ll need to invoke to process the message.

To that end Jasper gives you a couple possibilities. First, you can do the idiomatic, constructor injection approach like this:

public class ServiceUsingHandler
{
    private readonly IService _service;

    public ServiceUsingHandler(IService service)
    {
        _service = service;
    }

    public void Handle(Message1 message)
    {
        // do something with _service to handle this thing
    }
}

At the moment, Jasper would revert to spinning up a StructureMap nested container and uses that to build out the ServiceUsingHandler objects something like this:

// _root is a reference to the application's root
// container
using (var nested = _root.GetNestedContainer())
{
    var serviceUsingHandler = nested.GetInstance();
    var message1 = (Message1)context.Envelope.Message;
    serviceUsingHandler.Handle(message1, widget);
}

Using that approach enables Jasper to build objects of your handler classes with whatever dependencies you would need. Alternatively, you can also use “method injection” in your handlers like this:

public void Handle(
    Message2 message, 
    IService service, 
    Envelope envelope
)
{
    // handle the message
}

In the example above, Jasper “knows” how to resolve both the IService and Envelope dependencies before calling into the Handle() method. The Envelope object is Jasper’s version of an envelope wrapper that gives you more metadata about the current message. Instead of pushing everything through the constructor function, you can opt for potentially simpler and cleaner code by opting for method injection instead. In the case of the Envelope, that is not even available through the IoC container.

More about this in a later post, but ironically as the author of literally the oldest IoC container in .the .Net ecosystem, I’m trying hard to reduce Jasper’s usage of IoC containers at runtime.

The last thing I wanted to show here was Jasper’s concept of cascading messages that we used with some success in the earlier FubuMVC service bus. It’s very common for the handling of the original message to trigger additional “cascading” messages. In most service bus frameworks, that’d probably be something like this:

public class MessageHandler
{
    // Successfully handling Message1 will generate
    // a Message2 going out
    public Message2 Handle(Message1 message, IServiceBus bus)
    {
        bus.Send(new Message2());
    }
}

You can absolutely do that in Jasper as well, but we also support a policy where object(s) returned from a handler method are considered to be outgoing messages that are sent out as part of considering the message request complete. In its simplest usage, that may look like this:

public class MessageHandler
{
    // Successfully handling Message1 will generate
    // a Message2 going out
    public Message2 Handle(Message1 message)
    {
        return new Message2();
    }

    // Same thing, but async
    public Task Handle(Message1 message)
    {
        
    }
}

To get a little more complex, let’s say that your neck deep in CQRS jargon and when your service receives a “Command1” you raise one or more domain events that are handled separately. With cascading messages, that can look like this:

public IEnumerable<object> Handle(Command1 command)
{
    yield return new Event1();
    yield return new Event2();
}

In this case, each object returned is an outgoing message. I like the cascading message approach because it makes your message handlers easier to test with pure state-based testing.

There’s plenty more going on with this feature, but my wife really needs me to get out the office to go help with the little ones, so there’s going to have to more later;)