Jasper as an In-Memory Bus Part 1

I just finished the work in Jasper I’d described in Changing Jasper’s Direction and released a new Jasper 0.9 Nuget and a matching dotnet new template package with the newly streamlined structure. Hopefully, this represents the first big step on the march to a 1.0 release and production readiness by the end of this year. I’m going to start blogging a lot more examples on Jasper in hopes of building some community and possibly attracting some other contributors.

Today I want to talk about using Jasper as an in memory bus (like MediatR) within an ASP.Net Core application. To make Jasper less scary, let’s start with an ASP.Net Core application built with the dotnet new webapi template. After adding a dependency to Jasper via Nuget, you can add Jasper to your new application with this code:

    public class Program
    {
        public static void Main(string[] args)
        {
            CreateWebHostBuilder(args).Build().Run();
        }

        public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
            WebHost.CreateDefaultBuilder(args)
                .UseStartup()
                
                // This is all you have to do to use Jasper
                // with all its default behavior
                .UseJasper();
    }

Do note that this quietly replaces the default ASP.net Core DI container with Lamar that is absolutely required for Jasper usage. Also note that this is choosing to use Jasper with all its default behavior. Lastly, this simplified configuration approach will only discover message handlers within the same assembly with your main application. See this page for more information on configuring Jasper.

Now, let’s say we need to have an HTTP endpoint that will receive a command to create a new user within our system like this:

    public class CreateUser
    {
        public string FirstName { get; set; }
        public string LastName { get; set; }
        public string Email { get; set; }
    }

Since we’ve decided that we want the actual work to be done by a Jasper message handler instead of just burying it within the normal MVC controller cruft, let’s write our first Jasper message handler that will handle this message by just writing a message to the console:

    public class UserHandler
    {
        public void Handle(CreateUser user)
        {
            Console.WriteLine("Hey, I got a new user!");
        }
    }

Or to stay with using async anytime the handler involved any kind of IO, use an asynchronous version:

    public class UserHandler
    {
        public Task Handle(CreateUser user)
        {
            return Console.Out.WriteLineAsync("Hey, I got a user");
        }
    }

Now, switching to the MVC Core side, let’s add the controller action that receives a JSON POST for CreateUser and delegates to Jasper:

    public class UserController : ControllerBase
    {
        private readonly IMessageContext _messages;

        public UserController(IMessageContext messages)
        {
            _messages = messages;
        }

        [HttpPost("user")]
        public async Task CreateUser([FromBody] CreateUser user)
        {
            // Process the message NOW MISTER
            await _messages.Invoke(user);

            // OR

            // Put the incoming message in the in
            // memory worker queues and Jasper
            // will process it when it can
            await _messages.Enqueue(user);

            // OR, not sure why you'd want to do this, but say
            // you want this processed in 5 minutes
            await _messages.Schedule(user, 5.Seconds());

            return Ok();
        }
    }

The IMessageContext interface is the entry point for all messaging within Jasper and is automatically registered in your application’s Lamar container as ContainerScoped().

Alrighty then, it’s not super exciting in functionality, but this is actually a working application that hopefully shows off a decent getting started story for adding Jasper to an ASP.net Core system. In my next set of posts we’ll embellish the message handling to show off Jasper’s durable messaging, “outbox” support, and middleware.

In the meantime, please feel free to kick the tires on Jasper and let the community know what you think works and what needs some improvement in the Gitter room.

 

Jasper vs. MediatR

As strictly an in-memory messaging bus, Jasper supports a superset of MediatR’s functionality, but with a very different runtime architecture. In terms of implementation, MediatR is purposely simplistic and mostly relies on Inversion of Control (IoC) containers’ support for discovering and resolving generic types implementing its ​adapter interfaces likeIRequestHandler. Jasper, on the other hand, does its own command discovery (using Lamar’s built in type scanning) and eschews the usage of adapter interfaces or base classes in favor of naming conventions.

Selfishly speaking, MediatR has been a big headache to me with StructureMap and Lamar support because it encourages users to be too ambitious with generics to work around MediatR’s limited support for runtime composition. I certainly hope that Jasper’s middleware strategy — if it ever gets significant usage in the wild — generates far less user confusion.

As far as advantages, MediatR is obviously more mature, heavily used, and well understood by folks (as long as they stay sane on the generics usage). Plus some folks will prefer MediatR’s more explicit interfaces versus Jasper’s usage of conventions. In my opinion, Jasper’s potential advantages are:

  • Cleaner code because Jasper adapts itself to your code rather than forcing you to use its constructs
  • Higher performance, especially when it’s idiomatic Jasper HTTP handlers versus the combination of ASP.Net Core controller actions + MediatR.
  • Jasper supports quite a bit more functionality, and gives you much more control over message priority, error handling, and durability
  • Jasper composition and middleware design allows users to intelligently order middleware and provides many more extensibility scenarios
  • It’s rudimentary right now (just dump out the generated code), but Jasper provides out of the box diagnostics to show you what middleware is applied for which message types
Advertisements

Talking Integration Testing w/ .Net at CodeMash 2019

I’m officially approved by all pertinent bosses and spouses to go, so I’ll be speaking at CodeMash 2019 (think this is the 6th time for me). I’m doing a pre-compiler workshop this time around that’s going to be all new material. If you’re reading this and it’s not already January, feel very, very free to make requests or suggestions on scenarios you’d be interested in seeing as part of the workshop;)

Successful Integration Testing in .Net

Description

In the very early days of Agile development and Test Driven Development, we strove to structure our code such that we could write isolated unit tests with little or no coupling to infrastructure. Great, and it helped, but frequently that led to more complicated code internals and you still had issues from interactions with said infrastructure. What if instead, we could more effectively do quick, reliable integration testing? Could that simplify some of our code — especially when combined with microservice architectures? Do we need to reconsider where our efforts go in the old “testing pyramid?” How does ASP.Net Core support far better server side testing than previous versions? Where does HTTP contract testing fit in? Can Docker help us out? Should we pursue other techniques besides just trying to use Selenium? What about using alternative NoSQL style databases? Or if I’m stuck with relational databases, what can we do to soften the aggravations?

I’ll try to address all these questions and tackle automated testing best practices in this workshop.

Marten 3.0 is Released and Introducing the new Core Team

banner

Hey, look at that, Marten 3.0 is live on Nuget. It didn’t turn out to be a huge release, but we needed to accommodate the Npgsql 4.* dependency and I felt like that was a breaking change, so here we go. You can see the full list of bug fixes and changes in this list. Besides the usage of Npgsql 4, our biggest change was making the default schema object creation mode to this:

AutoCreateSchemaObjects = AutoCreate.CreateOrUpdate

Meaning that Marten even in its default mode will not drop any existing tables, even in development mode. You can still opt into the full “sure, I’ll blow away a table and start over if it’s incompatible” mode, but we felt like this option was safer after a few user problems were reported with the previous rules. See Schema Migrations and Patches for more information.

More exciting for me because I think it’s a mark of Marten becoming a real grownup open source project, Marten now has an official core contributor team who can review and approve pull requests:

All of them have been contributing to Marten for quite awhile and have been absolutely invaluable to supporting our community. Please join me in welcoming them all to the new team and looking forward to wherever Marten goes next.

 

Changing Jasper’s Direction

I usually won’t publish blog posts on weekends because nobody reads them, but in this case I’m just jotting down my own thoughts.

I’ve been working off and on for about 4-5 years on a new .Net framework called Jasper (see the “Jasper” tag for my previous posts). Jasper was originally envisioned as a better version of FubuMVC, including both its web framework side and its later functionality as a pretty full fledged service bus that could do publish/subscribe messaging through its own lightweight in process queue (LightningQueues) without any other additional infrastructure.

For the last half of 2017 and the early part of 2018 some of my colleagues and I worked a lot on Jasper specifically as a service bus that could be backwards compatible with older FubuMVC applications and be usable in a large, on premise deployed ecosystem. To that end, most of Jasper’s code deals with messaging concerns, including quite a bit of functionality that overlaps with big, grown up messaging tools like RabbitMQ or Azure Service Bus.

After getting back into Jasper again the past couple weeks I think these things:

  • The backwards compatibility with FubuMVC doesn’t matter anymore and could be eliminated
  • If I were using Jasper for messaging at any kind of scope, I’d want to be using Rabbit MQ or Azure Service Bus anyway
  • I’m personally way more interested myself in getting back to the HTTP side of things or learning Azure technologies through integration with Jasper
  • The codebase is big and probably a little daunting
  • In my opinion, the special thing about Jasper is its particular flavor of the Russian Doll runtime pipeline, extensibility, and the way it allows you to write very simple code in your application.
  • It’s extremely hard to get developers to use any kind of alternative framework, but it’s far less difficult to get developers to try out OSS libraries that complement, extend, or improve the mainstream framework they’re already working with.

I’m still mulling things over, but at this point I wanna switch directions so that Jasper is mostly about its runtime pipeline as a “command processor” that can be used 3 (maybe 4) ways:

  1. As a standalone, in memory service bus and command dispatcher (which it already does today, as is, and quite robustly, thank you) that you could embed in any kind of .Net Core application. For example, Jasper could be used in combination with ASP.Net Core the same way that folks use MediatR today.
  2. As an alternative way to write HTTP services in ASP.Net Core that’s effectively Jasper’s own lightweight HTTP router connected to Jasper’s runtime pipeline for command execution (it’s not documented at all other than a single blog post, but much of the basics are already in place). This could be used in place of or side by side with MVC Core or any other kind of ASP.Net Core middleware.
  3. As a connector between publish/subscribe queues like RabbitMQ or Azure Service Bus and Jasper’s command execution. Basically, Jasper handles all the things like serialization and messaging workflow described in Sure, You Can Just Use RabbitMQ — which Jasper’s existing messaging support generally does right now as is. What would change here is mostly subtraction as Jasper would rely much more on RabbitMQ/Azure Service Bus/etc. for message routing instead of the custom code that exists in Jasper’s codebase today.

That still sounds like a potentially huge tool, but there’s maybe a lot more in common between those 3 things than it appears. Jasper’s sharing all the command execution mechanics, the IoC integration, logging set up, and even sharing content negotiation (serialization) between items 1, 2, & 3 above.

Nuget wise, after whacking some code listed later in this post and the recent reorganization of the codebase anyway, I think Jasper is divided into these Nugets:

  • Jasper — The core library. Has the TCP transport (its tiny), the core runtime pipeline, the in memory bus and command execution functionality, the HTTP handler support, anything to do with extension discovery, and the integration into ASP.Net Core hosting. As much as possible, Jasper tries to rely on common pieces like the basic logging, hosting, and configuration elements of ASP.Net Core, so there’s no good reason in my mind to separate out the HTTP support from the core
  • Jasper.Persistence.Marten — integrates Marten/Postgresql into a Jasper application. Marten-backed saga persistence, durable messaging, and the outbox support
  • Jasper.Persistence.SqlServer — Sql Server backed message and saga persistence, including the outbox support. I’m not sure what to do with saga persistence here yet
  • Jasper.RabbitMQ — the RabbitMQ transport for Jasper
  • Jasper.AzureServiceBus — future
  • Jasper.TestSupport.Storyteller — tooling to host and monitor Jasper applications in Storytellerspecifications
  • Jasper.TestSupport.Alba — tooling to run Alba specifications directly against a Jasper runtime
  • Jasper.MvcCoreExtender — future, use MVC Core constructs in Jasper HTTP services and possibly act as a performance accelerator to existing MVC Core applications

So here’s my current thinking on what does or does not change:

Remove

  • The HTTP transport that uses Kestrel as another bus transport alternative. It needs a little more work anyway and I don’t see anyone using it
  • Anything to do with the dynamic subscription model, and that’s quite a bit
  • The Consul extension. Most of it is for the subscriptions that goes away, and the rest of it could be done by just piping Consul through ASP.Net Configuration
  • Request/Reply or Send/Await mechanics in the service bus. Only there for fubu compatibility we no longer care about. It’d be long gone as is but too many tests depend on it. Ugh.
  • The model binding support. I don’t think it’d be commonly used and I’d rather cede that to MVC Core
  • Node registration and discovery. Use what’s already in Azure or AWS for this kind of thing and let’s thin down Jasper

Keep

  • The lightweight TCP protocol stays, but it’s documented clearly as most appropriate for small to medium workloads or for local testing or for just a simple getting started story
  • The outbox mechanics w/ either Marten or Sql Server
  • The current in memory worker queues, with possible thought toward making them extensible for concerns like throttling in the future
  • The serialization and content negotiation
  • Jasper’s command line integration via Oakton. I think it’s turning out to be a cheap way to build in diagnostics and maintenance tasks right into your app.
  • The environment check support. I’m not aware of any equivalent in ASP.Net Core yet
  • Jasper’s internal HTTP router. I think it’s going to end up being much faster than the one built into ASP.net Core

Enhance or Add

Replace

  • The built in error handling/retry/circuit breaker stuff in Jasper is battle tested through years of usage in production as part of FubuMVC, but I want to see if there’s a way to replace it with Polly to shrink the codebase and add a lot more capabilities. It’s not a slam dunk though

Making the Jasper build faster and more reliable with Docker and other stuff

Maybe a little embarrassingly, most of my efforts in Jasper this summer has been around making all the automated test suites run faster and far more reliably. I first invested a dreadful amount of time toward making the main xUnit test library run tests in parallel.

Next, Jasper has extensions that integrate Consul, RabbitMQ, Sql Server, and Postgresql via Marten. As you can probably imagine, that makes integration testing a little bit of a mess with all the infrastructure dependencies. The prevailing trend — at least in my circles — these days is to try to leverage Docker for infrastructure needs within build automation. To that end, I first tried to use the Docker.Net library in the manner I wrote about in *A* way to use Docker for integration tests (which I need to update because our team had to tweak for CI usage).

That worked okay, but you’d still hit occasional timeouts and there were a few tests that needed both one of the databases and Rabbit MQ running. Instead, I switched to just having a tiny Docker compose file like this:

version: '3'
services:
  postgresql:
    image: "clkao/postgres-plv8:latest"
    ports:
     - "5433:5432"
  consul:
    image: "consul:latest"
    ports:
     - "8500:8500"
     - "8600:8600"
  rabbitmq:
    image: "rabbitmq:latest"
    ports:
     - "5672:5672"
  sqlserver:
    image: "microsoft/mssql-server-linux:2017-latest"
    ports:
     - "1433:1433"
    environment:
     - "ACCEPT_EULA=Y"
     - "SA_PASSWORD=P@55w0rd"
     - "MSSQL_PID=Developer"

which I’m showing mostly to call out “that was easy, why didn’t I do this ages ago?” In my build script for Jasper, it just makes sure to call “docker-compose up -d” once before running any of the integration test suites.

I also folded all the test libraries for the various integrations into a single test library called “IntegrationTests,” but used the xUnit [Collection] attributes and base context classes to effectively allow each of the extension libraries to be tested in parallel threads, but single threaded within the set of tests targeting each extension library.

All told, the combination of:

  1. Upgrading to netcoreapp2.1 for all test libraries
  2. Ensuring that the main xUnit testing library can run specifications in parallel
  3. Integrating the Docker infrastructure set up inside the build script itself
  4. Letting the tests for the Consul, Postgresql, MSSQL, and RabbitMQ integrations run in parallel in the combined integration testing library
  5. Moving any “flaky” tests that were vulnerable to timing or threading issues to Storyteller where it’s more robust
  6. Fixing the real underlying issues in the code around some of the “flaky” tests:-(

made the automated build far faster and more reliable than it was before I started that effort and I’m one of those people who thinks that’s extremely important for the success of a project.

 

*A* way to use Docker for integration tests

This approach was partially stolen from an approach I found in the Akka.Net code for integration testing. The Sql Server bits are taken from a blog post from David Neal. Lastly, I started down this path for integration testing inside the Jasper codebase, but have since switched to just using a Docker compose file in that case. 

Integration testing with databases kind of sucks. I know it, you know it, the American people know it. It’s also really valuable to do if you can pull it off, so here we are. What are some of the problems with testing databases? Let’s see:

  • It’s more effective when you have isolated databases per developer and environment
  • You should really have some kind of build automation that syncs your test database up with the current version of your schema
  • Ideally, you’d really like it to be as easy as possible for a developer to do a clean clone of your codebase and quickly have integration tests up and running without a lot of teeth gnashing over setting up either a local database or getting their own schema somewhere on a shared database server

With all that in mind, my current client project is trying to get away with running Sql Server in a Docker container that’s spun up on demand by the Docker.DotNet library within the integration tests.

The first step is a little base class helper I originally wrote (and discarded) to define a Docker container instance:

    internal abstract class DockerServer
    {
        public string ImageName { get; }
        public string ContainerName { get; }

        protected DockerServer(string imageName, string containerName)
        {
            ImageName = imageName;
            ContainerName = containerName; // + "-" + Guid.NewGuid().ToString();
        }

        public async Task Start(IDockerClient client)
        {
            if (StartAction != StartAction.none) return;


            var images =
                await client.Images.ListImagesAsync(new ImagesListParameters { MatchName = ImageName });


            if (images.Count == 0)
            {
                Console.WriteLine($"Fetching Docker image '{ImageName}'");

                await client.Images.CreateImageAsync(
                    new ImagesCreateParameters { FromImage = ImageName, Tag = "latest" }, null,
                    new Progress());
            }

            var list = await client.Containers.ListContainersAsync(new ContainersListParameters
            {
                All = true
            });

            var container = list.FirstOrDefault(x => x.Names.Contains("/" + ContainerName));
            if (container == null)
            {
                await createContainer(client);

            }
            else
            {
                if (container.State == "running")
                {
                    Console.WriteLine($"Container '{ContainerName}' is already running.");
                    StartAction = StartAction.external;
                    return;
                }
            }


            var started = await client.Containers.StartContainerAsync(ContainerName, new ContainerStartParameters());
            if (!started)
            {
                throw new InvalidOperationException($"Container '{ContainerName}' did not start!!!!");
            }

            var i = 0;
            while (!await isReady())
            {
                i++;

                if (i > 20)
                {
                    throw new TimeoutException($"Container {ContainerName} does not seem to be responding in a timely manner");
                }

                await Task.Delay(5.Seconds());
            }

            Console.WriteLine($"Container '{ContainerName}' is ready.");

            StartAction = StartAction.started;
        }

        public StartAction StartAction { get; private set; } = StartAction.none;

        private async Task createContainer(IDockerClient client)
        {
            Console.WriteLine($"Creating container '{ContainerName}' using image '{ImageName}'");

            var hostConfig = ToHostConfig();
            var config = ToConfig();

            await client.Containers.CreateContainerAsync(new CreateContainerParameters(config)
            {
                Image = ImageName,
                Name = ContainerName,
                Tty = true,
                HostConfig = hostConfig,
            });
        }

        public async Task Stop(IDockerClient client)
        {
            await client.Containers.StopContainerAsync(ContainerName, new ContainerStopParameters());
        }

        public Task Remove(IDockerClient client)
        {
            return client.Containers.RemoveContainerAsync(ContainerName,
                new ContainerRemoveParameters { Force = true });
        }

        protected abstract Task isReady();

        public abstract HostConfig ToHostConfig();

        public abstract Config ToConfig();

        public override string ToString()
        {
            return $"{nameof(ImageName)}: {ImageName}, {nameof(ContainerName)}: {ContainerName}";
        }
    }

I know, it’s a bit long, but in essence it just gives you the ability to define a simple Docker container you want to be running during tests and enough smarts to download missing images, start new containers on your box as necessary, and wait around until that Docker container is really ready.

To extend that approach to Sql Server, we use this class (partially elided):

    internal class SqlServerContainer : DockerServer
    {
        public SqlServerContainer() : base("microsoft/mssql-server-linux:latest", "dev-mssql")
        {
        }

        public static readonly string ConnectionString = "Server=127.0.0.1,1434;User Id=sa;Password=P@55w0rd;Timeout=5";

        // Gotta wait until the database is really available
        // or you'll get oddball test failures;)
        protected override async Task isReady()
        {
            try
            {
                using (var conn =
                    new SqlConnection(ConnectionString))
                {
                    await conn.OpenAsync();

                    return true;
                }
            }
            catch (Exception)
            {
                return false;
            }
        }

        // Watch the port mapping here to avoid port
        // contention w/ other Sql Server installations
        public override HostConfig ToHostConfig()
        {
            return new HostConfig
            {
                PortBindings = new Dictionary<string, IList>
                {
                    {
                        "1433/tcp",
                        new List
                        {
                            new PortBinding
                            {
                                HostPort = $"1434",
                                HostIP = "127.0.0.1"
                            }
                        }
                    },


                },

            };
        }

        public override Config ToConfig()
        {
            return new Config
            {
                Env = new List { "ACCEPT_EULA=Y", "SA_PASSWORD=P@55w0rd", "MSSQL_PID=Developer" }
            };
        }

        public static void RebuildSchema()
        {
            DropSchema();
            InitializeSchema();
        }

        public static void DropSchema()
        {
            // drops the configured schema from the database
        }

        public static void InitializeSchema()
        {
            // rebuilds the schema objects
        }
    }

Now, in tests we take advantage of the IClassFixture mechanism in xUnit.Net to ensure that our Sql Server container is spun up and executing before any integration test runs. First, add a class that just manages the lifetime of the Docker container that will be the shared context in integration tests:

    public class IntegrationFixture
    {
        private readonly IDockerClient _client;
        private readonly SqlServerContainer _container;
        
        public IntegrationFixture()
        {
            _client = DockerServers.BuildDockerClient();
            _container = new SqlServerContainer();

            _container.Start(_client).Wait(300.Seconds());


            SqlServerContainer.RebuildSchema();
        }
    }

With that in place, integration tests in our codebase need to implement IClassFixture<IntegrationFixture> and take that in with a constructor function like this:

    public class an_integration_test_class : IClassFixture<IntegrationFixture>
    {
        public an_integration_test_class(IntegrationFixture fixture)
        {
        }
        
        // Add facts
    }

If this is really common, you might slap together a base class for your integration tests that automatically implements the IClassFixture<IntegrationFixture> interface so you don’t have to think about that every time.

It’s too early to say this is a huge success, but so far, so good. We’ve had some friction around our CI build process with the Docker usage.

Do note that I purposely avoided shutting down the Docker container for Sql Server in any kind of testing teardown. I do that so that developers are able to interrogate the state of the database after tests to help troubleshoot inevitable testing failures.

Why this over using a Docker Compose file that would need to be executed before running any of the tests? I’m not sure I’d give you any hard and fast opinion on when or when not to do that, but in this case it’s helpful for Visual Studio.Net-centric devs to just be able to run tests from VS.Net without having to think about any other additional set up. I did choose to use Docker Compose files in Jasper where I had four different containers, and a handful of tests that need more than one of those containers.

Marten 2.10.0 and 3.0.0-alpha-2 are released.

The latest set of changes to Marten were just published as Marten 2.10.0 and Marten 3.0.0-alpha-2, with the only real difference being that the 3.0 alpha supports Npgsql 4. The complete list of changes is here. The latest docs have also been published. I counted 15 users total (not including me) in this release as either contributors or just folks who took the time to write up an actionable bug report (and don’t minimize the importance of that because it helps a ton). Thank you to all the Marten contributors and all the folks who answer questions in the Gitter room.

Do note that I quietly and unilaterally exercised my rights as the project’s benevolent dictator to finally remove Marten’s targeting for Netstandard1.* as it was annoying the crap out of me and I didn’t think many folks would care. As of 2.10.0, Marten targets net46 and netstandard2.0.

I hope that more serious work on the 3.0 release starts up again soon, but we’ll see.