Testing web services secured by JWT tokens with Alba v5

We’re working toward standing up a new OIDC infrastructure built around Identity Server 5, with a couple gigantic legacy monolith applications and potentially dozens of newer microservices needing to use this new identity server for authentication. We’ll have to have a good story for running our big web applications that will have this new identity server dependency at development time, but for right now I just want to focus on an automated testing strategy for our newer ASP.Net Core web services using the Alba library.

First off, Alba is a helper library for integration testing HTTP API endpoints in .Net Core systems. Alba wraps the ASP.Net Core TestServer while providing quite a bit of convenient helpers for setting up and verifying HTTP calls against your ASP.Net Core services. We will be shortly introducing Alba into my organization at MedeAnalytics as a way of doing much more integration testing at the API layer (think the middle layer of any kind of testing pyramid concept).

In my previous post I laid out some plans and proposals for a quickly forthcoming Alba v5 release, with the biggest improvement being a new model for being able to stub out OIDC authentication for APIs that are secured by JWT bearer tokens (I think I win programming bingo for that sentence!).

Before I show code, I should say that all of this code is in the v5 branch of Alba on GitHub, but not yet released as it’s very heavily in flight.

To start, I’m assuming that you have a web service project, then a testing library for that web service project. In your web application, bearer token authentication is set up something like this inside your Startup.ConfigureServices() method:

services.AddAuthentication("Bearer")
    .AddJwtBearer("Bearer", options =>
    {
        // A real application would pull all this information from configuration
        // of course, but I'm hardcoding it in testing
        options.Audience = "jwtsample";
        options.ClaimsIssuer = "myapp";
        
        // don't worry about this, our JwtSecurityStub is gonna switch it off in
        // tests
        options.Authority = "https://localhost:5001";
            

        options.TokenValidationParameters = new TokenValidationParameters
        {
            ValidateAudience = false,
            IssuerSigningKey = new SymmetricSecurityKey(Encoding.UTF8.GetBytes("some really big key that should work"))
        };
    });

And of course, you also have these lines of code in your Startup.Configure() method to add in ASP.Net Core middleware for authentication and authorization:

app.UseAuthentication();
app.UseAuthorization();

With these lines of setup code, you will not be able to hit any secured HTTP endpoint in your web service unless there is a valid JWT token in the Authorization header of the incoming HTTP request. Moreover, with this configuration your service would need to make calls to the configured bearer token authority (http://localhost:5001 above). It’s going to be awkward and probably very brittle to depend on having the identity server spun up and running locally when our developers try to run API tests. It would obviously be helpful if there was a quick way to stub out the bearer token authentication in testing to automatically supply known claims so our developers can focus on developing their individual service’s functionality.

That’s where Alba v5 comes in with its new JwtSecurityStub extension that will:

  1. Disable any validation interactions with an external OIDC authority
  2. Automatically add a valid JWT token to any request being sent through Alba
  3. Give developers fine-grained control over the claims attached to any specific request if there is logic that will vary by claim values

To demonstrate this new Alba functionality, let’s assume that you have a testing project that has a direct reference to the web service project. The direct project reference is important because you’ll want to spin up the “system under test” in a test fixture like this:

    public class web_api_authentication : IDisposable
    {
        private readonly IAlbaHost theHost;

        public web_api_authentication()
        {
            // This is calling your real web service's configuration
            var hostBuilder = Program.CreateHostBuilder(new string[0]);

            // This is a new Alba v5 extension that can "stub" out
            // JWT token authentication
            var jwtSecurityStub = new JwtSecurityStub()
                .With("foo", "bar")
                .With(JwtRegisteredClaimNames.Email, "guy@company.com");

            // AlbaHost was "SystemUnderTest" in previous versions of
            // Alba
            theHost = new AlbaHost(hostBuilder, jwtSecurityStub);
        }

I was using xUnit.Net in this sample, but Alba is agnostic about the actual testing library and we’ll use both NUnit and xUnit.Net at work.

In the code above I’ve bootstrapped the web service with Alba and attached the JwtSecurityStub. I’ve also established some baseline claims that will be added to every JWT token on all Alba scenario requests. The AlbaHost extends the IHost interface you’re already used to in .Net Core, but adds the important Scenario() method that you can use to run HTTP requests all the way through your entire application stack like this test :

        [Fact]
        public async Task post_to_a_secured_endpoint_with_jwt_from_extension()
        {
            // Building the input body
            var input = new Numbers
            {
                Values = new[] {2, 3, 4}
            };

            var response = await theHost.Scenario(x =>
            {
                // Alba deals with Json serialization for us
                x.Post.Json(input).ToUrl("/math");
                
                // Enforce that the HTTP Status Code is 200 Ok
                x.StatusCodeShouldBeOk();
            });

            var output = response.ResponseBody.ReadAsJson<Result>();
            output.Sum.ShouldBe(9);
            output.Product.ShouldBe(24);
        }

You’ll notice that I did absolutely nothing in regards to JWT set up or claims or anything. That’s because the JwtSecurityStub is taking care of everything for you. It’s:

  1. Reaching into your application’s bootstrapping to pluck out the right signing key so that it builds JWT token strings that can be validated with the right signature
  2. Turns off any external token validation using an external OIDC authority
  3. Places a unique, unexpired JWT token on each request that matches the issuer and authority configuration of your application

Now, to further control the claims used on any individual scenario request, you can use this new method in Scenario tests:

        [Fact]
        public async Task can_modify_claims_per_scenario()
        {
            var input = new Numbers
            {
                Values = new[] {2, 3, 4}
            };

            var response = await theHost.Scenario(x =>
            {
                // This is a custom claim that would only be used for the 
                // JWT token in this individual test
                x.WithClaim(new Claim("color", "green"));
                x.Post.Json(input).ToUrl("/math");
                x.StatusCodeShouldBeOk();
            });

            var principal = response.Context.User;
            principal.ShouldNotBeNull();
            
            principal.Claims.Single(x => x.Type == "color")
                .Value.ShouldBe("green");
        }

I’ve got plenty of more ground to cover in how we’ll develop locally with our new identity server strategy, but I’m feeling pretty good about having a decent API testing strategy. All of this code is just barely written, so any feedback you might have would be very timely. Thanks for reading about some brand new code!

Proposal for Alba v5: JWT secured APIs, more JSON options, and other stuff

Just typed up a master GitHub issue for Alba v5, and I’ll take any possible feedback anywhere, but I’d prefer you make requests on GitHub. Here’s the GitHub issue I just banged out copied and pasted for anybody interested:

I’m dipping into Alba over the next couple work days to add some improvements that we want at MedeAnalytics to test HTTP APIs that are going to be secured with JWT Bearer Token authentication that’s ultimately backed up by IdentityServer5. I’ve done some proof of concepts off to the side on how to deal with that in Alba tests with or without the actual identity server up and running, so I know it’s possible.

However, when I started looking into how to build a more formal “JWT Token” add on to Alba and catching up with GitHub issues on Alba, it’s led me to the conclusion that it’s time for a little overhaul of Alba as part of the JWT work that’s turning into a proposed Alba v5 release.

Here’s what I’m thinking:

  • Ditch all support < .Net 5.0 just because that makes things easier on me maintaining Alba. Selfishness FTW.
  • New Alba extension model, more on this below
  • Revamp the Alba bootstrapping so that it’s just an extension method hanging off the end of IHostBuilder that will create a new ITestingHost interface for the existing SystemUnderTest class. More on this below.
  • Decouple Alba’s JSON facilities from Newtonsoft.Json. A couple folks have asked if you can use System.Text.Json with Alba so far, so this time around we’ll try to dig into the application itself and find the JSON formatter from the system and make all the JSON serialization “just” use that instead of recreating anything
  • Enable easy testing of GPRC endpoints like we do for classic JSON services

DevOps changes

  • Replace the Rake build script with Bullseye
  • Move the documentation website from stdocs to VitePress with a similar set up as the new Marten website
  • Retire the existing AppVeyor CI (that works just fine), and move to GitHub actions for CI, but add a new action for publishing Nugets

Bootstrapping Changes

Today you use code like this to bootstrap Alba against your ASP.Net Core application:

var system = new SystemUnderTest(WebApi.Program.CreateHostBuilder(new string[0]));

where Alba’s SystemUnderTest intercepts an IHostBuilder, adds some additional configuration, swaps out Kestrel for the ASP.Net Core TestServer, and starts the underlying IHost.

In Alba v5, I’d like to change this a little bit to being an extension method hanging off of IHostBuilder so you’d have something more like this:

var host = WebApi.Program.CreateHostBuilder(new string[0]).StartAlbaHost();

// or

var host = await WebApi.Program.CreateHostBuilder(new string[0]).StartAlbaHostAsync();

The host above would be this new interface (still the old SystemUnderTest under the covers):

public interface IAlbaHost : IHost
{
    Task<ScenarioResult> Scenario(Action<Scenario> configure);

    // various methods to register actions to run before or after any scenario is executed
}

Extension Model

Right now the only extension I have in mind is one for JWT tokens, but it probably goes into a separate Nuget, so here you go.

public interface ITestingHostExtension : IDisposable, IAsyncDisposable
{
    // Make any necessary registrations to the actual application
    IHostBuilder Configure(IHostBuilder builder);

    // Spin it up if necessary before any scenarios are run, also gives
    // an extension a chance to add any BeforeEach() / AfterEach() kind
    // of hooks to the Scenario or HttpContext
    ValueTask Start(ITestingHost host);
   
}

My thought is that ITestingHostExtension objects would be passed into the StartAlbaHost(params ITestingHostExtension[] extensions) method from above.

JWT Bearer Token Extension

This will probably need to be in a separate Alba.JwtTokens Nuget library.

  • Extension method on Scenario to add a bearer token to the Authorization header. Small potatoes.
  • Helper of some sort to fetch JWT tokens from an external identity server and use that token on further scenario runs. I’ve got this prototyped already
  • a new Alba extension that acts as a stubbed identity server. More below:

For the stubbed in identity server, what I’m thinking is that a registered IdentityServerStub extension would:

  • Spin up a small fake OIDC web application hosted in memory w/ Kestrel that responds to the basic endpoints to validate JWT tokens.
  • With this extension, Alba will quietly stick a JWT token into the Authorization header of each request so you can test API endpoints that are secured by bearer tokens
  • Reaches into the system under test’s IHostBuilder and sets the authority url to the local Url of the stubbed OIDC server (I’ve already successfully prototyped this one)
  • Also reaches into the system under test to find the JwtBearerOptions as configured and uses the signing keys that are already configured for the application for the JWT generation. That should make the setup of this quite a bit simpler
  • The new JWT stub extension can be configured with baseline claims that would be part of any generated JWT
  • A new extension method on Scenario to add or override claims in the JWT tokens on a per-scenario basis to support fine-grained authorization testing in your API tests
  • An extension method on Scenario to disable the JWT header altogether so you can test out authentication failures
  • Probably need a facility of some sort to override the JWT expiration time as well

Using Alba to Test ASP.Net Services

TL;DR: Alba is a handy library for integration tests against ASP.Net web services, and the rapid feedback that enables is very useful

One of our projects at Calavista right now is helping a client modernize and optimize a large .Net application, with the end goal being everything running on .Net 5 and an order of magnitude improvement in system throughput. As part of the effort to upgrade the web services, I took on a task to replace this system’s usage of IdentityServer3 with IdentityServer4, but still use the existing Marten-backed data storage for user membership information.

Great, but there’s just one problem. I’ve never used IdentityServer4 before and it changed somewhat between the IdentityServer3 code I was trying to reverse engineer and its current model. I ended up getting through that work just fine. A key element of doing that was using the Alba library to create a test harness so I could iterate through configuration changes quickly by rerunning tests on the new IdentityServer4 project. It didn’t start out this way, but Alba is essentially a wrapper around the ASP.Net TestServer and just acts as a utility to make it easier to write automated tests around the HTTP services in your web service projects.

I ended up starting two new .Net projects:

  1. A new web service that hosts IdentityServer4 and is configured to use user membership information from our client’s existing Marten/Postgresql database
  2. A new xUnit.Net project to hold integration tests against the new IdentityServer4 web service.

Let’s dive right into how I set up Alba and xUnit.Net as an automated test harness for our new IdentityServer4 service. If you start a new ASP.Net project with one of the built in project templates, you’ll get a Program file that’s the main entry point for the application and a Startup class that has most of the system’s bootstrapping configuration. The templates will generate this method that’s used to configure the IHostBuilder for the application:

public static IHostBuilder CreateHostBuilder(string[] args)
{
    return Host.CreateDefaultBuilder(args)
        .ConfigureWebHostDefaults(webBuilder =>
        {
            webBuilder.UseStartup<Startup>();
        });
}

For more information on what role of the IHostBuilder is within your application, see .NET Generic Host in ASP.NET Core.

That’s important, because that gives us the ability to stand up the application exactly as it’s configured in an automated test harness. Switching to the new xUnit.Net test project, referenced my new web service project that will host IdentityServer4. Because spinning up your ASP.Net system can be relatively expensive, I only want to do that once and share the IHost between tests. That’s a perfect usage for xUnit.Net’s shared context support.

First I make what will be the shared test fixture context class for the integration tests shown below:

public class AppFixture : IDisposable
{
    public AppFixture()
    {
        // This is calling into the actual application's
        // configuration to stand up the application in 
        // memory
        var builder = Program.CreateHostBuilder(new string[0]);

        // This is the Alba "SystemUnderTest" wrapper
        System = new SystemUnderTest(builder);
    }
    
    public SystemUnderTest System { get; }

    public void Dispose()
    {
        System?.Dispose();
    }
}

The Alba SystemUnderTest wrapper is responsible for building the actual IHost object for your system, and does so using the in memory TestServer in place of Kestrel.

Just as a convenience, I like to create a base class for integration tests I tend to call IntegrationContext:

    public abstract class IntegrationContext 
        // This is some xUnit.Net mechanics
        : IClassFixture<AppFixture>
    {
        public IntegrationContext(AppFixture fixture)
        {
            System = fixture.System;
            
            // Just casting the IServiceProvider of the underlying
            // IHost to a Lamar IContainer as a convenience
            Container = (IContainer)fixture.System.Services;
        }
        
        public IContainer Container { get; }
        
        // This gives the tests access to run
        // Alba "Scenarios"
        public ISystemUnderTest System { get; }
    }

We’re using Lamar as the underlying IoC container in this application, and I wanted to use Lamar-specific IoC diagnostics in the tests, so I expose the main Lamar container off of the base class as just a convenience.

To finally turn to the tests, the very first thing to try with IdentityServer4 was just to hit the descriptive discovery endpoint just to see if the application was bootstrapping correctly and IdentityServer4 was functional *at all*. I started a new test class with this declaration:

    public class EndToEndTests : IntegrationContext
    {
        private readonly ITestOutputHelper _output;
        private IDocumentStore theStore;

        public EndToEndTests(AppFixture fixture, ITestOutputHelper output) : base(fixture)
        {
            _output = output;

            // I'm grabbing the Marten document store for the app
            // to set up user information later
            theStore = Container.GetInstance<IDocumentStore>();
        }

And then a new test just to exercise the discovery endpoint:

[Fact]
public async Task can_get_the_discovery_document()
{
    var result = await System.Scenario(x =>
    {
        x.Get.Url("/.well-known/openid-configuration");
        x.StatusCodeShouldBeOk();
    });
    
    // Just checking out what's returned
    _output.WriteLine(result.ResponseBody.ReadAsText());
}

The test above is pretty crude. All it does is try to hit the `/.well-known/openid-configuration` url in the application and see that it returns a 200 OK HTTP status code.

I tend to run tests while I’m coding by using keyboard shortcuts. Most IDEs support some kind of “re-run the last test” keyboard shortcut. Using that, my preferred workflow is to run the test once, then assuming that the test is failing the first time, work in a tight cycle of making changes and constantly re-running the test(s). This turned out to be invaluable as it took me a couple iterations of code changes to correctly re-create the old IdentityServer3 configuration into the new IdentityServer4 configuration.

Moving on to doing a simple authentication, I wrote a test like this one to exercise the system with known credentials:

[Fact]
public async Task get_a_token()
{
    // All I'm doing here is wiping out the existing database state
    // with Marten and using a little helper method to add a new
    // user to the database as part of the test setup
    // This would obviously be different in your system
    theStore.Advanced.Clean.DeleteAllDocuments();
    await theStore
        .CreateUser("aquaman@justiceleague.com", "dolphin", "System Administrator");
    
    
    var body =
        "client_id=something&client_secret=something&grant_type=password&scope=ourscope%20profile%20openid&username=aquaman@justiceleague.com&password=dolphin";

    // The test would fail here if the status
    // code was NOT 200
    var result = await System.Scenario(x =>
    {
        x.Post
            .Text(body)
            .ContentType("application/x-www-form-urlencoded")
            .ToUrl("/connect/token");
    });

    // As a convenience, I made a new class called ConnectData
    // just to deserialize the IdentityServer4 response into 
    // a .Net object
    var token = result.ResponseBody.ReadAsJson<ConnectData>();
    token.access_token.Should().NotBeNull();
    token.token_type.Should().Be("Bearer");
}

public class ConnectData
{
    public string access_token { get; set; }
    public int expires_in { get; set; }
    public string token_type { get; set; }
    public string scope { get; set; }
    
}

Now, this test took me several iterations to work through until I found exactly the right way to configure IdentityServer4 and adjusted our custom Marten backing identity store (IResourceOwnerPasswordValidator and IProfileService in IdentityServer4 world) until the tests pass. I found it extremely valuable to be able to debug right into the failing tests as I worked, and even needed to take advantage of JetBrains Rider’s capability to debug through external code to understand how IdentityServer4 itself worked. I’m very sure that I was able to get through this work much faster by iterating through tests as opposed to just trying to run the application and driving it through something like Postman or through the connected user interface.

Using Alba for Integration Testing ASP.Net Core Web Services

There’s a video of David Giard and I talking about Alba at CodeMash a couple years ago on YouTube if you’re interested.

Alba is a helper library for doing automated testing against ASP.Net Core web services. Alba started out as a subsystem of the defunct FubuMVC project for HTTP contract testing, but was later extracted into its own project distributed via Nuget, ported to ASP.Net Core, and finally re-wired to incorporate TestServer internally.

There are certainly other advantages, but I think the biggest selling point of adopting ASP.Net Core is the ability to run HTTP requests through your system completely in memory without any other kind of external web server setup. In my experience, this has made automated integration testing of ASP.Net Core applications far simpler compared to older versions of ASP.Net.

We could use FubuMVC’s OWIN support with or without Katana to perform the same kind of automated testing that I’m demonstrating here years ago, but hardly anyone ever used that and what I’m showing here is significantly easier to use.

Why not just use TestServer you ask? You certainly can, but Alba provides a lot of helper functionality around exercising HTTP web services that will make your tests much more readable and remove a lot of the repetitive coding you’d have to do with just using TestServer.

To demonstrate Alba, I published a small solution called AlbaIntegrationTesting to GitHub this morning.

Starting with a small web services application generated with the dotnet new webapi template, I added a small endpoint that we’ll later write specifications against:

public class Answers
{
    public int Product { get; set; }
    public int Sum { get; set; }
}

public class AdditionController : ControllerBase
{
    [HttpGet("math/{one}/{two}")]
    public Answers DoMath(int one, int two)
    {
        return new Answers
        {
            Product = one * two,
            Sum = one + two
        };
    }
}

Next, I added a new test project using xUnit.Net with the dotnet new xunit template. With the new skeleton testing project I:

  • Added a reference to the latest Alba Nuget
  • My preference for testing assertions in .Net is Shouldly, so I added a Nuget reference for that as well

Since bootstrapping an ASP.Net Core system is non-trivial in terms of performance, I utilized xUnit.Net’s support for sharing context between tests so that the application only has to be spun up once in the test suite. The first step of that is to create a new “fixture” class that holds the ASP.Net Core system in memory like so:

public class AppFixture : IDisposable
{
    public AppFixture()
    {
        // Use the application configuration the way that it is in the real application
        // project
        var builder = Program.CreateHostBuilder(new string[0])
            
            // You may need to do this for any static 
            // content or files in the main application including
            // appsettings.json files
            
            // DirectoryFinder is an Alba helper
            .UseContentRoot(DirectoryFinder.FindParallelFolder("WebApplication")) 
            
            // Override the hosting environment to "Testing"
            .UseEnvironment("Testing"); 

        // This is the Alba scenario wrapper around
        // TestServer and an active .Net Core IHost
        System = new SystemUnderTest(builder);

        // There's also a BeforeEachAsync() signature
        System.BeforeEach(httpContext =>
        {
            // Take any kind of setup action before
            // each simulated HTTP request
            
            // In this case, I'm setting a fake JWT token on each request
            // as a demonstration
            httpContext.Request.Headers["Authorization"] = $"Bearer {generateToken()}";
        });

        System.AfterEach(httpContext =>
        {
            // Take any kind of teardown action after
            // each simulated HTTP request
        });

    }

    private string generateToken()
    {
        // In a current project, we implement this method
        // to create a valid JWT token with the claims that
        // the web services require
        return "Fake";
    }

    public SystemUnderTest System { get; }

    public void Dispose()
    {
        System?.Dispose();
    }
}

This new AppFixture class will only be built once by xUnit.Net and shared between test unit classes using the IClassFixture<AppFixture> interface.

Do note that you can express some actions in Alba to take immediately before or after executing an HTTP request through your system for typical setup or teardown operations. In one of my current projects, we exploit this capability to add a pre-canned JWT to the request headers that’s required by our system. In our case, that’s allowing us to test the security integration and the multi-tenancy support that depends on the JWT claims at the same time we’re exercising controller actions and the associated database access underneath it. If you’re familiar with the test pyramid idea, these tests are the middle layers of our testing pyramid.

To simplify the xUnit.Net usage throughout the testing suite, I like to introduce a base class that inevitable accumulates utility methods for running Alba or common database setup and teardown functions. I tend to call this class IntegrationContext and here’s a sample:

public abstract class IntegrationContext : IClassFixture<AppFixture>
{
    protected IntegrationContext(AppFixture fixture)
    {
        Fixture = fixture;
    }

    public AppFixture Fixture { get; }

    /// <summary>
    /// Runs Alba HTTP scenarios through your ASP.Net Core system
    /// </summary>
    /// <param name="configure"></param>
    /// <returns></returns>
    protected Task<IScenarioResult> Scenario(Action<Scenario> configure)
    {
        return Fixture.System.Scenario(configure);
    }

    // The Alba system
    protected SystemUnderTest System => Fixture.System;

    // Just a convenience because you use it pretty often
    // in tests to get at application services
    protected IServiceProvider Services => Fixture.System.Services;

}

Now with the AppFixture and IntegrationContext in place, let’s write some specifications against the AdditionController endpoint shown earlier in this post:

public class DoMathSpecs : IntegrationContext
{
    public DoMathSpecs(AppFixture fixture) : base(fixture)
    {
    }

    // This specification uses the shorthand helpers in Alba
    // that's useful when you really only care about the data
    // going in or out of the HTTP endpoint
    [Fact]
    public async Task do_some_math_adds_and_multiples_shorthand()
    {
        var answers = await System.GetAsJson<Answers>("/math/3/4");
        
        answers.Sum.ShouldBe(7);
        answers.Product.ShouldBe(12);
    }
    
    // This specification shows the longhand way of executing an
    // Alba scenario and using some of its declarative assertions
    // about the expected HTTP response
    [Fact]
    public async Task do_some_math_adds_and_multiples_longhand()
    {
        var result = await Scenario(x =>
        {
            x.Get.Url("/math/3/4");
            x.ContentTypeShouldBe("application/json; charset=utf-8");
        });

        var answers = result.ResponseBody.ReadAsJson<Answers>();
        
        
        answers.Sum.ShouldBe(7);
        answers.Product.ShouldBe(12);
    }
}

Alba can do a lot more to work with the HTTP requests and responses, but I hope this gives you a quick introduction to using Alba for integration testing.

It’s an OSS Nuget Release Party! (Jasper v1.0, Lamar, Alba, Oakton)

My bandwidth for OSS work has been about zero for the past couple months. With the COVID-19 measures drastically reducing my commute and driving kids to school time, getting to actually use some of my projects at work, and a threat from an early adopter to give up on Jasper if I didn’t get something out soon, I suddenly went active again and got quite a bit of backlog things, pull requests, and bug fixes out.

My main focus in terms of OSS development for the past 3-4 years has been a big project called “Jasper” that was originally going to be a modernized .Net Core successor to FubuMVC. Just by way of explanation, Jasper, MO is my ancestral hometown and all the other projects I’m mentioning here are named after either other small towns or landmarks around the titular “Jasper.”

Alba

Alba is a library for HTTP contract testing with ASP.Net Core endpoints. It does quite a bit to reduce the repetitive code from using TestServer by itself in tests and makes your tests be much more declarative and intention revealing. Alba v3.1.1 was released a couple weeks ago to address a problem exposing the application’s IServiceProvider from the Alba SystemUnderTest. Fortunately for once, I caught this one myself while dogfooding Alba on a project at work.

Alba originated with the code we used to test FubuMVC HTTP behavior back in the day, but was modernized to work with ASP.Net Core instead of OWIN, and later retrofitted to just use TestServer under the covers.

Baseline

Baseline is a grab bag of utility code and extension methods on common .Net types that you can’t believe is missing from the BCL, most of which is originally descended from FubuCore.

As an ancillary project, I ripped out the type scanning and assembly finding code from Lamar into a separate BaselineTypeDiscovery Nuget that’s used by most of the other libraries in this post. There was a pretty significant pull request in the latest BaselineTypeDiscovery v1.1.0 release that should improve the application start up time for folks that use Lamar to discover assemblies in their application.

Oakton

Oakton is a command parsing library for .Net that was lifted originally from FubuCore that is used by Jasper. Oakton v2.0.4 and Oakton.AspNetCore v2.1.3 just upgrade the assembly discovery features to use the newer BaselineTypeDiscovery release above.

Lamar

Lamar is a modern, fast, ASP.Net Core compliant IoC container and the successor to StructureMap. I let a pretty good backlog of issues and pull requests amass, so I took some time yesterday to burn that down and the result is Lamar v4.2. This release upgrades the type scanning, fixes some bugs, and added quite a few fixes to the documentation website.

Jasper

Jasper at this point is a command executor ala MediatR (but much more ambitious) and a lightweight messaging framework — but the external messaging will mature much more in subsequent releases.

This feels remarkably anti-climactic seeing as how it’s been my main focus for years, but I pushed Jasper v1.0 today, specifically for some early adopters. The documentation is updated here. There’s also an up to date repository of samples that should grow. I’ll make a much bigger deal out of Jasper when I make the v1.1 or v2.0 release sometime after the Coronavirus has receded and my bandwidth slash ambition level is higher. For right now I’m just wanting to get some feedback from early users and let them beat it up.

Marten

There’s nothing new to say from me about Marten here except that my focus on Jasper has kept me from contributing too much to Marten. With Jasper v1.0 out, I’ll shortly (and finally) turn my attention to helping with the long planned Marten v4 release. For a little bit of synergy, part of my plans there is to use Jasper for some of the advanced Marten event store functionality we’re planning.

 

 

 

 

 

 

My (Big) OSS Plans for 2020

It’s now a yearly thing for me to blog about my aspirations and plans for various OSS projects at the beginning of the year. I was mostly on the nose in 2018, and way, way off in 2019. I’m hoping and planning for a much bigger year in 2020 as I’ve gotten much more enthusiastic and energetic about some ongoing efforts recently.

Most of my time and ambition next year is going toward Jasper, Marten, and Storyteller:

Jasper

Jasper is a toolkit for common messaging scenarios between .Net applications with a robust in process command runner that can be used either with or without the messaging. Jasper wholeheartedly embraces the .Net Core 3.0 ecosystem rather than trying to be its own standalone framework.

Jasper has been gestating for years, I almost gave up on it completely early last year, and purposely set it aside until after .Net Core 3.0 was released. However, I came back to it a few months ago with fresh eyes and full of new ideas from doing a lot of messaging scenario work for Calavista clients. I’m very optimistic right now about Jasper from a purely technical perspective. I’m furiously updating documentation, building sample projects, and dealing with last minute API improvements in an effort to kick out the big v1.0 release sometime in January of 2020.

Marten

Marten is a library that allows .Net developers to treat the outstanding Postgresql database as both a document database and an event store. Marten is mostly supported by a core team, but I’m hoping to be much more involved again this year. The big thing is a proposed v4 release that looks like it’s mostly going to be focused on the event sourcing functionality. There’s an initial GitHub issue for the overall work here, and I want to write a bigger post soon on some ideas about the approach. There’s going to be some new functionality, but the general theme is to make the event sourcing be much more scalable. Marten is a team effort now, and there’s plenty of room for more contributors or collaborators.

For some synergy, I’m also planning on building out some reference applications that use Jasper to integrate Marten with cloud based queueing and load distribution for the asynchronous projection support. I’m excited about this work just to level up on my cloud computing skills.

Storyteller

Storyteller is a tool I’ve worked on and used for what I prefer to call executable specification in .Net (but other folks would call Behavior Driven Development, which I don’t like only because BDD is overloaded). I did quite a bit of preliminary work last year on a proposed Storyteller v6 that would bring it more squarely into the .Net Core world, I wrote a post last year called Spitballing the Future of Storyteller that laid out all my thoughts. I liked where it was heading, but I got distracted by other things.

For more synergy, Storyteller v6 will use Jasper a little bit for its communication between the user interface and the specification process. It also dovetails nicely with my need to update my Javascript UI skillset.

 

Smaller Projects

Lamar — the modern successor to StructureMap and ASP.Net Core compliant IoC container. I will be getting a medium sized maintenance release out very soon as I’ve let the issue list back up. I’m only focused on dealing with problems and questions as they come in.

EDIT 1/6/2020 –> There’s a Lamar 4.1 release out!

Alba — a library that wraps TestServer to make integration testing against ASP.Net Core HTTP endpoints easier. The big work late last year was making it support ASP.Net Core v3. I don’t have any plans to do much with it this year, but that could change quickly if I get to use it on a client project this year.

Oakton — yet another command line parser. It’s used by Jasper, Storyteller, and the Marten command line package. I feel like it’s basically done and said so last year, but I added some specific features for ASP.Net Core applications and might add more along those lines this year.

StructureMap — Same as last year. I answer questions here and there, but otherwise it’s finished/abandoned

FubuMVC — A fading memory, and I’m pleasantly surprised when someone mentions it to me about once or twice a year

 

Environment Checks and Better Command Line Abilities for your .Net Core Application

EDIT Oct. 12th, 2021: Oakton.AspNetCore was folded into Oakton proper as part of the 3.0 release.

Oakton.AspNetCore is a new package built on top of the Oakton 2.0+ command line parser that adds extra functionality to the command line execution of ASP.Net Core and .Net Core 3.0 codebases. At the bottom of this blog post is a small section showing you how to set up Oakton.AspNetCore to run commands in your .Net Core application.

First though, you need to understand that when you use the dotnet run command to build and execute your ASP.Net Core application, you can pass arguments and flags both to dotnet run itself and to your application through the string[] args argument of Program.Main(). These two types of arguments or flags are separated by a double dash, like this example: dotnet run --framework netcoreapp2.0 -- ?. In this case, “–framework netcoreapp2.0” is used by dotnet run itself, and the values to the right of the “–” are passed into your application as the args array.

With that out of the way, let’s see what Oakton.AspNetCore brings to the table.

Extended “Run” Options

In the default ASP.Net Core templates, your application can be started with all its defaults by using dotnet run.  Oakton.AspNetCore retains that usage, but adds some new abilities with its “Run” command. To check the syntax options, type dotnet run -- ? run:

 Usages for 'run' (Runs the configured AspNetCore application)
  run [-c, --check] [-e, --environment <environment>] [-v, --verbose] [-l, --log-level <logleve>] [----config:<prop> <value>]

  ---------------------------------------------------------------------------------------------------------------------------------------
    Flags
  ---------------------------------------------------------------------------------------------------------------------------------------
                        [-c, --check] -> Run the environment checks before starting the host
    [-e, --environment <environment>] -> Use to override the ASP.Net Environment name
                      [-v, --verbose] -> Write out much more information at startup and enables console logging
          [-l, --log-level <logleve>] -> Override the log level
          [----config:<prop> <value>] -> Overwrite individual configuration items
  ---------------------------------------------------------------------------------------------------------------------------------------

To run your application under a different hosting environment name value, use a flag like so:

dotnet run -- --environment Testing

or

dotnet run -- -e Testing

To overwrite configuration key/value pairs, you’ve also got this option:

dotnet run -- --config:key1 value1 --config:key2 value2

which will overwrite the configuration keys for “key1” and “key2” to “value1” and “value2” respectively.

Lastly, you can have any configured environment checks for your application immediately before starting your application by using this flag:

dotnet run -- --check

More on this function in the next section.

Environment Checks

I’m a huge fan of building environment tests directly into your application. Environment tests allow your application to self-diagnose issues with deployment, configuration, or environmental dependencies upfront that would impact its ability to run.

As a very real world example, let’s say your ASP.Net Core application needs to access another web service that’s managed independently by other teams and maybe, just maybe your testers have occasionally tried to test your application when:

  • Your application configuration has the wrong Url for the other web service
  • The other web service isn’t running at all
  • There’s some kind of authentication issue between your application and the other web service

In the real world project that spawned the example above, we added a formal environment check that would try to touch the health check endpoint of the external web service and throw an exception if we couldn’t connect to the external system. The next step was to execute our application as it was configured and deployed with this environment check as part of our Continuous Deployment pipeline. If the environment check failed, the deployment itself failed and triggered off the normal set of failure alerts letting us know to go fix the environment rather than letting our testers waste time on a bad deployment.

With all that said, let’s look at what Oakton.AspNetCore does here to help you add environment checks. Let’s say your application uses a single Sql Server database, and the connection string should be configured in the “connectionString” key of your application’s connection. You would probably want an environment check just to verify at a minimum that you can successfully connect to your database as it’s configured.

In your ASP.Net Core Startup class, you could add a new service registration for an environment check like this example:

// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
    // Other registrations we don't care about...
    
    // This extension method is in Oakton.AspNetCore
    services.CheckEnvironment<IConfiguration>("Can connect to the application database", config =>
    {
        var connectionString = config["connectionString"];
        using (var conn = new SqlConnection(connectionString))
        {
            // Just attempt to open the connection. If there's anything
            // wrong here, it's going to throw an exception
            conn.Open();
        }
    });
}

Now, during deployments or even just pulling down the code to run locally, we can run the environment checks on our application like so:

dotnet run -- check-env

Which in the case of our application above, blows up with output like this because I didn’t add configuration for the database in the first place:

Running Environment Checks
   1.) Failed: Can connect to the application database
System.InvalidOperationException: The ConnectionString property has not been initialized.
   at System.Data.SqlClient.SqlConnection.PermissionDemand()
   at System.Data.SqlClient.SqlConnectionFactory.Permissi
onDemand(DbConnection outerConnection)
   at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1
 retry, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, 
DbConnectionOptions userOptions)
   at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry)
   at System.Data.SqlClient.SqlConnection.Open()
   at MvcApp.Startup.<>c.<ConfigureServices>b_
_4_0(IConfiguration config) in /Users/jeremydmiller/code/oakton/src/MvcApp/Startup.cs:line 41
   at Oakton.AspNetCore.Environment.EnvironmentCheckExtensions.<>c__DisplayClass2_0`1.<CheckEnvironment>b__0(IServ
iceProvider s, CancellationToken c) in /Users/jeremydmiller/code/oakton/src/Oakton.AspNetCore/Environment/EnvironmentCheckExtensions.cs:line 53
   at Oakton.AspNetCore.Environment.LambdaCheck.Assert(IServiceP
rovider services, CancellationToken cancellation) in /Users/jeremydmiller/code/oakton/src/Oakton.AspNetCore/Environment/LambdaCheck.cs:line 19
   at Oakton.AspNetCore.Environment.EnvironmentChecker.ExecuteAll
EnvironmentChecks(IServiceProvider services, CancellationToken token) in /Users/jeremydmiller/code/oakton/src/Oakton.AspNetCore/Environment/EnvironmentChecker.cs:line 31

If you ran this command during continuous deployment scripts, the command should cause your build to fail when it detects environment problems.

In some of Calavista’s current projects , we’ve been adding environment tests to our applications for items like:

  • Can our application read certain configured directories?
  • Can our application as it’s configured connect to databases?
  • Can your application reach other web services?
  • Are required configuration items specified? That’s been an issue as we’ve had to build out Continuous Deployment pipelines to many, many different server environments

I don’t see the idea of “Environment Tests” mentioned very often, and it might have other names I’m not aware of. I learned about the idea back in the Extreme Programming days from a blog post from Nat Pryce that I can’t find any longer, but there’s this paper from those days too.

Add Other Commands

I’ve frequently worked in projects where we’ve built parallel console applications that reproduce a lot of the same IoC and configuration setup to perform administrative tasks or add other diagnostics. It could be things like adding users, rebuilding an event store projection, executing database migrations, or loading some kind of data into the application’s database. What if instead, you could just add these directly to your .Net Core application as additional dotnet run -- [command] options? Fortunately, Oakton.AspNetCore let’s you do exactly that, and even allows you to package up reusable commands in other assemblies that could be distributed by Nuget.

If you use Lamar as your IoC container in an ASP.Net Core application (or .Net Core 3.0 console app using the new unified HostBuilder), we now have an add on Nuget called Lamar.Diagnostics that will add new Oakton commands to your application that give you access to Lamar’s diagnostic tools from the command line. As an example, this library adds a command to write out the “WhatDoIHave()” report for the underlying Lamar IoC container of your application to the command line or a file like this:

dotnet run --lamar-services

Now, using the command above as an example, to build or add your own commands start by decorating the assembly containing the command classes with this attribute:

[assembly:OaktonCommandAssembly]

Having this assembly tells Oakton.AspNetCore to search the assembly for additional Oakton commands. There is no other setup necessary.

If your command needs to use the application’s services or configuration, have the Oakton input type inherit from NetCoreInput type from Oakton.AspNetCore like so:

public class LamarServicesInput : NetCoreInput
{
    // Lots of other flags
}

Next, the new command for “lamar-services” is just this:

[Description("List all the registered Lamar services", Name = "lamar-services")]
public class LamarServicesCommand : OaktonCommand<LamarServicesInput>
{
    public override bool Execute(LamarServicesInput input)
    {
        // BuildHost() will return an IHost for your application
        // if you're using .Net Core 3.0, or IWebHost for
        // ASP.Net Core 2.*
        using (var host = input.BuildHost())
        {
            // The actual execution using host.Services
            // to get at the underlying Lamar Container
        }

        return true;
    }


}

Getting Started

In both cases I’m assuming that you’ve bootstrapped your application with one of the standard project templates like dotnet new webapi or dotnet new mvc. In both cases, you’ll first add a reference to the Oakton.AspNetCore Nuget. Next, break into the Program.Main()entry point method in your project and modify it like the following samples.

If you’re absolutely cutting edge and using ASP.Net Core 3.0:

public class Program
{
    public static Task<int> Main(string[] args)
    {
        return CreateHostBuilder(args)
            
            // This extension method replaces the calls to
            // IWebHost.Build() and Start()
            .RunOaktonCommands(args);
    }

    public static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .ConfigureWebHostDefaults(x => x.UseStartup<Startup>());
    
}

For what I would guess is most folks, the ASP.Net Core 2.* setup (and this would work as well for ASP.Net Core 3.0 as well):

public class Program
{
    public static Task<int> Main(string[] args)
    {
        return CreateWebHostBuilder(args)
            
            // This extension method replaces the calls to
            // IWebHost.Build() and Start()
            .RunOaktonCommands(args);
    }

    public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
        WebHost.CreateDefaultBuilder(args)
            .UseStartup<Startup>();
    
}

The two changes from the template defaults is to:

  1. Change the return value to Task<int>
  2. Replace the calls to IHost.Build() and IHost.Start() to use the RunOaktonCommands(args) extension method that hangs off IWebHostBuilder and the new unified IHostBuilder if you’re targeting netcoreapp3.0.

And that’s it, you’re off to the races.

Alba 3.1 supercharges your ASP.Net Core HTTP Contract Testing

I was just able to push a Nuget for Alba 3.1 that adds support for ASP.Net Core 3.0 and updated the documentation website to reflect the mild additions. Big thanks are in order to Lauri Kotilainen and Jonathan Mezach for making this release happen.

If you’re not familiar with Alba, in its current incarnation it’s a wrapper around the ASP.Net Core TestServer that adds a whole lot of useful helpers to make your testing of ASP.Net Core HTTP services be much more declarative and easier than it is with TestServer alone.

Alba is descended from some built in support for HTTP contract testing in FubuMVC that I salvaged and adapted for usage with ASP.Net Core a few years back. Finally, Alba has used TestServer rather than its own homegrown HTTP runner since the big 3.0 release this January.

My OSS Plans for 2019

I wrote a similar post last year on My OSS Plans for 2018, and it wasn’t too far off what actually happened. I did switch jobs last May, and until recently, that dramatically slowed down my rate of OSS contributions and my ambition level for OSS work. I still enjoy building things, I’m in what’s supposed to be a mostly non-coding role now (but I absolutely still do), and working on OSS projects is just a good way to try to keep my technical skills up. So here we go:

Marten — I’m admittedly not super active with Marten these days as it’s a full fledged community driven project now. There has been a pile up of Linq related issues lately and the issue backlog is getting a bit big, so it really wouldn’t hurt for me to be more active. I happen to love Stephen King’s Dark Tower novels (don’t bother with the movie), but he used to complain that they were hard for him to get into the right head space to write more about Roland and his Ka-tet. That’s basically how I feel about the Linq provider in Marten, but it’s over due for some serious love.

Lamar (was BlueMilk) — Users are reporting some occasional memory usage problems that I think point to issues in Roslyn itself, but in the meantime I’ve got some workarounds in mind to at least alleviate the issue (maybe, hopefully, knock on wood). The one and only big feature idea for this year I have in mind is to finally do the “auto-factory” feature that I hope will knock out a series of user requests for more dynamic behavior.

Alba — I took some time at CodeMash this year and did a pretty substantial 3.0 release that repositioned Alba as a productivity helper on top of TestServer to hopefully eliminate a whole lot of compatibility issues with ASP.Net Core. I don’t know that I have any other plans for Alba this year, but it’s the one tool I work on that I actually get to use at work, so there might be some usability improvements over time.

Storyteller — I’d really love to sink into a pretty big rewrite of the user interface and some incremental — but perfectly backward compatible — improvements to the engine. After a couple years of mostly doing integration work, I suddenly have some UI centric projects ahead of me and I could definitely use some refresher on building web UIs. More on this in a later post.

Jasper — I think I’m finally on track for a 1.0 release in the next couple months, but I’m a little unsure if I want to wait for ASP.Net Core 3.0 or not. After that, it’s mostly going to be trying to build some community around Jasper. Most of my side project time and effort the past three years has been toward Jasper, I have conceptual notes on its architecture that go back at least 5 years, and counting its predecessor project FubuMVC, this has been a 10 year effort for me. Honestly, I think I’m going to finish the push to 1.0 just to feel some sense of completion.

Oakton — I feel like it’s basically done

StructureMap — I answer questions here and there, but otherwise it’s finished/abandoned

FubuMVC — Dead, but little bits and pieces of it live on in Oakton, Alba, and Jasper

 

Alba 3.0 – TestServer on steroids and HTTP contract testing for .Net

Hey all, I was just able to make a 3.0 release of Alba with quite a few improvements. As introduced a couple years ago, Alba is a library you can use in combination with xUnit/NUnit to much more easily author integration tests against ASP.Net Core applications by running HTTP requests in process.

Some salient things:

Big Changes:

  • The core Alba Nuget package supports ASP.Net Core v2.1 and v2.2. Likewise, I’ve tested Alba with netcoreapp2.0 and netcoreapp2.1
  • The Alba.AspNetCore2 nuget has been deprecated
  • All support for ASP.Net Core < 2.0 and targets < netstandard2.0 have been eliminated
  • The latest Alba uses the ASP.Net Core TestServer under the covers. When I first released Alba I got plenty of “why didn’t you just use TestServer?”. So A.) now Alba does and that’s less code we have to support and much less weird incompatibilities to worry about, so yay! B.) Alba provides a lot of value and removes a lot of repetitive grunt work you’d have to deal with if you only used TestServer as is.
  • I added some syntactical sugar for common usages of “I only need to send and receive JSON to and from .Net objects” you can see in the new Getting Started tutorial

Do note that the advent of the Microsoft.AspNetCore.All metapackage makes it unfortunately likely that you’ll bump into assembly binding conflicts at runtime by consuming libraries like Alba. There are tips in the same Getting Started page for preventing or fixing these issues as they arise when you try to use Alba in your tests.

Show me a sample!

Hey, it’s silly, but let’s say you’ve got a Controller in your ASP.Net Core system that looks like this (with its related message DTO types):

    public enum OperationType
    {
        Add,
        Subtract,
        Multiply,
        Divide
    }
    
    public class OperationRequest
    {
        public OperationType Type { get; set; }
        public int One { get; set; }
        public int Two { get; set; }
    }

    public class OperationResult
    {
        public int Answer { get; set; }
        public string Method { get; set; }
    }
    
    
    public class MathController : Controller
    {
        [HttpGet("/math/add/{one}/{two}")]
        public OperationResult Add(int one, int two)
        {
            return new OperationResult
            {
                Answer = one + two
            };
        }

        [HttpPut("/math")]
        public OperationResult Put([FromBody]OperationRequest request)
        {
            switch (request.Type)
            {
                case OperationType.Add:
                    return new OperationResult{Answer = request.One + request.Two, Method = "PUT"};
                
                case OperationType.Multiply:
                    return new OperationResult{Answer = request.One * request.Two, Method = "PUT"};
                
                case OperationType.Subtract:
                    return new OperationResult{Answer = request.One - request.Two, Method = "PUT"};
                
                default:
                    throw new ArgumentOutOfRangeException(nameof(request.Type));
            }
        }
        
        [HttpPost("/math")]
        public OperationResult Post([FromBody]OperationRequest request)
        {
            switch (request.Type)
            {
                case OperationType.Add:
                    return new OperationResult{Answer = request.One + request.Two, Method = "POST"};
                    
                case OperationType.Multiply:
                    return new OperationResult{Answer = request.One * request.Two, Method = "POST"};
                
                case OperationType.Subtract:
                    return new OperationResult{Answer = request.One - request.Two, Method = "POST"};
                
                default:
                    throw new ArgumentOutOfRangeException(nameof(request.Type));
            }
        }

Inside of an xUnit.Net project for your web project, you can write a test for that controller that exercises the full HTTP stack of your application like so:

        [Fact]
        public async Task get_happy_path()
        {
            // SystemUnderTest is from Alba
            // The "Startup" type would be the Startup class from your
            // web application. 
            using (var system = SystemUnderTest.ForStartup<WebApp.Startup>())
            {
                // Issue a request, and check the results
                var result = await system
                    .GetAsJson<OperationResult>("/math/add/3/4");
                
                result.Answer.ShouldBe(7);
            }
        }

        [Fact]
        public async Task post_and_expect_response()
        {
            using (var system = SystemUnderTest.ForStartup<WebApp.Startup>())
            {
                var request = new OperationRequest
                {
                    Type = OperationType.Multiply,
                    One = 3,
                    Two = 4
                };

                var result = await system
                    .PostJson(request, "/math")
                    .Receive<OperationResult>();
                
                result.Answer.ShouldBe(12);
                result.Method.ShouldBe("POST");
            }
        }

        [Fact]
        public async Task put_and_expect_response()
        {
            using (var system = SystemUnderTest.ForStartup<WebApp.Startup>())
            {
                var request = new OperationRequest
                {
                    Type = OperationType.Subtract,
                    One = 3,
                    Two = 4
                };

                var result = await system
                    .PutJson(request, "/math")
                    .Receive<OperationResult>();
                
                result.Answer.ShouldBe(-1);
                result.Method.ShouldBe("PUT");
            }
        }