Tag Archives: Lamar

Using Oakton for Development-Time Commands in .Net Core Applications

All of the sample code in this blog post is on GitHub in the OaktonDevelopmentCommands project.

Last year I released the Oakton.AspNetCore library that provides an expanded command line experience for ASP.Net Core applications (and environment tests too!) that adds additional command line flags for the basic dotnet run behavior. Oakton.AspNetCore also gives you the ability to embed completely different named command directly into your application — either from your own application or Oakton.AspNetCore can pull in commands from Nuget libraries.

To make this concrete, I started a new sample project on GitHub with the dotnet new webapi template. To add the new command line experience, I added a Nuget reference to Oakton.AspNetCore and modified the Program.Main() method to this:

// I changed the return type to Task
public static Task Main(string[] args)
{
    return CreateHostBuilder(args)
        // This extension method makes Oakton the active
        // command line parser and executor
        .RunOaktonCommands(args);
}

To show the ability to add commands from an external library, I also swapped out the built in DI container with Lamar, and added a reference to the Lamar.Diagnostics Nuget to expose Lamar’s diagnostics reports via the command line.

Just to show the Lamar integration, I added just the one line of code to the host builder configuration you can see below:

public static IHostBuilder CreateHostBuilder(string[] args) =>
    Host.CreateDefaultBuilder(args)
        .UseLamar() // Overriding the IoC container to use Lamar
        .ConfigureWebHostDefaults(webBuilder =>
        {
            webBuilder.UseStartup();
        });

Now, to switch the actual command line to see the results of all of that, you can see Oakton’s built in command line help by typing the command dotnet run --?, which gives this output:

  ------------------------------------------------------------------------------
    Available commands:
  ------------------------------------------------------------------------------
         check-env -> Execute all environment checks against the application
          describe -> Writes out a description of your running application to either the console or a file
    lamar-scanning -> Runs Lamar's type scanning diagnostics
    lamar-services -> List all the registered Lamar services
    lamar-validate -> Runs all the Lamar container validations
               run -> Runs the configured AspNetCore application
          sayhello -> I'm a simple command that just starts up the app and says hello
  ------------------------------------------------------------------------------

A couple notes about the output above:

  • With the dotnet cli mechanics, the basic usage is dotnet [command] [optional flags to dotnet itself] -- [arguments to your application]. The double dashes marks the boundaries between arguments and flags meant for dotnet itself and arguments or flags for your application. So as an example, to run your application compiled as “Release” and using the “Testing” environment name for your .Net Core application, the command would be dotnet run --configuration Release -- --environment Testing.
  • The check-env, describe, and run commands come in from the base Oakton.AspNetCore library. The run command is the default, so dotnet run actually delegates to the run command.
  • All the lamar-******* commands come from Lamar.Diagnostics because Oakton.AspNetCore can happily find and apply commands from other assemblies in the project.
  • We’ll build out sayhello later in this post

Assuming that you are a Lamar (or a StructureMap) user, the first step to unravel most IoC configuration problems is the old WhatDoIHave() method to see what the service registrations are. To get at that data faster, you can use the dotnet run -- lamar-services command just to dump out the WhatDoIHave() output to the console or a text file.

However, adding Lamar.Diagnostics to your application adds some dependencies you may not want to be deployed. With a little help from the .Net Core SDK project system, we can just tell .Net to only use Lamar.Diagnostics at development time by using the <PrivateAssets> tag in the csproj file like this:

<PackageReference Include="Lamar.Diagnostics" Version="1.1.5">
    <PrivateAssets>all</PrivateAssets>
</PackageReference>

 

I don’t know how to do that without breaking into the csproj file, but once you do, the Lamar.Diagnostics assembly will not be deployed if you’re using dotnet publish to bundle up your application for deployment.

The “AspNetCore” naming is unfortunately a misnomer, because as of .Net Core 3.* the Oakton extension works for any project configured and bootstrapped with the new generic HostBuilder (purely console applications, worker applications, or any kind of web application).

Now, to add a custom command to the application directly into the application without polluting the deployed application, we’re just using some conditional compilation as shown in this super simple example:

// This is necessary to tell Oakton
// to search this assembly for Oakton commands
[assembly:Oakton.OaktonCommandAssembly]

namespace OaktonDevelopmentCommands
{
    // The conditional compilation here just keeps this command from
    // being present in the Release build of the application
#if DEBUG
    // This is also an OaktonAsyncCommand if you need to 
    // call async APIs
    [Description("I'm a simple command that just starts up the app and says hello")]
    public class SayHelloCommand : OaktonCommand<NetCoreInput>
    {
        public override bool Execute(NetCoreInput input)
        {
            // Super cheesy, just starting up the application
            // and shutting it right down
            using (var host = input.BuildHost())
            {
                // You do have access to the host's underlying
                // IoC provider, and hence to any application service
                // including the compiled IConfiguration as well
                
                Console.WriteLine("Hey, I can start up the application!");
            }

            // Gotta return true to let Oakton know that the command succeeded
            // This is important if you're using commands that need to report
            // success or failure to the command line.
            return true;
        }
    }
#endif
}

It’s hoke-y, but the command up above will only be compiled into your application if you are compiling as “Debug” — as you generally do when you’re working locally or running tests. When you deploy as a “Release” build, this command won’t be part of the compiled executable. As silly as it looks, this is being very useful in one of my client projects right now.

It’s an OSS Nuget Release Party! (Jasper v1.0, Lamar, Alba, Oakton)

My bandwidth for OSS work has been about zero for the past couple months. With the COVID-19 measures drastically reducing my commute and driving kids to school time, getting to actually use some of my projects at work, and a threat from an early adopter to give up on Jasper if I didn’t get something out soon, I suddenly went active again and got quite a bit of backlog things, pull requests, and bug fixes out.

My main focus in terms of OSS development for the past 3-4 years has been a big project called “Jasper” that was originally going to be a modernized .Net Core successor to FubuMVC. Just by way of explanation, Jasper, MO is my ancestral hometown and all the other projects I’m mentioning here are named after either other small towns or landmarks around the titular “Jasper.”

Alba

Alba is a library for HTTP contract testing with ASP.Net Core endpoints. It does quite a bit to reduce the repetitive code from using TestServer by itself in tests and makes your tests be much more declarative and intention revealing. Alba v3.1.1 was released a couple weeks ago to address a problem exposing the application’s IServiceProvider from the Alba SystemUnderTest. Fortunately for once, I caught this one myself while dogfooding Alba on a project at work.

Alba originated with the code we used to test FubuMVC HTTP behavior back in the day, but was modernized to work with ASP.Net Core instead of OWIN, and later retrofitted to just use TestServer under the covers.

Baseline

Baseline is a grab bag of utility code and extension methods on common .Net types that you can’t believe is missing from the BCL, most of which is originally descended from FubuCore.

As an ancillary project, I ripped out the type scanning and assembly finding code from Lamar into a separate BaselineTypeDiscovery Nuget that’s used by most of the other libraries in this post. There was a pretty significant pull request in the latest BaselineTypeDiscovery v1.1.0 release that should improve the application start up time for folks that use Lamar to discover assemblies in their application.

Oakton

Oakton is a command parsing library for .Net that was lifted originally from FubuCore that is used by Jasper. Oakton v2.0.4 and Oakton.AspNetCore v2.1.3 just upgrade the assembly discovery features to use the newer BaselineTypeDiscovery release above.

Lamar

Lamar is a modern, fast, ASP.Net Core compliant IoC container and the successor to StructureMap. I let a pretty good backlog of issues and pull requests amass, so I took some time yesterday to burn that down and the result is Lamar v4.2. This release upgrades the type scanning, fixes some bugs, and added quite a few fixes to the documentation website.

Jasper

Jasper at this point is a command executor ala MediatR (but much more ambitious) and a lightweight messaging framework — but the external messaging will mature much more in subsequent releases.

This feels remarkably anti-climactic seeing as how it’s been my main focus for years, but I pushed Jasper v1.0 today, specifically for some early adopters. The documentation is updated here. There’s also an up to date repository of samples that should grow. I’ll make a much bigger deal out of Jasper when I make the v1.1 or v2.0 release sometime after the Coronavirus has receded and my bandwidth slash ambition level is higher. For right now I’m just wanting to get some feedback from early users and let them beat it up.

Marten

There’s nothing new to say from me about Marten here except that my focus on Jasper has kept me from contributing too much to Marten. With Jasper v1.0 out, I’ll shortly (and finally) turn my attention to helping with the long planned Marten v4 release. For a little bit of synergy, part of my plans there is to use Jasper for some of the advanced Marten event store functionality we’re planning.

 

 

 

 

 

 

Lamar 4.1: Multithreading improvements, diagnostics, documentation updates, and some thoughts on troubleshooting

As promised in my previous post My (Big) OSS Plans for 2020, the very first thing out the gate for me this year is a bug fix release for Lamar.

Lamar 4.1 and its related libraries was released on Nuget late last week with a variety of bug fixes, a couple new features, and a documentation refresh at https://jasperfx.github.io/lamar — including some new guidance here and there as a direct reaction to GitHub issues. Continuing my personal theme of OSS interactions being more positive than not over the past couple years, I received a lot of help from Lamar users. This release was largely the result of users submitting pull requests with fixes or failing unit tests that reproduced issues — and I cannot stress enough how helpful those reproduction tests are for an OSS maintainer. Another user took the time to investigate how an error message could be greatly improved. Thank you to all the users who helped on this release with pull requests, suggestions, and feedback on GitHub.

All told, the libraries updated are:

  • Lamar 4.1.0 — Multi-threading issues were finally addressed, fixes for Lamar + ASP.Net Core logging, some finer grained control over type scanning registrations
  • Lamar.Microsoft.DependencyInjection v4.1.0 — Adds support back for IWebHostBuilder for .Net Core 3.0 applications
  • Lamar.Diagnostics v1.1.3 — More on this one below
  • LamarCompiler 2.1.1 — Just updated the Roslyn dependencies. Lamar itself doesn’t use this, so you’re very unlikely to be impacted
  • LamarCodeGeneration v1.4.0 — Some small additions to support a couple Jasper use cases
  • LamarCodeGeneration.Commands v1.0.2 — This isn’t documented yet, and is really just to support some diagnostics and pre-generation of code for Jasper

 

A Note on Troubleshooting

I have a partially written blog post slash treatise on troubleshooting and debugging. One of the things I try to suggest when troubleshooting a technical issue is to come up with a series of theories about why something isn’t working and figuring out the quickest way to prove or disprove that theory.

In the case of Lamar’s multi-threading issues addressed in this release and a very similar issue fixed previously, the “obvious” theory was that somewhere there was some issue with data structures or locking. Myself and several others tried to investigate Lamar’s internals down this path, but came up empty handed.

The actual root cause turned out to be related to the Expression construction and compilation inside of Lamar that allowed variables to bleed through threads in heavily multi-threaded usage.

So, I still think that my idea of “build a theory about why something is failing, then try to knock it down” is a good approach, but not 100% effective. I’m adding a section to that blog post entitled “don’t get tunnel vision” and talk about fixating on one theory and not considering other explanations;-)

Then again, some things are just hard some time.

Lamar.Diagnostics

Back in October I blogged about the new Oakton.AspNetCore package that extends the command line capabilities of standard .Net Core / ASP.Net Core applications with additional diagnostics. As a drop in extension to Oakton.AspNetCore, the new Lamar.Diagnostics package can be installed into a .Net Core application to give you ready access to all of Lamar’s built in diagnostics through the command line.

Your newly available commands from the root of your project with Lamar.Diagnostics are:

  1. dotnet run -- lamar-services — prints out the Container.WhatDoIHave() output to the console or a designated file
  2. dotnet run -- lamar-scanning — prints out the Container.WhatDidIScan() output to the console or a designated file
  3. dotnet run -- lamar-validate — runs the Container.AssertConfigurationIsValid() command

You can also opt into Lamar’s built in environment tests being used through Oakton.AspNetCore’s environment check capability.

In all cases, these commands work by calling IHostBuilder.Build() to build up your application — but doesn’t call Start() so none of your IHostedService objects will run — and calls the underlying Container methods. By doing this, you get ready access to the Lamar diagnostics against your application exacstly the way that your applicaiton is configured, without you having to add any additional code to your system to get at this diagnostic information.

 

 

My (Big) OSS Plans for 2020

It’s now a yearly thing for me to blog about my aspirations and plans for various OSS projects at the beginning of the year. I was mostly on the nose in 2018, and way, way off in 2019. I’m hoping and planning for a much bigger year in 2020 as I’ve gotten much more enthusiastic and energetic about some ongoing efforts recently.

Most of my time and ambition next year is going toward Jasper, Marten, and Storyteller:

Jasper

Jasper is a toolkit for common messaging scenarios between .Net applications with a robust in process command runner that can be used either with or without the messaging. Jasper wholeheartedly embraces the .Net Core 3.0 ecosystem rather than trying to be its own standalone framework.

Jasper has been gestating for years, I almost gave up on it completely early last year, and purposely set it aside until after .Net Core 3.0 was released. However, I came back to it a few months ago with fresh eyes and full of new ideas from doing a lot of messaging scenario work for Calavista clients. I’m very optimistic right now about Jasper from a purely technical perspective. I’m furiously updating documentation, building sample projects, and dealing with last minute API improvements in an effort to kick out the big v1.0 release sometime in January of 2020.

Marten

Marten is a library that allows .Net developers to treat the outstanding Postgresql database as both a document database and an event store. Marten is mostly supported by a core team, but I’m hoping to be much more involved again this year. The big thing is a proposed v4 release that looks like it’s mostly going to be focused on the event sourcing functionality. There’s an initial GitHub issue for the overall work here, and I want to write a bigger post soon on some ideas about the approach. There’s going to be some new functionality, but the general theme is to make the event sourcing be much more scalable. Marten is a team effort now, and there’s plenty of room for more contributors or collaborators.

For some synergy, I’m also planning on building out some reference applications that use Jasper to integrate Marten with cloud based queueing and load distribution for the asynchronous projection support. I’m excited about this work just to level up on my cloud computing skills.

Storyteller

Storyteller is a tool I’ve worked on and used for what I prefer to call executable specification in .Net (but other folks would call Behavior Driven Development, which I don’t like only because BDD is overloaded). I did quite a bit of preliminary work last year on a proposed Storyteller v6 that would bring it more squarely into the .Net Core world, I wrote a post last year called Spitballing the Future of Storyteller that laid out all my thoughts. I liked where it was heading, but I got distracted by other things.

For more synergy, Storyteller v6 will use Jasper a little bit for its communication between the user interface and the specification process. It also dovetails nicely with my need to update my Javascript UI skillset.

 

Smaller Projects

Lamar — the modern successor to StructureMap and ASP.Net Core compliant IoC container. I will be getting a medium sized maintenance release out very soon as I’ve let the issue list back up. I’m only focused on dealing with problems and questions as they come in.

EDIT 1/6/2020 –> There’s a Lamar 4.1 release out!

Alba — a library that wraps TestServer to make integration testing against ASP.Net Core HTTP endpoints easier. The big work late last year was making it support ASP.Net Core v3. I don’t have any plans to do much with it this year, but that could change quickly if I get to use it on a client project this year.

Oakton — yet another command line parser. It’s used by Jasper, Storyteller, and the Marten command line package. I feel like it’s basically done and said so last year, but I added some specific features for ASP.Net Core applications and might add more along those lines this year.

StructureMap — Same as last year. I answer questions here and there, but otherwise it’s finished/abandoned

FubuMVC — A fading memory, and I’m pleasantly surprised when someone mentions it to me about once or twice a year

 

Environment Checks and Better Command Line Abilities for your .Net Core Application

Oakton.AspNetCore is a new package built on top of the Oakton 2.0+ command line parser that adds extra functionality to the command line execution of ASP.Net Core and .Net Core 3.0 codebases. At the bottom of this blog post is a small section showing you how to set up Oakton.AspNetCore to run commands in your .Net Core application.

First though, you need to understand that when you use the dotnet run command to build and execute your ASP.Net Core application, you can pass arguments and flags both to dotnet run itself and to your application through the string[] args argument of Program.Main(). These two types of arguments or flags are separated by a double dash, like this example: dotnet run --framework netcoreapp2.0 -- ?. In this case, “–framework netcoreapp2.0” is used by dotnet run itself, and the values to the right of the “–” are passed into your application as the args array.

With that out of the way, let’s see what Oakton.AspNetCore brings to the table.

Extended “Run” Options

In the default ASP.Net Core templates, your application can be started with all its defaults by using dotnet run.  Oakton.AspNetCore retains that usage, but adds some new abilities with its “Run” command. To check the syntax options, type dotnet run -- ? run:

 Usages for 'run' (Runs the configured AspNetCore application)
  run [-c, --check] [-e, --environment <environment>] [-v, --verbose] [-l, --log-level <logleve>] [----config:<prop> <value>]

  ---------------------------------------------------------------------------------------------------------------------------------------
    Flags
  ---------------------------------------------------------------------------------------------------------------------------------------
                        [-c, --check] -> Run the environment checks before starting the host
    [-e, --environment <environment>] -> Use to override the ASP.Net Environment name
                      [-v, --verbose] -> Write out much more information at startup and enables console logging
          [-l, --log-level <logleve>] -> Override the log level
          [----config:<prop> <value>] -> Overwrite individual configuration items
  ---------------------------------------------------------------------------------------------------------------------------------------

To run your application under a different hosting environment name value, use a flag like so:

dotnet run -- --environment Testing

or

dotnet run -- -e Testing

To overwrite configuration key/value pairs, you’ve also got this option:

dotnet run -- --config:key1 value1 --config:key2 value2

which will overwrite the configuration keys for “key1” and “key2” to “value1” and “value2” respectively.

Lastly, you can have any configured environment checks for your application immediately before starting your application by using this flag:

dotnet run -- --check

More on this function in the next section.

 

Environment Checks

I’m a huge fan of building environment tests directly into your application. Environment tests allow your application to self-diagnose issues with deployment, configuration, or environmental dependencies upfront that would impact its ability to run.

As a very real world example, let’s say your ASP.Net Core application needs to access another web service that’s managed independently by other teams and maybe, just maybe your testers have occasionally tried to test your application when:

  • Your application configuration has the wrong Url for the other web service
  • The other web service isn’t running at all
  • There’s some kind of authentication issue between your application and the other web service

In the real world project that spawned the example above, we added a formal environment check that would try to touch the health check endpoint of the external web service and throw an exception if we couldn’t connect to the external system. The next step was to execute our application as it was configured and deployed with this environment check as part of our Continuous Deployment pipeline. If the environment check failed, the deployment itself failed and triggered off the normal set of failure alerts letting us know to go fix the environment rather than letting our testers waste time on a bad deployment.

With all that said, let’s look at what Oakton.AspNetCore does here to help you add environment checks. Let’s say your application uses a single Sql Server database, and the connection string should be configured in the “connectionString” key of your application’s connection. You would probably want an environment check just to verify at a minimum that you can successfully connect to your database as it’s configured.

In your ASP.Net Core Startup class, you could add a new service registration for an environment check like this example:

// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
    // Other registrations we don't care about...
    
    // This extension method is in Oakton.AspNetCore
    services.CheckEnvironment<IConfiguration>("Can connect to the application database", config =>
    {
        var connectionString = config["connectionString"];
        using (var conn = new SqlConnection(connectionString))
        {
            // Just attempt to open the connection. If there's anything
            // wrong here, it's going to throw an exception
            conn.Open();
        }
    });
}

Now, during deployments or even just pulling down the code to run locally, we can run the environment checks on our application like so:

dotnet run -- check-env

Which in the case of our application above, blows up with output like this because I didn’t add configuration for the database in the first place:

Running Environment Checks
   1.) Failed: Can connect to the application database
System.InvalidOperationException: The ConnectionString property has not been initialized.
   at System.Data.SqlClient.SqlConnection.PermissionDemand()
   at System.Data.SqlClient.SqlConnectionFactory.Permissi
onDemand(DbConnection outerConnection)
   at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1
 retry, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, 
DbConnectionOptions userOptions)
   at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry)
   at System.Data.SqlClient.SqlConnection.Open()
   at MvcApp.Startup.<>c.<ConfigureServices>b_
_4_0(IConfiguration config) in /Users/jeremydmiller/code/oakton/src/MvcApp/Startup.cs:line 41
   at Oakton.AspNetCore.Environment.EnvironmentCheckExtensions.<>c__DisplayClass2_0`1.<CheckEnvironment>b__0(IServ
iceProvider s, CancellationToken c) in /Users/jeremydmiller/code/oakton/src/Oakton.AspNetCore/Environment/EnvironmentCheckExtensions.cs:line 53
   at Oakton.AspNetCore.Environment.LambdaCheck.Assert(IServiceP
rovider services, CancellationToken cancellation) in /Users/jeremydmiller/code/oakton/src/Oakton.AspNetCore/Environment/LambdaCheck.cs:line 19
   at Oakton.AspNetCore.Environment.EnvironmentChecker.ExecuteAll
EnvironmentChecks(IServiceProvider services, CancellationToken token) in /Users/jeremydmiller/code/oakton/src/Oakton.AspNetCore/Environment/EnvironmentChecker.cs:line 31

If you ran this command during continuous deployment scripts, the command should cause your build to fail when it detects environment problems.

In some of Calavista’s current projects , we’ve been adding environment tests to our applications for items like:

  • Can our application read certain configured directories?
  • Can our application as it’s configured connect to databases?
  • Can your application reach other web services?
  • Are required configuration items specified? That’s been an issue as we’ve had to build out Continuous Deployment pipelines to many, many different server environments

I don’t see the idea of “Environment Tests” mentioned very often, and it might have other names I’m not aware of. I learned about the idea back in the Extreme Programming days from a blog post from Nat Pryce that I can’t find any longer, but there’s this paper from those days too.

 

Add Other Commands

I’ve frequently worked in projects where we’ve built parallel console applications that reproduce a lot of the same IoC and configuration setup to perform administrative tasks or add other diagnostics. It could be things like adding users, rebuilding an event store projection, executing database migrations, or loading some kind of data into the application’s database. What if instead, you could just add these directly to your .Net Core application as additional dotnet run -- [command] options? Fortunately, Oakton.AspNetCore let’s you do exactly that, and even allows you to package up reusable commands in other assemblies that could be distributed by Nuget.

If you use Lamar as your IoC container in an ASP.Net Core application (or .Net Core 3.0 console app using the new unified HostBuilder), we now have an add on Nuget called Lamar.Diagnostics that will add new Oakton commands to your application that give you access to Lamar’s diagnostic tools from the command line. As an example, this library adds a command to write out the “WhatDoIHave()” report for the underlying Lamar IoC container of your application to the command line or a file like this:

dotnet run --lamar-services

Now, using the command above as an example, to build or add your own commands start by decorating the assembly containing the command classes with this attribute:

[assembly:OaktonCommandAssembly]

Having this assembly tells Oakton.AspNetCore to search the assembly for additional Oakton commands. There is no other setup necessary.

If your command needs to use the application’s services or configuration, have the Oakton input type inherit from NetCoreInput type from Oakton.AspNetCore like so:

public class LamarServicesInput : NetCoreInput
{
    // Lots of other flags
}

Next, the new command for “lamar-services” is just this:

[Description("List all the registered Lamar services", Name = "lamar-services")]
public class LamarServicesCommand : OaktonCommand<LamarServicesInput>
{
    public override bool Execute(LamarServicesInput input)
    {
        // BuildHost() will return an IHost for your application
        // if you're using .Net Core 3.0, or IWebHost for
        // ASP.Net Core 2.*
        using (var host = input.BuildHost())
        {
            // The actual execution using host.Services
            // to get at the underlying Lamar Container
        }

        return true;
    }


}

Getting Started

In both cases I’m assuming that you’ve bootstrapped your application with one of the standard project templates like dotnet new webapi or dotnet new mvc. In both cases, you’ll first add a reference to the Oakton.AspNetCore Nuget. Next, break into the Program.Main()entry point method in your project and modify it like the following samples.

If you’re absolutely cutting edge and using ASP.Net Core 3.0:

public class Program
{
    public static Task<int> Main(string[] args)
    {
        return CreateHostBuilder(args)
            
            // This extension method replaces the calls to
            // IWebHost.Build() and Start()
            .RunOaktonCommands(args);
    }

    public static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .ConfigureWebHostDefaults(x => x.UseStartup<Startup>());
    
}

For what I would guess is most folks, the ASP.Net Core 2.* setup (and this would work as well for ASP.Net Core 3.0 as well):

public class Program
{
    public static Task<int> Main(string[] args)
    {
        return CreateWebHostBuilder(args)
            
            // This extension method replaces the calls to
            // IWebHost.Build() and Start()
            .RunOaktonCommands(args);
    }

    public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
        WebHost.CreateDefaultBuilder(args)
            .UseStartup<Startup>();
    
}

The two changes from the template defaults is to:

  1. Change the return value to Task<int>
  2. Replace the calls to IHost.Build() and IHost.Start() to use the RunOaktonCommands(args) extension method that hangs off IWebHostBuilder and the new unified IHostBuilder if you’re targeting netcoreapp3.0.

And that’s it, you’re off to the races.

 

Lamar and Oakton join the .Net Core 3.0 Party

Like many other .Net OSS authors, I’ve been putting in some time this week making sure that various .Net tools support the brand spanking new .Net Core and ASP.Net Core 3.0 bits that were just released. First up is Oakton and Lamar, with the rest of the SW Missouri projects to follow soon.

Oakton

Oakton is yet another command line parser tool for .Net. The main Oakton 2.0.1 library adds support for the Netstandard2.1 target, but does not change in any other way. The Oakton.AspNetCore library got a full 2.0.0 release. If you’re using ASP.Net Core v2.0, your usage is unchanged. However, if you are targeting netcoreapp3.0, the extension methods now depend on the newly unified IHostBuilder rather than the older IWebHostBuilder and the bootstrapping looks like this now:

public class Program
{
    public static Task<int> Main(string[] args)
    {
        return CreateHostBuilder(args)
            
            // This extension method replaces the calls to
            // IWebHost.Build() and Start()
            .RunOaktonCommands(args);
    }

    public static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .ConfigureWebHostDefaults(x => x.UseStartup<Startup>());
    
}

 

Oakton.AspNetCore provides an improved and extensible command line experience for ASP.Net Core. I’ve been meaning to write a bigger blog post about it, but that’s gonna wait for another day.

Lamar

The 3.0 support in Lamar was unpleasant, because the targeting covers the spread from .Net 4.6.1 to netstandard2.0 to netstandard2.1, and the test library covered several .Net runtimes. After this, I’d really like to not have to type “#if/#else/#endif” ever, ever again (and I do use ReSharper/Rider’s ALT-CTRL-J surround with feature religiously that helps, but you get my point).

The new bits and releases are:

  • Lamar v3.2.0 adds a netstandard2.1 target
  • LamarCompiler v2.1.0 adds a netstandard2.1 target (probably only used by Jasper, but who knows who’s picked it up, so I updated it)
  • LamarCodeGeneration v1.1.0 adds a netstandard2.1 target but is otherwise unchanged
  • Lamar.Microsoft.DependencyInjection v4.0.0 — this is the adapter library to replace the built in DI container in ASP.Net Core with Lamar. This is a full point release because some of the method signatures changed. I deleted the special Lamar handling for IOption<T> and ILogger<T> because they no longer added any value. If your application targets netcoreapp3.0, the UseLamar() method and its overloads hang off of IHostBuilder instead of IWebHostBuilder. If you are remaining on ASP.Net Core v2.*, UseLamar() still hangs off of IWebHostBuilder
  • Lamar.Diagnostics v1.1.0 — assuming that you use the Oakton.AspNetCore command line adapter for your ASP.Net Core application, adding a reference to this library adds new commands to expose the Lamar diagnostic capabilities from the command line of your application. This version targets both netstandard2.0 for ASP.Net Core v2.* and netstandard2.1 for ASP.Net Core 3.*.

 

The challenges

The biggest problem was that in both of these projects, I wanted to retain support for .Net 4.6+ and netstandard2.0 or netcoreapp2.* runtime targets. That’s unfortunately meant a helluva lot of conditional compilation and conditional Nuget references per target framework. In some cases, the move from the old ASP.Net Core <=2.* IWebHostBuilder to the newer, unified IHostBuilder took some doing in both the real code and in the unit tests. Adding to the fun is that there are real differences sometimes between the old, full .Net framework, netcoreapp2.0, netcoreapp2.1/2, and certainly the brand new netcoreapp3.0.

Another little hiccup was that the dotnet pack pathing support was fixed to what it was originally in the project.json early days, but that broke all our build scripts and that had to be adjusted (the artifacts path is relative to the current directory now rather than to the binary target path like is was).

At least on AppVeyor, we had to force the existing image to install the latest .Net SDK as part of the build because the image we were using didn’t yet support the brand new .Net SDK. I’d assume that is very temporary and I can’t speak to other hosted CI providers. If you’re wondering how to do that, see this example from Lamar that I got from Jimmy Byrd.

 

Other Projects I’m Involved With

  • StructureMap — If someone feels like doing it, I’d take a pull request to StructureMap.Microsoft.DependencyInjection to put out a .Net Core 3.0 compatible version (really just means supporting the new IHostBuilder instead of or in addition to the old IWebHostBuilder).
  • Alba — Some other folks are working on that one, and I’m anticipating an Alba on ASP.Net Core 3.0 release in the next couple days. I’ll write a follow up blog post when that’s ready
  • Marten — I’m not anticipating much effort here, but we should at least have our testing libraries target netcoreapp3.0 and see what happens. We did have some issues with netcoreapp2.1, so we’ll see I guess
  • Jasper — I’ve admittedly had Jasper mostly on ice until .Net Core 3.0 was released. I’ll start work on Jasper next week, but in that is going to be a true conversion up to netcoreapp3.0 and some additional structural changes to finally get to a 1.0 release of that thing.
  • Storyteller — I’m not sure yet, but I’ve started doing a big overhaul of Storyteller that’s gotten derailed by the .Net Core 3.0 stuff

 

Lamar 3.1 – New Diagnostics and more

Lamar 3.1.0 was published today with a few fixes, new diagnostics, and a newly improved equivalent to StructureMap’s old “Forward” registration. The full details are here.

There’s one big bug fix on multi-threaded usage that was impacting folks using Lamar with MediatR (it’s worth a later blog post about debugging problems because this took several folks and at least two attempts from me to finally diagnose and solve).

“Inverse” Registration

For the not uncommon need to have one single object be the implementation for multiple role interfaces within the container, there’s the new “inverse” registration mechanism as shown below:

[Fact]
public void when_singleton_both_interfaces_give_same_instance()
{
    var container = new Container(services =>
    {
        services.Use<Implementation>()
            .Singleton()
            .For<IServiceA>()
            .For<IServiceB>();
    });

    var instanceA = container.GetInstance<IServiceA>();
    var instanceB = container.GetInstance<IServiceB>();

    instanceA.ShouldBeTheSameAs(instanceB);
}

What you see above is registering the concrete type Implementation as the shared, underlying service for both IServiceA and IServiceBregistrations.

HowDoIBuild

Lamar has been able to show you the “build plan” for registrations since the beginning, but now there’s a helper Container.HowDoIBuild() method that gives you easier access to this information that can be helpful for troubleshooting IoC configuration issues.

Lamar.Diagnostics

More on this one later, but there’s also a brand new package called Lamar.Diagnostics that, in conjunction with Oakton.AspNetCore, adds command line access to all the Lamar diagnostic capabilities directly to your ASP.Net Core application.

 

 

Lamar v3 is Released: Faster, smaller, quicker cold starts, internal type friendly

Lamar has had some churn lately with me almost giving up on it, changing my mind, and look at that, publishing a very significantly improved version up to Nuget just now.

I just published Lamar v3.0 with some bug fixes and big internal improvements. This release is mostly significant because it eliminates Lamar’s previous reliance on Roslyn for runtime code generation in favor of a more common model that builds Expression trees and compiles dynamic lambdas — inevitably with the excellent FastExpressionCompiler for maximum performance and faster cold starts.

All told, Lamar v3.0:

  • Has much faster “cold start” times compared to ❤
  • Is faster across the board, but especially when internal types are involved. The very early user feedback is “holy moses that’s much faster”
  • I think that we’re through with compatibility with internal types being registered into the container because the dynamic Lambdas don’t have to care about the type visibility
  • Will use way less memory without the Roslyn dependency tree in memory
  • Has far fewer dependencies now that some users complained about

There are no API changes to Lamar itself, but there’s now a different set of Nuget libraries:

  1. LamarCompiler — only has Lamar’s old helper for executing the Roslyn runtime compilation, but this is no longer referenced by Lamar itself
  2. LamarCodeGeneration — the Frame / Variable model that Lamar uses to generate code and now Expression trees in memory. This is used by Lamar itself, could be used as a stand alone library, and has no direct coupling to LamarCompiler
  3. Lamar itself

A touch of history…

I’ve been absolutely exhausted for years with maintaining StructureMap and dealing with user questions and oddball problems. I also knew it had some pretty ugly performance issues and it had been a massive pain in the ass force fitting StructureMap’s long standing functionality into ASP.Net Core’s new compliance requirements (that appear to have been made up by the ASP.Net team out of whole cloth with ZERO input from the major OSS IoC tool authors). I also knew from some experimentation that I wasn’t going to be able to fix StructureMap very easily, so…

Lamar was an attempt by me to throw together a new IoC tool that would mostly mimic StructureMap‘s public API (everybody has opinions, but there are actually quite a few people who like StructureMap’s usability and it’s had ~6 million downloads) and a large subset of its functionality. At the same time, I did change its lifecycle semantics and behavior to be ASP.Net Core-compliant from the very get go to head off more problems with that.

Lamar’s original model was based around using dynamic code generation and compilation with Roslyn. Woohoo, it was easy to use and I was able to get a large subset of StructureMap’s old functionality working over Christmas break a couple years ago. Unfortunately, that model doesn’t play well at all with internal types and my limited workarounds were pretty clumsy. It turns out that lots and lots of people and common frameworks like to put internal types into IoC containers (looking at you ASP.Net Core). So here we are. Lamar switched over to using Expressions compiled into Funcs like basically every other IoC container I’m aware of, but hopefully it starts fulfilling my real goal for Lamar:

Get folks complaining about StructureMap off of that and make my life easier overall by reducing user problems.

Jasper v0.9.9 is Released!

Jasper is an open source project I’ve been furiously working on for the past couple years as what I have to admit is mostly a chance to resurrect the best parts of the earlier FubuMVC framework while solving its technical and usability shortcomings. Along the way, Jasper has evolved a bit away from its FubuMVC roots to be more consistent and compatible with ASP.Net Core. At this point, Jasper is a fancy command execution pipeline that lives inside of the ASP.Net Core ecosystem. Jasper can be used as any mix of a lightweight messaging framework, an in-memory service bus, or handling HTTP requests as an alternative ASP.Net Core framework.

I was just able to push a new v0.9.9 release of Jasper that I want to effectively be the last alpha release. At this point, I think the public API surface is pretty well set and only a handful of features left before making the big ol’ 1.0 release. I think the most important thing for Jasper is to try to get folks to take it for a spin, or glance through tutorials, and generally try to get some feedback and visibility about the project.

Here’s some links to get you started:

There’ll hopefully be plenty of blog posts on Jasper in the next couple weeks, starting with how Jasper’s usability contrasts with MVC Core in the HTTP space or NServiceBus/MediatR/etc. for messaging and command execution.

The Road to 1.0

I’ve got to put Jasper down for awhile to focus on some Lamar and help out a lot more with Marten for probably a couple months before I do much more on Jasper, but I’d still love to get 1.0 out by at least the end of the summer.

I’m thinking out loud in this section, so everything is subject to change. Before I flip the switch to the big, giant 1.0, I think these things might need to happen:

  • There are some optimizations I want to make to the database backed message persistence I couldn’t quite get to for v0.9.9.
  • I’m very tempted just to wait until the netcoreapp3.0 wave of updates comes out. It’s a near guarantee that ASP.Net Core v3.0 will break some of Jasper’s internals, and it’d be very helpful to slim Jasper’s package dependency tree down if Jasper could depend on the new generic host model instead of IWebHostBuilder when 3.0 unifies that model somewhat.
  • Jasper heavily depends on the Task Parallel Library from Microsoft, but that seems to be somewhat deprecated now. I might look to rewire Jasper’s internals to use the newer System.Threading.Channels instead. I haven’t done any research into this one yet.
  • If the HTTP support is going to live on, it’ll need some kind of Swagger/Swashbuckle integration for the Jasper HTTP API support
  • I’d like to spend some time on supporting the Techempower Benchmarks for Jasper and some performance optimization. My goal here is to make Jasper the fastest HTTP application framework for .Net Core and be just barely slower than the raw ASP.Net Core benchmarks (not sure how feasible that is, but let me dream on).
  • Jasper is going to have to adjust to whatever becomes of Lamar. I don’t think this is going to change Jasper at development time, but might introduce a new production build step to optimize Jasper application’s “cold start” times for better hosting in Docker kind of worlds.

Lamar stays and how that enfolds

I wrote a blog post at the end of last week about Lamar when I just happened to be feeling discouraged about a couple things (it happens sometimes, and I swear that having to support IoC tools exposes me to more difficult people than every other project I work on combined). I got some rest this weekend, a bit of positive reinforcement from other folks, and actually thought through how to fix the issues. Long story short, I’m not giving up on anything and here’s what I think the very doable game plan is for Lamar (and the closely related Jasper project):

Lamar

  • Short term: Get a small bug fix release out soon that has some options to “force” all the compilation upfront in one dynamic assembly. That’s gonna hurt the cold start time, but should help the memory usage. We’ll also look to see if there’s any places where Lamar is holding on unnecessarily to the Roslyn compilation objects to ensure that they can be garbage collected
  • Medium term: Introduce an alternative compilation model based on Expressions compiled to Lambdas with FastExpressionCompiler. This model will kick in automatically whenever there’s one or more internal types in the “build plan”, and could be opted into globally through a container level switch. I didn’t want to do this originally because the model just isn’t very fun to work with.  After thinking it through quite a bit over the weekend, I think it won’t be bad at all to retrofit this alternative to Lamar’s existing Frame and Variable model. This will knock out the performance issues with internal types and address all the memory issues.
  • Long term: Probably split up LamarCompiler a little bit to remove the actual code compilation to significantly slim down Lamar’s dependency tree and move Lamar to a purely Expression based model. Introducing the Expression model will inevitably make the exception stacktrace messages coming out of Lamar explode, so there might have to be an effort to rewrite them to make them more user friendly (I had to do this in StructureMap 3 several years ago).

 

Jasper

I’m very close to pulling the trigger on a Jasper v1.0, with the understanding that it’s inevitable that there will be a Jasper v2.0 later in the year to incorporate .Net Core 3.0. I don’t think that Jasper will have any issues with memory usage related to Lamar because it uses Lamar very differently than MVC in any flavor. The changes to Lamar will impact Jasper though, so:

Short term: Jasper v1.0 with Lamar as it is.

Medium term: Jasper gets a model where you can happily use the runtime codegen and compilation during development time while things are churning, but for production usage you have the ability to just drop the code that would be generated to disk, have that compiled into your system in the first place, and let Jasper use those types. Ultra fast production time cold start times, no worries at all about Roslyn doing bad things from memory. I’ve already done successful proof of concept development on this one.

Long term: profit.

As an aside, I got quizzed quite a bit about why Jasper has to be specific to Lamar as its IoC container and can’t just support whatever tool folks want to use. The reason is that Jasper uses Lamar’s very specific code generation in its pipeline to avoid using an IoC container at runtime whatsoever and also to avoid forcing users to have to conform to all kinds of Jasper specific adapter interfaces. I could maybe force Jasper to still pull this off with the built in DI container or another IoC container with Jasper-centric adapters to expose all of its metadata in a way such that the codegen understands it, but just ick.

If you took Lamar’s runtime codegen away, I think Jasper inevitably looks like a near clone of NServiceBus or Brighter both in its usability and runtime pipeline and why does the world need that?