I know full well that what I’m showing here is polarizing and some of you are going to hate it, but what we did does have some significant benefits and it’s possible it could be pulled off later with cleaner usage mechanics. When you read this and start saying, “isn’t this like Serilog?”, please just read to the end. 

I’ve frequently used the term “semantic logging” to describe the mechanism we still use in FubuMVC for our logging abstraction. By semantic logging, I mean that we log strong-typed messages for most framework audit and debug actions.

To make that concrete, this is part of our ILogger interface that FubuMVC uses internally:

    public interface ILogger
    {
        // LogTopic is yes, a standardized base
        // class with some basic properties for
        // timestamps and a Guid Id
        void DebugMessage(LogTopic message);
        void InfoMessage(LogTopic message);

        // This is the mechanism for conditional logging if
        // there is a registered listener for whatever T is
        void DebugMessage<T>(Func<T> message) where T : class, LogTopic;
        void InfoMessage<T>(Func<T> message) where T : class, LogTopic;
    }

Taking the service bus part of FubuMVC as an example, we have these log topic types that get logged at various points in message sending and handling:

1. EnvelopeSent — in service bus parlance, an “envelope” would be a message plus whatever metadata the service bus needs to get it to the right place.
2. EnvelopeReceived
3. MessageSuccessful
4. MessageFailed

For a concrete example from inside our service bus features, we log an EnvelopeSent log record any time a message is sent from the service bus client:

private void sendToChannel(Envelope envelope, ChannelNode node)
{
    var replyUri = _router.ReplyUriFor(node);

    var headers = node.Send(envelope, _serializer, replyUri: replyUri);
    _logger.InfoMessage(() => new EnvelopeSent(new EnvelopeToken
    {
        Headers = headers,
        Message = envelope.Message
    }, node));
}

As you can see above, we pass a Func<EnvelopeSent> into the logger.InfoMessage() method instead of directly building an EnvelopeSent object. We do that so that the logging can be optional, so that if nothing is listening for the EnvelopeSent log record type, that Func in never executed. Some of you might argue that this is a little less efficient at runtime than doing the if (_logger.ListensFor(typeof(EnvelopeSent)) _logger.InfoMessage(new EnvelopeSent{}); approach and you’d be correct, but I prefer using the Func approach to express the optional logging because I think it’s more terse and easier to read.

Internally, the actual “sinks” for these log records have to implement this interface (partially elided for space):

    public interface ILogListener
    {
        bool IsDebugEnabled { get; }
        bool IsInfoEnabled { get; }

        // Does this particular listener
        // care about a log record type?
        bool ListensFor(Type type);

        void DebugMessage(object message);
        void InfoMessage(object message);

    }

With this interface, you are able to subscribe for certain log record types being logged by the application.

We manage both ILogger and the full collection of ILogListener’s through our application’s IoC container. While there probably isn’t a more polarizing technique in the software world than using an IoC tool, in our case I think it’s been very advantageous for us because of how easy that makes it to add or remove log listeners and to govern the lifecycle or scoping of log listeners (think about having a log listener scoped to a single HTTP request that makes it stupidly easy to correlate log messages to a certain HTTP request).

The Downside

Some of my coworkers don’t like this approach because of the need to add all the extra Type’s to your code. To their point, I don’t think you’d bother to make every single log message its own special type, only things that are more valuable to key off of in system diagnostics or in tooling like our test automation support sample I’m showing in another section below. We do have some basic “Info(string message) or Debug(string message)” support too in our ILogger, but my advice for that kind of thing has been to go straight to some kind of low level logging tool like NLog or log4net.

The benefits are richer system monitoring because it’s so easy to intelligently parse and filter the information being logged.

Some Valuable Usages

• Our HTTP request tracing in FubuMVC depends on a custom ILogListener that tracks any and all log topic records to the record of that HTTP request. By being able to listen for only certain log topic Type’s, we can easily move our diagnostic trace level from nothing to a production mode where we only hang on to certain types of logging actions to a very verbose, record everything mode for local development. Ditto for our service bus support inside of FubuMVC (was FubuTransportation).
• Having the strong typed log record messages made it fairly simple to create powerful visualizations of different logging actions in FubuMVC’s built in diagnostics
• I actually think it’s easier to filter out what logging message types you care about listening for than trying to filter logging output by the classes or namespaces in your application like folks do with tools like log4net or NLog.
• We can inject custom ILogListener’s in automated testing harnesses to gather a lot of application diagnostics and embed these diagnostics directly into our acceptance testing output reports. That’s been extremely valuable for us to troubleshoot failing tests or understand performance problems in our code by having an easy correlation between each automated test and the logging records.  I’ll write a blog post about how we do that someday.
• In my very next blog post I’ll talk about how we use an ILogListener to monitor the activity of a service bus applications so that you can much more reliably coordinate automated test harnesses with the application code.

Replace with Serilog the next time around?

I’m very happy with what we were able to do in terms of its impact, and “semantic logging” has consistently been in my “keep from FubuMVC” column in our planning for whatever we do next. In the longer term though, I think I’d like look to look at using Serilog as our underlying logging mechanism and just figure out how we could inject custom listeners into Serilog to replace our diagnostic and test automation abilities. It would mean giving up on the strong typed messages that *I* like, but might be more palatable for other developers. Now that we’ve mostly replaced server side rendering with a React.js based UI in our diagnostics, it might make more sense anyway to do the visualizations against a JSON body.

Mostly though, that’s just a way to have less custom OSS code that I have to support and maintain;)

What about…?

Anytime I write a blog post like this I get a flood of “have you tried XYZ?” kind of questions about other tools that I may or may not have heard of. I’ll try to cover some of them:

• log4net? NLog? – now that NLog is strong named (boo!), I don’t have any preference for either and I would never in a million years consider coupling any of my OSS projects to either. In other news, strong naming still sucks and it’s something you have to consider when you release an OSS tool
• There is the Semantic Logging Block tool from Microsoft P&P. I haven’t used it, and my informal Twitter poll on it suggests that folks prefer Serilog over it.
• What about taking a dependency on Common Logging then doing whatever you want for the actual logging? I looked at doing this years ago when we first introduced logging into FubuMVC and decided against it. My thinking is that Common Logging suffers from a “boiling the ocean” kind of problem that bloats its API surface.
• Liblog? It’s probably a better alternative to Common Logging without some of the compatibility headaches brought on by strong naming conflicts, but doesn’t really address the idea of semantic logging
• I know that once upon a time the Glimpse guys were throwing around a similar idea for semantic logging (they weren’t using the same terminology though) with rich logging messages that you could consume in your own application monitoring tooling.

FubuMVC as an OSS project is still mostly dead, but we still use it very heavily at work and one of our teams asked me to explain how the authorization works. You’re probably never going to use FubuMVC, but we did some things that were interesting and successful from a technical perspective. Besides, it’s always more fun to learn from other people’s mistakes — in this case, my mistakes;)

Early in the project that originally spawned FubuMVC we spent almost two months tackling authorization rules in our application. We had a couple needs:

1. Basic authorization to allow or deny user actions based on their roles and permissions
2. Authorization rules based on the current business object state (rules like “an amount > 5,000 requires manager approval”).
3. The ability to extend the authorization of the application with custom rules and exceptions for a customer without any impact on the core code
4. Use authorization rules to enable, disable, show, or hide navigation elements in the user interface

The core architectural concept of FubuMVC was what we called the “Russian Doll Model” that allowed you to effectively move cross cutting into our version of middleware. Authorization was an obvious place to use a wrapping middleware class around endpoint actions. If a user has the proper rights, continue on. If the user does not, return an HTTP 403 and some kind of authorization failure response and don’t execute the inner action at all.

For some context, you can see the part of our AuthorizationBehavior class we used to enforce authorization rules:

protected override void invoke(Action action)
{
    if (!_settings.AuthorizationEnabled)
    {
        action();
        return;
    }

    var access = _authorization.IsAuthorized(_context);

    // If authorized, continue to the inner behavior in the 
    // chain (filters, controller actions, views, etc.)
    if (access == AuthorizationRight.Allow)
    {
        action();
    }
    else
    {
        // If authorization fails, hand off to the failure handler
        // and stop the inner behaviors from executing
        // the failure handler service was pluggable in fubu too
        var continuation = _failureHandler.Handle();
        _context.Service<IContinuationProcessor>().Continue(continuation, Inner); 
    }
}

Pulling the authorization checks into a separate middleware independent of the inner HTTP action had a couple advantages:

• It’s easier to share common authorization logic
• Reversibility — meaning that it was easy to retrofit authorization logic onto existing code without having to dig into the actual endpoint action code.
• The inner endpoint action code could be simpler by not having to worry about authorization (in most cases)

FubuMVC itself might have failed, but the strategy of using composable middleware chains is absolutely succeeding as you find it in almost all of the newer HTTP frameworks and service bus frameworks including the new ASP.Net Core MVC code.

Authorization Rights and Policies

The core abstraction in FubuMVC’s authorization  subsystem was the IAuthorizationPolicy interface that could be executed to determine if a user had rights to the current action:

    public interface IAuthorizationPolicy
    {
        AuthorizationRight RightsFor(IFubuRequestContext request);
    }

The “AuthorizationRight” class above was a strong typed enumeration consisting of:

1. “None” — meaning that the policy just didn’t apply
2. “Allow”
3. “Deny”

If multiple rules applied to a given endpoint, each rule would be executed to determine if the user had rights. Any rule evaluating to “Deny” automatically failed the authorization check. Otherwise, at least one rule had to be evaluated as “Allow” to proceed.

Being able to combine these checks enabled us to model both the simple, role and permission-based authorization, and also rules based on business logic and the current system state.

Simple Role Based Authorization

In the following example, we have an endpoint action that has a single “AllowRole” authorization rule:

        // This action would have the Url: /authorized/hello
        [AllowRole("Greeter")]
        public string get_authorized_hello()
        {
            return "Hello.";
        }

Behind the scenes, that [AllowRole] attribute is evaluated once at application startup time and adds a new AllowRole object to the underlying model for the “GET /authorized/hello” endpoint.

For some context, the AllowRole rule (partially elided) looks like this:

    public class AllowRole : IAuthorizationPolicy
    {
        private readonly string _role;

        public AllowRole(string role)
        {
            _role = role;
        }

        public AuthorizationRight RightsFor(IFubuRequestContext request)
        {
            return PrincipalRoles.IsInRole(_role) ? AuthorizationRight.Allow : AuthorizationRight.None;
        }

    }

Model Based Configuration

FubuMVC has an underlying model called the “behavior graph” that models exactly which middleware handlers are applicable to each HTTP route. Part of that model is an exact linkage to the authorization policies that were applicable to each route. In the case above, the “GET /authorized/hello” endpoint has a single AllowRole rule added by the attribute.

More powerfully though, FubuMVC also allowed you to reach into the underlying behavior graph model and add additional authorization rules for a given endpoint. You could do this through the marker attributes (even your own attributes if you wanted), programmatically if you had to, or through additive conventions like “all endpoints with a route starting with /admin require the administrator role.” We heavily exploited this ability to enable customer-specific authorization checks in the original FubuMVC application.

Authorization & Navigation

By attaching the IAuthorizationPolicy objects to each behavior chain, FubuMVC is also able to tell you programmatically if a user could navigate to or access any HTTP endpoint using the IEndpointService like this:

            IEndpointService endpoints = runtime
                .Get<EndpointService>();             

            var endpoint = endpoints
                 .EndpointFor<SomeEndpoint>(x => x.get_hello());

            var isAuthorized = endpoint.IsAuthorized;

We used the IEndpointService (and a simpler one not shown here called IAuthorizationPreviewService) in server side rendering to know when to show, hide, enable, or disable navigation elements based on whether or not a user would have rights to access those routes. By decoupling the authorization rules a little bit from the actual endpoint action code, we were able to define the authorization rules exactly once for every endpoint, then reuse the same logic in navigation accessibility that we did at runtime when the actual resource was requested.

This ability to “preview” authorization rights for HTTP endpoints was also useful for hypermedia endpoints where you used authorization rights to include or exclude additional links in the response body.

Lastly, having the model of what authorization rules applied to each route enabled FubuMVC to be able to present diagnostic information and visualizations of the middleware configuration for each HTTP endpoint. That kind of diagnostics becomes very important when you start using conventional policies or extensibility mechanism to insert authorization rules from outside of the core application.

Rule Object Lifecycle

FubuMVC, especially in its earlier versions, is awful for the number of object allocations it makes at runtime. Starting with FubuMVC 2, I tried to reduce that overhead by making the authorization rule objects live through the lifecycle of the application itself instead of being created fresh by the underlying IoC container on every single request. Great, but there are some services that you may need to access to perform authorization checks — and those services sometimes need to be scoped to the current HTTP request. To get around that, FubuMVC’s IAuthorizationPolicy takes in an IFubuRequestContext object which among other information contains a *gasp* service locator for the current request scope that authorization rules can use to perform their logic.

There’s been an almost extreme backlash against any and all usages of service locators over the past several years. Most of that is probably very justified, but in my opinion, it’s still very valid to use a service locator within an object that has a much longer lifetime than the dependencies it needs to use within certain operations. And no, using Lazy<T> or Func<T> builders injected in at the original time of creation will not work without making a potentially harmful dependency on things like the HttpContext for the scoping to work.

Please don’t dare commenting on this post with any form of “but Mark Seemann says…” I swear that I’ll reach through the interwebs and smack you silly if you do. Probably make a point of never, ever doing that on any kind of StructureMap list too for that matter.

What I’d do differently if there’s a next time

For the past couple years we’ve kicked around the idea of an all new framework we’re going to call “Jasper” that would be essentially a reboot of a core subset of FubuMVC on the new CoreCLR and all the forthcoming ASP.Net Core goodies. At some point I’ve said that I wanted to bring over the authorization model from FubuMVC roughly as is, but the first step is to figure out if we could use something off the shelf so we don’t have to support our own custom infrastructure (and there’s always the possibility that we’ll just give up and go to the mainstream tools).

The single biggest thing I’d change the next time around is to make it far easier to do one-off authorization rules as close as possible to the actual endpoint action methods. My thought has been to have something like a convention or yet another interface something like “IAuthorized” so that endpoint classes could happily expose some kind of Authorize() : AuthorizationRight method for one off rules.

Honestly though, I’m content with how the authorization model played out in FubuMVC. A big part of the theoretical plans for “Jasper” is be much, much more conscious about allocating new objects (i.e., use the IoC container much less) and adopting an async by default, all the way through approach to the runtime model.

As I said in my last post, we’re rebooting FubuMVC as a new project called Jasper. As an intermediate step, I’m working up a FubuMVC 3.0 release that among other things, simplifies the application configuration and bootstrapping so that it’s quicker and easier to stand up a brand new FubuMVC web or service bus application.

At it’s very, very simplest for the classic Hello World exercise, the newly streamlined bootstrapping looks like this (when we rebrand as Jasper, just replace “Fubu” with “Jasper” in the class names below):

    // This little endpoint class handles
    // the root url of the application
    public class HomeEndpoint
    {
        public string Index()
        {
            return "Hello, World";
        }
    }

    [TestFixture]
    public class RunHelloWorld
    {
        [Test]
        public void start_and_run()
        {
            // Bootstrap a basic FubuMVC applications
            // with just the default policies and services
            using (var runtime = FubuRuntime.Basic())
            {
                // Execute the home route and verify
                // the response
                runtime.Scenario(_ =>
                {
                    _.Get.Url("/");

                    _.StatusCodeShouldBeOk();
                    _.ContentShouldBe("Hello, World");
                });
            }
        }
    }

Of course, as we all know, real world development gets a lot hairier than cute little hello world examples and you’re not going to get away with the framework defaults in most cases. In the case of FubuMVC, you might want to add services to the application IoC container, change or add conventions and policies, configure features, turn on “opt in” features, or run an application in either the special “development” or “testing” mode.

Real World Configuration in FubuMVC 3.0

After all the dust settled, a FubuMVC application is completely described, configured, bootstrapped, and cleanly deactivated by just two classes:

1. FubuRegistry is used to express all of the configurable elements of a FubuMVC application
2. FubuRuntime holds all the runtime elements of a running FubuMVC application like the application container, the activation logs, the routing table, the root file path, the application mode (development/testing/normal) and “knows” how to cleanly shut down the running application.

A custom FubuRegistry might look something like this one:

    public class ExampleRegistry : FubuRegistry
    {
        public ExampleRegistry()
        {
            // Turn on some opt in features
            Features.Localization.Enable(true);
            Features.Diagnostics.Enable(TraceLevel.Production);

            // Change the application mode if you want
            Mode = "development";

            // Have the application use an embedded 
            // Katana host at port 5501
            HostWith<Katana>(5501);

            // Register services with the IoC container
            // using a superset of StructureMap's
            // Registry DSL
            Services.AddService<IActivator, MyActivator>();
        
            // For testing purposes, you may want 
            // to bootstrap the application from an external
            // testing library, in which case, you'd want
            // to override where FubuMVC looks for static
            // asset files like JS or CSS files
            UseParallelDirectory("MyApp");
            // or
            RootPath = "some other path";
        }
    }

To bootstrap the application specified by ExampleRegistry, you can use this syntax below to create a new FubuRuntime object:

            using (var server = FubuRuntime.For<ExampleRegistry>())
            {
                // do stuff with the application
                // or wait for some kind of signal
                // that you should shut it off
            }

If you don’t want to bother with your own subclass of FubuRegistry, you can forgo it and bootstrap a FubuRuntime with syntax like this:

            var runtime = FubuRuntime.Basic(_ =>
            {
                _.Features.Diagnostics.Enable(TraceLevel.Verbose);

                // I'm opting for NOWIN hosting this time
                // and letting FubuMVC try to pick an open
                // port starting from 5500
                _.HostWith<NOWIN>();
            });

There is a FubuRegistry involved in the code above, but you’re configuring it inside the lambda expression argument. More on that below in the section on the design patterns.

Bootstrapping Use Cases and When You Want Your Own FubuRegistry

Just for some background, here are the various use cases for bootstrapping a FubuMVC web or service bus application that I could think of:

1. Bootstrapping for ASP.Net hosting in the good ol’ Global.asax like this.
2. Spinning up an application adhoc inside of tests like this example from FubuMVC 2.2.
3. We have a tool we call Serenity (yes, it’s named for what you think it is) that we use to setup integration tests for FubuMVC with Storyteller.
4. Running an application with our fubu run development server
5. Hosting a FubuMVC application within a background service using our JasperService tool (was BottleServiceRunner) that’s just a small shim to TopShelf

The easiest way to use Serenity is to say “here, use this FubuRegistry,” while options #4 and #5 use type scanning to find either the one single FubuRegistry in your code or to use one by name to activate the hosted application. In these cases, it’s highly useful to have a custom FubuRegistry. Even without that, it’s also valuable to have your application configuration done in a way that’s easily reusable within automated tests as in use case #2 up above so that you’re tests are more reflective of how the actual application runs.

An aside on the design patterns I’ve used

Other people certainly have much more negative opinions, but I feel like learning, studying, and applying design patterns in code has been very useful to my career. At Oredev 2010 I gave a talk about Internal DSL’s in C# whose content is still very much relevant to the work I did the past month with FubuMVC’s bootstrapping.

The FubuMVC bootstrapping uses at least three different design patterns you can find described in Martin Fowler’s DSL book.

1. Fluent Interface — method chaining to create an internal DSL directly in C# “if you squint hard enough”
2. Object Scoping. The FubuRegistry (or StructureMap’s Registry) are examples of using “object scoping” to shorten the signatures of fluent interfaces to a base class. Most of your declarations are going to be in a custom fluent interface called in the constructor function of a FubuRegistry base class.
3. Nested Closure — The usage of FubuRuntime.Basic(x => {}) where you use an action that configures some option type as the single argument to a function. Nested closures are helpful where you need to allow a user to specify any number of optional parameters as the input to a discrete action. I discussed nested closures ages ago in an MSDN article.

I also remember going to a lot of talks at that Oredev on the new flashy HTML 5 technologies that have been long since passed by. The obvious takeaway is that conceptual knowledge tends to outlast the usefulness of knowledge about specific technologies.

Just for Comparison, FubuMVC before 3.0

When I was asked by some of our technical leadership to simplify the bootstrapping for FubuMVC 3.0, my first reaction was “it’s not that bad” — but this was what FubuMVC had previously:

• FubuRegistry & FubuRuntime were just a little smaller than they are today
• You used a static property called FubuMvcPackageFacility.PhysicalRootPath to change the root directory of the content files. And yes, the mutable static property was a problem.
• FubuApplication was a static class that you used to trigger a fluent interface that would gather up your FubuRegistry and allow you to choose an IoC adapter or use a pre-built IoC container in order to create a new FubuRuntime
• IApplicationSource was a reusable recipe for how to bootstrap a FubuRuntime that we previously used for fubu run or our older TopShelf service host.
• EmbeddedFubuMvcServer was a class we used to stand up a FubuMVC application hosted with Katana for testing or just embedding a web server for diagnostics in service bus applications running as a background service. Of course, that all got duplicated when we added NOWIN support and it duplicated some functionality from FubuRuntime anyway.
• FubuMode was a static class we used to detect and tell you whether the application should run normally or in “development” or “testing” mode. Statics are evil you say? Yeah well keep reading…
• PackageRegistry was a static class from our now defunct Bottles library that exposed information about the loaded extension assemblies in the application and the diagnostic logging from application bootstrapping.

As of now, we’ve completely eliminated all of the static classes and properties. All of the configuration is done through FubuRegistry, and all of the runtime information about a FubuMVC application is exposed off of FubuRuntime. So yeah, it’s quite a bit better now. It’s good to pay attention to the feedback of others because they see things you don’t or problems you just get too comfortable working around.

tl;dr: I had a convention based build automation approach based on Rake we used for the FubuMVC projects that I was proud of, but the world moved on and I’ve been replacing that with newer and hopefully better tools like Fixie, Paket, and gulp.js. What I do today with FubuRake I’ve generally used Rake as my build scripting tool for the last 7-8 years, even on .Net projects and I’ve been mostly happy with it. In a grandiose attempt to simplify the build scripts across the FubuMVC ecosystem, make all the homegrown build tools we’d created easier to adopt, and support a Ruby on Rails style “fubu new” approach to bootstrapping new FubuMVC projects, I created the FubuRake library as an addon to Rake. In its most simple form, a complete working build script using FubuRake can look like this below:

require 'fuburake'

FubuRake::Solution.new do |sln|
	sln.assembly_info = {
		:product_name => "FubuCore",
		:copyright => 'Copyright 2008-2015...'
	}
end

That simple script above generated rake tasks for:

1. Generating a “CommonAssemblyInfo.cs” file to embed semantic version numbers, CI build numbers, and git commit numbers into the compiled assemblies
2. Compiling code
3. Fetching, building, and publishing nuget with Ripple
4. Running unit tests with NUnit
5. Tasks to interact with our FubuDocs tool for documentation generation (and yes, I spent way more time building our docs tool than writing docs)
6. Tasks to create embedded resource files in csproj files as part of our Bottles strategy for modularizing large web applications

If you typed rake -T to see the task list for this script you’d see this output:

rake ci                # Target used for the CI server
rake clean             # Prepares the working directory for a new build
rake compile           # Compiles the solution src/FubuCore.sln
rake compile:debug     # Compiles the solution in Debug mode
rake compile:release   # Compiles the solution in Release mode
rake default           # **Default**, compiles and runs tests
rake docs:bottle       # 'Bottles' up a single project in the solution with...
rake docs:run          # Runs a documentation project hosted in FubuWorld
rake docs:run_chrome   # Runs the documentation projects in this solution i...
rake docs:run_firefox  # Runs the documentation projects in this solution i...
rake docs:snippets     # Gathers up code snippets from the solution into th...
rake ripple:history    # creates a history file for nuget dependencies
rake ripple:package    # packages the nuget files from the nuspec files in ...
rake ripple:publish    # publishes the built nupkg files
rake ripple:restore    # Restores nuget package files and updates all float...
rake ripple:update     # Updates nuget package files to the latest
rake sln               # Open solution src/FubuCore.sln
rake unit_test         # Runs unit tests for FubuCore.Testing
rake version           # Update the version information for the build

As you’ve probably inferred, FubuRake depended very heavily on naming and folder layout conventions for knowing what to do and how to build out its tasks like:

• Compile the one and only *.sln file under the /src directory using MSBuild with some defaults (.Net version = 4.0, compile target = Debug)
• Run all the NUnit tests in folders under /src that end in *.Tests or *.Testing. In the case up above, that meant “FubuCore.Testing”
• Build and publish all the *.nuspec files found in /packaging/nuget

You could, of course, explicitly override any of the conventional behavior. Any tool using conventions almost has to have an easy facility for overriding or breaking out of the conventions for one off cases. It was fine and great as long as you mostly stayed inside our idioms for project layout and you were okay with using NUnit and Ripple. All in all, I’d say that FubuRake was a mild technical success but time has passed it by. The location of MSBuild changed in .Net 4.5, breaking our msbuild support. Ripple was always problematic and it was frequently hard to keep up with Nuget features and changes. FubuDocs was a well intentioned thing, but the support for it that got embedded into FubuRake has file locking issues that sometimes forced me to shutdown VS.Net in order to run the build. If FubuRake is Roland in the Stephen King’s Dark Tower novels, I’d say that the world has moved on. Time for the world to move on I’ve been quiet on the blogging and even the Twitter front lately because I’ve been working very hard on a near rewrite of our Storyteller tool (more on this soon) that we use at work for executable specifications and integration testing. The new work includes a lot of performance driven improvements in the existing .Net code and a brand new user interface written as a Javascript single page application. At the end of last week my situation looked like this:

• I needed to start publishing pre-release nugets and we only do that from successful CI builds
• Our TeamCity CI build was broken because of some kind of problem with our Ripple tool that we use for Nuget management.
• I had started the new client in a completely separate Github repository and had been using a git submodule to include the Javascript code within the Storyteller .Net code repository for full integration
• I had no effective automation to attach the submodule if it was missing or do the initial npm install or any of the other hidden things that a developer would have to do before being able to work with the code. In other words, my “time to login screen” metric was terrible right as I’d love to start getting some other developers to start contributing to the project with feedback or patches.
• I’ve wanted to upgrade or replace several pieces of the build and test automation tools I use in my OSS projects for quite some time anyway — especially where there were opportunities to replace homegrown tools I no longer want to support in favor of actively maintained OSS projects.
• We have a heavy investment in Rake and Ruby for build automation both at work and on OSS projects. Now that we have so much dependency on Node.js tools for client asset work, we’ve gotten into this situation where project build scripts may include installing gems, nugets, and npm packages and it’s becoming a problem of technology overload and build times.

As a result, I’ve spent the last couple days swapping out tools and generally trying to make the new build automation a lot easier to use for other people. I’ve merged the client javascript code into the old Storyteller repository to avoid the whole git submodule mess. I created a new build script that does everything necessary to get both the Javascript and C# code ready for development work. And lastly, I took the very unusual step (for me) of trying to document how to use the code in a readme. All told I:

• Replaced our old NUnit-based SpecificationExtensions I ripped off from Scott Bellware ages ago with Shouldly. I chose Shouldly because I liked how terse it is, it was an easier switch than Should or Fluent Assertions would have been, and I love their error messages. It went smoothly
• Replaced Ripple with Paket for managing Nuget dependencies. Ripple is effectively dead, I didn’t want to invest any more time in it, and Paket has a strong, active community right now. Using Nuget out of the box is completely out of the question for me, so Paket was really the only alternative. So far, so good. I hit a few quirks with Paket using multiple feeds, but quickly learned to just be very particular about version numbers. The proof for me will be when we start using Paket across multiple upstream and downstream repositories like we did with our “floating” nuget dependencies in Ripple. It’s my hope though that tools like Paket or Ripple are no longer necessary in ASP.Net vNext but we’ll see.
• Built a small gulp.js script to compile the .Net code and run the C# unit tests. I leaned heavily at first on Mike O’Brien’s blogpost about building .Net projects with gulp. It didn’t go as smoothly as I’d like and I partially retreated to doing some things in npm scripts instead. I think this is definitely something I’m going to watch and reconsider moving forward. EDIT 3/25: I’ve already given up on gulp.js for the .Net build and I’m in the process of just using simple Javascript files called from NPM as a replacement. Oh well, no harm done.
• Completely replaced NUnit with Fixie, but used Fixie’s ability to emulate NUnit’s attributes and behavior as a temporary measure. The FubuMVC team has wanted to do this for quite a while because of Fixie’s crazy flexible feature set and performance — and this was the perfect opportunity to finally pull that off. I’m very happy with how this has turned out so far and I’m even seeing the promised performance improvement with the test suite consistently taking 75% of the runtime that NUnit was taking on my machine. Going forward, I’m going to look to slowly remove the NUnit attributes in favor of the cleaner Fixie idioms. I’m happy with Fixie right off the bat.
• Finally, to make sure it all “just works,” I created an overarching “npm run build” script with a matching “build.cmd” shortcut to do everything you need to do to build both the Javascript and C# code and pull down all of the various dependencies. So now, if you do a fresh clone of the Storyteller code and you already have both .Net 4.5 and Node.js >+ v10 installed, you should be able to execution “npm run build” and go straight to work. And I’ll keep patching things until that’s certified to be true by other developers;)

In case you’re wondering, the Javascript code is built with Webpack through npm scripts that also delegate to mocha and karma for testing. Ironically, I’m using gulp.js to build the .Net code but not the Javascript code, but that’s a subject for someone else to blog;)

More on Replacing Rake with Gulp

I personally like Ruby better than Javascript, especially for the kind of scripting you do for build and test automation. From time to time I still see folks wishing that the browsers would adopt Ruby as the embedded scripting language instead of Javascript, but that metaphorical ship has long since sailed and I think more developers are going to be familiar with JS than Ruby.

I gave some thought to just trying to use pure console tools for the build automation, and some of my coworkers want to investigate using make, but there’s just a little bit of programmatic manipulation here and there for picking up build versions and arguments from the CI server that I wanted to retain some kind of scripting language. I put this very question of replacing Rake to the StructureMap community and got myriad suggestions: F# based tools, C# tools using scriptcs, and Powershell based tools all made an appearance. My stance at this point is that the build script is best done with a low ceremony scripting language and preferably one that’s commonly understood so as to not be a barrier to entry. As much as I liked Rake personally, Ruby was a problem for us with many .Net developers. By choosing a Javascript based tool, we’re investing in what’s arguably the most widely used programming language going. And while this also forces developers to have a working installation of Node.js on their box, I think that’s going to be pretty common anyway.