A Better Development Web Server for .Net with FubuMVC 2.0

tl;dr: FubuMVC 2.0 includes an improved command line development web server for a better development time experience

First, a quick caveat. All the code in this post is part of the forthcoming FubuMVC 2.0 release and has not yet been publicly released and might not be for a couple more months. All the source code shown and referenced here is in the master branch of FubuMVC itself.

 

What’s the Problem?

If you ask me what I think are the best software development practices to come out of the past couple decades, I’d rattle off some things  like TDD/BDD, Continuous Integration, Continuous Delivery, and Iterative Development. If you take a step back and think about these “best practices” you’ll see that there’s a common thread of attempting to create more useful and definitely more rapid feedback cycles. The cruel truth about slinging code around and envisioning new software systems out of whole cloth is that it’s so very easy to be wrong — and that’s why software professionals have spent so much time and energy finding more ways to know when they’re efforts aren’t working and making it easier and less risky to introduce improvements.

Now, apply this idea of rapid feedback cycles to day to day web development. What I really want to do is to shorten the time between saving changes to any part of my web application, be it C# code or CSS/LESS or any kind of JavaScript, and seeing the impact of that change. Granted, I’d strongly prefer to use quick twitch unit tests to remove most of the potential problems in either the server side or client side code in isolation, but there are still plenty of issues where you’re going to need to do some manual testing with the complete application stack.

Enter “fubu run –watched”

Several web development frameworks include development servers that can automatically reload the application and sometimes even refresh a running browser when files in the application are changed. The Play framework in Java/Scala has the Play Console, we’re starting to use Mimosa.js and its watched server mode for pure client side development, and FubuMVC has our “fubu run” command line server that was originally inspired by PlayConsole.

While we’ve had this functionality for quite a while, I just pushed some big improvements yesterday that fixes our auto-refresh infrastructure. The new architecture looks like this:

Installing the fubu gem* places the fubu.exe on your box’s PATH, so you can just go straight to the directory that contains your FubuMVC application and type:

fubu run -o --watched

The “o” flag just directs the command to open your default browser to the Url where the application is going to be hosted. While the fubu run process always auto-reloads the application on recompilation, the “watched” flag adds some additional mechanics to refresh the current page in your browser whenever certain files change in your application.

So how does it work?

First off, fubu run has to create a separate AppDomain to host your FubuMVC application. If you’ve done .Net development for any length of time you know that there is no way to unload and replace a loaded AppDomain. By running a separate AppDomain we’re able to tear down and recreate new AppDomain’s when you recompile your application. The other reason to use a separate AppDomain is to make the application run just like it would in a production server, and that means making the new AppDomain be based on the directory of the application instead of wherever the fubu.exe happens to be and making the new AppDomain use the correct web.config file for the application.

The FubuMVC community has some special sauce in our Bottles framework called the RemoteServiceRunner that makes it easier to setup and coordinate multiple AppDomain’s in code (I’ll happily blog about later if anyone wants me to).  As shown in this code,  fubu run loads the second AppDomain based on the right location and copies over any missing assemblies to the application bin path for FubuMVC, Katana, and their dependencies.

The next step is to bootstrap your FubuMVC application in the second AppDomain. As of FubuMVC 1.0, the idiomatic way to describe your application’s bootstrapping is with an implementation of the IApplicationSource interface. If there’s only a single concrete class implementing this interface in all of your application binaries, fubu run is smart enough to use that to bootstrap your application exactly the way it would be (with some development time differences I’ll discuss below). The simplest possible implementation might look like this class:

    public class SimpleApplicationSource : IApplicationSource
    {
        public FubuApplication BuildApplication()
        {
            return FubuApplication
                .DefaultPolicies()
                .StructureMap();
        }
    }

Part of our Bottles infrastructure is an EventAggregator class that was specifically created in order to easily send messages bidirectionally between AppDomain’s opened by our RemoveServiceRunner class. fubu run uses this EventAggregator to send and receive messages to the new AppDomain to start an embedded Katana web server and bootstrap a new FubuMVC application using the IApplicationSource class for your application. Likewise, fubu run waits to hear messages back from the 2nd AppDomain about whether or not the application bootstrapping was successful.

Watching for Changes

If in the “–watched” mode, fubu run starts up a class called FubuMvcApplicationFileWatcher to watch for changes to certain files inside the application directory and call back to an observer interface when file changes trigger certain logical actions:

    public interface IApplicationObserver
    {
        // Refresh the browser
        void RefreshContent();

        // Tear down and reload the entire AppDomain
        void RecycleAppDomain();

        // Restart the FubuMVC application
        // without restarting the 
        // AppDomain
        void RecycleApplication();
    }

To make this concrete, changes in:

• .spark, .css, .js, or .cshtml files will trigger a refresh of the browser
• web.config, .dll, or .exe files will cause a full recycling of the AppDomain
• other *.config file changes will trigger an application recycle

 

Automatically Refreshing the Browser with WebSockets

The last piece of the puzzle is how we’re able to refresh the browser when the server content changes. In the currently released version of the fubu.exe tool I tried to use WebDriver to launch the browser in such a way that it would be easy to control the browser from fubu run without any impact on the html markup and the application itself. Let’s just say that didn’t work very well at all because of how easily WebDriver gets out of sync with rapid browser updates in real life.

For the FubuMVC 2.0 work, I went down a different path and used web sockets to send messages from the original fubu run process directly to the browser. My immediate and obvious goal was to pull this off without forcing any changes whatsoever onto the application’s HTML markup to support the auto-reloading. Instead, I used this work as an opportunity to revamp FubuMVC’s support for using OWIN middleware to use a new bit of custom middleware to squirt in a little bit of HTML markup into an HTML page’s <head> element after FubuMVC had rendered a page, but before the content is sent to the browser. While I’ll leave a discussion for how and why FubuMVC exposes OWIN middleware configuration much differently than other .Net frameworks for another day, it’s good enough to know that FubuMVC is only adding the auto-reloading content when the application detects that it is running inside of fubu run –watched.

On the client side, we just inject this wee bit of javascript code to listen at a supplied web sockets address (%WEB_SOCKET_ADDRESS%) for a message telling the page to refresh:

        var start = function () {
            var wsImpl = window.WebSocket || window.MozWebSocket;

            // create a new websocket and connect
            window.ws = new wsImpl('ws://localhost:%WEB_SOCKET_ADDRESS%');

            // when data is comming from the server, this metod is called
            ws.onmessage = function (evt) {
                location.reload();
            };

        }

	window.addEventListener('load', function(e){
	  start();
	});

Inside the fubu run process I used the lightweight Fleck library to run a web sockets server used to send messages to the browser. The code that uses Fleck is in the BrowserDriver class.

 

Roslyn for the Next Level of Awesomeness…

…or really just achieving parity with other web development platforms that already have a great auto-reload capability.

At this point, fubu run does not try to do the compilation for you based on file changes to *.cs files. I’m still not sure if I think that’s a good idea to do later or not. I’m perfectly okay with the “make changes, hit CTRL-SHIFT-B” workflow triggering a re-compilation which then in turn triggers fubu run to recycle the AppDomain and reload the application. Several folks have kicked around the idea of using the new, much faster Roslyn compiler behind the scenes to do the compilation and try to achieve a more rapid feedback cycle like you’d expect from a Node.js based solution. I think this would be a lot more appealing to me personally as my teams at work continue to break away from using Visual Studio.Net in favor of lightweight editors like Sublime.

 

Getting the Edge Nugets and Fubu.Exe

The edge nugets for FubuMVC 2 are on MyGet at https://www.myget.org/feed/Packages/fubumvc-edge. The fubu.exe gem can be manually downloaded from the artifacts of our TeamCity CI build.

 

 

* In an earlier post I discussed why we used Ruby Gems to distribute .Net executables instead of using Nuget. Long story short, Nuget by itself isn’t all that great for command line tools and Chocolately isn’t cross platform like Gems.