Thoughts on Running an OSS Project

One of my favorite development events is the every couple years Pablo’s Fiesta open spaces in Austin. However, I have to roll my eyes pretty hard every single time when one of the celebrity programmers here in town submits a session on running an OSS project for the express purpose of telling everyone exactly how great he is as an OSS lead.

While I’ve been heavily involved in OSS for years as the primary author and technical lead of StructureMap, FubuMVC / Jasper, and the recently rebooted Storyteller, I am certainly not the awesome OSS leader that the celebrity programmer above purports to be at every single Pablo’s Fiesta.

In my own OSS efforts I’ve:

  • Taken far too long to answer user questions or completely missed questions on user lists and emails
  • Probably been a little too impatient and testy with other developers a little too frequently
  • Left pull requests to rot without decent feedback
  • Consistently failed to write adequately useful docs and getting started tutorials
  • Never been terribly successful in building community around OSS projects

But hey, there’s oodles of conference talks and blog posts about being awesome at OSS, so how about instead some frank thoughts from a guy whose made a lot of mistakes at running OSS projects.

Bugs from Users

Let’s just work under the assumption that bugs are going to be reported from your users. My experience is that even when I’m doing my best to ratchet up the test coverage and software engineering discipline on my work enough users can always find scenarios that I didn’t anticipate or adequately accommodate in the code.

In a way, getting bug reports is a good thing because it means you actually have interested users and it’s frequently useful feedback. Because you’re frequently dealing with users remotely, I think the biggest challenge many times is to make sure that you fully understand the exact problem that the user is encountering. At one extreme, it’s becoming common for me to get bug reports in StructureMap that come with failing unit tests or example code that demonstrates exactly what the problem is and how to reproduce it. My cutesy saying to describe that is:

Blessed are those who attach failing unit tests to their bug reports, for their issues shall be addressed first

Even if you don’t get concrete examples from users, I think it’s valuable to try to do that yourself and get whoever reported the bug to look at your reproduction steps.

A couple other things about bugs:

  • I try not to close issues in GitHub from other people if there’s any question about whether or not a fix resolved their problems, but I’ll still flush out old issues that have gone dormant
  • Try to pin down every reported bug with an automated test of some kind that runs in your CI build to avoid regression problems. Bugs tend to be non-obvious edge cases, so it’s even more important than most code to have some kind of test coverage. You can see the result of that philosophy here in the StructureMap testing library.
  • This is a long term play, but I’m hoping that I’ll be able to embed diagnostics in my tools that users could use to export some kind of data describing their system to make my life a lot easier when I’m trying to diagnose problems with nothing but a random stack trace.

Taking Pull Requests

I’ve been getting some great pull requests to StructureMap recently for performance and some big features that I knew other people have wanted in the past but I frankly didn’t want to build. That’s been great, but in the past I’ve also brought in some pull requests that have come back to haunt me through support problems and structural problems in the code.

Some thoughts on pull requests:

  • Do a much better job than I do about giving feedback to submitters at least to say that “got it, thank you, I’ll try to get to this by…” 😉
  • And I’m sorry, but I cannot and should not take in a pull request with no tests if it impacts the code. I’m sure that your code did work when you used it, but tests are also there to demonstrate to other users how it should work and to keep me or yet more pull requests from breaking your code later
  • As much as you’d like to make everybody happy who takes the time to submit pull requests, you cannot automatically take in pull requests with spotty quality because at the end of the day you’re the one responsible. Take Harry S. Truman’s philosophy of “the buck stops here” in regards to pull requests.
  • Don’t be too quick to take in pull requests because “that looks fine to me.” I’ve been burned several times by not thinking through the implications and edge cases that get introduced by a pull request — especially when it seems well coded with tests and everything. The author of the pull request may be thinking tactically about his or her immediate problem, but you have to take the strategic view of that code change.
  • If a user is going to submit a pull request that makes a substantial change in direction or internals, I’d rather know about it early to avoid any hard feelings and wasted time on their part if you don’t want to take it in.
  • Go easy on stylistic elements of the code like naming and formatting, but my experience has been that other developers have been pretty reasonable when they get timely feedback about a pull request. I will from time to time take in a pull request that has some problems and just address those myself quietly off to the side. You have to walk a line between maintaining an acceptable level of quality and consistency within the codebase and not making it too much trouble for other folks to contribute.

Push vs Pull Features

While I can point to exceptions to this rule, I’ve consistently found that features built for a demonstrated need or use case on my project or something requested from other users have been more successful than features that started from “wouldn’t it be cool if…”

Play the Long Game

In my strong opinion and experience, the most reliable way to achieve high quality and great usability is to iterate over time in response to feedback and the problems encountered along the way. If you want your project to be good you better be ready to have a long enough attention span to improve it over time by responding to the problems that pop up and never being complacent about your current approach.

My 11-12 years and counting involvement with StructureMap is an extreme case. Maybe more relevant is my old Storyteller tool for acceptance test driven development in .Net. The first two versions of Storyteller have some severe usability problems and more friction in its usage than I care to admit. Later this month I’m going to do my first public presentation on the new Storyteller 3.0 version that’s vastly better so far in daily usage as a direct result of paying attention to all the lessons we learned in using and building it over 5-6 years.

Don’t be afraid to jettison old features that no longer make sense or you no longer want to support. My rule of thumb on StructureMap has been that any feature that makes me cringe when someone asks how to use it because I just know this is going to be trouble has to go in the next release.

Dealing with Angry Users

Let’s face it, software developers are not particularly known for having great interpersonal skills and primarily interacting through online mediums lets some of the worst behaviors leak through that would probably never happen in person. If you publish an OSS tool of any complexity it’s not unlikely that you’re going to get to deal with irate users at the end of their rope. You may be saying to yourself that your tool’s usage is so obvious that there won’t be any problem and I’m going to tell you that perfectly intelligent developers come from a lot of different backgrounds, think differently than you, and absolutely will be confused by something that’s obvious to you.

All I can tell you is to:

  • Remember that they’re probably not at their best right now and it’s almost a stereotype that developers who are unquestionably assholes online can easily be calm, pleasant people in real life
  • Not take it too personally and remember that even if it turns out to be your fault, you can’t be expected to be omniscient and cut yourself some slack
  • Really don’t worry too much about your version of “if this isn’t fixed right now I’m going to switch to AutoFac!” because that person is probably someone you just don’t need to be interacting with anyway. My internal response is usually that I’d be happy to dump that user onto someone else’s user list or gitter room;-)

I know I’ll get yelled at for this one in comments or Twitter, but my consistent experience is that the more strident and angry a developer is being online the more likely it is that they’re doing something stupid.

Ironically, some of the worst bugs and problems I’ve had uncovered by users have come from people that were very reasonable and patient. I suspect that might be because it’s just easier to communicate without the venom flying. I also know that I do try harder to help out folks that are being polite and patient.

Should I Follow My Project on Twitter or Stackoverflow?

I think it really depends on how thick your skin is and how important the stewardship of an OSS tool is to your career. I’ve gone both ways with Twitter, but I’m back to following references to StructureMap at least just to understand what people are saying about it and occasionally to lend a hand. At other times I’ve had to ignore Twitter because developers as a whole tend to be assholes on Twitter safely behind their cutesy little cartoon avatars and the online snark and griping was just too aggravating. I guess my advice is to make sure that you’re decoupling your mental and emotional well-being from any kind of online negativity from other people. You also might remember that developers probably tweet much more when they’re angry or frustrated.

No matter how competitive you happen to be, don’t you dare let it ruin your weekend when folks are extolling the benefits of a competitor tool instead of yours.

As for Stackoverflow, I have to admit that I purposely avoid following my tools on Stackoverflow because I just can’t handle the stress of trying to deal with all of the deluge. Granted, some of that is because StructureMap questions more frequently verge into needing to give software design and architectural advice more than answering simple API usage type of questions. I do try to stay on top of questions that are more or less directed to me from mailing lists, Gitter rooms, or Twitter.

If you’re making a big career bet on some kind of OSS tool, I think you’d better watch Stackoverflow just to understand what the usability problems are with your tool — and sadly enough, you may need to combat misinformation about your tool from other people.

Building an Awesome Community Around Your Project

Y’all will have to fill this section in yourself in the comments because I’ve got nothing.

Exploiting Generic Types with StructureMap

I’ve got a short window at work that I’m using to try to finally fill in the holes in StructureMap documentation. Everything I’m showing here is old, but some of it I’ve never documented or written about before and the usage of generic types has spawned a lot of questions on the StructureMap list over the years. 

The content of this blog post looks a lot better in the actual StructureMap documentation site here.

Example 1: Visualizing an Activity Log

I worked years ago on a system that could be used to record and resolve customer support problems. Since it was very workflow heavy in its logic, we tracked user and system activity as an event stream of small objects that reflected all the different actions or state changes that could happen to an issue. To render and visualize the activity log to HTML, we used many of the open generic type capabilities shown in this topic to find and apply the correct HTML rendering strategy for each type of log object in an activity stream.

Given a log object, we wanted to look up the right visualizer strategy to render that type of log object to html on the server side.

To start, we had an interface like this one that we were going to use to get the HTML for each log object:

    public interface ILogVisualizer
    {
        // If we already know what the type of log we have
        string ToHtml<TLog>(TLog log);

        // If we only know that we have a log object
        string ToHtml(object log);
    }

So for an example, if we already knew that we had an IssueCreated object, we should be able to use StructureMap like this:

            // Just setting up a Container and ILogVisualizer
            var container = Container.For<VisualizationRegistry>();
            var visualizer = container.GetInstance<ILogVisualizer>();

            // If I have an IssueCreated lob object...
            var created = new IssueCreated();

            // I can get the html representation:
            var html = visualizer.ToHtml(created);

If we had an array of log objects, but we do not already know the specific types, we can still use the more generic ToHtml(object) method like this:

            var logs = new object[]
            {
                new IssueCreated(), 
                new TaskAssigned(), 
                new Comment(), 
                new IssueResolved()
            };

            // SAMPLE: using-visualizer-knowning-the-type   
            // Just setting up a Container and ILogVisualizer
            var container = Container.For<VisualizationRegistry>();
            var visualizer = container.GetInstance<ILogVisualizer>();

            var items = logs.Select(visualizer.ToHtml);
            var html = string.Join("<hr />", items);

The next step is to create a way to identify the visualization strategy for a single type of log object. We certainly could have done this with a giant switch statement, but we wanted some extensibility for new types of activity log objects and even customer specific log types that would never, ever be in the main codebase. We settled on an interface like the one shown below that would be responsible for rendering a particular type of log object (“T” in the type):

    public interface IVisualizer<TLog>
    {
        string ToHtml(TLog log);
    }

Inside of the concrete implementation of ILogVisualizer we need to be able to pull out and use the correct IVisualizer<T> strategy for a log type. We of course used a StructureMap Container to do the resolution and lookup, so now we also need to be able to register all the log visualization strategies in some easy way. On top of that, many of the log types were simple and could just as easily be rendered with a simple html strategy like this class:

    public class DefaultVisualizer<TLog> : IVisualizer<TLog>
    {
        public string ToHtml(TLog log)
        {
            return string.Format("
{0}
", log);
        }
    }

Inside of our StructureMap usage, if we don’t have a specific visualizer for a given log type, we’d just like to fallback to the default visualizer and proceed.

Alright, now that we have a real world problem, let’s proceed to the mechanics of the solution.

Registering Open Generic Types

Let’s say to begin with all we want to do is to always use the DefaultVisualizer for each log type. We can do that with code like this below:

        [Test]
        public void register_open_generic_type()
        {
            var container = new Container(_ =>
            {
                _.For(typeof (IVisualizer<>)).Use(typeof (DefaultVisualizer<>));
            });

            
            Debug.WriteLine(container.WhatDoIHave(@namespace:"StructureMap.Testing.Acceptance.Visualization"));
            

            container.GetInstance<IVisualizer<IssueCreated>>()
                .ShouldBeOfType<DefaultVisualizer<IssueCreated>>();

            Debug.WriteLine(container.WhatDoIHave(@namespace: "StructureMap.Testing.Acceptance.Visualization"));
            


            container.GetInstance<IVisualizer<IssueResolved>>()
                .ShouldBeOfType<DefaultVisualizer<IssueResolved>>();
        }

With the configuration above, there are no specific registrations for IVisualizer<IssueCreated>. At the first request for that interface, StructureMap will run through its “missing family policies“, one of which is to try to find registrations for an open generic type that could be closed to make a valid registration for the requested type. In the case above, StructureMap sees that it has registrations for the open generic type IVisualizer<T> that could be used to create registrations for the closed type IVisualizer<IssueCreated>.

Using the WhatDoIHave() diagnostics, the original state of the container for the visualization namespace is:

===========================================================================================================================
PluginType            Namespace                                         Lifecycle     Description                 Name     
---------------------------------------------------------------------------------------------------------------------------
IVisualizer<TLog>     StructureMap.Testing.Acceptance.Visualization     Transient     DefaultVisualizer<TLog>     (Default)
===========================================================================================================================

After making a request for IVisualizer<IssueCreated>, the new state is:

====================================================================================================================================================================================
PluginType                    Namespace                                         Lifecycle     Description                                                                  Name     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
IVisualizer<IssueCreated>     StructureMap.Testing.Acceptance.Visualization     Transient     DefaultVisualizer<IssueCreated> ('548b4256-a7aa-46a3-8072-bd8ef0c5c430')     (Default)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
IVisualizer<TLog>             StructureMap.Testing.Acceptance.Visualization     Transient     DefaultVisualizer<TLog>                                                      (Default)
====================================================================================================================================================================================

Generic Registrations and Default Fallbacks

A powerful feature of generic type support in StructureMap is the ability to register specific handlers for some types, but allow users to register a “fallback” registration otherwise. In the case of the visualization, some types of log objects may justify some special HTML rendering while others can happily be rendered with the default visualization strategy. This behavior is demonstrated by the following code sample:

        [Test]
        public void generic_defaults()
        {
            var container = new Container(_ =>
            {
                // The default visualizer just like we did above
                _.For(typeof(IVisualizer<>)).Use(typeof(DefaultVisualizer<>));

                // Register a specific visualizer for IssueCreated
                _.For<IVisualizer<IssueCreated>>().Use<IssueCreatedVisualizer>();
            });


            // We have a specific visualizer for IssueCreated
            container.GetInstance<IVisualizer<IssueCreated>>()
                .ShouldBeOfType<IssueCreatedVisualizer>();

            // We do not have any special visualizer for TaskAssigned,
            // so fall back to the DefaultVisualizer<T>
            container.GetInstance<IVisualizer<TaskAssigned>>()
                .ShouldBeOfType<DefaultVisualizer<TaskAssigned>>();
        }

Connecting Generic Implementations with Type Scanning

It’s generally harmful in software projects to have a single code file that has to be frequently edited to for unrelated changes, and StructureMap Registry classes that explicitly configure services can easily fall into that category. Using type scanning registration can help teams avoid that problem altogether by eliminating the need to make any explict registrations as new providers are added to the codebase.

For this example, I have two special visualizers for the IssueCreated and IssueResolved log types:

    public class IssueCreatedVisualizer : IVisualizer<IssueCreated>
    {
        public string ToHtml(IssueCreated log)
        {
            return "special html for an issue being created";
        }
    }

    public class IssueResolvedVisualizer : IVisualizer<IssueResolved>
    {
        public string ToHtml(IssueResolved log)
        {
            return "special html for issue resolved";
        }
    }

In the real project that inspired this example, we had many, many more types of log visualizer strategies and it could have easily been very tedious to manually register all the different little IVisualizer<T> strategy types in a Registry class by hand. Fortunately, part of StructureMap’s type scanning support is the ConnectImplementationsToTypesClosing()auto-registration mechanism via generic templates for exactly this kind of scenario.

In the sample below, I’ve set up a type scanning operation that will register any concrete type in the Assembly that contains the VisualizationRegistry that closes IVisualizer<T> against the proper interface:

    public class VisualizationRegistry : Registry
    {
        public VisualizationRegistry()
        {
            // The main ILogVisualizer service
            For<ILogVisualizer>().Use<LogVisualizer>();

            // A default, fallback visualizer
            For(typeof(IVisualizer<>)).Use(typeof(DefaultVisualizer<>));

            // Auto-register all concrete types that "close"
            // IVisualizer<TLog>
            Scan(x =>
            {
                x.TheCallingAssembly();
                x.ConnectImplementationsToTypesClosing(typeof(IVisualizer<>));
            });

        }
    }

If we create a Container based on the configuration above, we can see that the type scanning operation picks up the specific visualizers for IssueCreated and IssueResolved as shown in the diagnostic view below:

==================================================================================================================================================================================
PluginType                     Namespace                                         Lifecycle     Description                                                               Name     
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ILogVisualizer                 StructureMap.Testing.Acceptance.Visualization     Transient     StructureMap.Testing.Acceptance.Visualization.LogVisualizer               (Default)
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
IVisualizer<IssueResolved>     StructureMap.Testing.Acceptance.Visualization     Transient     StructureMap.Testing.Acceptance.Visualization.IssueResolvedVisualizer     (Default)
                                                                                 Transient     DefaultVisualizer<IssueResolved>                                                   
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
IVisualizer<IssueCreated>      StructureMap.Testing.Acceptance.Visualization     Transient     StructureMap.Testing.Acceptance.Visualization.IssueCreatedVisualizer      (Default)
                                                                                 Transient     DefaultVisualizer<IssueCreated>                                                    
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
IVisualizer<TLog>              StructureMap.Testing.Acceptance.Visualization     Transient     DefaultVisualizer<TLog>                                                   (Default)
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
IVisualizer<TLog>              StructureMap.Testing.Acceptance.Visualization     Transient     DefaultVisualizer<TLog>                                                   (Default)
==================================================================================================================================================================================

The following sample shows the VisualizationRegistry in action to combine the type scanning registration plus the default fallback behavior for log types that do not have any special visualization logic:

        [Test]
        public void visualization_registry()
        {
            var container = Container.For<VisualizationRegistry>();

            Debug.WriteLine(container.WhatDoIHave(@namespace: "StructureMap.Testing.Acceptance.Visualization"));

            container.GetInstance<IVisualizer<IssueCreated>>()
                .ShouldBeOfType<IssueCreatedVisualizer>();

            container.GetInstance<IVisualizer<IssueResolved>>()
                .ShouldBeOfType<IssueResolvedVisualizer>();

            // We have no special registration for TaskAssigned,
            // so fallback to the default visualizer
            container.GetInstance<IVisualizer<TaskAssigned>>()
                .ShouldBeOfType<DefaultVisualizer<TaskAssigned>>();
        }

Building Closed Types with ForGenericType() and ForObject()

Working with generic types and the common IHandler<T> pattern can be a little bit tricky if all you have is an object that is declared as an object. Fortunately, StructureMap has a couple helper methods and mechanisms to help you bridge the gap between DoSomething(object something) and DoSomething<T>(T something).

If you remember the full ILogVisualizer interface from above:

    public interface ILogVisualizer
    {
        // If we already know what the type of log we have
        string ToHtml<TLog>(TLog log);

        // If we only know that we have a log object
        string ToHtml(object log);
    }

The method ToHtml(object log) somehow needs to be able to find the right IVisualizer<T> and execute it to get the HTML representation for a log object. The StructureMap IContainer provides two different methods called ForObject() and ForGenericType() for exactly this case, as shown below in a possible implementation of ILogVisualizer:

    public class LogVisualizer : ILogVisualizer
    {
        private readonly IContainer _container;

        // Take in the IContainer directly so that
        // yes, you can use it as a service locator
        public LogVisualizer(IContainer container)
        {
            _container = container;
        }

        // It's easy if you already know what the log
        // type is
        public string ToHtml<TLog>(TLog log)
        {
            return _container.GetInstance<IVisualizer<TLog>>()
                .ToHtml(log);
        }

        public string ToHtml(object log)
        {
            // The ForObject() method uses the 
            // log.GetType() as the parameter to the open
            // type Writer<T>, and then resolves that
            // closed type from the container and
            // casts it to IWriter for you
            return _container.ForObject(log)
                .GetClosedTypeOf(typeof (Writer<>))
                .As<IWriter>()
                .Write(log);
        }

        public string ToHtml2(object log)
        {
            // The ForGenericType() method is again creating
            // a closed type of Writer<T> from the Container
            // and casting it to IWriter
            return _container.ForGenericType(typeof (Writer<>))
                .WithParameters(log.GetType())
                .GetInstanceAs<IWriter>()
                .Write(log);
        }

        // The IWriter and Writer<T> class below are
        // adapters to go from "object" to <T>() signatures
        public interface IWriter
        {
            string Write(object log);
        }

        public class Writer<T> : IWriter
        {
            private readonly IVisualizer<T> _visualizer;

            public Writer(IVisualizer<T> visualizer)
            {
                _visualizer = visualizer;
            }

            public string Write(object log)
            {
                return _visualizer.ToHtml((T) log);
            }
        }
    }

The two methods are almost identical in result with some slight differences:

  1. ForObject(object subject) can only work with open types that have only one generic type parameter, and it will pass the argument subject to the underlying Container as an explicit argument so that you can inject that subject object into the object graph being created.
  2. ForGenericType(Type openType) is a little clumsier to use, but can handle any number of generic type parameters

Example #2: Generic Instance Builder

As I recall, the following example was inspired by a question about how to use StructureMap to build out MongoDB MongoCollection objects from some sort of static builder or factory — but I can’t find the discussion on the mailing list as I write this today. This has come up often enough to justify its inclusion in the documentation.

Say that you have some sort of persistence tooling that you primarily interact with through an interface like this one below, where TDocument and TQuery are classes in your persistent domain:

    public interface IRepository<TDocument, TQuery>
    {

    }

Great, StructureMap handles generic types just fine, so you can just register the various closed types and off you go. Except you can’t because the way that your persistence tooling works requires you to create the IRepository<,>objects with a static builder class like this one below:

    public static class RepositoryBuilder
    {
        public static IRepository<TDocument, TQuery> Build<TDocument, TQuery>()
        {
            return new Repository<TDocument, TQuery>();
        }
    }

StructureMap has an admittedly non-obvious way to handle this situation by creating a new subclass of Instance that will “know” how to create the real Instance for a closed type of IRepository<,>.

First off, let’s create a new Instance type that knows how to build a specific type of IRepository<,> by subclassing the LambdaInstance type and providing a Func to build our repository type with the static RepositoryBuilder class:

    public class RepositoryInstance<TDocument, TQuery> : LambdaInstance<IRepository<TDocument, TQuery>>
    {
        public RepositoryInstance() : base(() => RepositoryBuilder.Build<TDocument, TQuery>())
        {
        }

        // This is purely to make the diagnostic views prettier
        public override string Description
        {
            get
            {
                return "RepositoryBuilder.Build<{0}, {1}>()"
                    .ToFormat(typeof(TDocument).Name, typeof(TQuery).Name);
            }
        }
    }

As you’ve probably surmised, the custom RepositoryInstance above is itself an open generic type and cannot be used directly until it has been closed. You could use this class directly if you have a very few document types like this:

            var container = new Container(_ =>
            {
                _.For<IRepository<string, int>>().UseInstance(new RepositoryInstance<string, int>());

                // or skip the custom Instance with:

                _.For<IRepository<string, int>>().Use(() => RepositoryBuilder.Build<string, int>());
            });

To handle the problem in a more generic way, we can create a second custom subclass of Instance for the open type IRepository<,> that will help StructureMap understand how to build the specific closed types of IRepository<,> at runtime:

    public class RepositoryInstanceFactory : Instance
    {
        // This is the key part here. This method is called by
        // StructureMap to "find" an Instance for a closed
        // type of IRepository<,>
        public override Instance CloseType(Type[] types)
        {
            // StructureMap will cache the object built out of this,
            // so the expensive Reflection hit only happens
            // once
            var instanceType = typeof (RepositoryInstance<,>).MakeGenericType(types);
            return Activator.CreateInstance(instanceType).As<Instance>();
        }

        // Don't worry about this one, never gets called
        public override IDependencySource ToDependencySource(Type pluginType)
        {
            throw new NotSupportedException();
        }

        public override string Description
        {
            get { return "Build Repository<T, T1>() with RepositoryBuilder"; }
        }

        public override Type ReturnedType
        {
            get { return typeof (Repository<,>); }
        }
    }

The key part of the class above is the CloseType(Type[] types) method. At that point, we can determine the right type of RepositoryInstance<,> to build the requested type of IRepository<,>, then use some reflection to create and return that custom Instance.

Here’s a unit test that exercises and demonstrates this functionality from end to end:

        [Test]
        public void show_the_workaround_for_generic_builders()
        {
            var container = new Container(_ =>
            {
                _.For(typeof (IRepository<,>)).Use(new RepositoryInstanceFactory());
            });

            container.GetInstance<IRepository<string, int>>()
                .ShouldBeOfType<Repository<string, int>>();

            Debug.WriteLine(container.WhatDoIHave(assembly:Assembly.GetExecutingAssembly()));
        }

After requesting IRepository<string, int> for the first time, the container configuration from Container.WhatDoIHave() is:

===================================================================================================================================================
PluginType                         Namespace                           Lifecycle     Description                                          Name     
---------------------------------------------------------------------------------------------------------------------------------------------------
IRepository<String, Int32>         StructureMap.Testing.Acceptance     Transient     RepositoryBuilder.Build<String, Int32>()             (Default)
---------------------------------------------------------------------------------------------------------------------------------------------------
IRepository<TDocument, TQuery>     StructureMap.Testing.Acceptance     Transient     Build Repository<T, T1>() with RepositoryBuilder     (Default)
===================================================================================================================================================

New Construction Policies in StructureMap 4.0

This is taken from some brand spanking new StructureMap docs on construction policies. As part of the forthcoming StructureMap 4.0 release this month I’m hoping to flood my blog with little posts on StructureMap. The code samples look a lot better in the actual docs, mea culpa.

StructureMap has long supported conventional policies for registration based on type scanning and 3.0 introduced cleaner mechanisms for interception policies. The 4.0 release extends StructureMap’s support for conventional build policies with a new mechanism for altering how object instances are built based on user created meta-conventions using the IInstancePolicy shown below:

    /// <summary>
    /// Custom policy on Instance construction that is evaluated
    /// as part of creating a "build plan"
    /// </summary>
    public interface IInstancePolicy
    {
        /// <summary>
        /// Apply any conventional changes to the configuration
        /// of a single Instance
        /// </summary>
        /// <param name="pluginType"></param>
        /// <param name="instance"></param>
        void Apply(Type pluginType, Instance instance);
    }

These policies are registered as part of the registry dsl with the Policies.Add() method:

            var container = new Container(_ =>
            {
                _.Policies.Add<MyCustomPolicy>();
                // or
                _.Policies.Add(new MyCustomPolicy());
            });

The IInstancePolicy mechanism probably works differently than other IoC containers in that the policy is applied to the container’s underlying configuration model instead of at runtime. Internally, StructureMap lazily creates a “build plan” for each configured Instance at the first time that that Instance is built or resolved. As part of creating that build plan, StructureMap runs all the registered IInstancePolicy objects against the Instance in question to capture any potential changes before “baking” the build plan into a .Net Expression that is then compiled into a Func for actual construction.

The Instance objects will give you access to the types being created, the configured name of the Instance (if any), the ability to add interceptors and to modify the lifecycle. If you wish to add inline dependencies to Instances that are built by calling constructor function and setter properties, you may find it easier to use the ConfiguredInstancePolicy base class as a convenience:

    public abstract class ConfiguredInstancePolicy : IInstancePolicy
    {
        public void Apply(Type pluginType, Instance instance)
        {
            var configured = instance as IConfiguredInstance;
            if (configured != null)
            {
                apply(pluginType, configured);
            }
        }

        // This method is called against any Instance that implements 
        // the IConfiguredInstance interface
        protected abstract void apply(Type pluginType, IConfiguredInstance instance);
    }

For more information, see:

Example 1: Constructor arguments

So let me say upfront that I don’t like this approach, but other folks have asked for this ability over the years. Say that you have some legacy code where many concrete classes have a constructor argument called “connectionString” that needs to be the connection string to the application database like these classes:

        public class DatabaseUser
        {
            public string ConnectionString { get; set; }

            public DatabaseUser(string connectionString)
            {
                ConnectionString = connectionString;
            }
        }

        public class ConnectedThing
        {
            public string ConnectionString { get; set; }

            public ConnectedThing(string connectionString)
            {
                ConnectionString = connectionString;
            }
        }

Instead of explicitly configuring every single concrete class in StructureMap with that inline constructor argument, we can make a policy to do that in one place:

        public class ConnectionStringPolicy : ConfiguredInstancePolicy
        {
            protected override void apply(Type pluginType, IConfiguredInstance instance)
            {
                var parameter = instance.Constructor.GetParameters().FirstOrDefault(x => x.Name == "connectionString");
                if (parameter != null)
                {
                    var connectionString = findConnectionStringFromConfiguration();
                    instance.Dependencies.AddForConstructorParameter(parameter, connectionString);
                }
            }

            // find the connection string from whatever configuration
            // strategy your application uses
            private string findConnectionStringFromConfiguration()
            {
                return "the connection string";
            }
        }

Now, let’s use that policy against the types that need “connectionString” and see what happens:

        [Test]
        public void use_the_connection_string_policy()
        {
            var container = new Container(_ =>
            {
                _.Policies.Add<ConnectionStringPolicy>();
            });

            container.GetInstance<DatabaseUser>()
                .ConnectionString.ShouldBe("the connection string");

            container.GetInstance<ConnectedThing>()
                .ConnectionString.ShouldBe("the connection string");
        }

Years ago StructureMap was knocked by an “IoC expert” for not having this functionality. I said at the time — and still would — that I would strongly recommend that you simply don’t directly open database connections in more than one or a very few spots in your code anyway. If I did need to configure a database connection string in multiple concrete classes, I prefer strong typed configuration.

Example 2: Connecting to Databases based on Parameter Name

From another common user request over the years, let’s say that your application needs to connect to multiple databases, but your data access service in both cases is an interface called IDatabase, and that’s all the consumers of any database should ever need to know.

To make this concrete, let’s say that our data access is all behind an interface and concrete class pair namedDatabase/IDatabase like so:

        public interface IDatabase { }

        public class Database : IDatabase
        {
            public string ConnectionString { get; set; }

            public Database(string connectionString)
            {
                ConnectionString = connectionString;
            }

            public override string ToString()
            {
                return string.Format("ConnectionString: {0}", ConnectionString);
            }
        }

For a registration policy, let’s say that the parameter name of an IDatabase dependency in a constructor function should match an identifier of one of the registered IDatabase services. That policy would be:

        public class InjectDatabaseByName : ConfiguredInstancePolicy
        {
            protected override void apply(Type pluginType, IConfiguredInstance instance)
            {
                instance.Constructor.GetParameters()
                    .Where(x => x.ParameterType == typeof (IDatabase))
                    .Each(param =>
                    {
                        // Using ReferencedInstance here tells StructureMap
                        // to "use the IDatabase by this name"
                        var db = new ReferencedInstance(param.Name);
                        instance.Dependencies.AddForConstructorParameter(param, db);
                    });
            }
        }

And because I’m generally pretty boring about picking test data names, let’s say that two of our databases are named “red” and “green” with this container registration below:

            var container = new Container(_ =>
            {
                _.For<IDatabase>().Add<Database>().Named("red")
                    .Ctor<string>("connectionString").Is("*red*");

                _.For<IDatabase>().Add<Database>().Named("green")
                    .Ctor<string>("connectionString").Is("*green*");
                
                _.Policies.Add<InjectDatabaseByName>();
            });

For more context, the classes that use IDatabase would need to have constructor functions like these below:

        public class BigService
        {
            public BigService(IDatabase green)
            {
                DB = green;
            }

            public IDatabase DB { get; set; }
        }

        public class ImportantService
        {
            public ImportantService(IDatabase red)
            {
                DB = red;
            }

            public IDatabase DB { get; set; }
        }

        public class DoubleDatabaseUser
        {
            public DoubleDatabaseUser(IDatabase red, IDatabase green)
            {
                Red = red;
                Green = green;
            }

            // Watch out for potential conflicts between setters
            // and ctor params. The easiest thing is to just make
            // setters private
            public IDatabase Green { get; private set; }
            public IDatabase Red { get; private set; }
        }

Finally, we can exercise our new policy and see it in action:

            // ImportantService should get the "red" database
            container.GetInstance<ImportantService>()
                .DB.As<Database>().ConnectionString.ShouldBe("*red*");

            // BigService should get the "green" database
            container.GetInstance<BigService>()
                .DB.As<Database>().ConnectionString.ShouldBe("*green*");

            // DoubleDatabaseUser gets both
            var user = container.GetInstance<DoubleDatabaseUser>();

            user.Green.As<Database>().ConnectionString.ShouldBe("*green*");
            user.Red.As<Database>().ConnectionString.ShouldBe("*red*");

How I prefer to do this – my strong preference would be to use separate interfaces for the different databases even if that type is just an empty type marker that implements the same base. I feel like using separate interfaces makes the code easier to trace and understand than trying to make StructureMap vary dependencies based on naming conventions or what namespace a concrete type happens to be in. At least now though, you have the choice of my way or using policies based on naming conventions.

Example 3: Make objects singletons based on type name

Unlike the top two examples, this is taken from a strategy that I used in FubuMVC for its service registration. In that case, we wanted any concrete type whose name ended with “Cache” to be a singleton in the container registration. With the new IInstancePolicy feature in StructureMap 4, we could create a new policy class like so:

        public class CacheIsSingleton : IInstancePolicy
        {
            public void Apply(Type pluginType, Instance instance)
            {
                if (instance.ReturnedType.Name.EndsWith("Cache"))
                {
                    instance.SetLifecycleTo<SingletonLifecycle>();
                }
            }
        }

Now, let’s say that we have an interface named IWidgets and a single implementation called WidgetCache that should track our widgets in the application. Using our new policy, we should see WidgetCache being made a singleton:

        [Test]
        public void set_cache_to_singleton()
        {
            var container = new Container(_ =>
            {
                _.Policies.Add<CacheIsSingleton>();

                _.For<IWidgets>().Use<WidgetCache>();
            });

            // The policy is applied *only* at the time
            // that StructureMap creates a "build plan"
            container.GetInstance<IWidgets>()
                .ShouldBeTheSameAs(container.GetInstance<IWidgets>());

            // Now that the policy has executed, we 
            // can verify that WidgetCache is a singleton
            container.Model.For<IWidgets>().Default
                .Lifecycle.ShouldBeOfType<SingletonLifecycle>();
        }

 

The Surprisingly Valuable and Lasting Lessons I Learned from a Horrible Project

Let’s face it; we tend to learn lessons more profoundly from bad experiences, mistakes, and very unpleasant circumstances because we just don’t want to ever go through that again. Last year I tried to write a blog post about all the positive and useful things learned from Extreme Programming (XP) earlier in my career, but I kept getting stuck on one little problem. The only truly XP as described in Chairman Beck’s Little White Book* project I worked on was a horrible experience and probably the least productive team and most unpleasant working environment I have ever been a part of in almost 20 years of software development.

* Despite my cynical turn of phrase, I thought that Beck’s original XP book was well worth the read

Why bring this up now? I know that I learned some tremendously important lessons during and after that awful project that are still valuable. Plus, after all these years, writing this has made me consider what I might have done differently.

The Lessons

  • Own your development process and adapt
  • People communicate and learn in different ways and it pays to understand that and adapt your approach as a technical lead
  • A little team turnover is good, but too much is terrible
  • Build a level of trust and defuse tension between team members early
  • Intensive, detailed software processes and micromanagement down to the task level can lead to very poor technical results because it discourages developer initiative and adaptation. This burned us badly on the project I’m describing and I honestly worry about this on some of our projects at my work.

Why was it so bad?

I was the nominal technical lead of a team from a consulting company that was working at our customer site with a mixed team from our company and the customer’s own staff. We were mostly building new workflow and automated decision support tools on a very early version of .Net to replace some of their homegrown Powerbuilder and Oracle applications. I was the only member of our consultant team who had any prior experience working with .Net.

So exactly how bad was it? You know that joke about how “the beatings will continue until morale improves?” That was us. We had bi-weekly mandatory team building meetings after work that were generally pretty miserable.

As usual when consultants are parachuted in, the client staff resented us somewhat for being there — and I’m pretty sure they more than suspected that their employer intended to let them all go when our project advanced far enough. It probably didn’t help that my consultant team wasn’t terribly professional at the client site.

We had way too many strong personalities and opinions about how things should be done and endless arguments about the minutiae of how an XP project should be executed. We had absurdly long and contentious iteration kickoff and retrospective meetings. We had the dreaded embarrassment meeting at the end of each iteration where the project manager did his very best to humiliate the struggling development team.

Mostly though, I was embarrassed and frustrated by how little we seemed to be able to accomplish in the code compared to my previous (and subsequent) project experiences — even though I think it probably was one of the strongest teams I’ve ever been on in terms of the individuals involved. Some of it could be explained by having many folks that were new to the platform we were using or the XP practices we were trying to use (while I’m still a big fan of TDD, it can have a pretty nasty learning curve). Mostly though, I think that using XP in a very dogmatic way was our major undoing — in particular the requirement that all production coding had to be done as pair programming made it very difficult for me because I was by necessity working somewhat different hours than the others because I was traveling from much farther away.

I honestly don’t know how the project ended up because I was effectively kicked out by the project manager in what I felt at the time was kind of an act of mercy. I left the consulting company in question not long after, mostly because of personal issues but also because I was disillusioned with them after this experience.

I did learn some great technical lessons that might appear in another post on XP someday and I worked with some truly great individual developers — which really just made it so much more frustrating how ineffective this entire project team really was.

Regular readers of this blog know how much I hate words like zealot or dogma or pragmatism because of how developers so often use those words to deodorize the stench of sloppy self-justifications, but “dogma” and “zealot” do accurately apply here;-)

Enough of that, on to the lessons…

Own your development process

The most powerful lesson I learned on this terrible project was about proactively owning and adapting your team’s process and practices instead of just trying to muddle through when something isn’t working. For most of this project, we did our iteration estimation by having developers try to estimate in ideal hours. During iterations the developers would attempt to track how many of these nebulous “ideal hours” we had spent on each story. Needless to say, we had burned a lot of hours on arguing exactly how many “ideal hours” we were really spending and what the estimate for a user story really should be in iteration kickoff meetings. The killer for me was how the project manager would take all of these numbers to show in excruciating detail exactly how badly our struggling development team was missing our estimates on every single story in front of the client and the rest of the team.

As you can imagine, this resulted in long, nasty iteration kickoff meetings as the other developers and I tried to avoid the humiliation. Just a couple incidents I remember:

  1. Being pulled out of an iteration kickoff meeting because one of the BA’s was mad about how he thought we were inflating the estimates. It ended with me screaming at our account manager that I had to do that to protect myself while pointing at the project manager (if you’ve never met me, I’m a big guy and I’d bet that a very angry me might be a little bit upsetting).
  2. Going up to the project manager to show him how a big new subsystem had gone much better than expected and effectively being told that I had been sandbagging estimates. To say the least, I fumed a little bit about that one.

Fortunately, our resident older developer got up at a retrospective meeting one time and made a very simple suggestion: our stories seem to come in about three different sizes, so instead of worrying about ideal hours at all, let’s just estimate stories as small, medium, or large (shirt size estimation). My recollection is that as soon as we adopted the new estimation technique, iteration meetings became much quicker and smoother. As we eliminated the idiotic estimate tracking out of our retrospectives, our meetings became less contentious, faster, and more productive.

The big lesson I took away is to never, ever treat your software process as something that is set in stone. I also learned that software teams should be empowered to make corrections to the way they work.

I blogged about a similar theme when I was on Codebetter ages ago.

Interpersonal Tension and Lack of Trust

I guess I have to be honest here just in case it’s not obvious and admit that I still hold a pretty significant grudge toward that project manager. One of the first interactions I remember having with the project manager was him approaching me at some kind of after hours kickoff social and telling me that “he just got a bad vibe off of me.” To my detriment, I just hunkered down instead of trying to ask him what the hell he meant and finding a way to get past that. I certainly never trusted him, and he made it abundantly clear that he didn’t trust the way that I was leading the development effort.

I spent a lot of time on that project worrying about protecting myself from the project manager and his verbal abuse. The project manager spent a lot of time criticizing me for “over design” or trying to subvert the XP process. I responded by veering too far toward not doing any design at all and really stopped thinking about anything that wasn’t in the iteration plan so I would have to get more lectures about how things would work better if I just got in line and followed the XP process.

The end result of that was a lot of technical debt in our heavy client related to our navigation logic. I had known for a long time that we were writing bad spaghetti code that was going to hurt us and already had a design concept based on what’s now called the screen conductor pattern. If I were doing that project today, I would opt for the screen conductor design early on, but then I was so concerned with the charges of over design that I put it off, partly because it would have involved having to create a user story that had to be approved by the project manager. Once we finally had to admit that we were having a technical debt melt down, we played an official story to change to the architecture using the screen conductor abstraction idea. In my opinion, that work took much longer than it would have taken if we’d done it early on when it was apparent that we needed to do that.

To this day, I consider this episode of delaying the cleanup of the navigation logic one of the worst technical design decisions I’ve made – and it all could have been avoided if the project manager had trusted my judgement so that I could have acted earlier as I saw fit and if I had trusted him enough to actually communicate with him. I didn’t go to him with these concerns early because I thought he’d just mock me for wanting to build a framework.

The over design issue never really went away, not even when a very senior architect from our company was brought into check up on me and assured the PM and I that things were fine.

Oh, and the project manager in question? Last I knew he was a somewhat successful “Agile Coach” in the Chicago area. He might very well be much better for his clients today than he was for me a decade ago, but the normal cynicism toward “Agile Coach” consultants probably applies here.

Team Turnover

I think that having at least a little bit of team turnover is valuable. A reasonable trickle of new team members can add new knowledge, techniques, and perspective to a team that might be getting some “that’s the way we’ve always done it” blinders.

For example, during the project I’ve been discussing, we had a senior tester with some pretty heavy test automation skills come in for just a handful of iterations and did a lot to help us jumpstart our end to end automated testing (and taught me some things that I still use and recommend today).

On the downside, we had an absurd amount of churn in our team makeup from iteration to iteration. The client developers worked with us directly at first, then they were shunted off to the side for training (on a bunch of esoteric topics of no use to our project of course). Then our company added a second team overseas, and sent one of our core guys there to help get that started, but then we brought some of the overseas developers to our site for a while. Other developers bounced in and out of the team and my eventual replacement came in later as well. The end result was that we seemed to have an all new team makeup every single iteration and I think it hurt our ability to form any kind of chemistry, we didn’t really develop the “tribal knowledge” that’s so important for long running projects, and I felt like I was constantly spinning my wheels onboarding new developers to the team.

Different Learning and Communication Styles

To break the stream of negativity, a very useful lesson I learned on this project was to pay attention to how developers have different styles of learning and communication and to tailor your approach as a development lead to individuals.

Like many developers, I’m mostly a visual learner first, and kinesthetic learner second (not so much as my knees get worse, but I like to think through design problems while shooting baskets or walking around). One of the other senior developers on that project was almost strictly an auditory learner and just didn’t derive any value whatsoever from the boxes and arrows I kept trying to draw up on the whiteboard. I quickly learned that I just had to collaborate differently with him than I did with another senior developer who was more visual like I am.

See this article for more information on learning styles.

What would I do differently today?

I’ve been working on this post for about a week and I’ve spent quite a bit of time pondering what I could have done differently to have made that project a better experience. I know that an older, more self-confident me could probably handle that situation  far more effectively and probably get past both the interpersonal and process problems. Truth be told though, if I were in that kind of situation today I would probably just look for another job and not look back because life is just too short for another of these projects.

But back to the real exercise, what could I have done better?

  • Try to come to some kind of better working relationship and understanding with the project manager. What if I had tried to understand his concerns with me earlier? Hell, what if I’d gotten into his face when he jabbed at me instead of just letting him push me around like I probably did? Tell my management or the account manager that either I had to go or the project manager? That would have probably resulted in me getting moved off or even let go, but in retrospect, that would have been much better than hanging around for 9 months of punishment.
  • Deal with the accusations that I was indulging in over design early in the project
  • Question our process much more early on so we could have eliminated the “embarrassment meeting” anti-pattern early
  • As far as the architecture is concerned, I wish I had listened more to myself and dealt with the technical debt issue in our screen navigation early when the problem was first apparent

Streamlining FubuMVC Bootstrapping & the Design Patterns Used

As I said in my last post, we’re rebooting FubuMVC as a new project called Jasper. As an intermediate step, I’m working up a FubuMVC 3.0 release that among other things, simplifies the application configuration and bootstrapping so that it’s quicker and easier to stand up a brand new FubuMVC web or service bus application.

At it’s very, very simplest for the classic Hello World exercise, the newly streamlined bootstrapping looks like this (when we rebrand as Jasper, just replace “Fubu” with “Jasper” in the class names below):

    // This little endpoint class handles
    // the root url of the application
    public class HomeEndpoint
    {
        public string Index()
        {
            return "Hello, World";
        }
    }

    [TestFixture]
    public class RunHelloWorld
    {
        [Test]
        public void start_and_run()
        {
            // Bootstrap a basic FubuMVC applications
            // with just the default policies and services
            using (var runtime = FubuRuntime.Basic())
            {
                // Execute the home route and verify
                // the response
                runtime.Scenario(_ =>
                {
                    _.Get.Url("/");

                    _.StatusCodeShouldBeOk();
                    _.ContentShouldBe("Hello, World");
                });
            }
        }
    }

Of course, as we all know, real world development gets a lot hairier than cute little hello world examples and you’re not going to get away with the framework defaults in most cases. In the case of FubuMVC, you might want to add services to the application IoC container, change or add conventions and policies, configure features, turn on “opt in” features, or run an application in either the special “development” or “testing” mode.

Real World Configuration in FubuMVC 3.0

After all the dust settled, a FubuMVC application is completely described, configured, bootstrapped, and cleanly deactivated by just two classes:

  1. FubuRegistry is used to express all of the configurable elements of a FubuMVC application
  2. FubuRuntime holds all the runtime elements of a running FubuMVC application like the application container, the activation logs, the routing table, the root file path, the application mode (development/testing/normal) and “knows” how to cleanly shut down the running application.

A custom FubuRegistry might look something like this one:

    public class ExampleRegistry : FubuRegistry
    {
        public ExampleRegistry()
        {
            // Turn on some opt in features
            Features.Localization.Enable(true);
            Features.Diagnostics.Enable(TraceLevel.Production);

            // Change the application mode if you want
            Mode = "development";

            // Have the application use an embedded 
            // Katana host at port 5501
            HostWith<Katana>(5501);

            // Register services with the IoC container
            // using a superset of StructureMap's
            // Registry DSL
            Services.AddService<IActivator, MyActivator>();
        
            // For testing purposes, you may want 
            // to bootstrap the application from an external
            // testing library, in which case, you'd want
            // to override where FubuMVC looks for static
            // asset files like JS or CSS files
            UseParallelDirectory("MyApp");
            // or
            RootPath = "some other path";
        }
    }

To bootstrap the application specified by ExampleRegistry, you can use this syntax below to create a new FubuRuntime object:

            using (var server = FubuRuntime.For<ExampleRegistry>())
            {
                // do stuff with the application
                // or wait for some kind of signal
                // that you should shut it off
            }

If you don’t want to bother with your own subclass of FubuRegistry, you can forgo it and bootstrap a FubuRuntime with syntax like this:

            var runtime = FubuRuntime.Basic(_ =>
            {
                _.Features.Diagnostics.Enable(TraceLevel.Verbose);

                // I'm opting for NOWIN hosting this time
                // and letting FubuMVC try to pick an open
                // port starting from 5500
                _.HostWith<NOWIN>();
            });

There is a FubuRegistry involved in the code above, but you’re configuring it inside the lambda expression argument. More on that below in the section on the design patterns.

Bootstrapping Use Cases and When You Want Your Own FubuRegistry

Just for some background, here are the various use cases for bootstrapping a FubuMVC web or service bus application that I could think of:

  1. Bootstrapping for ASP.Net hosting in the good ol’ Global.asax like this.
  2. Spinning up an application adhoc inside of tests like this example from FubuMVC 2.2.
  3. We have a tool we call Serenity (yes, it’s named for what you think it is) that we use to setup integration tests for FubuMVC with Storyteller.
  4. Running an application with our fubu run development server
  5. Hosting a FubuMVC application within a background service using our JasperService tool (was BottleServiceRunner) that’s just a small shim to TopShelf

The easiest way to use Serenity is to say “here, use this FubuRegistry,” while options #4 and #5 use type scanning to find either the one single FubuRegistry in your code or to use one by name to activate the hosted application. In these cases, it’s highly useful to have a custom FubuRegistry. Even without that, it’s also valuable to have your application configuration done in a way that’s easily reusable within automated tests as in use case #2 up above so that you’re tests are more reflective of how the actual application runs.

 

An aside on the design patterns I’ve used

Other people certainly have much more negative opinions, but I feel like learning, studying, and applying design patterns in code has been very useful to my career. At Oredev 2010 I gave a talk about Internal DSL’s in C# whose content is still very much relevant to the work I did the past month with FubuMVC’s bootstrapping.

The FubuMVC bootstrapping uses at least three different design patterns you can find described in Martin Fowler’s DSL book.

  1. Fluent Interface — method chaining to create an internal DSL directly in C# “if you squint hard enough”
  2. Object Scoping. The FubuRegistry (or StructureMap’s Registry) are examples of using “object scoping” to shorten the signatures of fluent interfaces to a base class. Most of your declarations are going to be in a custom fluent interface called in the constructor function of a FubuRegistry base class.
  3. Nested Closure — The usage of FubuRuntime.Basic(x => {}) where you use an action that configures some option type as the single argument to a function. Nested closures are helpful where you need to allow a user to specify any number of optional parameters as the input to a discrete action. I discussed nested closures ages ago in an MSDN article.

I also remember going to a lot of talks at that Oredev on the new flashy HTML 5 technologies that have been long since passed by. The obvious takeaway is that conceptual knowledge tends to outlast the usefulness of knowledge about specific technologies.

 

Just for Comparison, FubuMVC before 3.0

When I was asked by some of our technical leadership to simplify the bootstrapping for FubuMVC 3.0, my first reaction was “it’s not that bad” — but this was what FubuMVC had previously:

  • FubuRegistry & FubuRuntime were just a little smaller than they are today
  • You used a static property called FubuMvcPackageFacility.PhysicalRootPath to change the root directory of the content files. And yes, the mutable static property was a problem.
  • FubuApplication was a static class that you used to trigger a fluent interface that would gather up your FubuRegistry and allow you to choose an IoC adapter or use a pre-built IoC container in order to create a new FubuRuntime
  • IApplicationSource was a reusable recipe for how to bootstrap a FubuRuntime that we previously used for fubu run or our older TopShelf service host.
  • EmbeddedFubuMvcServer was a class we used to stand up a FubuMVC application hosted with Katana for testing or just embedding a web server for diagnostics in service bus applications running as a background service. Of course, that all got duplicated when we added NOWIN support and it duplicated some functionality from FubuRuntime anyway.
  • FubuMode was a static class we used to detect and tell you whether the application should run normally or in “development” or “testing” mode. Statics are evil you say? Yeah well keep reading…
  • PackageRegistry was a static class from our now defunct Bottles library that exposed information about the loaded extension assemblies in the application and the diagnostic logging from application bootstrapping.

As of now, we’ve completely eliminated all of the static classes and properties. All of the configuration is done through FubuRegistry, and all of the runtime information about a FubuMVC application is exposed off of FubuRuntime. So yeah, it’s quite a bit better now. It’s good to pay attention to the feedback of others because they see things you don’t or problems you just get too comfortable working around.

 

 

Gutting FubuMVC and Rebooting as “Jasper”

tl;dr – FubuMVC and FubuTransportation (the service bus tooling we built on top of FubuMVC) are getting a full reboot with the name “Jasper” on the new DNX platform. This blog post tries to explain why we’d do such a silly thing and describe our current thinking on the technical direction to start getting some feedback. Just for fun, I’m also describing a lot of functionality that I’ve been ripping out of FubuMVC in preparation for the reboot for folks that are interested in how web development has changed since FubuMVC was conceived in 2008-9.

My wife loves watching all the home remodeling shows on HG TV. One of her favorites is a show called Love it or List It. The premise of the show is a couple that wants to move to a new house gets the opportunity to choose between staying in their old home after it has been remodeled by one of the show’s stars — or decides to sell the now remodeled home in favor of purchasing a different house that the other star of the show finds for them on the market. Last year I said that I was giving up on FubuMVC development when it became clear that it was going nowhere and our community support was almost completely gone.

My shop had some flirtations with other platforms and like many shops we have been supplementing .Net development with Node.js work, but this is our current situation as I see it:

  1. We’ve got a pretty big portfolio of existing FubuMVC-based applications, and the idea of rewriting them to a new platform or even just a different .Net application framework or architecture is daunting
  2. We’re very happy with how the FubuTransportation service bus built on top of FubuMVC has worked out for us in production, but we would like it to be sitting on top of a technical foundation that isn’t “dead” like FubuMVC
  3. We’d love to get to be able to “Docker-ize” our applications and quite possibly move our production hosting and day to day development off of Windows
  4. We’ve got a huge investment and coupling in test automation and diagnostics tooling tied to FubuMVC and FubuTransportation that’s providing value
  5. I think many of us are generally positive about the new .Net tooling (DNX, Roslyn, CoreCLR) — except for the part where they didn’t eliminate strong naming in the new world order:(

Taking those factors into account, we’ve decided to “love it” instead of “leaving it” with what I’ve been calling a “Casino Royale style reboot” of the newly combined FubuMVC and FubuTransportation.

I’m working this week to hopefully finish up an intermediate FubuMVC 3.0 release that largely consolidates the codebase, streamlines the application bootstrapping, improves the diagnostics, and eliminates a whole lot of code and functionality that no longer matters. When we finally make the jump to DNX, FubuMVC/FubuTransportation is going to be rebranded as “Jasper.”

IMG_1017

 

The Vision for Jasper

My personal hopes for Jasper are that we retain the best parts of FubuMVC, dramatically improve the performance and scalability of our applications, and solve the worst of the usability problems that FubuMVC and FubuTransportation have today. I’m also hoping that we end up with a decent foundation of technical documentation just for once. I’m not making any unrealistic goals for adoption beyond having enough community support to be viable.

We’re trying very hard to focus on what we consider to be our core competencies this time instead of trying to build everything ourselves. We’re going to fully embrace OWIN internally as a replacement for FubuMVC’s behavior model. We’re betting big on the future of OWIN servers, middleware, and community — even though I’ve been known to gripe about OWIN on Twitter from time to time. We’re also going to support integration and usage of some subset of ASP.Net MVC from within Jasper applications, with a catch. Some users have asked us to make Jasper an addon to ASP.Net MVC, but my strong opinion is that what we want to do with Jasper won’t work unless Jasper is in control of the middleware composition and routing.

Mainly though, we just need Jasper to provide enough benefits to justify the time we’re going to spend building it on work time;-)

What’s Changing from FubuMVC

  • We’re going to stop using the Routing module from ASP.Net in favor of a new routing engine for OWIN based on the Trie algorithm
  • Dropping support for System.Web altogether. It’s OWIN or nothing baby.
  • Dropping most of the server side rendering support, probably including our Spark view engine support. More on this below.
  • The OWIN AppFunc is going to be the new behavior. We’re keeping the behavior graph model for specifying which middleware goes where, but this time we’re planning to use Roslyn to compile code at runtime for composing all the middleware for a single route or message type into one OWIN AppFunc. We have high hopes that doing this will lead to easier to understand exception stack traces and a significantly more efficient runtime pipeline than the current FubuMVC model. We’ll also support MidFunc, but it won’t be encouraged.
  • Part of adopting OWIN is that we’ll be going async by default in all routes and message handling chains. Users won’t be forced to code this way, but it’s a great way to wring out a lot more scalability and many other .Net tools are already headed in this direction.
  • Jasper needs to be much easier in cases where users need to drop down directly to HTTP manipulation or opt out of the conventional FubuMVC behavior
  • FubuMVC on Mono was an unrewarding time sink. I’m very hopeful that with Jasper and the new cross platform support for .Net that coding on OS X and hosting on Linux will be perfectly viable this time around.

 

What Jasper is Keeping from FubuMVC

  • One of our taglines from FubuMVC was the “web framework that gets out of your way.” To that end, Jasper should have the least possible intrusion into your application — meaning that we’ll try hard to avoid cluttering up application code with fluent interfaces from the framework, mandatory base classes, marker interfaces, and the copious number of attributes that are all too common in many .Net development tools.
  • Keep FubuMVC’s Russian Doll Model and the “BehaviorGraph” configuration model that we use to compose pipelines of middleware and handlers per route or service bus message
  • Retain the “semantic logging” strategy we use within FubuMVC. I think it’s been very valuable for diagnostics purposes and frequently for testing automation support. The Glimpse guys are considering something similar for ASP.Net MVC that we might switch to later if that looks usable.
  • Continue supporting the built in diagnostics in FubuMVC/FT. These are getting some considerable improvement in the 3.0 release for performance tracking and offline viewing
  • Our current mechanisms for deriving url routes from endpoint actions and the reverse url resolution in FubuMVC today. As far as I know, FubuMVC is the only web framework on any platform that provides reverse url resolution for free without additional work on the user’s part.
  • The “one model in, one model out” architecture for expressing url endpoints — but for Jasper we’re going to support more patterns for cases where the one in, one out signature was annoying
  • The built in conventions that FubuMVC and FubuTransportation support today
  • Jasper will continue to support “meta-conventions” that allow users to create and use their own policies
  • The areas or slices modularity support that we have today with Bottles and FubuMVC, but this has already been simplified to only apply to server side code. Jasper is almost entirely getting out of the client side asset business.
  • Content negotiation, the authorization model, and the lightweight asset support from FubuMVC 2.0 will be optimized somewhat but mostly kept as is.
  • Definitely keep the strong-typed “Settings” pattern for application and framework configuration.

 

 

First, gut the existing code

Like I said in the beginning, my wife loves HG TV fix it up shows about remodeling houses. A lot of those episodes invariably include contractors tearing into an old house and finding all kinds of unexpected horrors lurking behind the dry wall and ripping out outdated 70’s shag carpet. Likewise, I’ve spent the last month or so at work ripping a lot of 70’s shag carpet type features and code out of FubuMVC. I’m ostensibly making an intermediate FubuMVC 3.0 release that we’ll use internally at work until next year when Jasper is ready and the dust seems settled enough on DNX, but I’ve also taken advantage of the time to clean as much junk out of the codebase as possible before transforming FubuMVC to Jasper.

The main goal of this release was to consolidate all the FubuMVC related code that is going to survive into Jasper into one single Github repository. As secondary goals, I’ve also streamlined the application bootstrapping, removed about a net of 10k lines of code, and I’ll be working this coming week on performance instrumentation inside FubuMVC’s diagnostics and some of the test automation support.

 

Consolidate and Simplify the Code Topology

FubuMVC’s ecosystem of add on projects and spun off tooling became massive and spread across about ~75 GitHub repositories at its peak. FubuMVC had — in my obviously biased opinion — a very effective architecture for modularity that led us to get a little too slaphappy with splitting features into separate assemblies and nugets. Doing development across related repositories though turned out to be a huge source of friction for us and no matter how much DNX may or may not improve that experience, we’re never going to try to do that again. In that vein, I’ve spent much of the past several weeks consolidating the codebase into many fewer libraries. Instead of just dropping assemblies into the application to auto-magically add new behavior or features to a FubuMVC application, those features ride with the core library but users need to explicitly opt into those features. I liked the “just drop the assembly file in” plugin abilities, but others prefer the explicit code. I’m not sure I have a strong opinion right now, but fewer repositories, libraries, and nugets definitely makes my life easier as the maintainer.

For previous FubuMVC users, I combined:

  • FubuTransportation, FubuMVC.Authentication, FubuMVC.AntiForgery, FubuMVC.StructureMap, and FubuMVC.Localization into FubuMVC.Core
  • FubuMVC.Diagnostics was combined into FubuMVC.Core as part of the 2.1 release earlier this year
  • FubuPersistence and FubuTransporation.RavenDb were all combined into FubuMVC.RavenDb
  • All the Serenity add ons were collapsed into Serenity itself
  • Some of Bottles was folded into FubuMVC and the rest thrown away. More on that later.

 

StructureMap Only For Now

FubuMVC, like very many .Net frameworks, had some abstractions to allow the tool to be used with multiple IoC containers. I was never happy with our IoC registration abstraction model, but our biggest problem was that FubuMVC was built primarily against StructureMap and its abilities and assumptions about open generic types, enumerable types, and lifecycle management and that made it very difficult for us to support other IoC containers. In addition to StructureMap, we fully supported Autofac and got *this* close with Windsor — but I’m not aware of anyone using other containers besides StructureMap with FubuMVC in a production application.

As of a couple weeks ago, I demolished the IoC abstractions in favor of just having FubuMVC use StructureMap directly. That change allowed me to throw away a lot of code and unit tests, eliminate a couple assemblies, remove some nontrivial duplication in reflection handling code between StructureMap and FubuMVC, and simplify the application bootstrapping.

In the longer run, if we decide to once again support other IoC containers, my thought is that Jasper itself will use StructureMap’s registration model and we’ll just have that model mapped into whatever the other IoC container is at bootstrapping time. I know we could support Autofac and probably Windsor. Ninject and SimpleInjector are definitely out. I don’t have the slightest clue about a Unity adapter or the other 20 or so .Net IoC tools out there.

The new IoC integration in ASP.Net MVC6 is surprisingly similar to FubuMVC’s original IoC integration in many aspects and I think is very likely to run into the exact same problems that we did in FubuMVC (some of the IoC containers out there aren’t usable with MVC6 as it is and their project maintainers aren’t happy with the ASP.Net team). That’s a subject for another blog post on another day though.

 

Backing off of Server Side Rendering

I know not everyone is onboard the single page application train, but it’s been very obvious to me over the past 2-3 years that server side html generation is becoming much less important as more teams use tools like Angular.js or React.js to do client side development while using FubuMVC to mostly expose Json over Http endpoints. We had some very powerful features in FubuMVC for server side html generation, but the world has moved on and among others, these features have been removed:

  • Html conventions – FubuMVC supported user-defined conventions for generating forms, labels, editors, and displays based on the signature of view models built around the HtmlTags library. While I think that our html convention support was technically successful, it’s no longer commonly used by our teams and I’ve removed it completely from FubuMVC.Core. Jimmy Bogard has pulled most of the convention support into HtmlTags 3.0 such that you can use the html convention generation in projects that don’t use FubuMVC. I’ve been surprised by how well the new TagHelpers feature in MVC6 has been received by the .Net community. I feel like our old HtmlTags-based conventions were much more capable than TagHelpers, but I still think that the time for that kind of functionality has largely passed.
  • Content extensions — a model we had early on in FubuMVC to insert customer specific markup into existing views without having to change those view files. It was successful, but it’s no longer used and out it goes.

 

 

 

 

StructureMap in 2015/16

I’ve been absurdly bogged down most of this calendar year rewriting the Storyteller tool we use at work for acceptance and regression testing. Now that I’m past that crunch, I’m finally able to focus on other things like StructureMap, this blog, and the still unfinished StructureMap documentation. That being said, here’s what’s going on with StructureMap now and in the near future.

Documentation

StructureMap and I have earned a bad reputation over the years for being under-documented and having out of date documentation. As of about an hour ago, I finished converting the existing StructureMap 3.0 documentation site to the far better “living documentation” tooling I built as part of my Storyteller work earlier this year. There are still far too many holes in that site, but I’m hoping to knock out a couple topics a week until they’re complete. With the new tooling, I think it would be much easier for other folks to contribute to the documentation effort.

Strong Naming and StructureMap. Sigh.

My position on strong naming with StructureMap 3 and beyond is that I do not want the main packages to be strong named. I have said that I would consider supporting a signed version of StructureMap 3 in a parallel nuget if someone did the pull request for that support. Lo and behold, someone finally did that, so look for the structuremap-signed package if you absolutely have to have a strong named version of StructureMap. That being said, I still fervently recommend that you do not use the signed version unless you absolutely have to in your situation.

StructureMap 3.2 in 2015

I’m just about to start some work on some new features and internal improvements for StructureMap that will make up a 3.2 release. Using Semantic Versioning rules, there should be no breaking API changes from 3.* to 3.2. Right now, I think the major changes are:

  1. Optimize the type scanning and convention registration. This was the one big subsystem that I left essentially untouched in the push to 3.0 and sure enough, it’s not holding up well for some users. I have some ideas about how to improve the performance and usability of the type scanning that I have already started in a private branch.
  2. Optimize and tighten up the “TryGetInstance()” runtime behavior under heavy multi-threaded loads. I don’t use this feature myself and I try to discourage its usage overall, but all the ASP.Net frameworks (MVC & Web API) use it in their integration and that’s been problematic to a couple folks.
  3. Some new syntactical sugar to verify specific container registrations
  4. New types of Container wide conventions or policies that act against the entire container
  5. Take advantage of default parameter values
  6. Features to make StructureMap easier to configure from mix and match sources for highly modular architectures

StructureMap 4 in 2016(?)

I *think* that StructureMap 4.0 is going to be all about the new stuff:

  • Use Roslyn runtime code generation in place of the current strategy of building then compiling Expression’s at runtime. I don’t know that this is going to result in faster code, but I am hopeful that it makes the guts of StructureMap’s internals more approachable. Really though, that’s because I want to use the Roslyn functionality on a proposed replacement for FubuMVC next year.
  • Maybe use Roslyn’s support for compiler symbols in place of the existing type scanning?
  • Support CoreCLR

Re-tooled the Codebase

I changed up the build automation tooling for StructureMap a couple weeks ago in an attempt to make the code easier to work with for more mainstream .Net developers.

I started from:

  • Rake (Ruby) as a build scripting tool
  • The old Ripple tool from the FubuMVC family of projects for Nuget support
  • The build script had to be executed at least once before working with the code to generate some required files and fetch all the necessary nuget requirements
  • Good ol’ fashioned NUnit for unit testing

The usage of Ruby for build automation has been off-putting to many .Net developers and the mass majority of .Net developers seem to prefer being able to open Visual Studio.Net and just go to work. Based on the twin desires to make the StructureMap build faster and easier for most .Net developers, the new build tooling changed to:

  • Use Paket as the nuget management tool with its auto-restore capability
  • After some conversations with the NancyFx team over the years, I stole their idea of using Rake but making it completely optional for potential contributors
  • Replaced NUnit with Fixie, a much better and faster testing library from some of the Headspring folks here in Austin

Succeeding with Automated Integration Tests

tl;dr This post is an attempt to codify my thoughts about how to succeed with end to end integration testing. A toned down version of this post is part of the Storyteller 3 documentation. 

About six months ago the development teams at my shop came together in kind of a town hall to talk about the current state of our automated integration testing approach. We have a pretty deep investment in test automation and I think we can claim some significant success, but we also have had some problems with test instability, brittleness, performance, and the time it takes to author new tests or debug existing tests that have failed.

Some of the problems have since been ameliorated by tightening up on our practices — but that still left quite a bit of technical friction and that’s where this post comes in. Since that meeting, I’ve been essentially rewriting our old Storyteller testing tool in an attempt to address many of the technical issues in our automated testing. As part of the rollout of the new Storyteller 3 to our ecosystem, I thought it was worth a post on how I think teams can be more successful at automated end to end testing.

Test Stability

I’ve worked in far too many environments and codebases where the automated tests were “flakey” or unreliable:

  • Teams that do all of their development against a single shared, development database such that the data setup is hard to control
  • Web applications with a lot of asynchronous behavior are notoriously hard to test and the tests can be flakey with timing issues — even with all the “wait for this condition on the page to be true” discipline in the world.
  • Distributed architectures can be difficult to test because you may need to control, coordinate, or observe multiple processes at one time.
  • Deployment issues or technologies that tend to hang on to file locks, tie up ports, or generally lock up resources that your automated tests need to use

To be effective, automated tests have to be reliable and repeatable. Otherwise, you’re either going to spend all your time trying to discern if a test failure is “real” or not, or you’re most likely going to completely ignore your automated tests altogether as you lose faith in them.

I think you have several strategies to try to make your automated, end to end tests more reliable:

  1. Favor white box testing over black box testing (more on this below)
  2. Closely related to #1, replace hard to control infrastructure dependencies with stub services, even in functional testing. I know some folks absolutely hate this idea, but my shop is having a lot of success in using an IoC tool to swap out dependencies on external databases or web services in functional testing that are completely out of our control.
  3. Isolate infrastructure to the test harness. For example, if your system accesses a relational database, use an isolated schema for the testing that is only used by the test harness. Shared databases can be one of the worst impediments to successful test automation. It’s both important to be able to set up known state in your tests and to not get “false” failures because some other process happened to alter the state of your system while the test is running. Did I mention that I think shared databases are a bad idea yet?*
  4. Completely control system state setup in your tests or whatever build automation you have to deploy the system in testing.
  5. Collapse a distributed application down to a single process for automated functional testing rather than try to run the test harness in a different process than the application. In our functional tests, we will run the test harness, an embedded web server, and even an embedded database in the same process. For distributed applications, we have been using additional .Net AppDomain’s to load related services and using some infrastructure in our OSS projects to coordinate the setup, teardown, and even activity in these services during testing time.
  6. As a last resort for a test that is vulnerable to timing issues and race conditions, allow the test runner to retry the test

Failing all of those things, I definitely think that if a test that is so unstable and unreliable that it renders your automated build useless that you just delete that test. I think a reliable test suite with less coverage is more useful to a team than a more expansive test suite that is not reliable.

You Gotta Have Continuous Integration

This section isn’t the kind of pound on the table, Uncle Bob-style of “you must do this or you’re incompetent” kind of rant that causes the Rob Conery’s of the world have conniptions. Large scale automation testing simply does not work if the automated tests are not running regularly as the system continues to evolve.

Automated tests that are never or seldom executed can even be a burden on a development team that still try to keep that test code up to date with architectural changes. Even worse, automated tests that are not constantly executed are not trustworthy because you no longer know if test failures are real or just because the application structure changed.

Assuming that your automated tests are legitimately detecting regression problems, you need to determine what recent change introduced the problem — and it’s far easier to do that if you have a smaller list of possible changes and those changes are still fresh in the developer’s mind. If you are only occasionally running those automated tests, diagnosing failing tests can be a lot like finding the proverbial needle in the haystack.

I strongly prefer to have all of the automated tests running as part of a team’s continuous integration (CI) strategy — even the heavier, slower end to end kind of tests. If the test suite gets too slow (we have a suite that’s currently taking 40+ minutes), I like the “fast tests, slow tests” strategy of keeping one main build that executes the quicker tests (usually just unit tests) to give the team reasonable confidence that things are okay. The slower tests would be executed in a cascading build triggered whenever the main build completes successfully. Ideally, you’d like to have all the automated tests running against every push to source control, but even running the slower tests suites in a nightly or weekly scheduled build is better than nothing.

Make the Tests Easy to Run Locally

I think the section title is self-explanatory, but I’ve gotten this very wrong in the past in my own work. Ideally, you would have a task in your build script (I still prefer Rake, but substitute MSBuild, Fake, Make, Gulp, NAnt, whatever you like) that completely sets up the system under test on your machine and runs whatever the test harness. In a less perfect world a developer has to jump through hoops to find hidden dependencies and take several poorly described steps in order to run the automated tests. I think this issue is much less problematic than it was earlier in my career as we’ve adopted much more project build automation and moved to technologies that are easier to automate in deployment. I haven’t gotten to use container technologies like Docker myself yet, but I sure hope that those tools will make doing the environment setup for automating tests easier in the future.

Whitebox vs. Blackbox Testing

I strongly believe that teams should generally invest much more time and effort into whitebox tests than blackbox tests. Throughout my career, I have found that whitebox tests are frequently more effective in finding problems in your system – especially for functional testing – because they tend to be much more focused in scope and are usually much faster to execute than the corresponding black box test. White box tests can also be much easier to write because there’s simply far less technical stuff (databases, external web services, service buses, you name it) to configure or set up.

I do believe that there is value in having some blackbox tests, but I think that these blackbox tests should be focused on finding problems in technical integrations and infrastructure whereas the whitebox tests should be used to verify the desired functionality.

Especially at the beginning of my career, I frequently worked with software testers and developers who just did not believe that any test was truly useful unless the testing deployment was exactly the same as production. I think that attitude is inefficient. My philosophy is that you write automated tests to find and remove problems from your system, but not to prove that the system is perfect. Adopting that philosophy, favoring white box over black box testing makes much more sense.

Choose the Quickest, Useful Feedback Mechanism

Automating tests against a user interface has to be one of the most difficult and complex undertakings in all of software development. While teams have been successful with test automation using tools like WebDriver, I very strongly recommend that you do not test business logic and rules through your UI if you don’t have to. For that matter, try hard to avoid testing business logic without using the database. What does this mean? For example:

  • Test complex logic by calling into a service layer instead of the UI. That’s a big issue for one of the teams I work with who really needs to replace a subsystem behind http json services without necessarily changing the user interface that consumes those services. Today the only integration testing involving that subsystem is done completely end to end against the full stack. We have plenty of unit test coverage on the internals of that subsystem, but I’m pretty certain that those unit tests are too coupled to the implementation to be useful as regression or characterization tests when that team tries to improve or replace that subsystem. I’m strongly recommending that that team write a new suite of tests against the gateway facade service to that subsystem for faster feedback than the end to end tests could ever possibly be.
  • Use Subcutaneous Tests even to test some UI behavior if your application architecture supports that
  • Make HTTP calls directly against the endpoints in a web application instead of trying to automate the browser if that can be useful to test out the backend.
  • Consider testing user interface behavior with tightly controlled stub services instead of the real backend

The general rule we encourage in test automation is to use the “quickest feedback cycle that tells you something useful about your code” — and user interface testing can easily be much slower and more brittle than other types of automated testing. Remember too that we’re trying to find problems in our system with our tests instead of trying to prove that the system is perfect.

Setting up State in Automated Tests

I wrote a lot about this topic a couple years ago in My Opinions on Data Setup for Functional Tests, and I don’t have anything new to say since then;) To sum it up:

  • Use self-contained tests that set up all the state that a test needs.
  • Be very cautious using shared test data
  • Use the application services to set up state rather than some kind of “shadow data access” layer
  • Don’t couple test data setup to implementation details. I.e., I’d really rather not see gobs of SQL statements in my automated test code
  • Try to make the test data setup declarative and as terse as possible

Test Automation has to be a factor in Architecture

I once had an interview for a company that makes development tools. I knew going in that their product had some serious deficiencies in their automated testing strategy. When I told my interviewer that I was confident that I could help that company make their automated testing support much better, I was told that testing was just a “process issue.” Last I knew, it is still weak for its support for automating tests against systems that use that tool.

Automated testing is not merely a “process issue,” but should be a first class citizen in selecting technologies and shaping your system architecture. I feel like my shop is far above average for our test automation and that is in no small part because we have purposely architected our applications in such a way to make functional, automated testing easier. The work I described in sections above to collapse a distributed system into one process for easier testing, using a compositional architecture effectively composed by an IoC tool, and isolatating business rules from the database in our systems has been vital to what success we have had with automated testing. In other places we have purposely added logging infrastructure or hooks in our application code for no other reason than to make it easier for test automation infrastructure to observe or control the application.

Other Stuff for later…

I don’t think that in 10 years of blogging I’ve ever finished a blog series, but I might get around to blogging about how we coordinate multiple services in distributed messaging architectures during automated tests or how we’re integrating much more diagnostics in our automated functional tests to spot and prevent performance problems from creeping into application.

* There are some strategies to use in testing if you absolutely have no other choice in using a shared database, but I’m not a fan. The one approach that I want to pursue in the future is utilizing multi-tenancy data access designs to create a fake tenant on each test run to keep the data isolated for the test even if the damn database is shared. I’d still rather smack the DBA types around until they get their project automation act together so we could all get isolated databases.

Retooling Build and Test Automation Tools

tl;dr: I had a convention based build automation approach based on Rake we used for the FubuMVC projects that I was proud of, but the world moved on and I’ve been replacing that with newer and hopefully better tools like Fixie, Paket, and gulp.js. What I do today with FubuRake I’ve generally used Rake as my build scripting tool for the last 7-8 years, even on .Net projects and I’ve been mostly happy with it. In a grandiose attempt to simplify the build scripts across the FubuMVC ecosystem, make all the homegrown build tools we’d created easier to adopt, and support a Ruby on Rails style “fubu new” approach to bootstrapping new FubuMVC projects, I created the FubuRake library as an addon to Rake. In its most simple form, a complete working build script using FubuRake can look like this below:

require 'fuburake'

FubuRake::Solution.new do |sln|
	sln.assembly_info = {
		:product_name => "FubuCore",
		:copyright => 'Copyright 2008-2015...'
	}
end

That simple script above generated rake tasks for:

  1. Generating a “CommonAssemblyInfo.cs” file to embed semantic version numbers, CI build numbers, and git commit numbers into the compiled assemblies
  2. Compiling code
  3. Fetching, building, and publishing nuget with Ripple
  4. Running unit tests with NUnit
  5. Tasks to interact with our FubuDocs tool for documentation generation (and yes, I spent way more time building our docs tool than writing docs)
  6. Tasks to create embedded resource files in csproj files as part of our Bottles strategy for modularizing large web applications

If you typed rake -T to see the task list for this script you’d see this output:

rake ci                # Target used for the CI server
rake clean             # Prepares the working directory for a new build
rake compile           # Compiles the solution src/FubuCore.sln
rake compile:debug     # Compiles the solution in Debug mode
rake compile:release   # Compiles the solution in Release mode
rake default           # **Default**, compiles and runs tests
rake docs:bottle       # 'Bottles' up a single project in the solution with...
rake docs:run          # Runs a documentation project hosted in FubuWorld
rake docs:run_chrome   # Runs the documentation projects in this solution i...
rake docs:run_firefox  # Runs the documentation projects in this solution i...
rake docs:snippets     # Gathers up code snippets from the solution into th...
rake ripple:history    # creates a history file for nuget dependencies
rake ripple:package    # packages the nuget files from the nuspec files in ...
rake ripple:publish    # publishes the built nupkg files
rake ripple:restore    # Restores nuget package files and updates all float...
rake ripple:update     # Updates nuget package files to the latest
rake sln               # Open solution src/FubuCore.sln
rake unit_test         # Runs unit tests for FubuCore.Testing
rake version           # Update the version information for the build

As you’ve probably inferred, FubuRake depended very heavily on naming and folder layout conventions for knowing what to do and how to build out its tasks like:

  • Compile the one and only *.sln file under the /src directory using MSBuild with some defaults (.Net version = 4.0, compile target = Debug)
  • Run all the NUnit tests in folders under /src that end in *.Tests or *.Testing. In the case up above, that meant “FubuCore.Testing”
  • Build and publish all the *.nuspec files found in /packaging/nuget

You could, of course, explicitly override any of the conventional behavior. Any tool using conventions almost has to have an easy facility for overriding or breaking out of the conventions for one off cases. It was fine and great as long as you mostly stayed inside our idioms for project layout and you were okay with using NUnit and Ripple. All in all, I’d say that FubuRake was a mild technical success but time has passed it by. The location of MSBuild changed in .Net 4.5, breaking our msbuild support. Ripple was always problematic and it was frequently hard to keep up with Nuget features and changes. FubuDocs was a well intentioned thing, but the support for it that got embedded into FubuRake has file locking issues that sometimes forced me to shutdown VS.Net in order to run the build. If FubuRake is Roland in the Stephen King’s Dark Tower novels, I’d say that the world has moved on. Time for the world to move on I’ve been quiet on the blogging and even the Twitter front lately because I’ve been working very hard on a near rewrite of our Storyteller tool (more on this soon) that we use at work for executable specifications and integration testing. The new work includes a lot of performance driven improvements in the existing .Net code and a brand new user interface written as a Javascript single page application. At the end of last week my situation looked like this:

  • I needed to start publishing pre-release nugets and we only do that from successful CI builds
  • Our TeamCity CI build was broken because of some kind of problem with our Ripple tool that we use for Nuget management.
  • I had started the new client in a completely separate Github repository and had been using a git submodule to include the Javascript code within the Storyteller .Net code repository for full integration
  • I had no effective automation to attach the submodule if it was missing or do the initial npm install or any of the other hidden things that a developer would have to do before being able to work with the code. In other words, my “time to login screen” metric was terrible right as I’d love to start getting some other developers to start contributing to the project with feedback or patches.
  • I’ve wanted to upgrade or replace several pieces of the build and test automation tools I use in my OSS projects for quite some time anyway — especially where there were opportunities to replace homegrown tools I no longer want to support in favor of actively maintained OSS projects.
  • We have a heavy investment in Rake and Ruby for build automation both at work and on OSS projects. Now that we have so much dependency on Node.js tools for client asset work, we’ve gotten into this situation where project build scripts may include installing gems, nugets, and npm packages and it’s becoming a problem of technology overload and build times.

As a result, I’ve spent the last couple days swapping out tools and generally trying to make the new build automation a lot easier to use for other people. I’ve merged the client javascript code into the old Storyteller repository to avoid the whole git submodule mess. I created a new build script that does everything necessary to get both the Javascript and C# code ready for development work. And lastly, I took the very unusual step (for me) of trying to document how to use the code in a readme. All told I:

  • Replaced our old NUnit-based SpecificationExtensions I ripped off from Scott Bellware ages ago with Shouldly. I chose Shouldly because I liked how terse it is, it was an easier switch than Should or Fluent Assertions would have been, and I love their error messages. It went smoothly
  • Replaced Ripple with Paket for managing Nuget dependencies. Ripple is effectively dead, I didn’t want to invest any more time in it, and Paket has a strong, active community right now. Using Nuget out of the box is completely out of the question for me, so Paket was really the only alternative. So far, so good. I hit a few quirks with Paket using multiple feeds, but quickly learned to just be very particular about version numbers. The proof for me will be when we start using Paket across multiple upstream and downstream repositories like we did with our “floating” nuget dependencies in Ripple. It’s my hope though that tools like Paket or Ripple are no longer necessary in ASP.Net vNext but we’ll see.
  • Built a small gulp.js script to compile the .Net code and run the C# unit tests. I leaned heavily at first on Mike O’Brien’s blogpost about building .Net projects with gulp. It didn’t go as smoothly as I’d like and I partially retreated to doing some things in npm scripts instead. I think this is definitely something I’m going to watch and reconsider moving forward. EDIT 3/25: I’ve already given up on gulp.js for the .Net build and I’m in the process of just using simple Javascript files called from NPM as a replacement. Oh well, no harm done.
  • Completely replaced NUnit with Fixie, but used Fixie’s ability to emulate NUnit’s attributes and behavior as a temporary measure. The FubuMVC team has wanted to do this for quite a while because of Fixie’s crazy flexible feature set and performance — and this was the perfect opportunity to finally pull that off. I’m very happy with how this has turned out so far and I’m even seeing the promised performance improvement with the test suite consistently taking 75% of the runtime that NUnit was taking on my machine. Going forward, I’m going to look to slowly remove the NUnit attributes in favor of the cleaner Fixie idioms. I’m happy with Fixie right off the bat.
  • Finally, to make sure it all “just works,” I created an overarching “npm run build” script with a matching “build.cmd” shortcut to do everything you need to do to build both the Javascript and C# code and pull down all of the various dependencies. So now, if you do a fresh clone of the Storyteller code and you already have both .Net 4.5 and Node.js >+ v10 installed, you should be able to execution “npm run build” and go straight to work. And I’ll keep patching things until that’s certified to be true by other developers;)

In case you’re wondering, the Javascript code is built with Webpack through npm scripts that also delegate to mocha and karma for testing. Ironically, I’m using gulp.js to build the .Net code but not the Javascript code, but that’s a subject for someone else to blog;)

More on Replacing Rake with Gulp

I personally like Ruby better than Javascript, especially for the kind of scripting you do for build and test automation. From time to time I still see folks wishing that the browsers would adopt Ruby as the embedded scripting language instead of Javascript, but that metaphorical ship has long since sailed and I think more developers are going to be familiar with JS than Ruby.

I gave some thought to just trying to use pure console tools for the build automation, and some of my coworkers want to investigate using make, but there’s just a little bit of programmatic manipulation here and there for picking up build versions and arguments from the CI server that I wanted to retain some kind of scripting language. I put this very question of replacing Rake to the StructureMap community and got myriad suggestions: F# based tools, C# tools using scriptcs, and Powershell based tools all made an appearance. My stance at this point is that the build script is best done with a low ceremony scripting language and preferably one that’s commonly understood so as to not be a barrier to entry. As much as I liked Rake personally, Ruby was a problem for us with many .Net developers. By choosing a Javascript based tool, we’re investing in what’s arguably the most widely used programming language going. And while this also forces developers to have a working installation of Node.js on their box, I think that’s going to be pretty common anyway.

Long Lived Codebases: Architecture Gardening, Rewrites, and Automated Tests

This post continues my series about my experiences maintaining and growing the StructureMap codebase over the last decade and change based on my talk this year at CodeMash. This series started with The Challenges, and this post is a direct continuation of the discussion on my last post on A Crime Against Computer Science. You’ll likely want to read the previous post first.

A Timeline

2009 – I introduced the nested container feature into StructureMap 2.6 as a purely tactical feature to fill an immediate need within a project at work. The implementation of nested containers, as I described in my my previous post, was highly problematic.

2010 – I attempted a complete, ground up rewrite of StructureMap largely to address the problems with the nested container. I burned out badly on OSS work for a while before I got very far into it and never picked the work up again.

2014 – StructureMap 3.0 was released with a considerably different architecture that largely fixed the known issues with the nested container feature (I measured a 100X improvement in nested container performance in a bigger application). The 3.0 work was done by applying a series of intermediate transformative changes to the existing 2.6 codebase until the architecture essentially resembled the vision of the earlier attempt to rewrite StructureMap.

Sacrificial Architecture and Continuous Design

I’ve beaten myself up quite a bit over the years for how bad the original implementation of the nested container was, but looking at it from another perspective, I was able to deliver some real value to a project in a hurry. The original implementation and the problems it incurred definitely informed the design decisions that I made in the 3.0 release. I can be a lot easier on myself if I can just treat the initial design as Martin Fowler’s Sacrificial Architecture concept.

My one true regret is just that it took so many years to permanently address the issue. It’s fine and dandy to write some throwaway code you know isn’t that great to satisfy an immediate need if you really do come back and actually throw it away before it does a lot of harm. And as an industry, we’re really good about replacing throwaway code, right? Right?

I’ve been a big believer in the idea of continuous design my entire career — even before the Agile programming movement came around to codify that idea. Simply put, I think that most of the examples of great technical work I’ve been a part of was the direct result of iterating and evolving an idea rather than pure creation. Some of that mental iteration can happen on paper or whiteboards as you refine your ideas. Other times it seems to take some initial stumbles in code before you really learn how to better solve your coding problem.

I’m not the greatest OSS project leader of all time by any means, but the one piece of advice that I can give out with a straight face is to simply avoid being over-extended. The gap in major StructureMap releases, and the continual lack of complete documentation, is a direct result of me having way too many OSS irons in the fire.

Play the Long Game

One of the biggest lessons I’ve learned in working with long lived codebases is that you can never act as if the architecture is set in stone.

In the case of a long lived codebase that changes in functionality and technology way beyond its initial vision, you need to constantly challenge the basic architecture, abstractions, and technical assumptions rather than just try to jam new features into the existing codebase. In the case of StructureMap, there have been several occasions where doing structural changes first has made new functionality easier to implement. In the case of the nested container feature, I tried to get away with building the new feature in even though the current architecture didn’t really lend itself to the new feature and I paid for that.

I think this applies directly to my day job. We don’t really start many new projects and instead continually change years old codebases. In many cases we know we’d like to make some big changes in the application architecture or switch out elements of the technical infrastructure (we’re getting rid of Angular 1.* if it’s the last thing I do), but there’s rarely time to just stop and do that. I think that you have to play the “long game” in those cases. You still need to continuously make and refine architectural plans for what you wish the system could be, then continuously move closer to your ideal as you get the chance in the course of normal project work.

In most of my OSS projects I’ve been able to think ahead to many future features and have a general idea of how I would want to built them or what structural changes would be necessary. In the case of the original nested container implementation, I was caught flat-footed. I don’t know how you completely avoid this, but I highly recommend not trying to make big changes in a hurry without enough time spent thinking through the implications.

Tactical vs. Strategic Thinking

I think the balance between thinking tactically and strategically in regards to software projects is a valuable topic that I don’t see discussed enough. Tilt too far to the tactical side and you get dramatically shitastic code going into production so that you can say that you kinda, sorta made some sort of arbitrary deadline — even though it’s going to cause you heartburn in subsequent releases. Tilt too far to the strategic side of things and you spend all your day going architect astronaut, fiddling with the perfect project automation, and generally getting nothing done.

In my case, I was only looking at the short term gains and not thinking about how this new nested container feature should fit into StructureMap’s architecture. The improved nested container implementation in 3.0 was only possible because I stepped back and reconsidered how the architecture could be changed to better support the existing feature. Since the nested container problems were so severe, that set of architectural improvements was the main driver for me to finally sit down and make the 3.0 release happen.

Whither the Rewrite?

To my credit, I did recognize the problems with the original nested container design early on. I also realized that the StructureMap internals needed to be quite different in order to improve the nested container feature that was suddenly so important in our application architecture. I started to envision a different configuration model that would enable StructureMap to create nested containers much more efficiently and eliminate the problems I had had with lifecycle bugs. I also wanted to streamline some unnecessary duplication between nearly parallel configuration models that had grown over the years in the StructureMap code.

In 2010 I started an entirely new Github repository for StructureMap 3. When I looked at the differences between my architectural vision for StructureMap and its current state in the 2.6 era, I thought that it was going to be far too difficult to make the changes in place so I was opting for a complete, ground up rewrite (don’t bother looking for the repository, I think I deleted it because people were getting confused about where the real StructureMap code was). In this case, the rewrite flopped — mostly because I just wasn’t able to devote enough time in a burst to rebuild all the functionality.

The killer risk for doing any kind of rewrite is that you just can’t derive much value from it until you’re finished. My experience, both on side projects and in enterprise IT, is that rewrite efforts frequently get interrupted or even completely shelved before they advance far enough to be usable.

At this point, I think that unless a project is small or at least has a limited and very well understood set of functionality, you should probably avoid attempts at rewriting. The StructureMap rewrite attempt failed because I was interrupted before it got any momentum. In the end, I think it was a much wiser choice to evolve and transform the existing codebase instead — all while staying within the existing acceptance level tests in the code (more on this below). It can be significantly harder to come up with the intermediate steps such that you can evolve to your end goal than just doing the rewrite, but I think you have a much better chance of delivering value without accidentally dropping functionality.

I’m the primary developer on a couple OSS projects that have gone through, or about to go through, a rewrite or restructure effort:

  • StructureMap 3.0 — As stated before, I gave up on the rewrite and transformed the existing code to a more effective internal model. I’m mostly happy with how this turned out.
  • Storyteller 3.0 — Storyteller is a tool for executable specifications I originally built in 2009. We use it heavily at work with some mixed success, but the WPF based user interface is clumsy and we’re having some severe throughput issues with it on a big project. Everybody wanted a complete rewrite of the WPF client into a web application (React.js with a Flux-like architecture), but I was still left with the rump .Net engine. I identified several structural changes I wanted to make to try to improve usability and performance. I tried to identify intermediate steps to take in the existing code, but in that case I felt like I needed to make too many changes that were interrelated and I was just getting tired of constantly pounding out “git checkout –force” and opted for a rewrite. In this case, I felt like the scope was limited, I could reuse the acceptance tests from the original code to get back to the same functionality, and that a rewrite was the easier approach. So far, so good, but I think this was an exception case.
  • FubuMVC 3 / “Jasper” — I know I said that I was giving up on FubuMVC and I meant it at the time, but for a variety of reasons we want to just move our existing FubuMVC .Net applications to the vNext platform this year and decided that it would be easier to modernize FubuMVC rather than have to rewrite so much of our code to support the new ASP.Net MVC/Web API combo. At the same time we want to transform FubuMVC’s old “Behavior” middleware model to just use the OWIN middleware signature throughout. After a lot of deliberation about a new, ground up framework, we’ve tentatively decided to just transform the existing FubuMVC 2 code and stay within the existing automated test coverage as we make the architectural changes.

So in three cases, I’ve opted against the rewrite twice. I think my advice at this point is to avoid big rewrites. Rewriting a subsystem at a time, sure, but not a complete rewrite unless the scope is limited.

Automated Testing is Good, Except When it Isn’t

Automated testing can most certainly help you create better designs and architectures by providing the all important quality of reversibility — and if you believe in or practice the idea of continuous design, you absolutely have to have reversibility. In other all too common cases, automated tests that are brittle can prevent you from making changes to the codebase because you’re too afraid of breaking the tests.

What I’ve found through the years of StructureMap development is that high level acceptance tests that are largely black box tests that express the desired functionality from a client perspective are highly valuable as regression tests. Finer grained tests that are tightly coupled to the implementation details can be problematic, especially when you want to start restructuring the code. When I was doing the bigger changes for StructureMap 3.0 I relied very heavily on the coarser grained tests. I would even write all new characterization tests to record the existing functionality before making structural changes when I felt like the existing test coverage was too light.

I’m not prepared to forgo the finer grained unit tests that are largely the result of using TDD to drive the low level design. Those tests are still valuable as a way to think though low level design details and keep your debugger turned off. I do think you can take steps to limit the coupling to implementation details.

If fine grained tests are causing me difficulty while making changes, I generally take one of a couple approaches:

  1. Delete them. Obviously use your best judgement on this one;)
  2. Comment the body of the test out and stick some kind of placeholder in instead that makes the test fail with a message to rewrite. I’ve fallen into the habit of using Assert.Fail(“NWO”) just to mean “rewrite to the new world order.”
  3. Write all new code off to the side to replace an entire subsystem. After switching over to using the new subsystem and getting all the acceptance level tests passing, go back and delete the old code plus the now defunct unit tests

Was I lulled to sleep by passing tests?

I had a great question from the audience at CodeMash about the poor initial implementation that caught me a little flat-footed. I was asked if “I had been lulled by passing tests into believing that everything was fine?” Quite possibly, yes. It’s certain that I wasn’t thinking about performance or how to first change the StructureMap internals to better support the new feature. I’m going to blame schedule pressure and overly tactical thinking at that time more than my reliance on TDD however.

There’s an occasional meme floating in the software world that TDD leads to developers just not thinking, and I know at the time of my talk I had just read this post from Ali Kheyrollahi and I was probably defensive about TDD. I’m very clearly in the pro-TDD camp of course and still feel very strongly that TDD can be a key contributor to better software designs. I’m also the veteran of way too many online debates about the effectiveness of TDD (see this one from 2006 for crying out loud). Unlike Ali, my experience is that there is a very high correlation suggesting that developers who use TDD are more effective than developers that don’t — but I think that might be more of a coincidental effect than a root cause.

That all being said, TDD in my experience is more effective for doing design at the small scale down at the class, method, or function level than it is as a technique for more larger scale issues like performance or extensibility. While I certainly wouldn’t rule out the usage of TDD, it’s just a single tool and should never be the sole design tool in your toolbox. TDD advocate or not, I do a lot of software design with pencil and paper using an amalgmation  of “UML as sketch” and CRC cards (I’m still a big fan of Responsibility Driven Design).

At the end of the day, there isn’t really any substitute for just flat out paying attention to what you’re doing. I probably should have done some performance testing on the nested container feature right off the bat or at least thought through the performance implications of all the model copying I was doing in the original implementation (see the previous post for background on that one).