Thoughts from CodeMash 2015

I had a fantastic time at CodeMash last week and I’d like to think all of the organizers for the hard work they do making one of the best development community events happen each year. I was happy with how my talk went and I will get around to what looks like a 3 part blog series adaptation of it later this week after I catch up on other things. I had a much lighter speaking load than I’ve had in the past, leading to many more opportunities and energy to just talk to the other developers there. As best I can recall, here’s a smattering of the basic themes from those discussions:

  • Microservices — I lived through DCOM and all the sheer lunacy of the first wave of SOA euphoria, making me very leery of the whole micro-service concept. I did spoke to a couple people that I respect who were much more enthusiastic about micro services than I am.
  • Roslyn — I feel bad for saying this, but I’m tempering my hopes for Roslyn right now — but that’s largely a matter of me having had outsize expectations and hopes for Roslyn. I’m disappointed by the exclusion of runtime metaprogramming for now and I think the earlier hype about how much faster the Roslyn compiler would be compared to the existing CSC compiler might have been overstated. The improved ability to introspect your code with Roslyn is pretty sweet though.
  • RavenDb — I still love RavenDb conceptually and it’s my favorite database development time experience, but the quality issues and the poor DevOps tooling makes it a borderline liability in production. From my conversations with other RavenDb users at CodeMash, this seems to be a common opinion and experience. I hate to say it, but I’m in favor of phasing out RavenDb at work in the next couple years.
  • Postgresql — Postgres has a bit of buzz right now and I’ve been very happy with my limited usage of it so far. I talked to several people who were interested in using postgres as a pseudo document database. I’m planning to do a lot more side work with postgresql in the coming year to see how easy it would be to use it as more of a document database and add a .Net client to the event store implementation I was building for Node.js.
  • Programming Languages — The trend that I see is a blurring of the line between static and dynamic typing. Static languages are getting better and better type inference and dynamic languages keep getting more optional type declarations. I think this trend can only help developers over time. I do wish I hadn’t overslept the introduction to Rust workshop, but you can’t do everything.
  • OWIN — One of the highlights of CodeMash for me was fellow Texan Ryan Riley‘s history of the OWIN specification set to the Pina Colada song. For as insane as the original version of OWIN was, I think we’ll end up being glad that the OWIN community persevered through all the silliness and drama on their list to deliver something that’s usable today. I do still think Ryan needs to add “mystery meat” into his description of OWIN though.
  • Functional Programming — In the past, over zealous FP advocates have generally annoyed me the same way that I bet I annoyed folks online when Extreme Programming hype back in the day. The last time I was at CodeMash I remember walking out of a talk that was ostensibly about FP because it was nothing but a very long winded straw man argument against OOP done badly. This time around I enjoyed the couple FP talks I took in and appreciated the candor from the FP guys I spoke with. I do wish the FP guys would stop looking at the FP vs. OOP or imperative development comparison as a zero sum game and spend more time talking about the specific areas and problems where FP is valuable and much less time bashing everything else.

 

 

My Talk on Long-Lived Codebases at Codemash

When I go to conferences I see a lot of talks that follow the general theme of I/my team/our product/our process/this tool is wonderful. In contrast to that, I’m giving a talk Friday at Codemash entitled “Lessons Learned from a Long-Lived Codebase” about my experiences with the evolution of the StructureMap codebase in its original production usage in 2004 all the way up to the very different tool that it is today. Many structural weaknesses in your code structure and architecture don’t become apparent or even harmful until you either need to make large functional changes you didn’t foresee or the code gets used in far higher volume systems. I’m going to demonstrate some of the self-inflicted problems I’ve incurred over the years in the StructureMap and what I had to do to ameliorate those issues in the hopes of helping others dodge some of that pain.

Because StructureMap is such an old tool that’s been constantly updated, it’s also a great microcosm of how our general attitudes toward software development, designing API’s, and how we expect our tools to work have changed in the past decade and I’ll talk about some of that.

In specific, I’m covering:

  • Why duplication of even innocuous logic can be such a problem for changing the system later
  • Abstractions should probably be based strictly on roles and variations and created neither too early nor too late
  • Why your API should be expressed in terms of your user’s needs rather than in your own tool’s internals
  • Trying to pick names for concepts in your tool that are not confusing to your users
  • For the usage of any development tool that’s highly configurable there’s a tension between making the user be very explicit about what they want to do versus providing a much more concise or conventional usage that is potentially too “magical” for many users.
  • How the diagnostics and exception messages in StructureMap have evolved in continuous attempts to reduce the cost of supporting StructureMap users;)
  • Using “living documentation” to try to keep your basic documentation reasonably up to date
  • Organizing whatever documentation you do have in regards to what the users are trying to do rather than being a dry restatement of the implementation details
  • The unfortunate tension between backward compatibility and working to improve your tool
  • Automated testing in the context of a long lived codebase. What kinds of tests allow you to improve the internals of your code over time, where I still see value in TDD for fine grained design, and my opinions about how to best write acceptance tests that came out of my StructureMap work over the years
  • Lastly, the conversation about automated testing is a great sequeway to talk about the great “Rewrite vs. Improve In-Place” discussion I faced before the big StructureMap 3.0 release earlier this year.

This is a big update to a talk I did at QCon San Francisco in 2008. Turns out there was plenty more lessons to learn since then;)

I’d be willing to turn this talk into a series of more concrete blog posts working in the before and after code samples if I get at least 5-6 comments here from people that would be interested in that kind of thing.

Some Thoughts on the new .Net

The big announcements on the new ASP.Net runtime were made a couple weeks ago, but I write very slowly these days, so here’s my delayed reaction.

 

Less Friction and a Faster Feedback Cycle Maybe? 

I’ve started to associate .Net “classic” with seemingly constant aggravations like strong naming conflicts, csproj file merge hell, slow compilation, slow nuget restores, and how absurdly heavyweight and bloated that Visual Studio.Net has become over the years. Plenty of it was self-inflicted, but I felt at the end that way too much of our efforts in FubuMVC development was really about working around limitations in the .Net ecosystem or ASP.Net itself (the existing System.Web infrastructure is awful for modularity in my opinion and clearly shows its age and the MSBuild model is just flat out nuts).

In contrast, my brief exposure to Node.js development has been a breathe of fresh air. My preferred workflow is to just start mocha in “watch” mode in a terminal window with Growl notifications on and continuously code in Sublime. The feedback cycle between making a change in code to test results is astonishing fast compared to .Net (admittedly less awesome when using lots of generator functions or using WebPack to bundle up React.js JSX files or running tests in Karma) and the editor is snappy.

ASP.Net vNext is very clearly a reaction to the improved development workflows you can get with Node.js or Golang. In particular, I’m positive or at least hopeful that:

  • The Roslyn compiler is fast enough that auto-test tools in .Net “feel” much snappier
  • The new package resolution functionality is much faster than the current Nuget
  • The new runtime is much more command line friendly. I do particularly like how easy they’ve made it to expose commands from Nuget packages in the project.json declaration. That was a big irritation to me in the existing Nuget that we eventually solved by moving to gems instead for CLI tools.
  • The new project.json model is so much cleaner and simpler than the existing MSBuild schema. I hope that translates to easier tooling for scaffolding and much less merge hell than .Net classic
  • I’m impressed with how much activity is happening around the OmniSharp project, because I’d like to be productive in C# without having to fire up Visual Studio.Net at all — or at least only when I want ReSharper to do a lot of heavy refactoring work.
  • The abomination that was strong naming is gone

 

The Spirit of Openness

I like that the ASP.Net team has posted their code for their next generation work on GitHub. The best part of that for me is that they moved the code over before it was completely baked and you can actually see what they’re up to. Even crazier to me is that the ASP.Net team seems to actually be listening to the feedback (I’ve felt in the past that the ASP.Net team hasn’t been all that receptive to feedback).

Compare that to how the tool that became Nuget was developed. Nuget was largely conceived and built in secret at the same time that a pair of community efforts were trying to build out a new package management strategy for .Net (Seb’s OpenWrap and the Nu project to adapt RubyGems to .Net that I was somewhat involved in). Contrary to the pleasant sounding legend that’s grown up around it, Nuget is not a continuation of a community project. Nuget partially took over the name and asked the guys doing Nu to just stop.

The Nuget episode is irritating to me, not just because of how they de facto killed off the community efforts, but because I think they made some design decisions that caused some severe problems for usage (I described the problems I saw and my recommendations for Nuget earlier this year). Maybe, just maybe, the usage and performance problems could have been solved by more contributions, awareness, and feedback from the larger community if Nuget had been built in the open.

 

Kill the Insider Groups

Most of us didn’t known anything about Nuget or the new ASP.Net vNext platform before it was announced (I did, but not through official channels), but Microsoft has several official “Insider” groups that get much more advance warning and theoretically provide feedback to Microsoft on proposals and early work. I’d like to propose right now that these self-selected “insider” groups be disbanded and much more of these conversations happen in more public forums. My reasoning is pretty simple: there are far more strong developers outside of Redmond and that list than there are on the official list. My fear is that those types of insider, true believer clubs can too easily turn into echo chambers with little exposure to the world outside of .Net — and I feel that my limited involvement with that list frankly confirms that suspicion.

I was not a member of the ASP.Net Insider list until somewhat recently. If I had been aware of the plans for “Project K” earlier, I would have shut down FubuMVC development much earlier than I did because so much of the improvements in ASP.Net vNext make some very large development efforts we did in FubuMVC irrelevant or unnecessary.

 

OSS and the Community Thing

I’m guilty of spreading the “.Net community sucks” meme, but that’s not precisely true, it’s just different than other communities. What makes .Net so much different than many other development communities is how the majority of it revolves around what one particular company is doing.

So much of .Net is open source now and they even take contributions. Awesome, great, but my very first reaction was that it doesn’t matter much because the .Net community as a whole isn’t as participatory as other communities and that would have to change before ASP.Net vNext being OSS matters. It’ll be interesting to me to see if that changes over time.

With few exceptions (NancyFx comes to mind), OSS projects in .Net struggle to get an audience and build a community. I just don’t think there are enough venues for community to coalesce around OSS efforts in .Net, and I think that’s the reason why we have so much duplication of OSS projects without any one alternative getting much involvement beyond the initial authors. I do suspect that the .Net OSS community is much better in Europe now than North America, but that might just me having become such a homebody over the past 5 years.

 

Mono is More Viable and Docker(!)

I thought that trying to support running FubuMVC on Mono was a borderline disaster, to the point where I didn’t want to touch Mono again afterward (it’s a longer conversation, but beyond the normal aggravations like the file system differences between Windows and *nix we hit several Mono bugs). With the recent announcements about open sourcing so much more of the .Net runtime, the PCL standardization, and the better collaboration between Microsoft and Xamarin on display in Miguel de Icaza’s blogpost on the .Net announcements, I think that building applications on server side Mono is going to be much more viable.

Like just about every development shop around, we’re very interested in using Docker for deployments. The ability to eventually deploy our existing .Net applications as Docker containers could be a very big win for us.

 

We wanna code on the Mac

I resisted switching over to Mac’s for years, even when so many of my friends were doing even .Net development on their Macs, just because I was too cheap. After a couple years now of using a Mac, I’d really prefer to stay on that side of things and hopefully give my Windows VM much more time off. Mac OS being a first class citizen for the new .Net and the progress on the OmniSharp tools for Sublime or MacVim is going to make the new ASP.Net vNext runtime a much easier sell in my shop.

 

I think we might just head back to the new .Net

We’ve been working toward eventually moving off of .Net and I have been trying to concentrate my side work efforts on Node.js related things, but the truth of the matter is that we have a huge amount of existing code on .Net and a heavy investment in .Net OSS tools that we just can’t make go away overnight. If the feedback cycle is really much better in the new runtime, we can code on Macs, and use Docker, our thinking is that we’re still better off in the end to just adopt the better .Net instead of rewriting the existing systems on another platform.

 

vNext & Me

I have no intentions of restarting FubuMVC itself, but we’ve put together a concept for a  partial reboot called “Jasper” that may or may not become reality someday. I do want to move our FubuTransportation service bus to vNext next year. At work we *might* try to extend the new version of ASP.Net MVC to act more like FubuMVC to make migrating our existing applications easier if we don’t try to do the Jasper idea. StructureMap 3 is already PCL compliant, so it *should* be easy to get up and going on vNext.

In the meantime, I think I’m going to start a blog series just reviewing bits of the code in the ASP.Net GitHub organizations to get myself more familiar with the new tooling.

 

Transaction Scoping in FubuMVC with RavenDb and StructureMap

TL;DR – We built unit of work and transactional boundary mechanics into our RavenDb integration with FubuMVC (with StructureMap supplying the object scoping) that I’m still happy with and I’d consider using the same conceptual design again in the future.

Why write this now long after giving up on FubuMVC as an OSS project? We still use FubuMVC very heavily at work, I’m committed to supporting StructureMap as long as it’s relevant, and RavenDb is certainly going strong. I’ve seen other people trying to accomplish the same kind of IoC integration and object scoping that I’m showing here on other frameworks and this might be useful to those folks. Mostly though, this post might benefit some of our internal teams and almost every enterprise software project involves some kind of transaction management.

Transaction Boundaries

If you’re working on any kind of software system that persists state, you probably need to be consciously aware of the ACID rules (AtomicityConsistencyIsolationDurability) to determine where your transaction boundaries should be. Roughly speaking, you want to prevent any chance of a system getting into an invalid state because some database changes succeeded while other database changes within the same logical operation fail. The canonical example is transfering money between two different bank accounts. For that use case, you’re probably making at least three different changes to the underlying database:

  1. Subtract the amount of the transfer from the first account
  2. Add the amount of the transfer to the second account
  3. Write some sort of log record for the transfer

All three database operations above need to succeed together, or all three should be rolled back to prevent your system from getting into an invalid state. Failing to manage your transaction boundaries could result in the bank customer either losing some of their funds or the bank magically giving more money to the consumer. Following the ACID rules, we would make sure that all three operations are within the same database transaction so that they all have to succeed or fail together.

Using event sourcing and eventual consistency flips this on its head a little bit. In that case you might just capture the log record, then rebuild the “read side” views for the two accounts asynchronously to reflect the transfer. Needless to say, that can easily generate a different set of problems when there’s any kind of lag in the “eventual.”

The Unit of Work Pattern

Historically, one of the easier ways to collect and organize database operations into transactions has been the Unit of Work pattern that’s now largely built into most every persistence framework. In RavenDb, you use the IDocumentSession service as the logical unit of work. Using RavenDb, we can just collect all the documents that need to be persisted within a single transaction by calling the Delete() or Store() methods. When we’re ready to commit the entire unit of work as a transaction, we just call the SaveChanges() method to execute all our batched up persistence operations.

One of the big advantages to using the unit of work pattern is the ability to easily coordinate transaction boundaries across multiple unrelated classes, services, or functions — as long as each is working against the same unit of work object. The challenge then is to control the object lifecycle of the RavenDb IDocumentSession object in the lifecycle of a logical operation. While you can certainly use some sort of transaction controller (little ‘c’) to manage the lifecyle of an IDocumentSession object and coordinate its usage within the actual workers, we built IDocumentSession lifecycle handling directly into our FubuMVC.RavenDb library so that you can treat an HTTP POST, PUT, or DELETE in FubuMVC or the handling of a single message in the FubuTransportation service bus as a logical unit of work and simply let the framework itself do all your transaction boundary kind of bookkeeping for you.

IDocumentSession Lifecycle Management 

First off, FubuMVC utilizes the StructureMap Nested Container feature to create an isolated object scope per HTTP request — meaning that any type registered with the default lifecyle that is resolved by StructureMap within a request (or message) will be the same object instance. In the case of IDocumentSession, this scoping means that every concrete type built by StructureMap will have the same shared instance of IDocumentSession.

To make it concrete, let’s say that you have these three classes that all take in an IDocumentSession as a constructor argument:

    public class RavenUsingEndpoint
    {
        private readonly IDocumentSession _session;
        private readonly Worker1 _worker1;
        private readonly Lazy<Worker2> _worker2;

        public RavenUsingEndpoint(IDocumentSession session, Worker1 worker1, Lazy<Worker2> worker2)
        {
            _session = session;
            _worker1 = worker1;
            _worker2 = worker2;
        }

        public string post_session_is_the_same()
        {
            _session.ShouldBeTheSameAs(_worker1.Session);
            _session.ShouldBeTheSameAs(_worker2.Value.Session);

            return "All good!";
        }
    }

    public class Worker1
    {
        public IDocumentSession Session { get; set; }

        public Worker1(IDocumentSession session)
        {
            Session = session;
        }
    }

    public class Worker2 : Worker1
    {
        public Worker2(IDocumentSession session) : base(session)
        {
        }
    }

When a FubuMVC request executes that uses the RavenUsingEndpoint.post_session_is_the_same() method, FubuMVC is creating a RavenUsingEndpoint object, a Worker1 object, a Lazy<Worker2> object, and one single IDocumentSession object that is injected both into RavenUsingEndpoint and Worker1. Additionally, if the Lazy<Worker2> is evaluated, a Worker2 object will be created from the active nested container and the exact same IDocumentSession will again be injected into Worker2. With this type of scoping, we can use RavenUsingEndpoint, Worker1, and Worker2 in the same logical transaction without worrying about how we’re passing around IDocumentSession or dealing with any sort of less efficient transaction scoping tied to the thread.

TransactionalBehavior

So step 1, and I’d argue the hardest step, was making sure that the same IDocumentSession object was used by each handler in the request. Step 2 is to execute the transaction.

Utilizing FubuMVC’s Russian Doll model, we insert a new “behavior” into the request handling chain for simple transaction management:

    public class TransactionalBehavior : WrappingBehavior
    {
        private readonly ISessionBoundary _session;

        // ISessionBoundary is a FubuMVC.RavenDb service
        // that just tracks the current IDocumentSession
        public TransactionalBehavior(ISessionBoundary session)
        {
            _session = session;
        }

        protected override void invoke(Action action)
        {
            // The 'action' below is just executing
            // the handlers nested inside this behavior
            action();

            // Saves all changes *if* an IDocumentSession
            // object was created for this request
            _session.SaveChanges();
        }
    }

The sequence of events inside a FubuMVC request using the RavenDb unit of work mechanics would be to:

  1. Determine which chain of behaviors should be executed for the incoming url (routing)
  2. Create a new StructureMap nested container
  3. Using the new nested container, build the entire “chain” of nested behaviors
  4. The TransactionalBehavior starts by delegating to the inner behaviors, one of which would be…
  5. A behavior that invokes the RavenUsingEndpoint.post_session_is_the_same() method
  6. The TransactionBehavior object, assuming there are no exceptions, executes all the queued up persistence actions as a single transaction
  7. At the end of the request, FubuMVC will dispose the nested container for the request, which will also call Dispose() on any IDisposable object created by the nested container during the request, including the IDocumentSession object.

How does the TransactionalBehavior get applied?

In the early days we got slap happy with conventions and the simple inclusion of the FubuMVC.RavenDb library in the applications bin path was enough to add the TransactionBehavior to any route that responded to the PUT, POST, or DELETE http verbs. For FubuMVC 2.0 I made this behavior “opt in” instead of being automatic, so you just had to either include a policy to specify which routes or message handling chains needed the transactional semantics:

    public class NamedEntityRegistry : FubuRegistry
    {
        public NamedEntityRegistry()
        {
            Services(x => x.ReplaceService(new RavenDbSettings { RunInMemory = true}));

            // Applies the TransactionalBehavior to all PUT, POST, DELETE routes
            Policies.Local.Add<TransactionalBehaviorPolicy>();
        }
    }

You could also use attributes for fine grained application if there was a need for that.

Explicit Transaction Boundaries

I fully realize that many people will dislike the API usage with the nested closure below, but it’s worked out well for us in my opinion — especially when it’s rarely used as the exception case rather than the normal every day usage.

All the above stuff is great as long as your within the context of an HTTP request or a FubuTransportation message that represents a single logical transaction, but that leaves plenty of circumstances where you need to explicitly control transaction boundaries. To that end, we’ve used the ITransaction service in FubuPersistence to mark a logical transaction boundary by again executing a logical transaction within the scope of a completely isolated nested container for the operation.

Let’s say you have a simple transaction worker class like this one:

    public class ExplicitWorker
    {
        private readonly IDocumentSession _session;

        public ExplicitWorker(IDocumentSession session)
        {
            _session = session;
        }

        public void DoSomeWork()
        {
            // do something that results in writes
            // to IDocumentSession
        }
    }

To execute the ExplicitWorker.DoSomeWork() method in its own isolated transaction, I would invoke the ITransaction service like this:

// Just spinning up an empty FubuMVC application
// FubuMVC.RavenDb is picked up automatically by the assembly
// being in the bin path
using (var runtime = FubuApplication.DefaultPolicies().StructureMap().Bootstrap())
{
    // Pulling a new instance of the ITransaction service
    // from the main application container
    var transaction = runtime.Factory.Get<ITransaction>();
            
    transaction.Execute<ExplicitWorker>(x => x.DoSomeWork());
}

At runtime, the sequence of actions is:

  1. Create a new StructureMap nested container
  2. Resolve a new instance of ExplicitWorker from that nested container
  3. Execute the Action<ExplicitWorker> passed into the ITransaction against the newly created ExplicitWorker object to execute the logical unit of work
  4. Assuming that the action didn’t throw an exception, submit all changes to RavenDb as a single transaction
  5. Dispose the nested container

Use this strategy when you need to execute transactions outside the scope of FubuMVC HTTP requests or FubuTransportation message handling, or anytime when it’s appropriate to execute multiple transactions for a single HTTP request or FT message.

Composable Generators in Javascript

I’m getting to work with Node.js for the first time and generally enjoying the change of pace from .Net — especially the part where my full Mocha test suite that includes integration tests runs faster than my souped up i7/SSD box can compile even the simplest .Net library. For my part, I’ve been playing with a new library to support event sourcing user defined with projections by leveraging Postgresql’s native JSON features and embedded Javascript support (not much to see yet, but the code is here if you’re curious).

Since it is Node.js, every action you do that touches the database or file system really wants to be asynchronous. I started by using promises with the Bluebird library, and it was okay for simple cases. However, the code became very hard to follow when I started chaining more than 3-4 operations together. Just this week I finally got a chance to try out the usage of the new ES6 generators feature with my codebase in the places where I was having to combine so many asynchronous operations and I’m very happy with the results so far.

To my eyes, generators are a big improvement in readability over promises. Consider two versions of the same method from my event store library. First up, this is the original version using promises to execute a stored procedure in Postgresql (yep, I said sproc) then transform the asynchronous result to return to the calling client:

	append: function(){
		var message = this.toEventMessage(arguments);

		return pg.connectAsync(this.options.connection).spread(function(client, release){
			return client.queryAsync('select pge_append_event($1)', [message])
				.finally(release);
		})
		.then(function(result){
			return Promise.resolve(result.rows[0].pge_append_event);
		});
	},

Now, see the newer equivalent method using generators with some help from the postgres-gen library:

	append: function(){
		var message = this.toEventMessage(arguments);

		return this.db.transaction(function*(t){
			return (yield t.query('select pge_append_event(?)', message)).rows[0].pge_append_event;
		});
	},

I’m going to claim that the generator version on the bottom is cleaner and easier to write and understand than the version that only uses promises above. Some of that is due to my usage of the postgres-gen library to smooth out the interaction with Postgresql, but I think that not having to nest Javascript functions is a big win (yes I know that Coffeescript or arrow functions with Traceur would help too by making the inline function syntax cleaner, but I’d still rather avoid the nesting anyway).

The biggest difference to me came when I wanted to start dynamically composing several asynchronous operations together. As I said earlier, I’m trying to build an event store library with user defined projections. To test the projection support I needed to repeatedly store a sequence of events and then fetch and evaluate the value of the calculated projection to verify the new functionality. I very quickly realized that embedding the original promise mechanics into the code was making the tests laborious to write and hard to read. To that end, I built a tiny DSL just for testing. You can see the raw code in Github here, but consider this sample:

	scenario('can update a view across a stream to reflect latest', function(x){
		var id = uuid.v4();

                // stores several pre-canned events to the event store
		x.append(id, 'Quest', e1_1, e1_2, e1_3);

                // does an assertion that the named view for the event
                // stream above exactly matches this document
		x.viewShouldBe(id, 'Party', {
			active: true,
			location: 'Baerlon',
			traveled: 16,
			members: ['Egwene', 'Mat', 'Moiraine', 'Perrin', 'Rand', 'Thom']
		});

                // do some more events and check the new view
		x.append(id, e1_4, e1_5);
		x.viewShouldBe(id, 'Party', {
			active: true,
			location: 'Shadar Logoth',
			traveled: 31,
			members: ['Egwene', 'Mat', 'Moiraine', 'Perrin', 'Rand']
		});

	});

Every call to the append() or viewShouldBe() methods requires an asynchronous operation to either post or query data from the underlying Postgresql database. Originally, I implemented the testing DSL above by creating an empty promise, then running that promise through all the “steps” defined in the scenario to chain additional promise operations. Earlier this week I got a chance to switch the testing DSL to using generators underneath the language and that made a big difference.

The first thing to know is that you can embed a generator function inside another generator by using the yield* keyword to designate that you want an inner generator function to be executed inline. Knowing that, the way that I implemented the testing DSL shown above was to create an empty array called steps as a member on a scenario object. In the testing expression methods like the append(), I just pushed a new generator function into the steps array like this:

	this.append = function(){
		var message = client.toEventMessage(arguments);
		self.lastId = message.id;

		// Adding a new generator function
		// to the steps array
		this.steps.push(function*(){
			yield client.append(message);
		});
	}

After the scenario is completely defined, my testing DSL just executes the steps as a single generator function as shown in the code below:

	// I'm using the Bluebird Promise.coroutine()
	// method to treat this single aggregated
	// generator method as a single promise
	// that can be happily executed and tracked
	// by the Mocha test harness
	this.execute = function(client){
		return Promise.coroutine(function*(){
			this.steps.forEach(function(s){
				yield* s;
			});
		});
	}

You can see the commit diff of the code here that demonstrates the difference between using just promises and using a generator to compose the asynchronous operation. The full implementation for the “scenario” testing DSL is here.

As an aside, if you’re playing Design Pattern Bingo at home, my testing DSL uses Fowler’s Nested Closure pattern to first define the steps of a test specification in a nested function before executing the steps. I’ve used that pattern successfully several times for little one off testing tools and seen plenty of other folks do similar things.

 

Using generators as a composable Russian Doll Model with Koa.js

My organization settled on a Node.js based stack for a rewrite of an older .Net system, but when we first started discussing a possible new platform Node.js was almost summarily dismissed because of its callback hell problems. Fortunately, I got a tip on twitter to look at the Koa.js framework and its support for generators to avoid the older callback hell problems and I was sold. Since our timeline is long enough that we feel safe betting on ES6 Harmony features, we’re starting with Koa.js.

One of the things I feel was successful in FubuMVC was our use of what I’ve always called the Russian Doll model. In this model, we could compose common behaviors like validation, authorization, and transaction management into reusable “behaviors” that could be applied individually or conventionally to the handlers for each HTTP route in a similar manner to aspect oriented programming (but with object composition instead of IL weaving or dynamic proxies). One of the things that appeals to me about Koa.js is their usage of generators as a middleware strategy that’s conceptually equivalent to FubuMVC’s old behavior model (the current Connect middleware is “single pass” where the Russian Doll model requires nesting the middleware handlers to do optional before and after operations).

At some point I’d like to experiment with building the FubuMVC “BehaviorGraph” model of configuring middleware per route for better fine grained control, but that’s a ways off.

React.js plays nicely with other tools and other thoughts

tl;dr: I’m not completely sure about React.js yet just because it’s such a different approach, but I really appreciate how easy it is to use it in conjunction with other client side tools.

My shop is looking very hard at moving to React.js for new development and I’m scrambling a bit to catch up as I’ve been mostly on the server side for years. Partially as a learning experience, I’ve been using React.js to rebuild the UI for our FubuMVC.Diagnostics package in FubuMVC 2.0 as a single page application. While my usage of React.js on the diagnostics is mostly about read only data displays, I did hit a case where I wanted a “typeahead” type search bar. While a google search will turn up a couple nascent projects to build typeahead type components purely with React.js, none looked to me to be nearly as mature and usable as the existing typeahead.js query plugin*.

Fortunately, it turns out to be very easy to use jquery plugins inside of React classes so I just used typeahead.js as is. For the StructureMap diagnostics, I wanted a search bar that allowed users to search for all the assemblies, namespaces, and types configured in the main application container with typeahead recommendations. Since I was going to want to reuse that search bar on a couple different little screens, I did things the idiomatic way React wants to be used anyway and created the small component shown below:

In the not unlikely chance that the code below is mangled and unreadable, you can find the source code in GitHub here.

var SearchBox = React.createClass({
	componentDidMount: function(){
		var element = this.getDOMNode();
		$(element).typeahead({
		  minLength: 5,
		  highlight: true
		},
		{
		  name: 'structuremap',
		  displayKey: 'value',
		  
		  // Just a little object that implements the necessary 
		  // signature to plug into typeahead.js
		  source: FubuDiagnostics.StructureMapSearch.findMatches,
		  templates: {
			empty: '<div class="empty-message">No matches found</div>',
			suggestion: _.template('<div><p><%= display%> - <small><%= type%></small></p><p><small><%= value%></small></p></div>')
		  }
		});
		
                // Behind the scenes, this is just delegating to Backbone's router
                // to 'navigate' the main pane of the page to a different view
		$(element).on('typeahead:selected', function(jquery, option){
			var url = 'structuremap/search-results/' + option.type + '/' + option.value;
			FubuDiagnostics.navigateTo(url);
		});
	},
	
	componentWillUnmount: function(){
		var element = this.getDOMNode();
		$(element).typeahead('destroy');
	},

	render: function(){
		return (
			<input type="search" name="search" ref="input" 
                          className="form-control typeahead structuremap-search" 
                          placeholder="Search the application container" />
		);
	}
});

 

There really isn’t that much to the code above, but the salient points are:

  • The componentDidMount() method fires just after React.js renders the component, giving me the chance to apply the typeahead jquery plugin to the DOM element.
  • In this case the component is a single HTML element, so I could just use the getDOMNode() method to get at the actual HTML DOM element. In more complicated components you can also use the refs collection.
  • I cleaned up and deactivated the typeahead plugin in the componentWillUnmount() method whenever React.js is tearing down the component to clean up after myself

To use the component above, I just embedded it into another component as more or less a custom tag in the jsx markup:

var Summary = React.createClass({
	render: function(){
		var items = [];
		
		this.props.assemblies.forEach(function(assem, i){		
			items.push(AssemblySummaryItem(assem));
			
			assem.namespaces.forEach(function(ns){
				items.push(NamespaceSummaryItem(ns));
			});
			
		});
		
		return (
			<div>
                        
                        // This is all you have to do to include it
			<SearchBox />
			
			<hr />
			
			<ul className="list-group">
				{items}
			</ul>
			
			</div>
		);
	}
});

 

So what do I think of React.js so far?

Um, well, React is interesting. I know that my initial reaction was similar to many others: “you seriously want me to embed HTML markup directly into JavaScript?” Once you get past that, it’s been very easy for straight forward usages of rendering JSON from the server into HTML displays. I won’t make any permanent judgement of React.js until I use it on something with much more complicated user interactions.

To speak to one of React.js’s major design philosophies, I’ve never liked two way model binding in any user interface technology I’ve ever used because I think it so easily obfuscates the behavior of screens and causes unexpected side effects. I’m not completely sure how well the one way data flow thing is going to work out in bigger applications, but I’m at least initially on board with the React.js team’s basic design philosophy.

One thing to consider is that React.js is much more of a library that you’d use as opposed to an all encompassing framework like Angular.js or Ember.js that you have to wrap your application around. The downside of that is that React.js doesn’t really do anything to encourage separation of concerns or help you structure your application beyond the view layer, but the old presentation patterns that came out of older technologies still apply and are still useful if you just treat React.js as the view technology. You’re just on your own to do your own thinking. The upside of React.js being just a library is how easy it is to use it in conjunction with other tools like the entire jquery ecosystem or bits of Backbone.js or even as something within frameworks like Angular or Ember.

I’ve also done a little bit of a proof of concept of using React.js as the view layer in a possible rewrite of the Storyteller UI into a pure web application. That work will probably generate much more interesting blog posts later as it’s going to require much deeper hierarchies of components, a lot more complicated user interaction, and will push us into exploring React.js usage with separated presentation patterns and much more unit testing.

At least in the circles I run in, there’s been a bit of a meme the past couple years that Functional Programming is the one true path to software programming and a corresponding parallel backlash against almost everything to do with OOP. If React.js becomes more popular, I think that it might help start a rediscovery of the better ways to do OOP. Namely a focus on thinking about responsibilities, roles, collaboration patterns, and object composition as a way to structure complicated React.js components instead of the stupid “Class-centric,” inheritance laden, data-centric approach to OOP that’s been so easy for the FP guys to caricature.

 

* I’m not really working much on the client side, so what do I really know? All the same, I think the backlash against jquery feels like a bit of throwing the baby out with the bathwater. Especially if you’re constraining the DOM manipulation to the purely view layer of your code.

Some Thoughts on Collective Ownership and Knowledge

This is a mild rewrite of an old blog post of mine from 2005 that I think is still relevant today. While this was originally about pair programming, I’ve tried to rewrite this to be more about working collaboratively in general. 

My shop is having some constructive internal discussions about our development process and practices. While I don’t think that Pair Programming is likely to be a big part of our daily routine, I think we could still use a bigger dose of collective ownership in our various code base’s and less silo-ing of developers. Since I’m a typical introverted developer who doesn’t handle hours of pair programming and I don’t think my attitude is uncommon at all, we need to look for ways to get the same type of shared understanding that you would get from 100% paired programming.

One of my suggestions is at a minimum to stop assigning coding stories to a single developer. Even if they aren’t pair programming all the time, they can still collaborate on the approach and split tasks so they can code in parallel. On one hand this should increase our team’s shared understanding of the codebase, and on the other hand it’ll definitely help us work discrete stories serially to completion instead of having so many half-done stories laying around at any one time.

 

“This is the Way We Do It”

My favorite metaphor for software design these days is a fishing tackle box.  I want a place for everything and everything in its place.  When I need a top water lure, I know exactly where to look.  I put data access code here, business logic there, and hook the two things up like this.  When I’m creating new code I want to organize it along a predictable structure that anyone else on the project will instantly recognize.  When other coders are making changes I want them to follow the same basic organization so I can find and understand their code later.

Each application is a little bit different so the application’s design is always going to vary.  In any agile project you should hit an inflection point in the team’s velocity that I think of as the “This is the Way We Do It” moment.  Things become smoother.  There are fewer surprises.  Story estimates become more consistent and accurate.  When any pair starts a new story, they understand the general pattern of the system structure and can make the mechanical implementation with minimal fuss.

You really want to get to this point as soon as you can.  There are two separate issues to address before you can reach this inflection point:

  1. Determining the pattern and architecture for the system under development
  2. Socializing the design throughout the development team

To the first point, pairing allows you bring to bear the knowledge and experience of everybody on the team to the work at hand.  It’s just not possible for any one developer to understand every technology and design pattern in the world. By having every developer active in the project design, you can often work out a workable approach faster than a solo architect ever could.  On the one project I’ve done with theoretical 100% pairing, we had a couple of developers with a lot of heavy client experience and me with more backend and web development experience.  By pairing together with our disparate knowledge we could rapidly create a workable general design strategy for the system as a whole by bringing a wider skill set to any coding task.

Right now I’m working some with a system that’s being built by two different teams that work in two very different ways with very different levels of understanding of some of the core architecture. That’s really not an ideal situation and it’s causing plenty of grumbling. There’s a couple specific subsystems that generate complaints about their usability. In one case it just needs to be a full redesign, but that really needs to take place with more than one or two people involved. In the other case, the problems might be from the team that didn’t build the subsystem not understanding how it works and why it was built that way — or it might be because the remaining developer who worked on it doesn’t completely understand the problems that the other team is having. Regardless of what the actual problem is, more active collaboration between the teams might help that subsystem be more usable.

If you’re a senior developer or the technical lead, one of your responsibilities is fostering an understanding of the technical direction to the other developers.  Nothing else I’ve ever done as a lead (design sessions, documentation, presentations, “do this,” etc.) beats working shoulder to shoulder with other developers as a mechanism for creating a shared understanding of the project strategy.  By making every developer be involved or at least exposed to the thinking behind the design, they’ll have much more contextual information about the design.

We have a great policy of doing internal brown bag presentations at our development office and it’s giving our guys a lot more experience in presentation skills and opportunities to learn about new tools and techniques along the way. While I’d like to see that continue and I like learning about programming languages we don’t yet use, I also think our teams should probably speak about their own work much more often to try to create better shared understanding of the architectures and code structures that they use every day.

Part of any explanation or knowledge sharing about your system’s architecture, structure, and build processes needs to include a discussion of why you’ve chosen or arrived at that architecture. One unpleasant fact I’ve discovered over and over again is that the more detailed instructions you give to another developer, the worse the code is that comes back to you.  I simply can’t do the thinking for someone else, especially if I’m trying to do all the thinking upfront independently of the feedback you’d naturally get from working through the problem at hand.  If the developer doing the work understands the “why” of your design or instructions, they’ll often do a better job and make improvements as they go — and that’s especially important if you gave out instructions that turn out to be the wrong approach and they should really do something different altogether.

For example, I’ve spent the last couple years learning where some of the more rarely used cooking utensils and gadgets in our kitchen go. Instead of trying to memorize where each thing I find unloading the dishwasher goes, my wife has explained her rationale for how the kitchen is organized and I’m somewhat able to get things put up in the right place now. Organizing code isn’t all that different.

 

Improving Our Code vs. Defensiveness about My Code

Don’t for one second discount the psychological advantages of pair programming or even just a more collaborative coding process.  Formal or even just peer code reviews can be nasty affairs.  They can often amount to a divide between the prosecution and the accused.  In my admittedly limited experience, they’ve been largely blown off or devolve into meaningless compliance checks with coding style standards.  Even worse is the fact that they are generally used as a gating process just prior to the next stage of the waterfall, eliminating the usefulness because it’s too late to make any kind of big change.

The collective ownership achieved with pair programming can turn this situation on its head.  My peers and I can now start to talk about how to improve our code instead of being defensive or sensitive to criticism about my code.  Since we’ve all got visibility now into the majority of the code, we can have informed conversations about the technical direction overall.  The in depth code review happens in real time, so problems are caught sooner.  Add in the shifting of different coders through different areas of the code and you end up with more eyes on any important piece of code. The ability to be self-critical about existing code, without feeling defensive, helps to continuously improve the system design and code.  I think this is one of the primary ways in which agile development can lead to a better, more pleasant workplace.

Why I hate the word “Pragmatic” and other rants

I told myself that when I started my new blog that I’d try not to rant as much as I did in the CodeBetter days, but oh well, here we go anyway:

Software is a young profession and judging from the rate of change and improvement over my 16 years as a software developer, we’re still learning a lot of new lessons — or relearning old lessons for ourselves that some other group of developers already learned years ago. The great part about that ever changing landscape is that constantly learning new tools and techniques is so much more fun than my first job as an engineer where I mostly certified piping system designs with the “boiler code” that dated back to the 1800’s. The downside is that I think that developers as a whole are terrifyingly bad at sharing information and debating or reasoning about the merits of various alternatives — especially if we feel threatened by other developers promoting tools or techniques that we don’t yet know or use.

I want to ban these words and phrases from technical discussions

I’m betting that many of you are familiar with Godwin’s Law:

“As an online discussion grows longer, the probability of a comparison involving Nazis or Hitler approaches 1.”

From my perspective, the importance of Godwin’s Law in practice is that conversations effectively lose all usefulness when highly charged words and hyperbolic comparisons start getting tossed around.  Much like George Carlin’s Seven Dirty Words, I’d like to create a new list of words and phrases that should be banned in technical conversations at least until we learn to use them a little more wisely:

  • Pragmatic — If you follow me on Twitter you surely know that I despise the word pragmatic as it’s usually wielded as a magical talisman by inarticulate developers the world over to win debates and justify whatever random decision they’ve made to other developers. More on this in the next section.
  • Fetish — As in “I don’t get your fetish for static/dynamic typed languages.” What you’re doing is shutting your mind off to the merits of different ideas than you’re own.
  • Zealot — see the above.
  • Dogmatic — Dogmatic is generally used as a pejorative term, but sometimes it’s not. Some techniques are most effective when they’re applied consistently. Again, I think developers mostly use this word to denigrate or delegitimize the differing opinions of other developers.
  • Academic — Denigrating some technique as being more theory than something that’s practical to use. You might be right and my experience in a couple different fields is that there is some real distance between theory and reality, but using the word “academic” to describe a technique you don’t use is a sign of a closed mind. Sample usage: “OWIN is a kernel of a good idea wrapped in a lot of academic masturbation.” I’m admittedly guilty of using this quite frequently in reference to enthusiastic proponents of functional programming.
  • “Why don’t/didn’t you just” — I’m bad about using this on other developers. Sample usage: “instead of unraveling the Gordian Knot painstakingly by hand, why didn’t you just cut through it with the sword laying over there that no one told you about?.” The cruel reality of software development is that frequently the simplest solution involving the least work is not the obvious solution.
  • “It’s obvious” — My ex-wife absolutely hated it when her professors in college used the phrase “it should be obvious to the most casual observer” when their point wasn’t clear at all. It’s just a way of either putting down your co-workers or using loaded language to win an argument. Contributed by my colleague Matt Smith.
  • “Git r’ Done” — Used non-ironically it’s just a cutesy synonym for “being pragmatic.” I should probably ban “git ‘r done” because in my circles it’s code for bad developers that don’t care about any kind of quality as long as the immediate work gets done sorta, kinda on time and mostly “works.”
  • “Use the right tool for the job” — This is an empty calories kind of statement and a cliche. I usually interpret this as “I just want to be left alone to use what I already know and I’m not interested in continuing this conversation”
  • “We just have to get this out the door” — I think there’s a real tension and a necessary balance between making decisions tactically for immediate problems and thinking strategically for the long term. While being an architect astronaut is a sign that you’re too far into the strategic side of things, using the phrase “we just have to get this out the door” is often an indication that your team might be veering way too far in favor of short sighted tactical decisions while missing opportunities to improve their productivity by making changes in process, tools, or a redesign of a bad subsystem. My fear — and my experience backs this up — is that teams that use this phrase and moreover believe it, have shut off their minds and just started to bang harder and faster on their keyboard. My favorite analogy for this is always the saying “I’m too busy chopping trees to stop and sharpen my axe!
  • Shiny Object — I’m guilty of this one. As in, “you just want to use XYZ because it’s the new shiny object”
  • Cargo Cult — All the analogies to cargo cults and teams just going through the motions of doing Agile development were kind of funny a couple years ago, but I think the term “cargo cult” is over used mostly as a way to put down other people at this point. If you’ve been living under a rock and haven’t heard this term, I liked James Shore’s Cargo Cult Agile from back in the day.

If you use any of these words and phrases to another developer, you might have wasted a chance to learn or teach something new.

Good Decision Making, Modeling, and being Pragmatic

Taking the definition from Merriam Webster, being pragmatic is:

dealing with the problems that exist in a specific situation in a reasonable and logical way instead of depending on ideas and theories

I think that developers often conflate being pragmatic with doing things well. Being reasonable and logical sounds like a great way to be successful, but you can be pragmatic until the cows come home and still make bad decisions all day long if you’re using bad information or not accounting for other factors. Making good decisions isn’t so much about being pragmatic, dogmatic or zealous so much as it is about constructing the right mental model that represents more of the important factors that go into the decision.

As I alluded to earlier, my educational and early professional background is engineering. My primary job was often about creating models to accurately simulate the real world in order to validate decisions or optimize designs. Needless to say, the better and more accurate your models are, the better your real results.

I thoroughly enjoyed a special projects class I took my last year of school where we designed thermodynamic systems based on optimizing either cost or capacity. To do the optimization, you had to model the various factors (length of a connecting pipe, depreciation, rate of heat loss, etc.) with equations that could be solved with optimization software to find local minimums or maximums.

 

Since it’s just math, two teams working on the same project should always arrive at the same designs and decisions, right? Right?

What we found in the course of doing these projects was (unsurprisingly) that:

  • The calculated optimal design could be much different if you added or ignored certain factors. Maybe my team accounted for the pressure drop across a mile of pipeline while the other team assumed that it was constant. Maybe the other team made different assumptions than we did.
  • The calculated optimal design could also be much different for shorter timeframes than it was for longer timeframes.

To make this concrete in terms of software development, take the decision about whether or not to write unit tests for any given piece of code:

  • Writing the extra code to automate unit tests takes more time than just writing the production code. A big point against the unit tests.
  • Only end to end or integration tests would be able to definitively determine problems in the code in question. Don’t write the unit test.
  • An end to end test would be very expensive to write. Write unit tests for what you can easily unit test.
  • Writing automated unit tests might force changes to the way you structure your code — which might be a forcing function in favor of better factored code which is good or additional complexity that you don’t want
  • Without unit tests you might not catch problems until later when the problem will be harder to find and fix. A point in favor of unit tests.
  • Using a debugger is an inefficient way to solve problems in code and foregoing unit testing increases the risk of long debugging sessions. Writing automated unit tests frequently reduces the amount of time spent using a debugger.
  • Automated tests of any kind definitely save time in regression tests. A point in favor of writing the tests in the longer term.
  • The existence of automated tests can lead to better reversibility so that you can make changes safely later. A point that might be very important or not at all.

If you base your decision about whether or not to write unit tests on only one of these bullet points and not a combination of many or all of them, your decision making isn’t as likely to optimal in the end. Also, your decision making should probably come to different results on a one week project versus a multi-year project.

Again, making better decisions isn’t just about being “pragmatic,” it’s about creating a mental model that more completely and accurately accounts for the positives and negatives of any given decision and also being cognizant of the timeframe in which you need to optimize. If you make the decision about whether or not  Making decisions pragmatically but with incomplete information or understanding of your problem is still bad.

Going back to the point about being too focused on making incremental deadlines and being too focused on the tactical, I’ve far too frequently seen teams forgo solving some sort of technical debt issue that is slowing them down because they had to make their next iteration. In the very short term, they might be right — but if the time saved over X number of iterations is greater than the time it takes to solve and fix the technical debt issue then you need to stop and fix that damn technical debt problem.

To make that last paragraph more concrete with a real life example from my shop, let’s say that your team is working with a system where each and every new vertical feature requires changes to code in six different git repositories. Would you continue to pound away on new features to maintain your current velocity, or stop for a little bit to combine all the repositories into a single logical repository where it’s much easier to make changes in one place and very likely increase your velocity afterward?

Lullaby Language 

I asked my Twitter followers for more examples of words and phrases we should ban and a large number of replies contained the word “just.”  As in the time a business analyst told me that if we were to use our brand new website application as the front end for a system built by another branch of our company that neither of us had other seen it would “just be some refactoring.”  Using the word “just” is a perfect example of Lullaby Language that we should probably avoid using because it gives us and our listeners a potentially false sense of security.

Another loaded word is “should,” as in “the new build should work” with an unspoken admission that they hadn’t tested the exact scenario in question. I had a very enjoyable co-worker years ago that would react comically to statements like that with “you said ‘should!'” (for my current colleagues, this is the guy that got me started on the finger on the nose gesture you’ve all seen by now).

Other examples from the peanut gallery were:

  • “should be a quick fix/change” — putting pressure on you to turn it around quickly with no real forethought about how big it really is
  • “this is too simple/small for testing/versioning” — famous last words…
  • “The deadline is already set. Can we make it?” — I really enjoyed Jez Humble’s talk at NDC London last year about how we should concentrate more on delivering value instead of just trying to hit deadlines

 

Non Technical Folks and Estimation

I got a lot of other complaints on Twitter that mostly boiled down to project managers and other non-technical folks setting deadlines, time limits, or estimates for developers. I’m a big believer in the idea that developers should do most of the estimation work — and even then, those estimates should be treated as unproven working theories. The only time I think a project manager can get away with doing estimation is when there’s plenty of historical data for the project on similar stories and the composition of the team is stable.

Like many developers in Austin of my generation, I started my development career at a large captive IT shop that was famous for generating truly awesome horror stories. The very last thing I did there was to help kick off a new project to replace an existing web application that our business wasn’t happy with and we felt would be easier to rewrite than improve (it build dynamic HTML inside of VB6 COM objects hosted in ASP classic). Since that very year I had had my bonus slashed because of how far off my estimate was for a previous project, I was appropriately paranoid about getting it right this time around. To that end, I spent a lot of time using use case points (this was so long ago that RUP was still cool) to both define the scope and a first estimate. Because I’ve told this story so many times, I still remember the basic details of the proposed system:

  1. A small new database schema to be what we’d now call the read side model — maybe 2 primary master/detail tables plus the normal contingent of lookup data tables
  2. A pretty complicated web screen with a lot of different user interactions, and this was way before things like jQuery or Angular.js
  3. About a dozen integration points to legacy systems that we had had trouble with in previous projects

When we finally got a project manager I tried to present my use case’s and estimates to the new project manager but I was basically told “don’t worry about it hon, I’ll make the project schedule.”

The next day I got the email with the new Excel schedule attached. As memory serves me, it was an old fashioned waterfall schedule that roughly went:

  • 40 days for logical database design
  • 20 days for physical database design
  • 10 days for the integration work
  • 10 days for the web application

I was apoplectic because it was pretty well exactly opposite of what I came up with in every way and pushed all the technical risk to the very end of the project — but I quit the very next week anyway so it was okay;) The PM wasn’t amused when I drew two boxes and one line on her whiteboard and said “there, the logical database design is done!”

 

 

Software Development Horror Stories and Scar Tissue

My best friend (also a developer) and I meet for lunch almost every Friday. As experienced developers often do, we got to swapping horror stories this past Friday. I had so much fun telling some of my favorite stories that I thought I’d repeat some of them here just to break my blogging drought. As I started doing just that, it occurred to me that there’s a deeper theme here about how we process our bad experiences, what we learn from them, and maybe being a little more nuanced instead of making a knee jerk “guard rail to guard rail” decision about whatever we deem to be the cause of those bad experiences.

Just think, how many times have you said or heard another developer say some form of the following two statements:

  1. I’ll never use [IoC containers/NoSql/Design Patterns/mock objects] ever again because this one time at band camp I saw some people on a project (not me, of course) try to use that and it was horrible! Never again!
  2. We threw the baby out with the bathwater on [UML/Pair Programming/Stored Procedures/something that’s valuable sometimes but not always]

UML here is the obvious example of a technique that was horribly abused in the past and now largely discredited. While I lived through the age of absurd UML overuse and I’ve made fun of UML only architects, I still think that some knowledge of UML can be very useful to a developer. Specifically, I’ll still sketch class diagrams to understand a subsystem or a sequence diagram to think through a tricky interaction between classes or subsystems. With multi core processors and the push for so much more parallelization, I think that activity diagrams come back as a way to think through the timing and coordination between tasks running in different threads.

The Centralized Architecture Team

Not long after I started my current job we had a huge phone call to discuss some potential changes to our development methods and projects. Someone suggested that we form a dedicated, centralized architecture team and I shot it down vehemently and colorfully. Why you ask? Because…

The most useless I’ve ever felt in my career was the 18 months I spent as the junior most member of a centralized architecture team in a large enterprise IT organization. My shop at the time had decided to double down on formal waterfall processes with very well defined divisions of specialized responsibility. Architects made high level designs, technical leads made lower level designs, and the developers coded what they were told. As you’ve probably guessed, we architects were not to be doing any actual coding. As you’ve probably also guessed, it didn’t work at all and most teams faked their compliance with the official process in order to get things done.

I firmly believe that at the core of all successful software development is a healthy set of feedback mechanisms to nudge a team toward better approaches. A non-coding architect who doesn’t stick around for the actual coding doesn’t have any real feedback mechanism to act as a corrective function on his or her grandiose technical visions.

I witnessed this first hand when I was asked to come behind one of my architect peers on what should have been a relatively simple ETL project. My peer had decided that the project should consist of a brand new VB6 application with lots of elaborate plugin points and Xml configuration to accommodate later features. The team who actually had to deliver the project wanted to just use the old Sql Server DTS infrastructure to configure the ETL exchanges and be done with it in a matter of days — but my peer had told them that they couldn’t do that because of corporate standards and they’d have to build the new infrastructure he’d designed on paper.

When I took over as the “architect,” the team was struggling to build this new ETL framework from scratch. I was able to clear the usage of DTS with one quick phone call and sheepishly told the project manager that he had my official blessing to do the project in the way that he and his team already knew was the easiest way forward. I felt like an ass. The team had the project well in hand and would have already been done if not for the “guidance” of my architecture team.

I’ve had other bad experiences with centralized architecture teams over the years in several other large companies. In each negative case I think there was a common theme — the centralized architecture team was too far removed from the actual day to day work (and many of the members of those teams knew that full well and worked in constant frustration) . There’s some obvious potential goodness in sharing technical knowledge and strategy across projects, but my opinion is that this is best done with a virtual team of technical leaders from the various project teams in an organization sharing solutions and problems rather than a dedicated team off by themselves, brown bag presentations, and just rotating developers between teams once in a while.

I ended up replacing my non-coding architect peer on a couple later projects and every single time I walked into a situation where the new team hated the very idea of an “architect” because of their scar tissue. Good times.

 

Web Services Everywhere!

The undisputed technical leader of the same centralized architecture team had a grandiose SOA vision of exposing all of our business processes as loosely coupled SOAP web services. I was dubious of some of the technical details at the time (see Lipstick on a Pig), but there was a definite kernel of solid thinking behind the strategy and I didn’t have any choice but to get on board the SOA train anyway.

A couple months in, our SOA visionary gave a presentation to the rest of the team upon the completion of the first strategic building block web service that we should be using for all new development from then on. The new standard data service exposed an HTTP endpoint that accepted SOAP compliant Xml with a single body element that would contain a SQL string. The web service would execute the SQL command against the “encapsulated” database and return a SOAP compliant Xml response representing the data or status returned by the SQL statement in the request Xml.

That’s right, instead of making our applications couple themselves to well understood, relatively efficient technologies like JDBC or ODBC and an application database, we were logically going to follow the approach of using one big damn database that would only be accessed by Xml over HTTP and each application would still be tightly coupled to the structure of that database (kind of what I call the “pond scum application” anti-pattern where little applications hang off a huge shared database).

My older, more experienced colleagues mostly nodded sagaciously murmuring that this was a good approach (I’m sure now that they all thought that the approach sounded nuts but were intimidated by our visionary and didn’t call out the quite naked emperor in the room). I sputtered and tried to explain later to my boss that this was the worst possible approach we could possibly take, but I was told to change my attitude and get with the program.

My lesson learned from that experience? The most experienced person in the room isn’t automatically right and the new shiny development idea isn’t magically going to make things better until you really understand it. And yes, dear colleagues reading this, I do realize that now *I’m* the most experienced person in the room and I’m not always automatically right when we disagree.

Would I use SOA after I saw it done in the dumbest way possible? Yes, in certain situations like this one — but I still contend that it’s better to focus on building well factored code that’s easy to extend without the overhead of distributed services (easier testing, easier debugging, just flat out less code) when you can get away with it.

 

Database Batch Programming Woes 

Most of my horror stories come from my very first “real” IT job. My original team was hired en masse to be the support team for a new set of supply chain automation applications built by a very large consulting company (I’ll give you one guess who I’m mocking here). Just before the consultants rolled off the project I was enlisted to help their technical lead try to speed up a critical database batch script that was too slow and getting much slower a mere 6 weeks after the project was put into production. So what was wrong that made my database guru friend sputter when I told this story? As best I recall:

  1. The 3rd party ERP system that fed the batch process had some custom Java code with a nasty N+1 problem where they were doing a logical database join in memory
  2. The batch process ran several times a day and started by first inserting data into a table from the result of a cross join query (still the only time I’ve ever seen that used in the wild) of every possible part our factories used, all of our offsite logistics centers, and each factory line where the parts would be needed — regardless of whether or not that permutation made any sense whatsoever.
  3. That giant table built from the cross joins was never archived or cleaned up, and the very expensive full table scan queries in the next stage of the batch process against that table were becoming exponentially slower just a couple months into the life of the system.
  4. Somewhere in this batch process was a big query where two tables had to be joined by calculated values, one created by string concatenation of 3 fields in table #1 and a matching string created by concatenating 5 different fields in table #2 to make the correct inner join (I’m not making this up).

Long story short, within about 6 months my new team wrote a new replacement system from scratch with two orders of magnitude throughput over the original consultant’s code that didn’t suffer from degrading performance over time by simply being a bit smarter about our database code. Our team got a bit of acclaim and some internal awards because of how happy the business was with our new system (as a first time technical lead I made my own share of WTF’s on that project too, but they were less damaging). Lessons learned?

  • Success is easiest when you start with a low bar
  • Building a quality system is so much easier to do well when you have some sort of prior art to learn from — even if that prior art is poorly done
  • Don’t be egregiously stupid about how you do your database access
  • Being smart isn’t perfectly helpful without a modicum of technical knowledge or experience (the very young consultants responsible for the horrible database designs were all very sharp, but very green).

I think you can interpret that second lesson in a couple useful ways. One, it might help to lighten up on whatever team went before you. Two, don’t be so hard on yourself when you’re having to go back over and replace some kind of software design you did previously with something much better. Doing it wrong first might have taught you how to it better the second time around.

I left that company a couple years later to go be a fancy pants traveling consultant and I very distinctly remember seeing that consulting company’s huge ad with Tiger Woods plastered all over O’Hare airport in Chicago talking about how they’d helped my former employer with their supply chain automation and grumbling how *I’d* built that, not them.

 

 

FubuMVC Lessons Learned — “fubu new”, Standardization, and Polyglot Programming

EDIT:  Someone on Twitter got upset about my “or an inferior IoC tool” comment.  Most of you probably know that I’m also the author of StructureMap, and I did mean that as a joke.  In a later post about IoC integration I’ll happily admit that making FubuMVC too StructureMap-centric was a problem with adoption (we did support Autofac and had Windsor support all but done as well).  To be blunt, I think that the IoC tools in .Net are largely an interchangeable commodity item and any of the mainstream ones will get the job done just fine.

I’m still up for doing this series of technical lessons learned about FubuMVC to wring some value out of the whole process, but I feel like I’ve said enough mea culpas about our failure on documentation. From now on, instead of dog piling on me in the comments, could you just say “Docs sucked++” and then get on with whatever else it is you want to say?

Today’s blogpost is partially about the danger of deviating from .Net orthodoxy, but mostly a lamentation on my part on missed opportunities and unrealized potential.  I’m actually quite proud and happy with most of what I’m describing in this post, but it was too late to matter much and might not have ever been widely accepted.

Quickstart from the Command Line

I’ve always admired elements of Ruby on Rails, especially the almost magical “rails new” project skeleton creation that sets you up with an entire code tree with a standard build script that exposes common build tasks for testing, database migrations, and even deployment recipes — but the word “standard” in that last sentence is a very key concept.  Much of the value of “rails new” is enabled by standardizing the layout and naming conventions of a Rails codebase to make it cheaper to write reusable command line tooling.

We knew from the very beginning that we’d eventually want our very own “fubu new” analogue.  Our community built a simple one initially that would just clone a pre-canned Git repository and do some simple string replacement for your project name.  Great, and it added value immediately.  However, the FubuMVC ecosystem got bigger as we built:

  • Ripple as a better command line manager for Nuget packages in continuous integration
  • Bottles as our modularization strategy, including a command line bottles tool to package up web content files as a pre-compile step in your build script
  • FubuDocs as a command line tool to author and publish technical documentation
  • fubu run as a Katana based development server that’s more efficient in development than anything IIS or VS based
  • FubuTransportation as our FubuMVC based message bus
  • A slew of command line testing tools

To unlock the usefulness of all those tools above to new users, not to mention just getting a simple application codebase up and running fast, we embarked on a new effort to create a vastly improved “fubu new” story that would allow you to mix and match options, different project types, and integrate many of those tools I listed above.

At the time of this post, you can stand up a brand new FubuMVC application using the Spark view engine from scratch to do grown up development by following these steps at the command line assuming that you already have Ruby 1.9.3+ installed:

  1. gem install fubu
  2. fubu new MyApp –options spark

If you do this, you’ll see a flurry of activity as it:

  1. Builds a new VS.Net solution file
  2. Csproj files for the main application and a matching test library
  3. Creates the necessary classes to bootstrap and run a minimal FubuMVC application
  4. Invokes gems to install Rake, FubuRake, Ripple, FubuDocs, and the bottles command line tool
  5. Invokes Ripple’s equivalent to Nuget Package Restore to bring down all the necessary Nugets
  6. Opens the new Visual Studio solution

The result is a code tree that’s completely ready to do grown up development with a build script containing these tasks:

rake ci                # Target used for the CI server
rake clean             # Prepares the working directory for a new build
rake compile           # Compiles the solution src/MyApp.sln
rake compile:debug     # Compiles the solution in Debug mode
rake compile:release   # Compiles the solution in Release mode
rake default           # **Default**, compiles and runs tests
rake docs:bottle       # 'Bottles' up a single project in the solution with...
rake docs:run          # Runs a documentation project hosted in FubuWorld
rake docs:run_chrome   # Runs the documentation projects in this solution i...
rake docs:run_firefox  # Runs the documentation projects in this solution i...
rake docs:snippets     # Gathers up code snippets from the solution into th...
rake myapp:alias       # Add the alias for src/MyApp
rake myapp:chrome      # run the application with Katana hosting and 'watch...
rake myapp:firefox     # run the application with Katana hosting and 'watch...
rake myapp:restart     # touch the web.config file to force ASP.Net hosting...
rake myapp:run         # run the application with Katana hosting
rake ripple:history    # creates a history file for nuget dependencies
rake ripple:package    # packages the nuget files from the nuspec files in ...
rake ripple:restore    # Restores nuget package files and updates all float...
rake ripple:update     # Updates nuget package files to the latest
rake sln               # Open solution src/MyApp.sln
rake unit_test         # Runs unit tests for MyApp.Testing
rake version           # Update the version information for the build

This entire rake script is (I’ve added some explanatory comments for this blog post):

require 'fuburake'

@solution = FubuRake::Solution.new do |sln|
        # This is unnecessary if there's only one sln file in the code
	sln.compile = {
		:solutionfile => 'src/MyApp.sln'
	}
				 
        # This feeds the CommonAssemblyInfo.cs file we use
        # to embed version information into compiled assemblies
        # on the build server
	sln.assembly_info = {
		:product_name => "MyApp",
		:copyright => 'Copyright 2013. All rights reserved.'
	}
	
        # These are defaults now, but I still left it in the
        # template
	sln.ripple_enabled = true
	sln.fubudocs_enabled = true
end

# This one line of code below creates rake tasks as a convenience
# to run our development server in various modes
FubuRake::MvcApp.new({:directory => 'src/MyApp', :name => 'myapp'})

 

The terseness of the rake script above relies very heavily on a standardized code tree layout and naming conventions.  As long as you could accept the FubuMVC idiomatic code tree layout, I feel like we succeeded in making our ragbag collection of command line tools easy to setup and use.  Great, awesome, it’s done wonders for the Rails community and I’ve now seen and used working analogues with Scala’s Play framework and Mimosa.js’s “mimosa new” command.  A couple problems though:

  • There’s never been a successful effort in .Net to identify and codify idioms for laying out a code repository and we’ve found that many folks are loathe to change their own preferences (“src” versus “source”, “lib” vs “bin”, and so on).  It might be easier if TreeSurgeon had succeeded way back when.
  • I think it was just too little, too late.  I think that OSS projects, once announced, have a short window of opportunity to be awesome or else.  I felt good about the improved “fubu new” experience just in time for NDC London this past December — but that was almost 4 years after FubuMVC started.
  • VS.Net templating was a lot of work and dealing with different project types, VS versions, and versions of .Net added overhead
  • Nuget and our own Ripple tool are still problematic
  • While common in other development communities, this is not a common approach for .Net developers

 

Why not Visual Studio Templates or Project Template Nugets?

We did support Visual Studio project templates, but I always felt that would only be for folks that just want to play with the framework a little bit.  The feedback we got from other OSS projects that did invest heavily in VS templates was uniformly negative and I wasn’t enthusiastic about doing much with them.  It was a tremendous amount of work to build our templating engine (FubuCsProjFile), but at least what we got was something easy to author and possible to cover with automated testing (a never-ending shortcoming of so many tools originating from Redmond).

We supported a project template with Nuget very early on and even finagled out a “install nuget, press F5” experience, but I always found it to be very limited and problematic.  Again, testability was an issue.  If I had it to all do over again, I’d still choose our command line approach with the mix and match selection of options (Spark or Razor?  StructureMap or an inferior IoC tool?  Want RavenDb support?  Bootstrap?  Html Conventions?).

 

The Polyglot Thing is a Double Edged Sword

There’s always some sort of meme running around in developer circles that’s meant to make the cool edgy kids feel superior to the unwashed masses.  A couple years ago there was a lot of hype about polyglot programming where you would happily mix and match different programming languages and paradigms in a single solution based on their relative strengths.

For better or worse, the FubuMVC projects were a mild example of polyglot programming and that repeatedly scared people off.  We used Ruby’s Rake tool for our project automation, we partially replaced Nuget with gems for distributing binaries that we used in the build scripts.  For 2.0, we’re happily ditching our original asset pipeline in favor of the Node.js based Mimosa.js tool.  At other times we also used the Python-based Sphinx for some early documentation efforts.

While I think that Rake is an outstanding tool and very easy to use, the simple need for a Ruby tool drove many people away — which is a shame because many other .Net OSS tools besides FubuMVC prefer Rake-based build scripts over the other alternatives.

It’s not hard to understand people being hesitant about having to learn non-mainstream .Net tools, it’s unfortunate.  One of the big problems with trying to do ambitious OSS work on .Net is that .Net simply isn’t the home of the best tools.  We repeatedly found Nuget to be lacking (a blog post for a later day) and MSBuild is, in my opinion, a complete non-starter for build automation.

As an aside, I’ve frequently seen members of the ASP.Net team complain on Twitter about having to install Ruby just to build some OSS project when they turn right around and require you to install all kinds of Visual Studio.Net add ons and templates just to build their code.

 

Aborted Plans for FubuMVC 2.0:

For 2.0 I wanted us to push forward with more templating options and introduce a new standalone web application that you could run from some sort of “fubu new –interactive” mode to visually discover and select the options you wanted to use in your own solution.  I also wanted to stop bundling the template library into the fubu.exe tool itself in favor of using a git repository that could be easily auto-updated by fubu.exe as we extended or improved our templates.

I thought we had a strong concept for the bootstrapping, but after getting one hard look at Typesafe Activator in Scala, it’s pretty obvious that we would never have been able to match that level of polish.

I also wanted to upgrade our standard testing tools from whatever old version of NUnit we’ve been using for years to something based on the excellent Fixie tool.

 

Conclusion

In a lot of ways, I think the .Net ecosystem — even though it’s over a decade old — is immature compared to other development platforms.  I feel like we had to do way too much bespoke infrastructure (Bottles for packaging web content, Ripple for a more usable Nuget story, FubuCsProjFile for csproj file manipulation) to pull off the “fubu new” story.  I wonder a bit if what we did might be easier in a couple years when Nuget matures and the new OneGet package manager gains traction.

I feel like Visual Studio.Net was a significant hurdle in everything we tried to do with our bootstrapping story.  I think .Net would be able to innovate much faster if our community would be much more accepting of lighter weight command line tools instead of demanding much more time intensive Visual Studio integration.

My colleagues at work and I are likely moving to the Play/Akka stack on Scala and a very common refrain around the office this week is excitement over being able to use lighter weight tools like Sublime and SBT instead of being forced to work with a heavyweight IDE like VS.