Retrospective on my OSS Career

Tomorrow is my last day at Willis Towers Watson (WTW) after 5+ years, and I felt like a bit of a retrospective was in order — partially to convince myself that I actually accomplished something in that time;) I did plenty of other things while I was there, but supporting and building OSS tools has been a major part of my official role.

It’s stupid long, but I think this is a decent timeline of the highlights and lowlights of my OSS career. I think I’m about to slow way down on OSS work after this year and in my new role, so I kind of want to reflect back a little bit on what I have done the past 15 years or so before turning the page.

2002-2004

I was in a miserable position as a non-coding architect at a Big IT shop, and I was scared to death that that meant my career was already over. I came up with a crazy scheme to build out an awesome new ORM for this new .Net thing as a way to prove to prospective employers that I really could write code. That work went nowhere, but I did actually manage to land a different job at ThoughtWorks.

From following the internal discussion lists there I learned about dependency injection and the very early PicoContainer IoC tool that had been built by some folks at TW. I realized that I could repurpose some of the code from the wreckage of my failed ORM project to be an IoC tool for .Net. Add some encouragement from TW colleagues, and StructureMap was born (and to this day, refuses to die).

2008-2011

The details aren’t important, but I went through some dark, then much happier but chaotic, personal times and work — especially OSS work — was an escape for me.

I made the big StructureMap 2.5 release that Jimmy Bogard took to calling my Duke Nukem Forever release that was supposed to be StructureMap’s “Python 3000” release that fixed all the original structural problems and put it permanently on a stable base for the future.

Narrator’s Voice: Everybody including Jeremy hated the new StructureMap 2.5 configuration DSL, he immediately added better alternatives that weren’t documented, people yelled at him for years about the missing documentation, and StructureMap was certainly not “done”

A couple of us at Dovetail started FubuMVC, an alternative framework for ASP.Net because we thought we were smarter than the real ASP.Net team working on MVC and had a different vision for how all of that should work.

We also rebooted my older Storyteller project I had worked on for years as improved tooling for the old FitNesse testing engine, and finally moved it toward being a full blown replacement for FitNesse.

 

2012

I did a big pre-compiler workshop at CodeMash 2012 that I thought went great and I was encouraged.

I joined WTW at the end of 2012 after my one and only (and last) startup experience turned sour. The corner of WTW where I work was at the time the biggest user of FubuMVC — the very large, very ambitious OSS web framework that was my own personal white whale for a long time. A couple of us believed that this would be a perfect opportunity to keep FubuMVC going as an OSS project because we genuinely believed that it would be successful enough that we would be able to make a living doing consulting for shops wanting to use it.

At the time, we were still building new features into FubuMVC that were almost immediately being put into real projects and it was a fun time.

Narrator Voice: It did not turn out the way Jeremy planned

2013

Some of my WTW colleagues and I did a big workshop at CodeMash 2013 on the big FubuMVC 1.0 release that flopped hard and I failed to reach the obvious conclusion that the gig was up and it was time to move on. Having stayed up all night the night before reading A Memory of Light didn’t help either, but c’mon, we’d waited 20+ years to get to the end of that thing.

I poured a lot more energy that year into FubuMVC for command line tooling and a full blown templating engine that made for some really cool conference talks that year, but didn’t really get used that much.

The big win for 2013 was building out an addon called FubuTransportation that used a lot of the basic infrastructure in FubuMVC to be a fairly robust service bus that still underpins quite a few systems at WTW today. That was my first real exposure to asynchronous messaging, and I take some significant pride in what we built.

2014

I basically had to admit that FubuMVC had failed as an OSS project and I was admittedly kind of lost the rest of the year. I wrote a lot about why it failed and what I thought we had learned along the way. The biggest lesson to me was that I had never done a good job promoting, explaining, or documenting FubuMVC and its usage. Part of my theory here is that if we’d had more usage and visibility early, we could have more quickly identified and addressed usability issues. I swore that if I ever tried to do something like FubuMVC ever again that I’d do that part of the project much better.

It did give me a chance to swing back to StructureMap and finish the big 3.0 release that made some sweeping improvements to the internals, improved a serious performance problem that impacted our systems, killed off some old bugs, and fixed some pretty serious flaws. I genuinely believed that the 3.0 release would put it permanently on a stable base for the future.

Narrator’s Voice: Jeremy was wrong about this

I also got to play around with an eventually abandoned project called pg-events that was meant to be a Node.js based event store backed by Postgresql. It didn’t go anywhere by itself, but it absolutely helped form the basis for what became the Marten event sourcing functionality that’s actually been a success.

Later that year we started seeing some significantly encouraging information about “Project K” that eventually became .Net Core. All of that made myself and my main contributors much more bullish about .Net again and in a fit of pure stubbornness, I started to envision how I would build a much better version of FubuMVC that took advantage of all the new stuff like Roslyn and fixed the technical flaws in FubuMVC. I started referring to that theoretical project as “Jasper” after my original hometown in Missouri.

2015

My shop was in border line insurrection over how our Storyteller integration testing was going. I gave a big talk outlining what I thought some of the challenges and problems were including options to switch to SpecFlow or just use xUnit, but to my surprise, they chose to just have a much improved Storyteller that became the Storyteller 3.0 release.

I pretty well did a near ground up rewrite of the testing engine focusing on performance and ease of use. For the user interface, I used this work as a chance for me to learn React.js that we had just adopted at work as a replacement for Angular.js. I had a blast doing UI work for the first time in years and I’m genuinely pretty proud of how it all turned out. Storyteller 3 went into usage later that year and it’s mostly still going in our older projects that haven’t converted yet to .Net Core.

In late 2015, we knew that we needed to get our biggest system off of RavenDb before the next busy season. My boss at the time had a theory that we could exploit Postgresql’s new JSON support to act as a document database. I took on the work to go spike that out and see if we could build a library that could be swapped in for our existing RavenDb usage in that big application. The initial spiking went well, and off it went.

At one point my boss asked me what name I was using for the Postgresql as doc db library, because he was concerned that I’d choose a bad name like “Jasper.Data” — which of course was exactly what the project name I was temporarily using. I mumbled “no,” quickly googled what the natural predators of ravens are, and settled on the name “Marten.”

Because of the bitter taste of FubuMVC that still remained, I swore that I would do a much better job at the softer things in an OSS project and tried to blog up a storm about the early work. The Marten concept seemed to resonate with quite a few folks and we had interest and early contributions almost off the bat that did a lot to make the project go faster and be far more successful than I think it would have otherwise.

I still wasn’t *completely* done with FubuMVC, and did a pretty large effort to consolidate all the remaining elements that we still used at work into a single library in the FubuMVC 3.0 release. I spent a lot of time streamlining the bootstrapping as well for quicker feedback cycles during integration testing. A lot of this work helped inform the internals of Jasper I’ll talk about later.

In late 2015 Kristian Hellang worked with me to make StructureMap work with the new ASP.Net Core DI abstractions and compliance specifications. While we were doing that, I also snuck in some work to overhaul StructureMap’s type scanning support based on years of user problems. With that work done, I pushed the StructureMap 4.0 release in the belief that I had now overhauled everything in StructureMap from the old 2.* architecture and that it was done for all time.

Narrator’s Voice: Jeremy was wrong. While the 4.* release was definitely an improvement, users still managed to find oddball problems and the performance definitely lagged newer IoC tools

2016

We used Marten in production, just in time for our busy season. It had about the expected level of issues pop up in its shakedown cruise, but I still felt pretty good about how it went. Adoption in the outside world steadily creeped up and I got to do several podcasts that year about Marten.

Unfortunately, Marten caused quite a bit of conflict between myself and our centralized database team that ultimately contributed to me deciding to leave. I lost some enthusiasm for Marten because of this, and my activity within the Marten community declined because of it.

2017

OSS wise, this year was going to be all about Jasper for me. FubuMVC was a web framework first that had messaging functionality bolted on later. Jasper on the other hand, was going to be a much smaller tool to replace the older FubuMVC/FubuTransportation messaging in a way that would play nicely with the newer ASP.Net Core bits.

First though, I needed to clean my plate of all other outstanding work so I could concentrate on just Jasper:

  • Oakton was a bit of leftover fubu code we’d used for years for command line parsing that I converted to .Net Core and documented
  • Alba is also some leftover fubu code for declarative testing of HTTP handlers that I adapted for .Net Core, documented, and published to Nuget
  • Storyteller 4 moved Storyteller to ASP.Net Core. That turned out to be a huge chunk of work, but it was a great learning experience. I also added quite a few improvements for usage by non-developers that might not have paid off.
  • Storyteller 5 took a little better advantage of the new dotnet cli and made debugging specifications a lot easier
  • Marten 2.0 was a huge effort to reduce object allocations within Marten and improve runtime performance. It also cleaned up some internal trash and made it a lot easier going forward to add new types of database schema objects that have definitely paid off

Finally, I got some time to bear down and start working on Jasper that’s really been my main passion project for 3-4 years. I had a bunch of new colleagues in our architecture team at work that were interested in Jasper, and I thought we made a huge amount of progress fast. I didn’t do as much work to publicize it like I did with Marten because I just didn’t have a good feeling about my company’s continued support for Jasper after what happened with Marten.

2018

I decided I was absolutely fed up with supporting StructureMap and it wasn’t viable to make the large scale changes it would take to fix the performance and other remaining issues. As an offramp for SM users and as part of Jasper, I started the year by yanking some code out of Jasper into a new library I first called “BlueMilk,” and now Lamar that is meant to be a much more performant and smaller replacement for StructureMap.

I’m also working full speed right now on Jasper with an expected 0.8 and 0.9 update coming in the next couple weeks. I don’t have a lot of interest yet, but I’m feeling very, very good about the technical direction, the usability, and the performance right now. I’m not 100% sure what WTW will do with it, but I’m committed to continuing with both Jasper and Lamar. I’m aiming for a 1.0 release of both by this fall.

And Beyond…

Several folks have asked me what will happen with Jasper or Marten when I start my new position next week. I honestly don’t know, but I fully intend to continue supporting Jasper, Marten, Storyteller, and Lamar. I don’t expect that I’ll have the bandwidth to write nearly as much OSS code as I have the past 5 years or so at WTW, but I wanted to slow down anyway.

 

What I’ve Learned

Heh, maybe not enough. I learned a lot of specific technical lessons and I’m a far better technical developer because of my OSS work. For more specifics though, I’d say:

  • Only do OSS work if you’re passionate about it and generally enjoy what you’re doing
  • My OSS work has absolutely had a positive impact on my career, just indirectly. My interview for my new position included a presentation on Marten. My previous position came about directly because of my OSS projects
  • It’s also an opportunity cost if you could have been learning something valuable in the time you spent on OSS work
  • I’ve met a lot of cool people through my OSS work and I have relationships I wouldn’t have had otherwise
  • Make your documentation easy to update and easy to contribute to by other folks
  • Talk about what you’re doing early and often

 

 

 

Advertisements

5 thoughts on “Retrospective on my OSS Career

  1. skoon

    ” reading A Memory of Light didn’t help either, but c’mon, we’d waited 20+ years to get to the end of that thing.”

    Truth! (sniff)

    Reply
  2. Pingback: Dew Drop – May 11, 2018 (#2723) | Morning Dew

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s