EDIT: Someone on Twitter got upset about my “or an inferior IoC tool” comment. Most of you probably know that I’m also the author of StructureMap, and I did mean that as a joke. In a later post about IoC integration I’ll happily admit that making FubuMVC too StructureMap-centric was a problem with adoption (we did support Autofac and had Windsor support all but done as well). To be blunt, I think that the IoC tools in .Net are largely an interchangeable commodity item and any of the mainstream ones will get the job done just fine.
I’m still up for doing this series of technical lessons learned about FubuMVC to wring some value out of the whole process, but I feel like I’ve said enough mea culpas about our failure on documentation. From now on, instead of dog piling on me in the comments, could you just say “Docs sucked++” and then get on with whatever else it is you want to say?
Today’s blogpost is partially about the danger of deviating from .Net orthodoxy, but mostly a lamentation on my part on missed opportunities and unrealized potential. I’m actually quite proud and happy with most of what I’m describing in this post, but it was too late to matter much and might not have ever been widely accepted.
Quickstart from the Command Line
I’ve always admired elements of Ruby on Rails, especially the almost magical “rails new” project skeleton creation that sets you up with an entire code tree with a standard build script that exposes common build tasks for testing, database migrations, and even deployment recipes — but the word “standard” in that last sentence is a very key concept. Much of the value of “rails new” is enabled by standardizing the layout and naming conventions of a Rails codebase to make it cheaper to write reusable command line tooling.
We knew from the very beginning that we’d eventually want our very own “fubu new” analogue. Our community built a simple one initially that would just clone a pre-canned Git repository and do some simple string replacement for your project name. Great, and it added value immediately. However, the FubuMVC ecosystem got bigger as we built:
- Ripple as a better command line manager for Nuget packages in continuous integration
- Bottles as our modularization strategy, including a command line bottles tool to package up web content files as a pre-compile step in your build script
- FubuDocs as a command line tool to author and publish technical documentation
- fubu run as a Katana based development server that’s more efficient in development than anything IIS or VS based
- FubuTransportation as our FubuMVC based message bus
- A slew of command line testing tools
To unlock the usefulness of all those tools above to new users, not to mention just getting a simple application codebase up and running fast, we embarked on a new effort to create a vastly improved “fubu new” story that would allow you to mix and match options, different project types, and integrate many of those tools I listed above.
At the time of this post, you can stand up a brand new FubuMVC application using the Spark view engine from scratch to do grown up development by following these steps at the command line assuming that you already have Ruby 1.9.3+ installed:
- gem install fubu
- fubu new MyApp –options spark
If you do this, you’ll see a flurry of activity as it:
- Builds a new VS.Net solution file
- Csproj files for the main application and a matching test library
- Creates the necessary classes to bootstrap and run a minimal FubuMVC application
- Invokes gems to install Rake, FubuRake, Ripple, FubuDocs, and the bottles command line tool
- Invokes Ripple’s equivalent to Nuget Package Restore to bring down all the necessary Nugets
- Opens the new Visual Studio solution
The result is a code tree that’s completely ready to do grown up development with a build script containing these tasks:
rake ci # Target used for the CI server rake clean # Prepares the working directory for a new build rake compile # Compiles the solution src/MyApp.sln rake compile:debug # Compiles the solution in Debug mode rake compile:release # Compiles the solution in Release mode rake default # **Default**, compiles and runs tests rake docs:bottle # 'Bottles' up a single project in the solution with... rake docs:run # Runs a documentation project hosted in FubuWorld rake docs:run_chrome # Runs the documentation projects in this solution i... rake docs:run_firefox # Runs the documentation projects in this solution i... rake docs:snippets # Gathers up code snippets from the solution into th... rake myapp:alias # Add the alias for src/MyApp rake myapp:chrome # run the application with Katana hosting and 'watch... rake myapp:firefox # run the application with Katana hosting and 'watch... rake myapp:restart # touch the web.config file to force ASP.Net hosting... rake myapp:run # run the application with Katana hosting rake ripple:history # creates a history file for nuget dependencies rake ripple:package # packages the nuget files from the nuspec files in ... rake ripple:restore # Restores nuget package files and updates all float... rake ripple:update # Updates nuget package files to the latest rake sln # Open solution src/MyApp.sln rake unit_test # Runs unit tests for MyApp.Testing rake version # Update the version information for the build
This entire rake script is (I’ve added some explanatory comments for this blog post):
require 'fuburake'
@solution = FubuRake::Solution.new do |sln|
# This is unnecessary if there's only one sln file in the code
sln.compile = {
:solutionfile => 'src/MyApp.sln'
}
# This feeds the CommonAssemblyInfo.cs file we use
# to embed version information into compiled assemblies
# on the build server
sln.assembly_info = {
:product_name => "MyApp",
:copyright => 'Copyright 2013. All rights reserved.'
}
# These are defaults now, but I still left it in the
# template
sln.ripple_enabled = true
sln.fubudocs_enabled = true
end
# This one line of code below creates rake tasks as a convenience
# to run our development server in various modes
FubuRake::MvcApp.new({:directory => 'src/MyApp', :name => 'myapp'})
The terseness of the rake script above relies very heavily on a standardized code tree layout and naming conventions. As long as you could accept the FubuMVC idiomatic code tree layout, I feel like we succeeded in making our ragbag collection of command line tools easy to setup and use. Great, awesome, it’s done wonders for the Rails community and I’ve now seen and used working analogues with Scala’s Play framework and Mimosa.js’s “mimosa new” command. A couple problems though:
- There’s never been a successful effort in .Net to identify and codify idioms for laying out a code repository and we’ve found that many folks are loathe to change their own preferences (“src” versus “source”, “lib” vs “bin”, and so on). It might be easier if TreeSurgeon had succeeded way back when.
- I think it was just too little, too late. I think that OSS projects, once announced, have a short window of opportunity to be awesome or else. I felt good about the improved “fubu new” experience just in time for NDC London this past December — but that was almost 4 years after FubuMVC started.
- VS.Net templating was a lot of work and dealing with different project types, VS versions, and versions of .Net added overhead
- Nuget and our own Ripple tool are still problematic
- While common in other development communities, this is not a common approach for .Net developers
Why not Visual Studio Templates or Project Template Nugets?
We did support Visual Studio project templates, but I always felt that would only be for folks that just want to play with the framework a little bit. The feedback we got from other OSS projects that did invest heavily in VS templates was uniformly negative and I wasn’t enthusiastic about doing much with them. It was a tremendous amount of work to build our templating engine (FubuCsProjFile), but at least what we got was something easy to author and possible to cover with automated testing (a never-ending shortcoming of so many tools originating from Redmond).
We supported a project template with Nuget very early on and even finagled out a “install nuget, press F5” experience, but I always found it to be very limited and problematic. Again, testability was an issue. If I had it to all do over again, I’d still choose our command line approach with the mix and match selection of options (Spark or Razor? StructureMap or an inferior IoC tool? Want RavenDb support? Bootstrap? Html Conventions?).
The Polyglot Thing is a Double Edged Sword
There’s always some sort of meme running around in developer circles that’s meant to make the cool edgy kids feel superior to the unwashed masses. A couple years ago there was a lot of hype about polyglot programming where you would happily mix and match different programming languages and paradigms in a single solution based on their relative strengths.
For better or worse, the FubuMVC projects were a mild example of polyglot programming and that repeatedly scared people off. We used Ruby’s Rake tool for our project automation, we partially replaced Nuget with gems for distributing binaries that we used in the build scripts. For 2.0, we’re happily ditching our original asset pipeline in favor of the Node.js based Mimosa.js tool. At other times we also used the Python-based Sphinx for some early documentation efforts.
While I think that Rake is an outstanding tool and very easy to use, the simple need for a Ruby tool drove many people away — which is a shame because many other .Net OSS tools besides FubuMVC prefer Rake-based build scripts over the other alternatives.
It’s not hard to understand people being hesitant about having to learn non-mainstream .Net tools, it’s unfortunate. One of the big problems with trying to do ambitious OSS work on .Net is that .Net simply isn’t the home of the best tools. We repeatedly found Nuget to be lacking (a blog post for a later day) and MSBuild is, in my opinion, a complete non-starter for build automation.
As an aside, I’ve frequently seen members of the ASP.Net team complain on Twitter about having to install Ruby just to build some OSS project when they turn right around and require you to install all kinds of Visual Studio.Net add ons and templates just to build their code.
Aborted Plans for FubuMVC 2.0:
For 2.0 I wanted us to push forward with more templating options and introduce a new standalone web application that you could run from some sort of “fubu new –interactive” mode to visually discover and select the options you wanted to use in your own solution. I also wanted to stop bundling the template library into the fubu.exe tool itself in favor of using a git repository that could be easily auto-updated by fubu.exe as we extended or improved our templates.
I thought we had a strong concept for the bootstrapping, but after getting one hard look at Typesafe Activator in Scala, it’s pretty obvious that we would never have been able to match that level of polish.
I also wanted to upgrade our standard testing tools from whatever old version of NUnit we’ve been using for years to something based on the excellent Fixie tool.
Conclusion
In a lot of ways, I think the .Net ecosystem — even though it’s over a decade old — is immature compared to other development platforms. I feel like we had to do way too much bespoke infrastructure (Bottles for packaging web content, Ripple for a more usable Nuget story, FubuCsProjFile for csproj file manipulation) to pull off the “fubu new” story. I wonder a bit if what we did might be easier in a couple years when Nuget matures and the new OneGet package manager gains traction.
I feel like Visual Studio.Net was a significant hurdle in everything we tried to do with our bootstrapping story. I think .Net would be able to innovate much faster if our community would be much more accepting of lighter weight command line tools instead of demanding much more time intensive Visual Studio integration.
My colleagues at work and I are likely moving to the Play/Akka stack on Scala and a very common refrain around the office this week is excitement over being able to use lighter weight tools like Sublime and SBT instead of being forced to work with a heavyweight IDE like VS.
The Shade Tree Developer 
Sorry to see you leave .NET development. I think just not I’m only beginning to understand your grasp of programming. I think .NET’s problem is that there are a lot of 9-5 ‘programmers’. And organizations unwillingness to use OSS.
Let’s just say that it’s a little disheartening to do demos and get the feedback “that looks great, but we can’t use anything that isn’t from Microsoft”
I don’t understand why people can’t use anything not from Microsoft. I understand if they are already invested in ASP.NET MVC and don’t want to rewrite an app in something else.
I didn’t read this as “I’m leaving .NET,” but more as “I enjoyed my journey, loved some parts, didn’t love others; focusing on other platforms for now.”
Almost a year ago, I semi-intentionally started a project where I’m working exclusively with OSS stuff (Scala, Python, node) after doing .NET for almost 10 years. I had done side projects in other languages/platforms before, mostly Java and Ruby (Rake FTW!), but it’s been an refreshing and eye-opening experience.
Suffice to say, I’ve been using Scala for a year (doing real-time stuff with Storm), Akka is amazing, and I don’t feel like I’m missing out on something; if anything, exposure to new ideas will make me be a better developer no matter what platform I choose to build with next.
Have fun with the new toys!
Have you taken a look at F#? Would be interesting to hear feedback from an ex .NET-er that has used Scala. Seems like F# and Scala have a lot of similar ideas/goals.
We looked at F# for awhile, but the decision was really based on wanting to get off of .Net altogether. I don’t have a strong opinion about Scala vs. F# as a language. My opinion is that Akka is a killer feature for Scala. The Scala community seems to be much more mature than F#’s — but maybe that’s just my impression.
I think Rake for build scripts is a much better option than MSBuild. And frankly I don’t understand why people won’t use it. For example, we’re thinking about using Grunt.js tasks for building and deploying .NET applications – but that makes total sense since we already use Grunt to package and deploy Node.js app.
I suppose in general .NET developers don’t want to learn other languages. I was repeatedly asked about why I want to learn Scala if I know C# – and THAT says a lot. Not so many devs think outside the box, which is really sad – mostly they just use standard stack and don’t know about anything else.
I took a long look at using FuBu MVC. In fact I probably spend too much time looking at it. I disliked ASP.NET MVC (still do but much less) and wanted something else and I believed in Jeremy Miller and the rest of the FuBu Team Yes the docs sucked, but the kicker was I could never get any of the samples to run out of the box. Everything that was an example or sample didn’t run for me. If an example doesn’t run why should I switch over to FuBu MVC? Let alone trust it? Samples/Examples not runnable could have been for many reasons including (1) examples not using current codebase (2) examples using things that need to be installed or setup in Visual Studio. If the docs were updated I probably could have figured things out but they weren’t. We’ve moved on to ServiceStack.Net. They’ve done an excellent job. Samples/Examples work and it’s constantly being updated. What originally got us “sold” on ServiceStack.Net is that we could submit a pull request and usually that day or the next it was implemented, whether it was for a bug or a feature. FuBu MVC website made it look dead in the water with nothing current. So in summary, keeping recent activity on the website and runnable samples is the best approach for the next XYZ. PS> .NET still rocks not sure why all the ruby love. I think y’all complicated things way too much in certain aspects.
So that was *exactly* the kind of comment I was asking folks to stop making. For the infinite fucking time, I get it, we did a shitty job on docs, websites, examples, and sticker bombing every random passerby at technical conferences.
I guess I wasn’t trying to emphasize that, my bad and I apologize as that wasn’t my point. For clarity, “samples/examples” I’m referring to are VS project samples not doc samples/snippets. My main emphasis was that I couldn’t get any of the sample VS projects or example VS projects to run. I could care less about the docs if the samples/examples worked. I want to nuget/clone/download a project and it should just work, if not I may spend time or I may not. I **REALLY** wanted to use FUBU MVC. I HATED the initial ASP.NET MVC. I LOVE everyone on the FUBU team and follow just about everyone and why I spent A LOT of time trying to get the samples / examples VS projects to work.
I happened across your twitter feed with @jen20 @ventaur. We’re supposed to be doing KISS. You know, Keep It Simple Stupid… Making a dependency on a project to use an a package, like rake, outside does not make things simple. It just another thing that we have to learn and maintain. No OSS project should require a developer to use an external tool not already built into .NET, unless it resides in NuGet AND it is seamless to use in the experience. At least if they want any following from the core .NET developers. I have no problem if my team wants to use rake or grunt or whatever, but they shouldn’t make the project depend upon it. I look at it as an “unnecessary external dependency” that adds complexity and that will increases a developers initial “learning” phase of the project and cause stumbling during the project because something changes.
The very short answer is that nuget was terrible for distributing CLI tools, especially tools you want to use in your build. Chocolately is not cross platform, so we went with gems and by and large it worked out very well from the technical side of things.
Yeah I suppose it all depends upon the project. I have found that having the “no external tool” rule has made for some very elegant and simple solutions so far. I do think NuGet can be leveraged in a way that keeps simplicity while relying on external dependencies. Most people use Web Essentials 2013 by Mads and community. Most people wouldn’t know that it uses node to do some things. That is the right way to do an external dependency, i.e. not even know that it’s there.
We knew from the very beginning that we were never going to get much from mainstream .Net developers, and Rake is commonly used in the circles I run in.
Those should be bigger circles. I don’t get this aversion we have in the .NET community to change. Package managers for Ruby and NodeJS are miles ahead of Nuget. Instead of embracing tools like yeoman and bower we try and cram even more into Nuget.
FWI Rake/Ruby increases the ‘surface area’ or knowledge a .NET developer and the business needs to know. The business and developer team has now created a dependency on another language that now needs to be maintained and another vector for build/debug exceptions. Usually the person implementing Rake ends up being a single or two people and then that person quits or moves on making life difficult and setting the whole team and business behind. This dependency creeps into the hiring making things more difficult to find the appropriate hire. A large surface area is not agile IMHO, it just complicates everything including business processes and hires. It also increases training to get a new hire up to speed. I want efficiency and new hires to hit the ground running. Right now the only exception I have to this is NodeJS, b/c it’s fundamentally JS and my web developers know JS. I realize Austin TX may have access to polyglot developers, but not every metro area is Austin, SF, Washington, etc. If FUBU team absolutely needs to create a dependency then NodeJS should be used, not Rake. Also, checkout Mads Kristensen, et. al.SideWaffle.com Maybe that has a richer templating.
I’m not completely disagreeing with you, but think about this too:
* FubuMVC itself is not coupled to Ruby whatsoever. You don’t have to use Rake to develop a FubuMVC application. There is at least one team that develops w/ FubuMVC using Grunt (but they want to get off of .Net anyway)
* We use Rake internally, and it was the automatic choice for the *initial* support
* If we were to support a different build scripting tool, which one would you pick? Most of the .Net guys I know consider MSBuild to be a non-starter. Psake isn’t any more common than Rake. Fake is new and the new scriptcs based tools have no traction yet.
* I’m aware of side waffle, but I don’t think it completely overlaps with the templating we built into FubuCsProjFile. I wanted to look at Sidewaffle for micro-generation within VS down the road, but that’s obviously not going to happen now;)
* We are using Node.js tools for the asset work, but keep in mind that FubuMVC dates back to ’08 before node.js was a big thing. See my comment too about an OSS project only having a short window of opportunity to take off and we are way, way past ours
Thank you for your comments,
Jeremy
From your comments, I gather that you looked at F# FAKE, but didn’t got for it. You mention its recency as a reason for that; was it your primary reason? FAKE is awesome, and arguably introduces much less friction in a .NET project than Rake (no need to install Ruby or anything).
On a different note, thanks for taking the time to share your retrospective thoughts on Fubu. Technical content is great, but you don’t hear that often a larger perspective on longer projects, discussing what went well and not so well. So thanks!
I didn’t look very hard at Fake. We talked about possibly rewriting our FubuRake helpers into Fake as a learning exercise but it didn’t go anywhere. We already had a longstanding investment in Rake and I didn’t see any advantage to switching — especially if we were just gonna pack up and quit anyway;)
Ruby is a one time install. I have a hard time buying that it’s a source of friction. Having gems/bundler integrated into Rake was an advantage as well. I think the .Net tools lag a little bit on that one.
@cthames: your comment show the exact problem with .NET community and that is preferring short term gain with long term pain over the opposite. You and most devs like you just don’t realize how much easier it is to learn the tool and benefit from it in the long run. I guess it comes down to not realizing the pain you are taking each day with all the inferior tools from MS cause you don’t have any comparison with other ecosystems cause you never learn them and that’s the vicious circle you live in and even don’t realize that. You just take for granted what MS gives and never question or bother to peek outside for better solutions.
IMO I think anything NodeJS based would be the best option outside of MSBuild at this time. MSFT is using NodeJS, IIS has Node. Node runs MSFT services like Visual Studio Online. It would have the most MSFT support for the long term. Especially now with TypeScript, the build could be type safe with intellisense and other editors (like Sublime Text) support TypeScript so it wouldn’t just be VS only and could gain support by non .NET/MSFT people. Another reason for NodeJS is that it *might* be able to be integrated into Kudu. (https://github.com/projectkudu/kudu)
Note: We use Kudu (yes still uses MSBuild) on some small to medium websites that aren’t very complex and it works wonderfully publishing to Azure. We have a separate build process it goes through before it can be published but would be nice if later this could be integrated. I don’t know Kudu internals but I would assume this would be possible given uses NodeJS.
BTW to provide date time context of my FUBU MVC issues, I was testing FUBU MVC back right before the 1.0 version up till the 1.0 release, so yes I understand Rake was probably the best choice at that time.
FYI: Besides Grunt there is also Gulp. Not sure how they compare as I’ve only played with Grut sparingly, but both are js. https://github.com/gulpjs/gulp
There is also a session by John Papa if anyone is going to AngleBrackets in Orlando, FL in April 13-16, 2014 http://anglebrackets.org/sessions.aspx
@cthames: yeah, MS promised to create vsdoc files for jQuery when they put it in the web project template and… where are they? MS support, buahahahaah 🙂 And again you are leaning on MS so much and that’s why you’re afraid of Ruby and Rake 😛
I wanted to thank you for doing this series of posts. I’ve read both posts and all comments completely and it’s great, thought provoking stuff. You and your commenters are making me question and examine my own sacred cows about programming and .NET.