How I’m Documenting OSS Projects

This post talks about the “living documentation” authoring support in Storyteller 3.0. I’m working toward finally making an official 3.0 release of the completely rebooted Storyteller tool just in time for a webinar for JetBrains on the 21st (I don’t have the link yet, but I’ll blog it here later). Expect as much Storyteller content as I can force myself to write for the next month.

OSS projects succeed or fail not just upon their technical merits or ease of usage. Effective documentation and samples matter too, and this is something that I haven’t always done well. When I restarted work on Storyteller last year I made a pledge to myself that any new OSS work that I attempted would not fail due to bad or missing documentation. I’m going to claim that you can already see the result of that attitude in the latest online docs for Marten, StructureMap, and Storyteller.

Doing Documentation Badly

I have unfortunately earned myself a bad reputation for doing a very poor job of documenting my OSS projects in the past:

• I tried gamely to create comprehensive documentation for StructureMap as part of a big 2.5 release in 2008, but I foolishly did it with static html and the API very quickly got out of sync with the static content and that documentation probably caused more harm than good by confusing users. Only in the past couple months has the StructureMap documentation finally gotten completely updated for the latest release.
• I cited the lack of quality documentation as the primary reason why I think that FubuMVC, the single largest effort of my technical career by far, failed as an OSS project. Sure, there were other issues, but more documentation might have led to many more users which surely would have led to much more usable feedback and improvement to the tooling.

 

Better Documentation with Storyteller 3.0

After my experiences with StructureMap and FubuMVC documentation, I knew I needed to some kind of “living documentation” approach that would make it easy to keep the documentation in sync with an API that might be rapidly evolving and relatively painless to incrementally update and publish. In addition, I really wanted to be able to get contributions from other folks with the documentation content. And finally, I wanted to be able to host the documentation by using GitHub gh-pages, and that means that I needed to export the documentation as static HTML. I’m not doing this yet, but since it might be nice to embed the HTML documentation inside of Nuget’s or in a downloadable zip file, I also want to be able to export the documentation as HTML that could be browsed from the file system.

To that end, the new Storyteller has some tooling inspired by readthedocs (what ASP.Net uses for the vNext documentation now) to author, publish, and easily maintain “living documentation” that can stay in sync with the actual API.

The key features are:

1. The actual documentation content is authored as Markdown text files because that’s now a de facto standard for technical documentation, many developers understand it already, and it does not require any special editor.
2. Storyteller can derive a navigation structure from the markdown file structure with or without some hints to make it easier to grow a documentation website as a project grows
3. A system for embedding code samples taken directly from the actual source code explained in the next section
4. A preview tool that will allow you to run the documentation project in a browser that auto-reloads based on your edits. The auto-reloading mechanism scans for changes to code samples and content files like CSS or custom JS files in addition to the Markdown content files. The preview tool has some keyboard shortcuts as well to open the underlying Markdown page being rendered in your default editor for .md files.
5. “Skinning” support to theme your documentation site with some preprocessors to enable navigation links using the navigation derived from the file structure (next/preview/home etc.). If you look at the sample documentation websites I linked to at the beginning of this post, you’ll see that they all use roughly the same theme and layout that is very obviously based on Bootstrap. Storyteller itself does not require Bootstrap or that particular theme, it’s just that I’m not the world’s greatest web designer and I kept reusing the first theme layout that I got to look okay;-)
6. A command line tool for quickly generating the final HTML contents and exporting to a directory. This tool supports different publishing modes for generating internal links for hosting at an organization level (http://structuremap.github.io), at the project level (http://jasperfx.github.io/marten), or for browsing from a file system. In real usage, I export directly to a clone of a gh-pages Git branch on my box, then manually push the changes to GitHub.

 

Code Samples

The single biggest problem I had with technical documentation in the past has been embedding code samples into HTML files. Both from the standpoint of how awkward that was in the past and in keeping the code samples up to date with changing API’s — and API’s tend to change fast when documentation efforts inevitably reveal deficiencies in that API.

The approach I use in Storyteller 3 is to make the documentation pull code samples directly out of the actual code — preferably from unit test code. By doing this, you pretty well force the documentation code samples to be synchronized with the actual code API’s. As long as the tests holding the code samples pass in an automated build, the code samples should be valid. You can see the result of this approach most clearly in the StructureMap docs.

Mechanically, Storyteller pulls this off by scanning code files in the repository and looking for comments in the actual code that marks an embeddable code sample. In C#, that looks something like this:

        // SAMPLE:  ActionMethod
        [FormatAs("Start with the number {number}")]
        public void StartWithTheNumber(int number = 5)
        {
            _number = number;
            say();
        }

        // END:  ActionMethod

In a Markdown file, I can embed the code sample above with a preprocessor like so:

<[sample: ActionMethod]>

When Storyteller generates the HTML from the Markdown file, it will embed the textual contents between the // SAMPLE and // ENDSAMPLE comments above in a <div /> that is formatted by Prism.js.

Today I’m supporting code samples from C#, HTML files, Xml files (boo, right?), and JavaScript. It’s not really much effort to add other languages if that’s valuable later (I used to support Ruby too, but we were going to move away from it and I dropped that).

Iterating from Failure == Success?

As an aside, I think that achieving great results in most software projects is more about iteration and responding to the feedback from early attempts than about crafting a great plan or having the perfect idea upfront. In the case of the Storyteller documentation generation, I learned a great deal from some earlier attempts inside the FubuMVC ecosystem to solve the same kind of living documentation solutions called FubuDocs and FubuMVC.CodeSnippets. Both of those projects were failures in and of themselves, but if the Storyteller documentation generation turns out to be successful, it will be directly attributable to what I learned from building those two earlier tools.