Category Archives: Uncategorized

Storyteller 4.0 is Out!

Storyteller is a long running project for authoring human readable, executable specifications for .Net projects. The new 4.0 release is meant to make Storyteller easier to use and consume for non technical folks and to improve developer’s ability to troubleshoot specification failures.

After about 5 months of effort, I was finally able to cut the 4.0 Nugets for Storyteller this morning and the very latest documentation updates. If you’re completely new to Storyteller, check out our getting started page or this webcast. If you’re coming from Storyteller 3.0, just know that you will need to first convert your specifications to the new 4.0 format. The Storyteller Fixture API had no breaking changes, but the bootstrapping steps are a little bit different to accommodate the dotnet CLI.

You can see the entire list of changes here, or the big highlights of this release are:

  • CoreCLR Support! Storyteller 4.0 can be used on either .Net 4.6 projects or projects that target the CoreCLR. As of now, Storyteller is now a cross platform tool. You can read more about my experiences migrating Storyteller to the CoreCLR here.
  • Embraces the dotnet CLI. I love the new dotnet cli and wish we’d had it years ago. There is a new “dotnet storyteller” CLI extensibility package that takes the place of the old ST.exe console tool in 3.0 that should be easier to set up for new users.
  • Markdown Everywhere! Storyteller 4.0 changed the specification format to a Markdown plus format, added a new capability to design and generate Fixture’s with markdown, and you can happily use markdown text as prose within specifications to improve your ability to communicate intentions in Storyteller specifications.
  • Stepthrough Mode. Integration tests can be very tricky to debug when they fail. To ease the load, Storyteller 4.0 adds the new Stepthrough mode that allows you manually walk through all the steps of a Storyteller specification so you can examine the current state of the system under test as an aid in troubleshooting.
  • Asynchronous Grammars. It’s increasingly an async-first kind of world, so Storyteller follows suit to make it easier for you to test asynchronous code.
  • Performance Assertions. Storyteller already tracks some performance data about your system as specifications run, so why not extend that to applying assertions about expected performance that can fail specifications on your continuous integration builds?

 

Other Things Coming Soon(ish)

  • A helper library for using Storyteller with ASP.Net Core applications with some help from Alba. I’m hoping to recreate some of the type of diagnostics integration we have today with Storyteller and our FubuMVC applications at work for our newer ASP.net Core projects.
  • A separate package of Selenium helpers for Storyteller
  • An extension specifically for testing relational database code
  • A 4.1 release with the features I didn’t get around to in 4.0;)

 

How is Storyteller Different than Gherkin Tools?

First off, can we just pretend for a minute that Gherkin/Cucumber tools like SpecFlow may not be the absolute last word for automating human readable, executable specifications?

By this point, I think most folks associate any kind of acceptance test driven development or truly business facing Behavioral Driven Development with the Gherkin approach — and it’s been undeniably successful. Storyteller on the other hand, was much more influenced by Fitnesse and could accurately be described as a much improved evolution of the old FIT model.

SpecFlow is the obvious comparison for Storyteller and by far the most commonly used tool in the .Net space. The bottom line for me with Storyteller vs. SpecFlow is that I think that Storyteller is far more robust technically in how you can approach the automated testing aspect of the workflow. SpecFlow might do the business/testing to development workflow a little better (but I’d dispute that one too with the release of Storyteller 4.0), but Storyteller has much, much more functionality for instrumenting, troubleshooting, and enforcing performance requirements of your specifications. I strongly believe that Storyteller allows you to tackle much more complex automated testing scenarios than other options.

Here is a more detailed list about how Storyteller differs from SpecFlow:

  • Storyteller is FOSS. So on one hand, you don’t have to purchase any kind of license to use it, but you’ll be dependent upon the Storyteller community for support.
  • Instead of parsing human written text and trying to correlate that to the right calls in the code, Storyteller specifications are mostly captured as the input and expected output. Storyteller specifications are then “projected” into human readable HTML displays.
  • Storyteller is much more table centric than Gherkin with quite a bit of functionality for set-based assertions and test data input.
  • Storyteller has a much more formal mechanism for governing the lifecycle of your system under test with the specification harness rather than depending on an application being available through other means. I believe that this makes Storyteller much more effective at development time as you cycle through code changes when you work through specifications.
  • Storyteller does not enforce the “Given/When/Then” verbiage in your specifications and you have much more freedom to construct the specification language to your preferences.
  • Storyteller has a user interface for editing specifications and executing specifications interactively (all React.js based now). The 4.0 version makes it much easier to edit the specification files directly, but the tool is still helpful for execution and troubleshooting.
  • We do not yet have direct Visual Studio.Net integration like SpecFlow (and I’m somewhat happy to let them have that one;)), but we will develop a dotnet test adapter for Storyteller when the dust settles on the VS2017/csproj churn.
  • Storyteller has a lot of functionality for instrumenting your specifications that’s been indispensable for troubleshooting specification failures and even performance problems. The built in performance tracking has consistently been one of our most popular features since it was introduced in 3.0.

 

Marten 1.3 is Out: Bugfixes, Usability Improvements, and a lot less Memory Usage

I just uploaded Marten 1.3.0 to Nuget (but note that Nuget has had issues today with the index updating being delayed). This release is mostly bugfixes, but there’s some new functionality, and significant improvements to performance on document updates and bulk inserts. You can see the entire list of changes here with some highlights below.

I’d like to thank Marten contributors Eric Green, James Hopper, Michał Gajek, Barry Hagan, and Babu Annamalai for their contributions in this release. A special thanks goes out to Szymon Kulec for all his efforts in both Marten and Npgsgl to reduce Marten’s memory allocations.

Thanks to Phillip Haydon There’s a slew of new documentation on our website about Postgresql for Sql Server folks.

What’s New?

It wasn’t a huge release for new features, but these were added:

  1. New “AsPagedList()” helper for fetching documents by page
  2. Query for deleted, not deleted, or all documents marked as “soft deleted
  3. Indexes on Marten’s metadata columns
  4. Querying by the document metadata

What’s Next?

The next release is going to be Marten 2.0 because we need to make a handful of breaking API changes (don’t worry, it’s very unlikely that most users would hit this). The big ticket item is a lot more work to reduce memory allocations throughout Marten. The other, not-in-the-slightest-bit-sexy change is to standardize and streamline Marten’s facilities for database change tracking with the hope that this work will make it far easier to start adding new features again.

The Different Meanings of “I take pull requests”

Years ago when I was in college and staying at my grandparent’s farm, my uncle rousted me up well after midnight because he could see headlights in our pasture. We went to check it out to make sure no one was trying to steal cattle (it’s very rare, but does happen) and found one of my grandparent’s neighbors completely stuck in a fence row and drunkenly trying to get himself out. I don’t remember the exact “conversation,” but his vocabulary was pretty well a single four letter expletive used as noun, verb, adjective, and adverb and the encounter went pretty quickly from potentially scary to comical.

Likewise, when OSS maintainers deploy the phrase “I take pull requests,” they mean a slew of very different things depending on the scenario or other party.

In order of positive to negative, here are the real meanings behind that phrase if you hear it from me:

  • I think that would be a useful idea to implement and perfectly suitable for a newcomer to the codebase. Go for it.
  • I like that idea, but I don’t have the bandwidth to do that right now, would you be willing to take that on?
  • I don’t think that idea is valuable and I wouldn’t do it if it were just me, but if you don’t mind doing that, I’ll take it in.
  • You’re being way too demanding, and I’m losing my patience with you. Since you’re clearly a jerk, I’m expecting this to make you go away if you have to do anything for yourself.

Introducing Alba for integration testing against ASP.Net Core applications

My shop has started to slowly transition from FubuMVC to ASP.Net Core (w/ and w/o MVC) in our web applications. Instead of going full blown Don Quixote and writing my own alternative web framework like I did in 2009, I’m trying to embrace the mainstream concentrate on tactical additions where I think that makes sense.

I’ve been playing around with a small new project called Alba that seeks to make it easier to write integration tests against HTTP endpoints in ASP.Net Core applications by adapting the “Scenario” testing mechanism from FubuMVC. I’ve pushed up an alpha Nuget (1.0.0-alpha-28) if you’d like to kick the tires on it. Right now it’s very early, but we’re going to try to use it at work for a small trial ASP.Net Core project that just started. I’m also curious to see if anybody is interested in possibly helping out with either coding or just flat out testing it against your own application.

A Quick Example

First, let’s say we have a minimal MVC controller like this one:

    [Route("api/[controller]")]
    public class TextController : Controller
    {
        [HttpGet]
        public string Get()
        {
            // I'm an MVC newb, and I'm sure there's a better way
            HttpContext.Response.Headers
                .Append("content-type", "text/plain");

            return "Hello, world";
        }
    }

With that in place, I can use Alba to write a test that exercises that HTTP endpoint from end to end like this:

    public class examples : IDisposable
    {
        private readonly SystemUnderTest theSystem;

        public examples()
        {
            theSystem = SystemUnderTest.ForStartup<Startup>();
        }

        public void Dispose()
        {
            theSystem.Dispose();
        }


        [Fact]
        public async Task sample_spec()
        {
            var result = await theSystem.Scenario(_ =>
            {
                _.Get.Url("/api/text");
                _.StatusCodeShouldBeOk();
                _.ContentShouldContain("Hello, world");
                _.ContentTypeShouldBe("text/plain");
            });

            // If you so desire, you can interrogate the HTTP
            // response here:
            result.Context.Response.StatusCode
                .ShouldBe(200);
        }
    }

A couple points to note here:

  • The easiest way to tell Alba how to bootstrap your ASP.net application is to just pass your Startup type of your application to the SystemUnderTest.ForStartup<T>() method shown above in the constructor function of that test fixture class.
  • Alba is smart enough to set up the hosting content path to the base directory of your application project. To make that concrete, say your application is at “src/MyApp” and you have a testing project called “src/MyApp.Testing” and you use the standard .Net idiom using the same name for both the directory and the assembly name. In this case, Alba is able to interrogate your MyApp.Startup type, deduce that the “parallel” folder should be “MyApp.Testing,” and automatically set the hosting content path to “src/MyApp” if that folder exists. This can of course be overridden.
  • When the Scenario() method is called, it internally builds up a new HttpContext to represent the request, calls the lambda passed into Scenario() to configure that HttpContext object and register any declarative assertions against the expected response, and executes the request using the raw “RequestDelegate” of your ASP.Net Core application. There is no need to be running Kestrel or any other HTTP server to use Alba — but it doesn’t hurt anything if Kestrel is running in side of your application.
  • The Scenario() method returns a small object that exposes the HttpContext of the request and a helper object to more easily interrogate the http response body for possible further assertions.

Where would this fit in?

Alba itself isn’t a test runner, just a library that can be used within a testing harness like xUnit.Net or Storyteller to drive an ASP.Net Core application.

One of the things I’m trying to accomplish this quarter at work is to try to come up with some suggestions for how developers should decide which testing approach to take in common scenarios. Right now I’m worried that our automated testing frequently veers off into these two non-ideal extremes:

  1. Excessive mocking in unit tests where the test does very little to ascertain whether or not the code in question would actually work in the real system
  2. End to end tests using Selenium or Project White to drive business and persistence logic by manipulating the actual web application interface. These tests tend to be much more cumbersome to write and laborious to maintain as the user interface changes (especially when the developers don’t run the tests locally before committing code changes).

Alba is meant to live in the middle ground between these two extremes and give our teams an effective way to test directly against HTTP endpoints. These Scenario() tests originally came about in FubuMVC because of how aggressive we were being in moving cross cutting concerns like validation and transaction management to fubu’s equivalent to middleware. Unit testing an HTTP endpoint action was very simple, but you really needed to exercise the entire Russian Doll of attached middleware to adequately test any given endpoint.

How is this different than Microsoft.AspNetCore.TestHost?

While I’ve been very critical of Microsoft’s lack of attention to testability in some their development tools, let me give the ASP.Net team some credit here for their TestHost library that comes out of the box. Some of you are going to be perfectly content with TestHost, but Alba already comes with much more functionality for common set up and verifications against HTTP requests. I think Alba can provide a great deal of value to the .Net ecosystem even with an existing solution from Microsoft.

I did use a bit of code that I borrowed from an ASPNet repository that was in turn copy/pasted from the TestHost repository. It’s quite possible that Alba ends up using TestHost underneath the covers.

We’re hiring senior developer/architects

EDIT 1/5: We’re still hiring for Salt Lake City or Phoenix. I can probably sell a strong remote candidate in the U.S., but I can’t get away with remote folks in Europe (sorry).

 

Here’s the job posting.

We’re (Extend Health, part of Willis Towers Watson) doing a little bit of reorganization with our software architecture team and how it fits within the company. As part of that, we’re looking to grow the team with open slots in our main Salt Lake City office and our new Phoenix office. We might be able to add more remote folks later (I’m in Austin, and another member is in Las Vegas), but right now we’re looking for someone to be local.

Who we’re looking for

Let me say upfront that I have a very conflicted relationship with the term “software architect.” I’ve been a member of the dreaded, centralized architect team where we mostly just got in the way and I’ve had to work around plenty of architecture team’s “advice.”  This time around, I want our new architecture team to be consistently considered to be an asset to our development teams while taking care of the strategic technical goals within our enterprise architecture.

More than anything, the architecture team needs to be the kind of folks that our development teams want to work with and can depend on for useful advice and help. We’re not going to be landing huge upfront specifications and there won’t be much UML-spewing going on. You will definitely be hands on inside the code and it’s likely you’ll get to work on OSS projects as part of your role (check out my GitHub profile to get an idea of the kinds of work we’ve done over the years).

You’re going to need to have deep software development experience and been in roles of responsibility on software teams before. You’re going to need to have strong communication skills because one of your primary duties is to help and mentor other developers. A good candidate should be thoughtful, always on the lookout for better approaches or technologies, and able to take on all new technical challenges. It’s not absolutely required, but a healthy GitHub or other OSS profile would be a big plus. The point there is just to look for folks that actually enjoy software development.

You’ll notice that I’m not writing up a huge bullet list of required technical acronyms. I’m more worried about the breadth and depth of your experience than an exact fit with whatever tools we happen to be using at the moment. That being said, we’re mostly using .Net on the server side (but with a heavy bias toward OSS tools) and various Javascript tools in the clients with a strong preference for React.js/Redux in newer development. We do a lot of web development, quite a bit of distributed messaging work, and some desktop tools used internally. Along the way you’ll see systems that use document databases, event sourcing, CQRS, and reactive programming. I can safely promise you that our development challenges are considerably more interesting than the average .Net shop.

Marten 1.2 — Improved Linq support and way more polish

Marten is a library for .Net that turns Postgresql into a document database and event store.

I just published the Marten 1.2 release to Nuget. While I hoped to fit a lot more new functionality into this release, 1.2 really just adds a lot more polish to Marten by fixing several bugs, makes some performance improvements based on my company’s trial by fire usage of Marten during our peak “season”, and by largely reworking the internals of the Linq support.

Marten continues to have a vibrant community of interested folks and contributors that are helping push the project on. Probably missing some names, but I’d like to call out James Hopperjokokko, Barry Hagan,  Alexander Langer, and Robin van der Knaap for their contributions to this release. I’d also like to thank all of you who have opened and commented on Github issues to help improve Marten. If this all keeps up long enough, I may finally stop being so cynical about OSS on the .Net platform;)

Here’s the entire list of changes from the GitHub milestone. The highlights of the 1.2 release are:

  • Support for the SelectMany() operator in Linq queries (this story spurred an absurd amount of rework in our Linq support that I think will make it easier to add more features in subsequent releases)
  • Distinct() Linq query support
  • Named parameter usage in user supplied queries
  • Better logging and exception messages
  • Marten’s sequential Guid algorithm was corrected to order consistently with Postgresql. This should result in better write performance in Marten usage with Guid id’s.
  • Marten tries harder to warn you when you use unsupported Linq operators
  • Several improvements to querying against child collections
  • The ability to use event metadata in the built in aggregation projections
  • Cleaned up some of the database connection mechanics to stop mixing blocking and async calls and makes Marten much more aggressive about closing database connections

 

What’s Next?

I’m not 100% sure I want to commit to another new release before the holiday season, but 1.3 is looking like it’s going to be a lot of improvements for querying against multiple documents, new types of Select() transformations, and working over the internals to optimize performance.

The tentative list of 1.3 enhancements can be seen here.