Spitballing the Future of Storyteller

I just made a little tactical release of Storyteller v5.3.5 (it took a couple “fix” releases to really knock things out) that at least addresses the critical Nuget dependency issues users were experiencing against the latest versions. While in the code, it was quickly obvious that the Storyteller codebase was in some serious need of refurbishing and modernization. Hence, this post laying out some envisioning for the next wave of Storyteller development.

Storyteller is an OSS behavior driven development / executable specifications / integration testing tool I’ve worked on and led in some form or another for years. The obvious comparison is to Cucumber-based tools (like SpecFlow in the .Net world), but Storyteller was much more influenced by FitNesse and Ward Cunningham’s original FIT library from way back. While Cucumber has sucked up almost all the Oxygen in the room for tools like this, I think Storyteller’s model has some significant advantages over Cucumber tools for more data-intensive scenarios and does a lot more to help you tackle more complicated integration testing scenarios.

It’s never been super successful in terms of adoption or mindshare, but I and others do use it successfully. I personally enjoy working on Storyteller because it’s the only significant UI/UX work I get to do (my various day jobs have all wisely kept me out of any kind of UI design for the past decade).

Sometime later this year I’d like to turn my attention back to Storyteller to:

1. Overhaul and modernize the user interface. There’s definitely some room for ease of use improvements. There’s also some missing features that would help for the next bullet. Mostly though, it’s a way for me personally to level up on client side development as it’s been a couple years now since I’ve done any substantial Javascript work
2. Improve the usability of Storyteller for non-developers and make Storyteller work better throughout the entire software development lifecycle. Storyteller today is very oriented toward technical developers and automated testing more than true Behavior Driven Development as a requirements communication and capture mechanism
3. Find ways to parallelize specification execution. Storyteller specifications today have to run single file, and bigger suites of heavy integration tests have proven to be very slow to run in real usage (it’s not Storyteller itself so much as things like database access and any usage of Selenium). I’d like to explore some possible ways to parallelize execution in CI pipelines. More on this later in the post.

I’d love to attract some other contributors or folks that would be willing to play with early versions and give feedback. I’d especially love help and feedback on any new user interface work.

Storyteller’s Current Architecture

If you use Storyteller’s interactive user interface today in its latest incarnation (v5), there are three processes running including the browser. Here’s a quickie visual representation of the current architecture:

The Specification Project

The actual Storyteller specifications project is analogous to an xUnit.Net/NUnit/MSTest project in that it holds all the test fixture code and specifications (tests). In Storyteller v5, this project is a console application that delegates to Storyteller in its Project.Main() entry point method to perform any kind of application set up, tear down, and Storyteller configuration you may need.

This project probably has references to your actual application assemblies. In the case of an ASP.Net Core system, this might completely bootstrap and run your entire application. It also includes your Storyteller Fixture classes that sit in between the specifications and your application code.

This executable is all you need to run Storyteller specifications from the command line as you would in continuous integration builds. This same application runs as a test execution “agent” in a buddy process to the dotnet storyteller process when you’re executing specifications from the interactive specification runner. The dotnet storyteller process starts and stops the Specification project process so you can rebuild the system under test without having to stop and restart the Storyteller UI.

Offhand, I don’t think that the Fixture model will change too much in a new version and I think I’d like it to be a goal that your existing Fixture class code would compile immediately after an upgrade. I am very interested in doing a significant overhaul to how Storyteller bootstraps or shuts down the system under test and other environmental set up actions. So far I’ve identified a few opportunities for improvement:

• Replace the existing ISystem mechanism with a new fluent interface model that will closely emulate the ASP.Net Core WebHostBuilder model that I think most .Net developers will be familiar with in the near future.
• I bolted the async method behavior on to the Fixture model later on, and that shows. As part of the bullet point above, I want to do a big simplification of the actual specification execution internals
• Some facilities to start or stop Docker containers as part of the system bootstrapping
• A facility to start and stop arbitrary command line operations as part of bootstrapping. For an example, I have a project starting soon in an existing system where we’ll have to use IISExpress for hosting the web application locally, and I’m hoping this ability would facilitate full stack integration testing of that system by starting letting Storyteller manage the IISExpress process
• Meaningful setter properties on Fixture classes
• Some method injection in Fixture methods might be helpful
• Be better about passing around the CancellationToken for the current executing specification so you can be smarter about timeouts

dotnet storyteller

dotnet storyteller is a separate executable that is added to your project today through the dotnet tool extensibility. This tool runs a small ASP.Net Core application hosted locally and served via Kestrel (a “backend for frontend”). All of the client side assets (minus some on CDN’s) are embedded resources within this executable. This process communicates with the browser front end through JSON messages sent via web sockets. The communication with the running specification “buddy” process is done through JSON messages sent through raw sockets.

Honestly, more of the evident improvements will be in the actual user interface, but there’s a few things I’d like to do in the “Backend for Frontend” AspNetCore application:

• Incorporate Jasper for both the HTTP handling, an in-memory command processor, and the communication with the Specification process through sockets. Mostly this is a way to prove out Jasper, but it would also remove quite a bit of one-off code in Storyteller today.
• Possibly convert this to being a global tool rather than using it as a local dotnet tool.
• Maybe consider making this application be an Electron application? I’ve done enough research to know that’s feasible.

Storyteller Editor Browser Application

The actual Storyteller client running in the browser is a fairly complicated React/Redux application. After the initial requests for the HTML and client assets, all other communication between the browser application and the ASP.Net Core backend is through JSON messages sent via web sockets.

The current stack is:

• React.js v14 — the UI was started in v11, and you can unfortunately see a lot of different styles of building React.js components as React itself was churning so hard
• Redux + react-redux — I like this combination, especially on the receiving end of web sockets. This was all retrofitted in after the application work had started and I had some growing pains with this approach. I think the Redux work could be much simpler with a redo
• Webpack — it works great when it’s all set up, but it’s a brutal tool to use for someone like me who only moonlights in the JS world once in a while
• Immutable.js — It did what it was supposed to do, and I think it works pretty well in combination with Redux
• Postal.js — I think it was a great event aggregator in its day, but most of what it originally did in a homegrown Flux implementation was later subsumed by Redux
• react-bootstrap — Former colleagues of mine were active in this project at the time of Storyteller v3 development when the UI was rebuilt, and it was a natural fit
• Mocha/Chai/Karma for testing

I very briefly looked at possibly incorporating Blazor as the user interface tooling, but it’s model just doesn’t appeal much to me and it won’t have a huge amount of mindshare in the near future. I guess that WPF would be more viable in the .Net Core 3.0 stack, but I’ve never cared for it. Right now I’m leaning very heavily toward continuing to develop the user interface as a browser based application primarily built around React + Redex.

My current thinking about a future web stack is:

• React.js vLatest — I personally like React.js, and there’s plenty of usable code in the existing application that I don’t feel it’s justified to switch to Vue.js or anything else at this point
• Redux + react-redux — I looked at MobX and it’s interesting, but I actually like Redux and there’s existing code. I’m not buying into React’s Context API for everything
• Parcel.js — I don’t think Storyteller needs all the functionality of Webpack, and I’ve been very impressed with what I’ve seen with Parcel.js so far in admittedly limited prototyping
• Reactstrap or Material UI — I think that it would be easier to move to Reactstrap because the existing UI is built around Bootstrap and there are so many existing Bootstrap themes, but I like the editor support better in Material UI at a glance. I think I’m ever so slightly leaning toward Material UI and I’d love to hear from other folks.
• Jest + Enzyme for testing. Time to modernize, and I’d really like to leave Karma behind. I think I’d happily continue to use Chai though. I’d really like to be more familiar with the most recent approaches to testing React components before I need to oversee real client side development at work
• Typescript? I don’t think it’s absolutely necessary, but it would have helped the first time around in the more complicated data structures and the Redux reducers. Mostly this would be a chance for me to get into TypeScript I think.

Headless or Review Mode

Today you pretty well have to run your specification project to be working with Storyteller at all. That’s fine for developers who would naturally have all the prerequisites for the system under test on their box and be comfortable working with the code. That isn’t a great situation for the mass majority of testers and absolutely not at all good for non technical business experts.

For Storyteller v4, a couple former colleagues and I built some functionality where you could define all new grammar/fixture language for your project in a specialized markdown format, then be able to write specifications using the interactive editor using those stubbed grammars.

As an alternative to running fully connected against the “system under test”, I’d like Storyteller to also have a “headless mode” that would allow you to review and edit specifications, but not have to actually run the application. That architecture would look like this:

In this mode, Storyteller could run without the actual specification project process, so you’d need absolutely nothing but a web browser and the dotnet storyteller application running. You’d have to previously export the definition of the Fixture/Grammar language for your system under test to Storyteller-flavored Fixture markdown files.

• As an important follow up to that headless mode, I really want to embed “language designers” directly into the Storyteller UI that would allow users to make up fixture/grammar language usage on the fly.
• Storyteller already has some exposed functionality to generate the C# code for missing grammars and fixtures. My hope is that the combination of the headless mode, being able to define the specification language any time within the editor, and finally being able to generate skeleton code to match that language will hugely improve a user’s ability to use Storyteller as a requirements elicitation tool in addition to being an integration testing tool.

 

Parallelized Specification Execution

Folks have wanted some ability to parallelize Storyteller specification execution seemingly forever. I’ve come up with a couple ideas so far:

1. Use a model very similar to xUnit.Net where you can mark Fixture classes by whether they’re completely stateless or somehow “know” what fixtures can run simultaneously. The engine would then “know” how to schedule and parallelize executions
2. In the Storyteller bootstrapping, have some way to effectively set up multiple specification runners in the specification project. You might have to do some extra work in your setup to opt into this model. Things like using different database schemas per runner or other stateful resources
3. Docker all the things! For really big and slow specification suites, what if you could use something like an Azure Elastic Job to temporarily put out a whole bunch of parallel Storyteller specification runs, farm work out to all these runners, and combine the results in one place for an effective CI run. I’m assuming Docker would come into play as a way to spin up Storyteller agents.

 

What next?

Honestly, I need to get busy with Marten’s backlog first, but I really wanted to get some of these ideas out and onto paper, so to speak. I’d really like to gather as much feedback as possible from Storyteller’s current userbase and maybe try to attract some new folks.