Context is Important

I’ve been thinking recently about how Agile changed software development in important ways and getting back to some of the important lessons I felt that we as an industry learned from Extreme Programming back in the day. Depending on my time and ambition level, I might try to spit out some essays on team organization, feedback, engineering practices, and continuous delivery. It should be implied anyway, but I need to be clear that all the contents of this blog are my opinions alone and do not reflect that of my employer (Calavista). I.e., I’m the only one to blame;)

I’m hopefully near the end of what’s been a very frustrating project. There’s more to the story of course, but I ascribe many of the challenges we have faced on the decision to execute the project with a true waterfall lifecycle. Being forced back to waterfall development has naturally led me to reflect on what I think are the crucial differences between that model and Agile development techniques and remember just why Agile development was such a revolutionary movement (before it was all watered down by Scrum, but that’s a subject for another day). That reflection includes feedback cycles, team and organization mechanics, project management, and reversibility — but for now, let’s just talk about context.

Picking up the definition of “context” from Dictionary.com:

the set of circumstances or facts that surround a particular event, situation, etc.

In terms of software development, understanding the “context” of your week means knowing how your work fits into the greater tableau of the system and ecosystem. It also means understanding the “why” behind the technical decisions that have been made so far and the architectural approach.

It turns out that understanding the context around the software system you are building is very important. Thank you, Captain Obvious right?

Now, let’s try to use the word “context” in some sentences about software development:

• As an architect, I make more effective decisions when I understand the context of how our work fits into the greater enterprise ecosystem. One of the things I think I’ll be doing in the next couple weeks is to give a little presentation to our client on how our dev lead and I think the architecture or our project should be streamlined in the future by consolidating responsibilities, simplifying the topology, and reducing integration chattiness. Why didn’t I and others just make better decisions upfront you ask? Conway’s Law* is a partial explanation, and our future recommendations assume that the organization in question will eliminate that artificial barrier in the future. Mostly though, we made decisions based very early on a limited view of a specific initiative in isolation. Now that we know much more about other system interactions and related projects going on at our client, we see a path for a much simpler and more reliable architecture — but we could only get to that point by seeing the bigger picture rather than dealing with “build this console app to move data from point A to point B.”
• As any kind of technical leader, I can better set up other developers for success by explaining the context around their tasks. I learned a brutal lesson early as a technical lead. The more specific instructions I gave to the other developers working on our project, the worse the code they wrote came back. When I just had a discussion with developers about what we were trying to achieve and how their tasks tied into the greater workflow of the project, the results were very obviously better. It turns out that developers can much better make decisions when they understand the “why” of the work that they’re doing.
• As an OSS author and maintainer, I do a much better job helping users with questions when I understand what the users are trying to accomplish rather than just trying to answer specific API questions. It’s time consuming to ask a lot of questions, but the results are usually better in the end. It does lead to a lot of “why would you possibly need to do that?” questions on my part before I understand what their scenario really is. It’s not uncommon to give much easier solutions to their real issues once I know the context behind the original question. That really depends on catching me at a moment when I’m not too busy or stressed out by other things and the other person has enough patience to go back and forth, but hey, it’s an imperfect world.
• As a developer, I’m much faster fixing bugs and addressing questions when the context of the problem is fresh in my mind. I distinctly remember making the argument back in the early to mid-00’s that developers would be able to fix bugs much faster in an Agile process where testing happens simultaneously and the code is always fresh in your mind versus waterfall development where you’re frequently needing to address code you haven’t worked with in several months. After going back to waterfall development the past 12 months, I can attest to this argument being very much a true fact.
• As a tester, I can more effectively troubleshoot problems and write actionable defect reports when I understand the environmental context of the functionality I’m testing. Things inevitably go wrong, and good testers help solve those problems when they can investigate the failures, find the relevant logs, and give developers the information they need in order to further diagnose or fix the problems.
• As an architect or any kind of technical leader making technical recommendations and guidance to others, I should explain the “why” and context behind any guidance. This is absolutely vital because my experience has been that folks will eventually face scenarios where your guidance or advice does not apply very well or might even be flat out harmful. You can’t just say “always do this or that.” You need to explain why you gave the recommendations to hopefully let folks know when they should break out and look for other solutions rather than force fitting a “standard architecture.” My strong advice is to always include some context about why any particular reference architecture came about so that developers can be more effective rather than blindly following along.
• As a developer (really as any possible role), I am less effective when I have to deal with a lot of context switching during the work day. I’ve long been in roles where I need to try to answer questions from other folks and I’ve always prided myself on my ability to quickly shift contexts and help folks out, but it definitely wears on me and I’m not as sharp as I am when it’s questions about something I’m actively working on at the time. Personally, I see my productivity lately going down the drain as I’m having to do an unusual amount of context switching between different projects.

Context and the Agile vs Waterfall Argument

I’m both a long time advocate of Agile methods and a long time critic of waterfall methods, and the issue of context and context shifting really drives home the differences to me. I feel like there’s both much more sharing of project context in Agile teams and less context shifting than you frequently see in waterfall organizations.

In an Agile team there’s simply much more face to face interaction (or at least should be) between different disciplines as opposed to waterfall style toss-the-documents-over-the-wall communication. You would hope that would lead to much more shared understanding of why decisions are made and understanding of the context around the team’s work.

In a typical Agile shop, you would try to create a self-contained, multi-disciplinary team that is completely focused on one stream of work. Developers, analysts, scrum masters, testers, and whatnot all working together and bearing down on a single backlog.

In an extreme waterfall shop**, a project team may really be a larger, virtual team of individuals who are all simultaneously working on other unrelated work streams. After all, waterfall theory says that the analysts should be able to hand over the requirements document and then move on to another project. Testers shouldn’t even come into play until all the coding is done, so they’re not even around while the requirements are being formulated and the initial coding is happening. The architect, if you have one, might not be fully engaged after the initial technical specification was handed off (dunno if any organization still tries to do that, but I experienced that in my first role as an architect).

The folks in a waterfall shop are probably doing a lot more harmful context shifting than their counterparts in an Agile shop. The waterfall folks may also be getting much less information about their project because it’s constrained through documentation rather than through higher bandwidth, interactive discussions between disciplines. If you’ve never encountered this before, I highly recommend taking a read through Alistair Cockburn’s paper on Communicating, Cooperative Teams.

 

* Just a reminder that Conway’s Law is neither good, bad, or an anti-pattern. It simple is. Better shops will be aware of Conway’s Law and purposely and intentionally organize themselves to support their desired enterprise architecture.

** My description of a waterfall shop may sound like a straw man argument, but it really exists even today and I’ve run into this model in multiple big shops now.