As an experienced software developer and the owner of a software tooling company I’d really like to continue to grow, I’m fully embracing the central role that AI assisted development now takes. I can’t say that I completely like how things have changed and we’re still clearly adapting as we go.
Just to complain a bit and try to make sense of our new world order, here’s where I think the bottle necks in software development have moved to, at least for me:
CI via GitHub Actions. We the entire OSS community are hammering GitHub in a probably unsustainable way and CI runs are now the ultimate bottle neck for Critter Stack development. This of course is after we got very aggressive with AI tools to expand the testing that runs in CI to the point where Wolverine currently has 46 CI gates and Marten has 32 (some of that is from matrix testing on different JSON serializers, .NET versions, and PostgreSQL versions). That explosion in usage plus GitHub implementing some rate limiting on the number of available action runners is definitely an impediment. More and more, I’m leaning back into using local testing runs to gate commits where I think that is suitably low risk. I think it’s also worth thinking hard about making CI runs be more selective about what CI gates are necessary for different pull requests or commits as another way of reducing usage.
I don’t know how I feel about this yet, but we’re occasionally opting for much bigger pull requests like this one that spans 8-9 issues. I know there’s a bit of a backlash against pull requests right now for plenty of reasons I’m quite sympathetic to, but pull requests provide tracking and visibility into progress and release notes for us. I’d like to keep using pull requests for that reason only for right now, but it’s something we need to evaluate changing.
Test execution time. Most of the Critter Stack tools almost have to be tested through their integration with infrastructure like databases and message brokers. To make that much worse, we have to frequently stand up and trash .NET applications with different configurations to test more use cases. The advent of AI has also made it much easier to build in far, far more test coverage to explore more permutations, “soak” tests, and other exploratory testing. Yea for more test coverage! Boo for the increased time to run test suites. Also, and this is a good problem to have, AI is making it much easier for our users to add actionable issues for normally hard to observe problems in the tools they find from their own exploratory testing (which is also easier to do now and you should!) — and those issues tend to require specialized testing harnesses to recreate their exact problem and tend to be expensive tests to run.
My own energy level. The increased mental overload from context switching, code reviews, and generally trying to make decisions fast enough and well enough to keep the agents spinning makes me burn out faster during the day. I take more breaks now than I did before.
Development box resources. I got unexpectedly lucky by needing to replace my main development box earlier this year right before memory prices spiked. In all seriousness, memory and CPU utilization for locally running Docker containers and our heavyweight test suites “limits” how many parallel agents I run — but that number is admittedly more than I can generally handle mentally anyway.
Nuget.org indexing time. I doubt this impacts many people nearly as much as it does me, but we’re frequently having to do cross repository work in the Critter Stack lately, and the latency of Nuget.org indexing newly published packages is noticeable, if better than I think it was this time last year.
Sorry, I don’t have a ton of answers to any of these problems or expect them to be the exact same issues you might be experiencing yourself. But hey, it’s at least good to start seeing where the bottlenecks are and concentrate our efforts to solving those problems first I guess.
And apropos of nothing, my colleague is lounging on the job again today and I’m pretty sure that anyone walking into my office is going to know right off the bat that this is the lair of a Gen X guy who finally has the posters he wish he’d had as a kid.
