Do you recall Eliyahu Goldratt’s “Theory of Constraints” (ToC)? It’s a relatively simple (& industry-agnostic) management paradigm focused on workflow efficiency. Simple, yet with massive practical implications. The key theses of this paradigm (that is fully compatible with Systems' Theory) go something like this:

1. The efficiency of the (work) system is limited by a very small number of constraints (at least one).
2. There’s very little (or no) sense in optimizing anything but those constraints.
3. The optimization process should be iterative: after each iteration, reassess whether the constraints have changed, adjust your focus (if needed), and keep optimizing.

Yeah, that’s about it.

But why did I decide to resurface it now? I believe ToC is a very helpful tool when speculating on the future of building software systems in the post-LLM era. More on that below ...

ToC & building software

Let’s treat a software product development team as a work system. What determines its efficiency?

Side-note: Please notice that in this consideration I’m NOT interested in EFFECTIVENESS (what’s the value of what we’ve produced) but EFFICIENCY (how quickly/cheaply we’ve produced it). I agree that the former is more important than the latter, but the two aspects are highly independent and require separate consideration.

OK, so back to efficiency constraint … Obviously, that has to be something that:

1. is on the critical path (of the flow of work)
2. is time/resource-consuming
3. is scarce and/or expensive to scale (or scales out/up very ineffectively)

All pieces of the puzzle seem to click in the correct places - the main constraint of efficiency in writing software used to be (surprise, surprise!) … writing the code.

It is definitely the center of the critical path; it used to take a lot of time (as it was always the key component of all work estimations); options to scale it out were always limited (”9 women in 1 month …”, the cost of developers’ work hours due to supply-vs-demand imbalance, etc.).

New rules of the game

But this era is pretty much over. Due to continuously improving LLMs, the code writing (or rather: code generation) is now:

• incomparably faster
• apparently significantly cheaper (but LLM tokenomics is very unstable these days, so situation may change to some degree)

Does it mean we’ve removed the major constraint and need to reassess the work system's state again? Yes, definitely - that’s exactly my point, but it comes with a major caveat:

writing code in the pre-LLM era != generating code with LLMs

Long story short, it’s a matter of reliability and trust:

• People are more consistent and predictable (than Generative AI) - once taught a way to do something, they rarely “hallucinate” (as LLMs still do).
• People have much deeper & broader “implicit” context - due to the collective mass of their experience, they know (w/o saying) what’s unacceptable (e.g., dropping the production database).
• Manual code inspection (self-assessment, peer code review) was significantly faster than manual code writing (so the former was rarely a constraint) …
• … but 100% LLM-powered code inspection isn't reliable enough to guarantee production-grade quality/safety (in general), so we can’t fully “outsource” it to Generative AI.

As a result:

1. Code inspection can’t be fully delegated to LLMs/automations.
2. But it doesn't make sense to do it fully manually (it’s tedious and scales poorly).
3. Code inspection now requires a much more deliberate & sophisticated approach than before (automatic guardrails, evals, adversary techniques, competitive techniques, etc.).
4. The granularity and frequency of the feedback loop in “new” inspections changes: there should be more, but smaller iterations (for humans to match their cognitive load with LLM’s context window size).

What does it mean in practice? We’ve developed a very powerful, high-output brute-force mechanism for generating code: LLMs. But it needs much better means of control - which we still DON’T HAVE - as we’re in the early days of developing them (& good practices around them).

But it’s realistic to expect those new mechanisms to become the new efficiency constraint. In other words, how quickly we ship software won’t be determined by how fast we generate code, but by how long (& how many iterations!) it takes us to validate it.

Walking all this around

Frankly, I don’t expect us to tackle this new constraint anytime soon (unless we develop something much closer to real, self-improving AGI than LLMs). And that means the industry will try (naturally) to walk around this constraint. Here’s what I expect to see in the forthcoming few years:

1. Smaller granulation of software - single-purpose apps that do just one thing (to limit complexity, so it’s easier to validate).
2. Immutable software - because it’ll be simpler to re-generate (the new version of) software from scratch (a prompt) than modify/refactor the already existing version.
3. Short-lived, single-purpose software - I have a problem now, I create a tool to solve it, and ditch it afterward (as we did with Excel spreadsheets).
4. Private software (as opposed to SaaS) - limited liability, limited range of (negative) impact, better fit for the needs.
5. Re-discovering the value of standardized integration - safe & controlled data interchange will (again) become a billion-dollar problem.

I’m very far from saying the “traditional” SaaS will disappear - my take is that we will get far more “categories” of software. And the software creation will commoditize a lot, becoming a modern equivalent of past-era Excel spreadsheets.

This new era of “Shadow IT 2.0” will not only help us to create all the software we always wanted to use (but couldn’t get it, for any reason), but it will also have deep social implications - e.g., regarding trust in software.

Oh, interesting times are coming. Again.