TL;DR Many IT people (mainly executives & various managers) are guilty of building the walls between business and engineering parts of organisation: by introducing gated demand management, communication via specification, cooperation through proxy people and siloeing the competences they hope to achieve more order & control, additionally reducing the operational risk. What they get instead is crappy products, nose-diving trust, peaking inertia, "local" ownership & problems with knowledge sharing. Building software-based products is all about tight & CONTINUOUS co-operation (incl. feedback) between Business & IT.

As we live in times where Agile Manifesto has almost the status of biblical ten commandments & you can't throw a stone without hitting any Scrum Masters, some fundamental principles of Agility may seem just too trivial to elaborate on. But strangely enough, reality proves such assumptions wrong - I've recently learned (the hard way) that even today, people still keep falling into one of traps I've faced myself (the painful way) in the very beginning of my professional journey. Sounds like a great candidate for a blog post - let me tell you a short story about the thing I got the most wrong when I've started building software (commercially), circa 16 years ago ...

"Stay awhile and listen ..."

What you should know about me is that I was always half-engineer & half-humanist. Abstract thinking & problem decomposition were always very natural for me, but so where also describing things with oral and written words. "Pouring" words into paper were never an issue, quite the contrary - I was able to shape thoughts into paragraphs of text fluently, w/o further rework, already with proper structure, narration flow, w/o repetitions or inconsistencies.

Everyone has some talents - this one was mine.

As I was aware of my strengths (plus very daring & super-confident), I've found this mixture of engineering & humanist skills perfect for creating top-notch software - IMO I had everything needed to:

  • extract the domain knowledge from sponsor / expert / client / user
  • create accurate, complete, multi-Level-of-Detail (from the general to the detail) specification of what (& how) has to be done (that also had to be decomposable into chunks of work)
  • plan & execute (or delegate to team-mates & monitor) work
  • collect the ticks on User Acceptance Test (UAT) checklist for all the items from the specification

It seemed so easy, practically unthinkable that it may have failed at any point.
I felt being born exactly for this job: all pieces were fitting in, I was totally in control.

All I needed others to do, was:

  1. to provide me knowledge to create the specs (experts/users)
  2. to create & test code based on my specifications (rest of the team)
  3. to confirm that what we've crafted meets these specs (experts/users again)

Evangelion

The specs were crucial.

They were the contract, the axiom, the foundation of reality, the "lever & place to stand to move the Earth" - pure essence of knowledge reflected by written characters. Good specs meant (IMO) that good software (as the result) was just inevitable.

And frankly, my specs were awesome. Objectively, possibly the best I've ever seen.

  • superbly formatted
  • sensibly structured
  • annotated with glossaries & F.A.Q.s
  • with multitudes of drawings & diagrams (incl. legends)
  • with a proper usage techniques for colour / shape / different kinds & sizes of fonts

I've always covered both positive & negative conditions, I was able to reflect top-level & in-detail perspectives in a way that enabled smooth transition. Everything had its ids, tags & mnemonics which I zealously kept up-to-date. Creating these documents were never a burden for me, quite the contrary - this was my idea of doing the best to make sure that the final product will be as awesome as possible.

I believed there were the key success factor. Building actual software based on these specs was supposed to be an easy part.

Once specification (we called it "Detailed Designs") was ready & signed off by people who have provided the knowledge & requirements (more about that later ...), I actually didn't need them (experts/users) - everything was in the specification already, so presence of these people would be just an annoyance, right? "You guys, just get back once software is completed, for a full round of User Acceptance Tests ..."

Epiphany

It's still a bit embarrassing, but looking back, after all these years, I can tell you for sure:

  1. The worst software products I've (& my teams) built were based on these super-detailed & signed-off specifications (let's call such situations "1st scenario").

  2. The best ones were based on actual hand-to-hand collaboration, when we've set ourselves a goal, time cut-off ("pool of mandays") & evolved towards it with feedback delivered (by users/experts) even before particular chunk of app was fully developed (I'' refer to that as "2nd scenario").

What was usually happening in situations based on the 1st scenario?

Users, sponsors or domain experts were just overwhelmed with the specifications - it was not possible to grok them through completely. Not because the author was smarter than them, but because author has modeled the specs in HIS intuitive manner, he went the full conceptual path himself & coded down what in HIS opinion was necessary to implement software. But this was NOT the full picture! It was an OUTCOME of the whole thought process, the rest remained in HIS head - the whole path he followed to get to this final step. To recreate this journey was just beyond capacity of these people - who had different backgrounds, different experience, different perception - it's FAR easier to propose a new solution (from scratch) than to fully understand, embrace & adopt someone else's one.

Soooo, they had this consultant (me), coming with skillfully crafted specification (60 ... 80 ... 100 pages, sometimes even more) they could not fully validate, but also couldn't reject as there were no obvious flaw! When they made spot validations of random places in it (which already required significant effort), they couldn't question anything - it was correct enough, from the perspective of their current knowledge & understanding of the problem. Which of course proved nothing - tiny details may be fine & in place, but it doesn't mean they make a coherent, valuable & usable product in total.

Needless to say, my teams were always able to develop according to these specifications. And the end-products were meeting everything that was written down up-front, but ... the effects were not exactly as the users have been expecting ...

Assumptions & statements that made sense during the Detail Design phase have been losing when confronted with reality. We were not able to predict some behaviours, effect, synergies. Usability limped. What was supposed to be easy & natural, was complicated & irritating. As technical people we were biased towards solutions that either were simpler for implementing or followed generalization principles / engineering design patterns - these rarely go aligned with real-life domains.

My teeth ache when I remind myself some crap we have been creating.
Always on time.
Always according to the Detailed Design.
And always proud of ourselves.

What was usually happening in situations based on the 2nd scenario?

I was taught the 2nd scenario (2006-2007) the hard way (as my initial reaction was to object):

"Just shut the f@#k up & do as told, you'll see it will work."

The key concept was "wygrzewanie" (it's the polish word) - direct translation do not make much sense, but these are the best English paraphrases (/analogies) that come to my mind:

  • "incubating" - sitting on eggs in order to keep them in optimal conditions for hatching
  • "annealing" - heating and cooling slowly the metal material, in order to toughen & harden it

The idea was to have business / user representatives directly in the teams full-time since the day 1. They were doing the actual (but informal) UATs continuously - when there was anything (half-)baked-enough to get any feedback, they were straight on it. And it was everyone's (both devs' & these users') obligation to make this happen & raise the red flag when (/if) it was not happening. It was the most important point in the (usually rather loose) "Definition of Done".

Q1: But wasn't it irritating for devs to have someone behind their back all the time?

Nearly-continuous feedback doesn't mean that business users were interrupting devs' work in random moments & demanding access to something what was still in disarray. Feedback was based mainly on what was deployed by Continuous Integration engine (refreshed several times per day), so all the devs had to do was keeping the WIP (number of features in development) limit low, making sure that their changes do not affect overall stability of test environment & being transparent about their near-future plans (so users knew what to expect in the forthcoming days & can plan accordingly).

Q2: How was it affecting the commitments (agreed scopes, plans & "deadlines")?

Indeed, there was always a chance of an infinite loop of grumbling, correcting & generally running in circles, BUT these projects were the joint endeavours (& joint responsibility) of both Business & IT, who worked TOGETHER. Each of the parties had the right (& obligation!) to escalate the issue - whether it was "it doesn't meet the goals! it is not usable!" or "we're not progressing! we won't have enough time to complete other crucial functionalities!" and these issues were resolved by joint force of operational managers from both (engineering & product) sides. And it worked. Why? Because there was no "WE & THEM" - responsibility was on BOTH PARTIES, project was owned by BOTH PARTIES, same success criteria were for BOTH PARTIES.

Screwing something (bad code, wrong decision, poor testing, conceptual gap) was LITERALLY meaningless when compared with the deadliest sin that could have been committed - not raising hand when shit was not being done (end-to-end, incl. this feedback). Resemblance to Toyota's Andon Cords is not accidental. This was of course not about pointing fingers & finding scapegoats, but about everyone feeling accountable for the final end-product.

Summarizing all that ...

  1. Building good software products is all about feedback (from all kinds of stakeholders). There is NO replacement for that, no short-cut. Just integrate early, validate early, validate often, rinse & repeat.
  2. Written functional specifications can be helpful as the supplement, but they don't invalidate the prior statement (about feedback) & their value provided does NOT increase with the level of detail covered (& the number of pages ...) - quite the contrary.
  3. Specification of deltas (changes) tends to get outdated very quickly, it's hard not only to browse & search through (after some time, when you get next wave of deltas), but also it's far more challenging to understand as this requires re-building (conceptually) the context that reigned at the moment of introducing the change. What's much more effective is documenting the conceptual MODEL of the domain (To-Be state, evergreen), preferably as a Living Documentation, directly linked to code & its metadata.
  4. "Waterfall-izing" (sequentializing) stages of software delivery is by far not as bad as splitting & separating responsibilities of not only technical specialties (e.g. front-end & back-end), but also technical VS business specialties - this is a team sport, there should be NO local success criteria.
  5. The more distance you put between "requestors"/users & "providers"/developers, the worse is the final product (& the more time it takes to deliver it); this distance doesn't have to be physical (kilometres), but it is frequently also about:
  • communication via intermediaries (proxy people)
  • gating requirements (in complex, multi-step "demand management" processes)
  • shift in time (something was specified months ago, doesn't reflect reality of today)
  1. Continuous feedback has 1 more important "side effect" - a healthy pressure for concluding things; instead of having everything 90% done until the very last hours of sprint/release/project, business people & users create a tension to mark items as complete ("production-ready"), so they don't have to be re-visited & team can focus on next items from backlog ...

As a conclusion of this (far too lengthy, sorry) blog post, one more comment. I've heard zealots of formal, gated approach who've been mocking the "annealing" approach by saying that:

"There's nothing that prevents devs from asking business people a question or asking them to "click through" something as a form of early validation. No-one will retaliate if they do, they know that, but still - they do not do it, so this continuous-coop mumbo-jumbo clearly doesn't work!

But documents for formal approval have their deadlines, agreed formats, they follow the User Story pattern - rules are clear for everyone, people know their roles & places, they can plan their work better, there are fewer distractions, disrupting changes (!) & overall chaos."

The answer is freaking simple. The role of a leader/manager is to shape / adjust the work system (work environment) to support positive behaviors & practices; to remove the impediments & obstacles; to dissolve to comfort zones that do nothing but restrain people from achieving their potential. If you leave a dysfunctional system & just tell people: "you're free to do it using the best principles & the most catchy slogans", it will just not happen - they won't feel real organisation's support, but they will feel environmental resistance instead.

Share this post