This blog post is all about: cognitive bias (one particular one), how quickly we get used to "the new normal", how being insatiable when it comes to knowledge can make us ashamed, why we get annoyed when some junior pops up with "microservices now!" idea, why leader in the trenches may help (& what does it really mean).

Habit of fooling ourselves

You've heard/read about cognitive bias, didn't you? If not, you should definitely read up - there's zillion of resources on them available on-line, e.g.:

To keep the long story short, cognitive bias are frequently encountered (across whole humanity) thinking fallacies at the junction of perception, feelings and intellect. I like 2 other definitions as well:

"Inherent thinking errors that humans make in processing information." (RationalWiki)

and

"A systematic pattern of deviation from norm or rationality in judgement." (Wikipedia)

Dunning-Kruger Effect

There are many of them (cognitive bias), but today I'd like to focus on a single, particular one - so-called "Dunning-Kruger Effect" (in short: DKE) - it's an observed psychological phenomenon, according to which ...

  1. ... inexperienced, beginner people tend to notoriously overestimate their abilities and expertise ...
  2. ... AND experienced, highly skilled veterans habitually underestimate their capabilities & efficiencies

It's a good moment for a pause - what would you say? Have you observed any of these 2 effects? In yourself? Your co-workers? Anyone else?

I'd like refer to the 2nd part of the DKE here - the one that affects experienced, knowledgeable, highly skilled (aka "senior") specialists - architects, engineering managers, technical area leaders, etc.

"Normal" != "Normal"

So, my observation is that in many cases the bigger the knowledge/experience/skill gap (between these veterans & the rest of the team/organisation), the more these veterans are struggling with the collaboration & the less clear (for them) it is to find why.

Let's try to figure it out then.

Humans in general & engineers in particular have a huge problem with switching to someone else's perspective ("to get into someone else's shoes") - we're egocentric by nature & we assume everyone's pretty much like us - e.g. if something is obvious for me, it should be also obvious to you.

A person who've mastered the engineering to some degree have probably read A books, finished B projects, attended C talks/conferences, completed D programming katas, spent E hours on skunkworks that never got released (but was a learning lesson anyway), etc.. Really passionate engineers are pretty much insatiable - it doesn't matter how big are A,B,C,D,E, ... they are still looking for more. And MORE. As the main output from learning is ... more questions & deeper understanding of our own ignorance.

So their freshly updated state of knowledge/experience becomes "the new normal" & they are looking for more - "the new normal" is not satisfactory for them:

  • it's obvious, it's easy (now, once it has been mastered)
  • they see its deficiencies, gaps - they already know the questions they'd like to have answered

The problem is that thanks to DKE we think that our "normal" is everyone else's "normal":

  1. "it's so easy/interesting/important, so probably everyone else knows it already as well"
  2. "this conclusion is so logical, I can't believe I didn't come up with it earlier - everyone else probably did, bah"
  3. "this topic was presented on few conferences already, I'm probably the last one who've watched it/read up on it ... what a shame"

We were all juniors once

That's why we get so annoyed when a fresh team member comes up with a recently heard idea of splitting everything into microservices (& a face flushed with excitation ;>):

"Seriously? Doesn't (s)he understand what's the price? That we lack monitoring capabilities? That we need clean modular split first? That we have too many internal dependencies? That it would only increase hosting & operations costs, because our problems are somewhere completely else? Come on, it's OBVIOUS!"

We completely forget we were exactly at this very place (of a bespoken fresh team member) not more than a year (or two, or three, ...) ago ...

DKE's effect(s) on collaboration

OK, so sometimes we're delusional about what others know (& what they don't) - what's the fuss?

Things get really bad once we have to collaborate:

Veterans drive the path/the vision using the terms, concepts & (what's the most important ...) the capital of accumulated professional experiences they've got - with the level of detail THEY NEED, without clearly documented "thinking route" (how they got to the final conclusion(s)) - all of that assuming that "it's easy, no big deal, everyone should get that as it wasn't even really challenging to come up with all that".

Other members of the team ... pick up the work & execute it as well as they can - many of them not even realising we have an issue here. Why so?

  1. many are over-optimistic - yet too ignorant to see the obstacles before hitting the ground ... (the 1st sub-variant of DKE effect)
  2. some fall into Rashomon Effect trap (& don't bother to clear the hasty assumptions out up-front)
  3. all the others are so used to the culture of machismo in Software Engineering that they'll never admit they don't understand something / need help with something ("I'll learn on the way", "I'll check on web in the evening") - it'd be the sign of weakness & no-one wants "to expose the underbelly to lurking predators" ...

Yea, we - humans - suck in so many ways when it comes to collaboration.

Countermeasure

You know all the negative effects that follow, right? Irritated veterans are accusing others of being distracted, not paying attention, low commitment, etc. In the end they "save the day" by doing the challenging stuff themselves ("it's faster & more reliable that way"), effectively siloing the knowledge & skills even more. Gap widens, the junior team members get frustrated because they are not learning, stuck in doing the repetitious tasks - they also feel they have no visibility (& ofc no impact) on "the big picture".

There's one simple way to fix this.

Super obvious, almost "cliche-ish", but for many: too inconvenient, too unnatural, slowing down, pulling us out of comfort zone.

LEAD IN THE TRENCHES.

Lead by working with people directly. Don't let your seniority make you work LESS with other people - in fact do it MORE. Forget about being an individual contributor (and individual efficiency/productivity) - regardless of how smart your are you can up-scale yourself to a certain limit, but you can out-scale yourself upon whole team (& more) in much easier & effective way.

  • you've come up with a design idea -> pair-program with someone to illustrate it with an example.
  • you've identified a risk/issue -> host an ad-hoc hands-on mini-workshop so others can "burn their hands" as well
  • you're distributing "obvious" (to you ...) work -> start with asking others to describe it with their very own words, provide first feedback after 1 hour of their work (instead of 1 week ...), make sure that 1st tasks will be about sketching something that requires a deeper understanding - e.g. crafting the intention-based interface

You'll be amazed how ... painful it will be in the first place. There will be so many questions (many of them surprising), so many misalignment & misunderstandings, knowledge gaps to be filled, etc.

But IT'S GOOD!

You've read all those books, tried all those techniques/technologies yourselves - now what is left is just to pass this knowledge in the comprehensible, approachable way to your co-workers, so you guys can build something TOGETHER ;P. It's a necessary learning exercise. When would you prefer these things to come out instead?

Share this post