You've probably heard the popular quote, "If you can't measure it, you can't manage it." attributed to W. Edwards Deming. Few disagree with the statement, but identifying and applying correct measurements (metrics) and making sense out of them is not trivial.
Some people have written full books on that topic; my ambitions are far less spectacular :) I intend to equip you with a very simple mental model that simplifies working with metrics in nearly any scenario. Frankly, I didn't invent this model. I daresay there's no single author, but you can find it in numerous books and articles. However, surprisingly few people seem to be familiar with it, so I've decided (out of pure laziness) to describe it in a blog post. Since now, I'll have a single place to refer people to ;)
The idea of this mental model is to classify metrics into particular groups (metric categories). Each group has different characteristics - it determines your impact on the metric (how direct it is), how the metric's value corresponds to (expected) business value, and what conclusions you may draw from it (e.g., to apply corrective actions).
The model
This model distinguishes the following metric categories:
- input (/action) metric - the measurement of effort - it quantifies the actions/behaviors that have been executed; you have full control over this kind of metric - however, there's no guarantee it will help you achieve higher-level goals (it may have no or even negative impact)
- output metric - the measurement of direct results of input actions; you don't control it directly, but the more effective tactic for inputs (efforts were chosen adequately and executed correctly), the higher the output value
- outcome metric - it measures what we want to get (in the end) due to outputs; your control over outcomes is very indirect - you perform inputs to get outputs, and these outputs (plus other, "environmental" factors) turn into outcomes (bear in mind - it can take time, there may be side effects and the causality may not be 1:1)
- Northern Star metric - it's a particular type of outcome metric; How does it differ from the other outcome metrics? In many cases, they can be (at least partially) contradictory, and their importance may vary (including the perception of this importance). So it's handy to have a single, "joint" metric that "contains" (or "covers") all other outcome metrics, so you can long-term base your strategic plans and reasoning on it
I believe you already see a pattern here.
Every organization (company, unit, or squad) should have its Northern Star metric (NSm). It has to represent its primary goal (of existence) and answer the most basic question - how are we doing on that? It can't be "too present" or "too visible". It should be measurable, specific, comprehensible, and non-conflicting (especially with your day-to-day activities/priorities).
Identifying NSm should be pretty straightforward (if you have a clear strategy, understand your value streams, and keep the proper organizational focus). But how to move from here on? Well, by crawling backward. First, you identify outcomes that contribute to NSm, then outputs that should drive these outcomes, and finally - inputs to generate outputs.
There's no single solution to solve this riddle. The further you get in this metrics tree (backward), the more options you get (to pick from), but also the less direct the impact on the final goal (NSm) and (at last but not least) the higher control over the metric (e.g., you have 100% control over input).
Reality check
Let's bring some examples to illustrate the idea.
Let's imagine you're a part of the sales organization that offers a SaaS product. The NSm of your unit may be "monthly sales of business subscription in geo Y". Outcome metric that contributes to this NSm may be "adoption of new feature X (paid extension)". The output that drives this outcome could be the "number of customers familiar/trained with the new feature X". And finally - this output is an effect of action (input) metric - "number of organized open training sessions".
And another one:
This time you're a Tech Leader of a big engineering team, and the NSm you guys agreed upon is the "number of successful product releases per week" (to minimize the cycle time needed to roll new features out). This NSm is supported by several outcomes, e.g., "average time of automated regression tests" or "%-tage of failed deployments". These outcomes are the product of particular inputs ("how many SPs are allocated on architectural improvement in our CI/CD pipeline") turned into outputs ("number of actual architecture improvements in the CI/CD pipeline").
Tips'n'tricks
OK, it looks deceptively simple. What to do to make sure you reap as much as possible from that model?
- Simplifying the consideration by having just the outcome metric (even if the best imaginable one), may turn out insufficient - especially if you're not meeting it and you lack clarity on how to get there.
- Suppose you have properly classified input, output, and outcome metrics (and relationships between them). In that case, you'll precisely see where's the leak (disproportion between what gets in and out) in your "pipeline". You'll be able to calculate the leverage, e.g., the increase in inputs by X% has caused a Y% increase in outputs - is that good enough? or should we try another approach?
- Having a crystal-clear Northern Star metric spurs innovative, out-of-the-box thinking; otherwise, you may be conceptually stuck with a single, given "input" -> "output" pipeline.
- Metrics are helpful as long as they are truly used as a primary management tool (single definition, up-to-date, adopted in communication by all involved parties)
- The metrics people do not have real ownership over or that do not represent reality (for any reason) are a huge organizational issue. Typically it's a testimony of either flawed culture or deep misalignment between various levels of management and individual contributors.
