“Measurement is the first step that leads to control and eventually to improvement.
If you can’t measure something, you can’t understand it.
If you can’t understand it, you can’t control it.
If you can’t control it, you can’t improve it.”
— H. James Harrington
Measurements are a tool that appear in many approaches to help get results. This is something that can be found in both self-improvement techniques, as well as in business with KPIs and analogous approaches.
Measurements act as a focus tool: it’s not just what we measure, but also about all the other things that we can ignore because they’re not influencing the measurement. It could also be argued that measurements in this sense are effective because they force people to do a deep dive in what actually they are trying to do, and hold themselves accountable to it.
Unfortunately, there’s also strong evidence that measurements can be dangerous:
“The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
— Campbell’s Law
“When a measure becomes a target, it ceases to be a good measure.”
— Goodhart’s Law
What Campbell’s Law describes is a known phenomenon: how many social policies went south because the measurement was wrong? How many stories of sales teams that did bad sales because they were just after the numbers and associated bonuses? How many managers are caring more about the share price than about the business they are running?
This apparent contradiction is resolved adding another variable: time.
In the short term we can afford to ignore everything but the thing being measured. Focusing on it is effective.
However, once some time passes the measure becomes stronger than the goal it was meant to improve and people slowly start forgetting about the true end goal. They hyper-focus on the metric alone: corruption appears.
This degradation is rarely intentional: imagine a group of people that have a goal. To focus, they write down a metric and set up the tooling to measure it. They all know what the original goal is, so they can use the measurement effectively.
But over time, it’s the measure and not the original goal that keeps appearing in reports, messages, communications, and discussions. Revisiting the goal often isn’t done. As such, slowly, the group changes. The measurement takes over.
The effect is accelerated even more if the original people that set the goal change. This happens easily in businesses with new hires coming in and senior people moving to different endeavors. It’s not just limited to businesses – this happens to any group.
Another strong accelerator of this effect are cheaters: people that play the metric to their own advantage. No metric ever that is immune to cheating, sometimes with dreadful consequences (see: Cobra Effect).
So, how do you balance these two contrasting dynamics? Keep reminding everyone involved the original goal, review, and possibly change the metrics every few months.
One approach that does this and is already well described is OKRs:
- They have a qualitative goal.
- They have specific metrics associated.
- They are changed regularly.
Even if you don’t want to use OKRs, there’s at least one think you can do: change any metrics that is too old. This will force to regularly re-evaluate whether what you measure still helps you achieve the results you’re aiming for.
- NHS “four-hour targets”, set in 2005, working for a few years, then failing to drive results around 2010. While there are many other variables at play, still it’s a good example on how time works in changing the effectiveness of targets, and with examples of how the corruption of the metric happens.
Thanks to Leif Singer for the inspiration and review.