Reading — Numbers Under Pressure

Numbers Under PressureBefore You Point at a Number · Rule 3

Before You Point at a Number

Rule 3Chapter I

List the three easiest ways to move the measure without improving the outcome, and judge the measure on that list.

Why this rule exists

Every proxy has decoupling routes; the question is only how cheap and how plausible they are. Forcing yourself to name three turns a vague worry into a concrete comparison you can make between candidate metrics. A measure whose three easiest cheats are hard, visible, and embarrassing is a much better measure than one whose cheats are a dropdown change.

Numbers Under Pressure

In practice

Do this in writing, with the people who run the process rather than only the people who designed it — they know the real shortcuts. Rate each route on effort, detectability, and how defensible it would look if written in an email. Compare candidate metrics on their worst route, not their intended use, and prefer the one whose cheats are expensive or conspicuous.

When it doesn't apply

If you can't find three routes, the metric may be unusually tight, or your group may be too far from the work; check with the people doing it before concluding it is safe.

Keep reading

5 of 17 · 25%