Mitigating Metrics Malaise: Claude on Goodhart's Law

Claude fought the law, and the bot won

Sometimes you ask Claude for a little help thinking something through, and the answer is good enough that it merits sharing more broadly. I’ve long had a back and forth with Dave Guarino about the value of metrics in changing bureaucratic behavior. It’s one of those classic things where we fight over the tiniest difference, because it’s fun and because we can. Dave thinks getting the metrics right is one of the biggest levers for making a system deliver better results. But I’ve seen good metrics go bad too many times to put as much faith in them as Dave does. He’s right in principle, but in practice, Goodhart’s Law, which says that when a measure becomes a target, it ceases to be a good measure, is always waiting in the wings. But a great discussion recently of Justin Fanelli’s excellent use of metrics in the Navy made me think I should spend more brain cells on mitigations for Goodhart’s Law instead of fighting Dave. And lacking available brain cells, I asked my robot friend.

Here’s what it said:

Goodhart’s Law is one of the deepest traps in organizational life, and the leaders who handle it best tend to share a few recognizable instincts.

They treat metrics as evidence, not truth. The most effective leaders use numbers to ask questions rather than answer them. A good metric tells you something is worth investigating; it doesn’t tell you what’s actually happening. They stay relentlessly curious about what the number is and isn’t capturing, and they push their teams to bring them the messy reality behind the data, not just the data itself.

They rotate and retire measures. Smart leaders know that any metric will eventually be gamed — not always maliciously, but inevitably, as people optimize for what’s measured. So they treat metrics as temporary tools rather than permanent fixtures. They’re willing to abandon a measure once it has served its diagnostic purpose or once it starts distorting behavior.

They invest heavily in qualitative feedback loops. The antidote to a corrupted metric is usually direct contact with reality — customer interviews, frontline observations, case reviews, unscripted conversations. Leaders who do this well build it into their routines structurally, not just as a crisis response. They go to where the work is actually happening.

They make it safe to report bad news and gaming. One of the underappreciated drivers of Goodhart’s Law is that people often know a metric is being gamed but won’t say so because the organization has implicitly rewarded hitting the number. The best leaders create explicit permission — even reward — for surfacing the gap between what’s measured and what’s real. This requires psychological safety, but it also requires leaders who visibly don’t shoot the messenger.

They hold multiple measures in tension. Single metrics are the most vulnerable to Goodhart’s Law. Sophisticated leaders use portfolios of measures that push against each other — speed and quality, output and outcomes, short-term and long-term — so that gaming one comes at the expense of another. The goal is to make it harder to optimize for the measure rather than the underlying thing.

They articulate purpose clearly enough that the metric becomes less important. This is perhaps the deepest solution: when people genuinely understand what they’re trying to accomplish and why it matters, they’re more likely to flag when the metric is pointing in the wrong direction. Mission clarity doesn’t eliminate Goodhart’s, but it changes the culture in which it operates.

The leaders who fail at this are usually the ones who respond to metric failure by adding more metrics or tightening accountability around the existing ones. The leaders who succeed understand that the problem is structural — that measurement and reality inevitably diverge — and they build organizations that can navigate that gap rather than pretend it doesn’t exist.

I shared this with Dave, and he thought it was pretty good too. The only problem now is that we need to find something else to fight about. I’m not terribly worried about that, though.

I hope you find this useful. Please share what you’ve seen work in the comments.

We love XKCD so much. Creative Commons Attribution-NonCommercial 2.5 License.