In short, describe the actions you want people to take rather than the results you think those actions should achieve. Or perhaps more fundamentally, you should know what people are doing and you won't know that when you are only looking at an opaque metric.

I'm still not convinced. Goodhart’s Law is rooted in human behavior—once people know what’s being measured, they’ll optimize for that, often distorting the system or the data to hit targets. The article's solution boils down to “just do it right” by refining metrics and improving systems, but that’s easier said than done. It ignores the fact that people will always game metrics if their rewards depend on them. Plus, it conflates data-driven decision-making with performance evaluation, which are very different. The psychology behind Goodhart’s Law isn’t solved by more metrics tweaking.

This doesn't feel well elucidated, but I've been thinking about Goodhart's law in other area's of life -- e.g. Owning a home is cool and can enable some cool things. However, when home ownership becomes the goal, it's becomes easy to disregard a lot of life giving things in pursuit of owning a home.

This seems to pop up in a lot of areas and I find myself asking is X thing a thing I really desire or is it something that is a natural side effect of some other processes.

What I find striking is the emphasis on tracking and managing inputs (to develop a healthy causal model for management). This is in contrast with advice that's commonly passed around (ostensibly attributable to OKRs, High Output Management, etc -- though I haven't carefully read the original sources), with heavy emphasis on the output but eschew focusing on input metrics. While it avoids the failure mode of focusing on causally irrelevant metrics, it also assumes that the right causal model has already been discovered and there's no learning process here. This impedes the development of an agile organization that is learning constantly (growth mindset) instead promoting a "fixed" mindset where people/teams can either execute well or not, and the only control lever is hiring/firing/promoting those who seem to "get it". Fantastic and thought-provoking article!

This is all well and good, but unfortunately depends on the people pushing for the metric/system to give a shit about what the metric is supposed to improve. There are still far too many that prefer to slap 1 or 2 careless metrics on an entire team, optimize until they're promoted, then leave the company worse off.

Reminds me of one of my favorite Dilbert cartoons: https://www.reddit.com/r/ProgrammerHumor/comments/k5hka0/bug...

Goodhart's law can diagnose an issue, but it prescribes no solutions.

However, it's still better to recognise a problem, so you can at least look into ways of improving the situation.

"...when you’re incentivising organisational behaviour with metrics, there are really only three ways people might respond: 1) they might improve the system, 2) they might distort the system, or 3) they might distort the data."

This is wrong, and the wrongness of it undermines the whole piece, I think:

- A fourth way people respond is to oppose the choice of target and/or metric; to question its value and lobby to change it.

- A fifth way people respond is to oppose the whole idea of incentives on the basis of metrics (perhaps by citing Goodhart's Law... which is a use of Goodhart's Law).

Goodhart's Law is useful not just because it reminds us that making a metric a target may incentivize behavior that makes THAT metric a poor indicator of a good system, but also because choosing ANY metric as a target changes everyone's relationship with ALL metrics-- it spells the end of inquiry and the beginning of what might be called compliance anxiety.

There is a very good essay in the first comment by "Roger" dated Jan-2023, reproduced below. Skip the primary essay and work from this:

"I really appreciated this piece, as designing good metrics is a problem I think about in my day job a lot. My approach to thinking about this is similar in a lot of ways, but my thought process for getting there is different enough that I wanted to throw it out there as food for thought.

One school of thought 9https://www.simplilearn.com/tutorials/itil-tutorial/measurem...) I have trained in is that metrics are useful to people in 4 ways:

    1. Direct activities to achieve goals
    2. Intervene in trends that are having negative impacts
    3. Justify that a particular course of action is warranted
    4. Validate that a decision that was made was warranted

My interpretation of Goodhart’s Law has always centered more around duration of metrics for these purposes. The chief warning is that regardless of the metric used, sooner or later it will become useless as a decision aid. I often work with people who think about metrics as a “do it right the first time, so you won’t have to ever worry about it again”. This is the wrong mentality, and Goodhart’s Law is a useful way to reach many folks with this mindset.

The implication is that the goal is not to find the “right” metrics, but to instead find the most useful metrics to support the decisions that are most critical at the moment. After all, once you pick a metric, 1 of 3 things will happen:

    1. The metric will improve until it reaches a point where you are not improving it anymore, at which point it provides no more new information.
    2. The metric doesn’t improve at all, which means you’ve picked something you aren’t capable of influencing and is therefore useless.
    3. The metric gets worse, which means there is feedback that swamps whatever you are doing to improve it.

Thus, if we are using metrics to improve decision making, we’re always going to need to replace metrics with new ones relevant to our goals. If we are going to have to do that anyway, we might as well be regularly assessing our metrics for ones that serve our purposes more effectively. Thus, a regular cadence of reviewing the metrics used, deprecating ones that are no longer useful, and introducing new metrics that are relevant to the decisions now at hand, is crucial for ongoing success.

One other important point to make is that for many people, the purpose of metrics is not to make things better. It is instead to show that they are doing a good job and that to persuade others to do what they want. Metrics that show this are useful, and those that don’t are not. In this case, of course, a metric may indeed be useful “forever” if it serves these ends. The implication is that some level of psychological safety is needed for metric use to be more aligned with supporting the mission and less aligned with making people look good."

its not a "law" of course. should not be called a law