Jerry Z. Muller – The Tyranny of Metrics
This short book is one of the most useful I’ve read in recent years. I will be citing it often. Measurement is a good and useful thing, but it has its limits. Muller’s job in this book is to remind people of those limits. For example, improving school test scores sounds like a good idea, and was a key part of President George W. Bush’s No Child Left Behind education bill. But teachers started teaching to the test, ruining the purpose. This on-the-ground was entirely predictable, but regulators were so intent on using metrics to measure performance, they didn’t think it through.
One key point has to do with social science research. Few journals will publish papers that don’t measure anything–but not everything is measurable. This means that when policymakers and pundits are evaluating a policy, they can leave out important policy impacts. Either they dismiss non-measurable concerns because there is no published empirical research on it, or such concerns never enter their minds in the first place. Like a drunk looking for his lost keys, they only look where the light shines. Better to admit that things exist outside of that light. Better still to create one’s own light and see what is out there. Statistical significance matters, but it should not replace human judgment. It is a complement, not a substitute.
Inertia Wins! 