Book Review: “Weapons of Math Destruction. Read It!

by John W Rodat on April 14, 2017

Cathy O’Neil’s Weapons of Math Destruction, How Big Data Increases Inequality and Threatens Democracy, is a terrific and important book. 

O’Neil has the credentials and the cred. Her Ph.D. in mathematics is from Harvard and she subsequently taught at Barnard. She took her analytical skills to D. E. Shaw, a hedge fund and then to other private sector organizations.

O’Neil sees the risks in overuse, especially of opaque mathematical models, not so much from a technical perspective, but because of the economic and political power of such models. She also sees the risks of how using such models and analyses creates feedback loops, especially vicious cycles that disadvantage the disadvantaged. And she’s acutely critical – rightly so in my judgement – of analyses and measurement systems that may be technically sound, but are far from conceptually sound. 

Example: measurement of teacher performance based on how their students do on high-stakes, non-routine tests. O’Neil’s explanation is not mere rant. Because she understands the technical and conceptual framework, she sees and helps the reader see how fragile and unreliable (thus, unfair) the system is. (And, I would add that, as a matter of public policy, how counterproductive such systems are.) O’Neil tells the story of Tim Clifford, an experienced teacher whose model-based score in one year was 6 out of 100. Protected by tenure, but baffled by his dismal score, he continued teaching the same way. The following year his score was 96 out of 100. The arithmetic might have been right. A measurement system that produces those kinds of swings is dangerous to rely on.

(Note that education is not my domain, but health care has been and I’ve observed that education policy makers could and should have paid attention to the lessons about measurement systems that health care folks learned with a 20-30 year head start. Perhaps I’ll expand on that one of these days.)

Don’t assume that because O’Neil is very much the mathematician, that she can’t write. She can. She has a nice, accessible style and uses storytelling well to make her points. And anyone who goes by the moniker, mathbabe takes her work, but not herself too seriously.

I’d put this book in a group with Michael Lewis’s Moneyball, Nate Silver’s The Signal and the Noise as eminently readable books on modern analysis. Lewis is about “why,” Silver’s mostly about “how” and O’Neil is about “be careful” about using today’s tools.

Don’t just take my word for it. Here’s Evelyn Lamb’s review in Scientific American.

For the lay reader who wants to know what some of the downside are, for the policymaker who wants to measure performance, but wants also to be fair and responsible, and for the data maven who’s concerned about the ethical parameters of their work, Weapons of Math Destruction is a must read.

Data WMD O Neil

Previous post: