Forget about Obama versus Romney. The real contest in the 2012 election was about analytics.
Politics is just the latest front in a war being fought in businesses and consulting firms around the world, with this round pitting “quants” such as The New York Times blogger Nate Silver against the intuition of pundits like James Carville or Karl Rove. The quants bring data, computers and formal models. The pundits – though they do use data – rely more on gut feelings, industry experience and personal contacts.
In the latest skirmish, the quants won. They predicted the election outcome far more accurately than the pundits did. Therein lies a lesson for executives and policymakers alike: Wisdom and intuition may actually be hurting your firm or organization.
Consider the Federal Reserve, arguably the most powerful economic institution on the planet. The Fed’s staff economists do for economic statistics what Silver does for poll numbers, crunching piles of data into a very accurate forecast. By contrast, the members of the Federal Open Market Committee are more like Carville, wisened by years of experience and equipped with anecdotes from their industry contacts.
A surprising study by economists Christina Romer and David Romer found that we would be better served if the members of the FOMC simply withheld their judgment.
This doesn’t mean the quants can declare victory. As it turns out, there is an even better forecaster: crowds. Political prediction markets, such as Intrade, which rely on the wisdom of crowds rather than any individual uber-pundit, have historically done a better job of predicting elections than even very sophisticated statistical models do.
The implications go far beyond elections. Research has shown that prediction markets can forecast economic, business and sporting outcomes better than relevant experts can.