Nate Silver has an important post, Herman Cain and the Hubris of Experts. It’s not really about Herman Cain. Rather, it’s about the reality that pundits tend to underestimate uncertainty and complexity. Saying you don’t know isn’t as satisfying as making a definitive categorical assertion. This manifests particularly in the domains of sports and politics because there are clear and distinct criteria to assess predictive power. Politicians win or lose elections, while teams win or lose games. And yet despite the long history of minimal value-add on the part of pundits they persist in both domains. Why? I think it’s pretty obviously a cognitive bias toward storytelling. Similarly, in the 1930s the Alfred Cowles concluded that financial newsletters didn’t help their readers “beat the market,” but he also assumed these newsletters would persist. There was a psychological need for them.
The key here is to change the attitude of the pundit class. The populace will always have a preference for stories with plausible and clean conclusions over radical uncertainty. Not surprisingly many professional pundits reacted with hostility to Silver’s observation that they’re quite often wrong. I don’t venture into political punditry often, but when the Democrats passed health care reform I predicted that Mitt Romney would have no shot to win the the Republican nomination. The facts in this case seemed so clear. Romney was going to be walloped over and over again over his record on health care reform when he was governor. I was wrong. Romney may not win, but obviously he’s a contender. My logic was simple and crisp, but the logic was wrong. That’s why you let reality play out. If what was “on paper” determined national elections, then we’d be talking about President Hillary Clinton.
Of course political journalists that engage in analysis still have a role to play. Don’t newspapers have horoscopes and style sections?