November 30, 2005
— Dr. Reo Symes The New Yorker does a good rundown on Expert Political Judgment: How Good Is It? How Can We Know?, a new book by Berkeley academic, Philip Tetlock.
As the title suggests, its all about how accurate the predictions the political and economic pundits we see and read actually turn out to be. And Tetlock isnt just some crank, shooting off his mouth. Hes pretty rigorous:
[H]is conclusions are based on a long-term study that he began twenty years ago. He picked two hundred and eighty-four people who made their living commenting or offering advice on political and economic trends, and he started asking them to assess the probability that various things would or would not come to pass, both in the areas of the world in which they specialized and in areas about which they were not expert. By the end of the study, in 2003, the experts had made 82,361 forecasts
And his conculsion? The experts are full of it.
eople who make prediction their businesspeople who appear as experts on television, get quoted in newspaper articles, advise governments and businesses, and participate in punditry roundtablesare no better than the rest of us.
Tetlock also found that specialists are not significantly more reliable than non-specialists in guessing what is going to happen in the region they study. Knowing a little might make someone a more reliable forecaster, but Tetlock found that knowing a lot can actually make a person less reliable.
And the big shots are the worst:
Tetlock claims that the better known and more frequently quoted they are, the less reliable their guesses about the future are likely to be. The accuracy of an experts predictions actually has an inverse relationship to his or her self-confidence, renown, and, beyond a certain point, depth of knowledge. People who follow current events by reading the papers and newsmagazines regularly can guess what is likely to happen about as accurately as the specialists whom the papers quote.
Tetlock points to a number of explanations, most centering on the psychological foibles inherent in assessing uncertainty, along with simple stuff like the fact theres really no puditocracy penalty for getting it wrong.
Anyway, the piece is relatively short, not one of those New Yorker book/articles and well worth the read.
(N.b. My predictions were not part of the study, and remain, of course, rock solid certainties. You can continue to bet the farm on the Good Doctor's forecasts)
(h/t Jane Galt)
Coulter pointed this out in Slander, how CNN threatened to replay a clip of her being wrong about how long the impeachment trial would last. Gee, who won that battle?
I mean, if you are going to offer your credibility up to pose WP as a WMD, then you have to expect to never be right. Even if WP=WMD will get you NYT cred. BTW, does the WP=WMD mean that Iraq had WMDs? So Bush was right all along!
Posted by: joeindc44 at November 30, 2005 08:19 AM (8TcCs)
Posted by: polynikes at November 30, 2005 08:34 AM (m2CN7)
I wish there would have been some way to bet on whether a military draft would be installed during the Bush presidency. I would have laid at least a grand against it, and I'm certain not one lefty would have taken the bet. Even they know how full of shit they are.
Posted by: The Warden at November 30, 2005 08:45 AM (Zxtyv)
Posted by: Moonbat_One at November 30, 2005 09:23 AM (ivB4r)
Posted by: Yaron at November 30, 2005 10:07 AM (rj3dl)
Like that scene in Deuce Bigalo. "That's a huge bitch!"
Posted by: fugazi at November 30, 2005 10:57 AM (KyUPY)
The author of the study said that one thing surprised him. He expected that judges actually believed the experts. In fact, he found that judges recognized perfectly well, that the experts were blowing smoke. They went with the expert recommendations because they had nothing better, and it shifted the responsibility off them and on to the "experts."
Posted by: Bob Hawkins at December 01, 2005 09:51 AM (eZ0vq)
Posted by: Mark V. at December 01, 2005 01:02 PM (/653L)
Posted by: SEO Pakistan at November 01, 2011 04:08 AM (Q7nl0)
Posted by: davidtan at November 03, 2011 08:32 AM (E0PR6)
62 queries taking 0.0221 seconds, 197 records returned.
Powered by Minx 1.1.6c-pink.