Amazing blunder
drowns 'Signal'

On Nate Silver's book
The signal and the noise: why so many predictions fail -- but some don't
(Penguin 2012)

Nate Silver plainly believes knowledge is power. People who use statistical methods without incorporating relevant information tend to be less accurate at prediction than those who use both, he says.

This finding is born out by baseball forecasters, he writes. Though his own statistical method was fairly good, he concedes scouts spotted potential in players that his method did not. But on the other hand, scouts tended to overlook potential disclosed by statistics. Which is why, Silver says, teams these days combine statistical methods with scouting reports when choosing players.

Silver, a reformed internet poker shark, gives his general readers a taste of Bayesian probability, without going into much depth on the mathematics or the controversy concerning that approach.

Much of his book is given over to examples of how statistical forecasters often mistake signal for noise. He praises the improvement in weather forecasts but damns economic modeling.

And, after a well-researched look at the global warming debate, we discover: Silver is also an expert on terrorism.

He spoke with Donald Rumsfeld and read a book Rumsfeld recommended that explained why America was unprepared for the Pearl Harbor attack, which essentially said that though all the dots were there, they were improperly connected. Next, we come to how the attacks of  9/11 could have caught America off-guard. Only the intellectually lazy would think of conspiracy. Statistics confirm that failure to connect dots could easily be a true explanation.

Silver has committed the very blunder he so earnestly warns about earlier in his book: Failure to incorporate important information. He seems to have decided that because a probabilistic explanation can be used to endorse the official line, it's obvious that no further work need be done.

And he hasn't done that work.

In a seemingly well-researched chapter on global warming, Silver carefully examines the controversy. Not so for 9/11. Only one side of the controversy -- the government's -- is considered. Statistics, uninformed by relevant information establishing coverup, is then used to rationalize the government's claims. Most of his information concerning 9/11 appears to have come from the 9/11 commission report; Silver doesn't seem to realize that the commission's co-chairmen and its general counsel have distanced themselves from the report because of the CIA's concealment of evidence.

Why ruin an otherwise pretty good book with a penultimate chapter that misleads the gullible with false insight? The best one can say is hubris: Silver has committed one of the sins he so sternly warns against: overconfidence, which, as he says, leads to the forcing of data to fit a curve or the "seeing" of a signal that might not be there.