This is going to sound crazy, but a 100 percent chance something will happen is not the same thing as a sure thing. Mathematicians and statisticians have a name for this: “almost sure.”
Fortunately, you do not need to understand the technical definition of “almost sure” to understand a statistical concept of rather more practical importance: None of the major election forecasts actually predicted Hillary Clinton was going to win on Tuesday.
No, not a single one.
You’re going to read and hear a lot about why the polls got it so wrong, and, indeed, there were some intriguing failures. There’s the fact that Donald Trump did an average of two percentage points better than polls suggested in states Mitt Romney won in 2012, as statistician and political scientist Andrew Gelman has already pointed out. Models of who was going to vote may have been wrong, which could skew forecasts. There is also something known, colloquially, as the Bradley effect, a special sort of “social desirability bias,” as psychologists call it — basically, some people weren’t willing to tell pollsters that they were planning to vote for Trump, because they didn’t think that’s what pollsters wanted to hear. And then there’s the possibility of voter suppression in key states like North Carolina.
But even after we take all that into account, there’s a basic truth pundits and the public don’t seem to completely grasp: probabilities. No forecast ever predicted a Clinton win. They estimated the probability of a Clinton win, based on polling data. And just because that probability was over 50 percent or 80 percent or whatever doesn’t mean she was a sure thing (or even an almost sure thing).
To understand that point, it’s crucial to understand what a poll really is — namely, considerably more random than they seem. To make this concrete, look at Wisconsin, where 1.41 million people voted for Trump and 1.38 million voted for Clinton. A typical poll selects 1,000 people at random and asks how they intend to vote. By pure bad luck, many such polls will contact more Clinton voters than Trump voters. In fact, with Wisconsin’s numbers, Clinton should have come out on top in about 40 percent of all such polls, even if she was destined to lose.
So a poll doesn’t tell you who is going to win; it gives you a rough picture of who’s likely to win. Statisticians turn these rough pictures into probabilities, which it can be helpful to think of in terms of gambling odds. If a poll in one state says Clinton leads Trump 51 percent to 49, for example, the probability of a Clinton win is about 72 percent. The corresponding odds are about five to two.
If you’ve toughed this out so far, here’s what this all means. Based on the final pre-election polls, FiveThirtyEight put the probability of a Clinton win at around 71 percent. Five to two odds. Even at 80 percent, about where The Upshot put Clinton’s chances, the odds are still only four to one. Maybe you’d bet on that, but, if so, you shouldn’t be surprised if your horse doesn’t win.
That Clinton lost, then, isn’t on its own evidence that the forecasts were wrong. Nobody every claimed she was a sure, or even almost sure, thing.
