It stands as one of the great “famous last words” moments in recent U.S. political history. Shortly after losing the 2016 presidential election to Donald Trump, Hillary Clinton told New York Magazine:
“I don’t know how we’ll ever calculate how many people thought it was in the bag, because the percentages kept being thrown at people — ‘Oh, she has an 88 percent chance to win!’”
Clinton’s comments capture one of most intriguing and significant findings of a study last year titled “Projecting Confidence: How the Probabilistic Horse Race Confuses and Demobilizes the Public.” Written by a Dartmouth government professor, Facebook data scientist, and University of Pennsylvania communications professor, it found that during the 2016 presidential election, overconfident forecasts of Clinton’s victory from virtually every prominent source of polling led to lower voter turnout from Democrats and Independents.
The study’s argument basically puts a twist on the observer effect, a psychological phenomenon in which people change one or more aspects of their behavior in response to knowing that they are being observed. It referenced past work that suggested people are less likely to vote when they believe the outcome of an election is certain — especially if the certainty favors their preferred candidate.
To be clear, this is not made out to be a seismic factor. The study cited data showing that voters who expected one candidate to win by quite a bit were only about two and a half percent less likely to vote than those who believed a race to be close. However in an election that was effectively decided by just 107,000 votes across three key states, even a relatively small reduction in voter turnout could have had huge consequences in the final outcome.
“Clinton lost by so few votes that it is certainly possible that probabilistic forecasts caused enough Democrats to stay home that it affected the outcome,” wrote Yphtach Lelkes, a professor of communications at the University of Pennsylvania and one of the study’s authors.
Clinton herself and many of her would-have-been voters are fully aware of the implications of this phenomenon. “Not so much anymore, but in the immediate aftermath, from after the election to probably the first of the year,” she told New York Magazine, “I had people literally seeking absolution.”
The phenomenon has been exacerbated in recent years, the study argues, by probabilistic forecasts employed in modern horse race coverage of elections. These forecasts aggregate polling data from a variety of sources and produce a specific probability of winning. They are much more prominent in media outlets with left-leaning audiences, but they can be easily misunderstood. Giving Clinton a 71.4 percent chance to defeat Trump (one of the more conservative probabilistic forecasts in 2016) tended to lower perceptions that the election was competitive, when in reality it simply meant that Clinton should be expected to win a head-to-head matchup seven times out of 10.
What does this mean for the health of America’s democracy moving forward, and how can polling be presented in such a way that it doesn’t depress, but rather encourages voter turnout regardless of party or ideology?
“We’re experimenting with ways to convey uncertainty that won’t turn people off [from voting],” said G. Elliott Morris, a data journalist at the Economist who works on election forecasting. “But I think that is still a problem that forecasters are going to have… I don’t know how we get around some of the societal implications of our work.”