Predicting the outcome of elections is an inherently chancy endeavor.
“If you look into the crystal ball,” says an experienced pollster, “you’ve got to be ready to eat ground glass.”
But pollsters’ job is getting harder. The number of people willing to answer their questions is plummeting.
Of every ten people in rich countries they contact by telephone, at least nine now refuse to talk.
Far more intractable is the bias that creeps in when samples are not representative of the electorate. Taking bigger samples does not help. The margins of error cited by pollsters refer to the caution appropriate to sampling error, not to this flaw, which is revealed only on polling day.
New political fault lines are complicating their efforts to find representative groups to question, and voters’ changing behavior blindsides them as they try to discern the truth behind polling responses.
Old political allegiances are weakening and public opinion is becoming more fickle.
Confidence in polling has been shaken. Pollsters are scrambling to regain it.
Sam Wang, a neuroscience professor at Princeton and part-time psephologist, kept a pre-election promise to eat an insect on live television if Mr Trump won more than 240 electoral-college votes.
Statistical models of election outcomes attempt to quantify the uncertainty in polls’ central findings by generating probability estimates for various outcomes. Some put Hillary Clinton’s chance of victory against Mr Trump above 99%.
To deal with non-response bias, pollsters try to correct their samples by a process known as weighting. The idea is simple: if one group is likelier to respond to a survey than another, giving a lower weight to the first group’s answers ought to set matters right.
But adjusting weights is also one of the ways pollsters can do what political scientists call “herding”. If one weighting scheme produces a seemingly outlandish result, the temptation is to tweak it. “There’s an enormous pressure for conformity,” says Ann Selzer, an American pollster. Polls can thus narrow around a false consensus, creating unwarranted certainty about the eventual outcome.
To make weighting work, pollsters must pull off two difficult tricks. The first is to divide their samples into appropriate subgroups. Age, sex, ethnicity, social class and party affiliation are perennial favorites. The second is to choose appropriate weights for each group. This is usually done with the help of a previous election’s exit poll, or the most recent census.
But the old political dividing lines are being replaced by new ones. Increasingly, samples must be weighted to match the voting population for a much larger set of characteristics than was previously needed. Levels of education, household income and vaguer measures such as people’s feelings of connection to their communities have all started to be salient.
Earlier this century, online betting exchanges beat pollsters before several big elections. Economists argued that the forecasts made by punters with money on the line were likely to be more considered than the sometimes offhand responses given to pollsters.
Far more intractable is the bias that creeps in when samples are not representative of the electorate. Taking bigger samples does not help. The margins of error cited by pollsters refer to the caution appropriate to sampling error, not to this flaw, which is revealed only on polling day.
Spotting new electoral rifts and changing electoral habits will require much more data (and data science) than pollsters now use. And picking up changing social attitudes means measuring them, too—which will take never-ending checks and adjustments, since those measurements will suffer from the same problems as pre-election polls. Pollsters will also have to improve their handling of differential turnout and undecided voters. Most accept self-reported intention to vote, which turns out to be a poor guide. And they often assume that undecided voters will either stay away or eventually split the same way as everyone else, which seems not to have been the case in recent contests.
And dealing with declining response rates will probably require new ways to contact prospective voters. During the early days of internet polling, many feared that online samples were bound to be unrepresentative, mainly because they would include too few older people.
A striking example came in 1936, when Literary Digest, a weekly American magazine, asked its affluent readers whom they would vote for in that year’s presidential election. Nearly 2 million replied. But the sample, though large, was horribly biased. Based on it, Literary Digest forecast a landslide for Alf Landon. He went on to lose all but two states to Franklin Roosevelt.
https://www.economist.com/international/2017/06/17/britains-election-is-the-latest-occasion-to-bash-pollsters
