Sunday, July 7, 2019

Interview: Sam Wang on what went wrong with polling last year

Editor's note: This interview was originally published in April 2017.

Story by Joseph Ford Cotto

"Like I've said before .... polls are only good for strippers and cross-country skiers," Former Alaska Governor Sarah Palin told Fox News host Bill O'Reilly two days before last year’s presidential election.

While I have criticized her more than a fair bit during years gone by, it is undeniable that she turned out to be a sage among pundits.

When Palin made her remark, virtually all national opinion surveys – save two highly important yet conspicuously underreported ones – indicated an impending win for Hillary Clinton. Reuters predicted she was set to win 247 electoral votes outright and favored to seize so many more that her chance of victory hovered at 90 percent.
United Press International, in conjunction with the polling group CVoter, reported that her Electoral College lead over Trump was substantial: 259 to 209.

After the race was called, Clinton barely eked out 232 votes. Her vaunted 'firewall' of Michigan, Pennsylvania, and Wisconsin crumbled into ash. Only Minnesota and New Hampshire stood -- and not by any means tall. Clinton struggled to win either, despite the former having gone to every Democratic nominee since Richard Nixon's 1972 earth-slide over George McGovern.

Just a few hours earlier, Clinton fans were jubilant over her seemingly assured victory. By the morning after, perhaps more tears were shed than at any other time in twenty-first century America.

As Election Day loomed, the Princeton Election Consortium's Dr. Sam Wang – a neuroscientist and prolific author – declared that Clinton enjoyed a 99 percent probability of winning. The platinum-grade forecaster Moody's Analytics also claimed she would triumph in the Electoral College.

Wang recently spoke with me about what happened last year. Our conversation is included below.


Joseph Ford Cotto: More than any other reason, why did polling not prove an effective guide to predicting an outcome for the 2016 presidential election, unlike in previous races?

Dr. Sam Wang: The polling error was closely linked with Republican states, where Trump had been lagging Mitt Romney's 2012 numbers by a large margin. The overall pattern suggests that Republican voters were hanging back from supporting Trump. In the end their partisan loyalty won out.

Cotto: Crunching numbers, especially in our social media-driven, sound bite-prone age, is touted as the begin all, end all of event forecasting. In 2016, perhaps more attention was given to polls than in any previous election. Might the press have generated such a demand for polling data that quantity increased at the expense of quality?

Wang: In fact, the quantity of polling in 2016 decreased compared with previous elections.

Cotto: Historical data, analyzed and complied into two studies by SUNY Stonybrook's Helmut Norpoth and American University's Allan Lichtman, respectively, indicated that Donald Trump would win. Many, both in the media and otherwise, chose to ignore these academics even though they have solid track records. After 2016, some might say that history is a better election forecaster than opinion surveys. What is your perspective on this?

Wang: Your question contains inaccurate information. Look at Norpoth and Lichtman's statements before the election closely. I believe they said they were predicting the winner of the popular vote, which was Hillary Clinton.

Cotto: Perhaps the most-castigated of all polls from the 2016 election was the one USC Dornsife conducted for the LA Times. It turned out to be one of the only surveys which correctly forecast the race. How did this poll evade the failure of nearly all others?

Wang: Your question is again incorrect. Dornsife was one of the two least accurate polls of 2016, tied with SurveyMonkey. Dornsife predicted a Trump popular vote win by several percentage points, yet Mr. Trump lost the popular vote. That would make Dornsife's overall error about 4-5 percentage points.

Cotto: Since it is growing more difficult to take scientific surveys, might the events of 2016 spell an eventual end for the polling industry?

Wang: No, but probably they can do what they have always done, which is adopt new methods as they become available.


Editor's note: Wang also offered a general explanation of how polling related to election returns, and it is this: "In my view the error was not that large, but of enormous consequence. In state polls, Trump overperformed by 3 points across the board, 6 points in Republican-leaning states. National polls were not that far off - they said HRC was likely to win the popular vote by a few points, and she did."