Last week’s presidential election has once again prompted a wave of soul searching by the polling industry, as many polls greatly overestimated the support president-elect Joe Biden had in battleground states.
One of those states? Our neighbor Wisconsin. Going into the election, polling averages from FiveThirtyEight and RealClearPolitics put Biden 8.4 points and 6.7 points ahead of President Donald Trump, respectively, in the Badger State.
Preliminary election results put Biden’s actual margin of victory at 0.7 percentage points in Wisconsin, making it a much closer race than polls, on average, had suggested. Polling averages were also off in Ohio, which had the race close, but went for Trump by nearly 10 percentage points, and Pennsylvania, which was a closer race than anticipated.
Interestingly, though, the polling average was not far off the mark in Minnesota, where FiveThirtyEight and RealClearPolitics had Biden ahead 9.2 points and 4.3 points, respectively, in their polling averages going into Election Day. The final result? Biden +7.1.
For now, pollsters say it’s too early to tell for sure why the industry, on average, missed the mark in Wisconsin and other states, but not places like Minnesota and Georgia. But early analysis points to a few possibilities.
The submerged Trump vote — a twist on the ‘shy Trump voter’ theory
After many polls underestimated Trump’s support in 2016, some poll watchers who saw forecasts indicating a likely Biden win were wary the industry would miss the mark again in 2020.
One theory about how that could happen was the theory of the shy Trump voter: that the president’s supporters might be embarrassed to tell pollsters that they were planning to vote for Trump in survey interviews, resulting in an underestimate of his strength.
Many pollsters agree there’s little evidence for that specific theory. This year, compared to years prior, the number of respondents who indicated to pollsters they were undecided was relatively low.
“Nationally in 2016, there were about 12 percent undecided between Clinton and Trump, and this year in 2020 it was half that, about 6 percent,” said Rob Daves, the principal researcher at Minneapolis-based Daves and Associates Research and the former director of the Star Tribune’s Minnesota Poll. “So the undecideds are smaller, which means people are coughing up the information about who they support to pollsters.”
What many pollsters are starting to think was more likely at play was what they call differential nonresponse, which happens when one group of people is systematically less likely to respond to a poll than others.
That group? Certain Trump supporters.
“What we have heard from the conservative media for the past decade, and certainly in this election, or the last four years with Trump as president, we’ve heard about fake news,” Daves said. If Trump supporters are internalizing that message, they may be less interested in responding to polls, many of them sponsored by news organizations.
James Lee, the president and CEO at Susquehanna Polling and Research came closer than most pollsters to the final result in Wisconsin.
Asked what his polls might have done differently than others that were more off the mark — some estimating Biden’s lead in the double digits in Wisconsin in the weeks leading up to the election — Lee said adjustments had to be made to account for Trump support that wasn’t showing up in the surveys. Susquehanna didn’t poll Minnesota this year, but it did find a similar phenomenon in Pennsylvania.
“We were finding that Biden was leading in sections of Wisconsin and Pennsylvania that completely defied what happened in 2016,” Lee said. “That’s where we noticed the big red flag, and I’m not saying that metaphorically, that was the submerged Trump vote.”
Such was the case, for example, in the area around Green Bay, which Trump won 60-40 in 2016, Lee said. This year, Lee became concerned when he saw polls suggested Biden was not just chipping away at Trump’s support but winning by a margin that seemed unrealistic given his 2016 results. Susquehanna made adjustments to correct for that.
“We came up with a formula to adjust those regions based on enthusiasm,” Lee said. “We also looked at the percent of people who refused to take those polls. You can weight a poll all day long and still have it out of whack if the wrong people took the poll in the first place.”
If this phenomenon is in fact at play, there’s some evidence it may be tied closely to having Trump on the ticket, said Charles Franklin, a Marquette University Law School professor who runs the Marquette poll: when Trump wasn’t on the ticket in 2018, midterm polls were fairly accurate.
“In 2018, we were within 1.1 percentage points of the governor’s margin, and within one-tenth of a point of the Senate margin, and in 2020, in the Democratic primary this spring, we were within 2 points of both Biden and of Sanders’ votes,” Franklin said.
The percent of Republicans in Marquette’s sample increased between 2012 and 2016, and has been steady since 2017, so there isn’t evidence that Republicans as a group have become less willing to talk to pollsters, Franklin said. This suggests the possibility that the Trump voters who aren’t showing up in polls are less strongly tied to politics and the Republican party than they are to Trump.
“My hypothesis is that this is that small group, maybe 4 to 5 percent who support Trump, think of themselves as independents or maybe independents who lean Republican but are not broadly engaged in politics outside of their support for Trump. Therefore, they don’t turn out to vote in midterms, they certainly don’t vote in a Democratic presidential primary, and therefore the polls [in those races] are pretty good,” Franklin said.
The degree to which the submerged Trump vote caused polls to be off in some states versus others is completely unclear at this point, but Daves said it shouldn’t be discounted yet.
“Now does that explain what went on in Wisconsin? The difference between Wisconsin and Minnesota? Maybe not, but we can’t discount it,” he said.
How much did higher turnout help Trump in Wisconsin?
Marquette’s polls also got closer to the Wisconsin result than the average presidential poll did: in a late-October poll, Marquette estimated Biden’s lead at 4 percentage points among registered voters and 5 percentage points among likely voters.
Franklin said another factor that might have been at play in the underestimation of Trump’s support was the turnout model pollsters were using. Franklin’s high-turnout model estimated 3.2 million Wisconsin voters. Actual turnout was closer to 3.3 million.
Both Biden and Trump received more votes than their 2016 equivalents in Wisconsin. But evidence in Franklin’s sample suggested higher turnout in Wisconsin would tighten the race by one point, helping Trump.
“Though we are talking about just a one-point shift in the result, that still puts us in the right direction and it’s consistent with the actual turnout that we had,” Franklin said.
Given the partisan breakdown of early versus Election Day votes (Democrats were more likely to vote early) suggested by polls, it’s possible pollsters missed a late, Election Day surge for Trump.
“Given the sharp party split in early voters and Election Day voters, one way for the polling to consistently be off across the country would be if there was, in fact, an unexpected surge for Trump on Election Day that we weren’t catching beforehand,” Franklin said.
It’s speculation at this point, but Wisconsin had more room to up turnout than Minnesota did, so it’s possible pollsters, in part, more correctly estimated the size of Minnesota’s electorate than Wisconsin’s.
In 2016, Wisconsin’s turnout was 67 percent of estimated eligible voters. Minnesota’s was 75 percent. This year, Wisconsin’s turnout reached 72 percent, compared to 78 percent in Minnesota.
More to learn
There are about as many possible explanations for why polls in some states were more accurate than others as stars in the sky, Daves said.
It could have been any of the above, it could have been different pollsters with different methodologies dominating different states. But while it’s not clear yet which factors influenced polling accuracy this year, and to what degree they were at play, it is clear something was off by-and-large, particularly in some states.
As it did in 2016, the American Association for Public Opinion Research has appointed a task force to evaluate the performance of polls this year and make suggestions based on its findings.
Ideally, the polling average is pretty close to the actual vote margin, with some polls falling on one side of the average, and some on the other, Franklin said. This year, the polling average in some states was far off the vote total, and many battleground state polls overestimated Biden.
“This time the average error is quite large, relatively speaking, to past better performances by the polls [and] almost all, if not every one of the averages is at least a little bit pro-Biden.”