Does excluding cell phones give Dayton-Emmer-Horner polls the wrong number?

Last week, the University of Minnesota’s Humphrey Institute and MPR released a gubernatorial poll showing DFLer Mark Dayton and Republican Tom Emmer deadlocked with 34 percent each. It was the first survey in months that didn’t have Emmer trailing.

As disappointed partisans are wont to do, some DFLers seized on what they said was a methodological failing: the poll only included voters with landlines, excluding those who’ve gone cell phone-only. The implication: if surveys included voters who’d “cut the cord,” Dayton would likely be the leader.

Polls are political catnip, not governance. Still, they theoretically influence everything from fundraising to voter attitudes, and media outlets spend tens of thousands of dollars on them. According to a May 2010 Pew Research Center report, 23 percent of adults (and 25 percent of households) are now cell-only. Fully 49 percent of 25- to 29-year-olds have cell phones and no landlines.

So why exclude them from your survey?

“The main reason for not including cell phones in our sample is cost,” says Joanne Miller, an associate professor of political science who works on the HHH survey.

Unlike many politicians, Miller is up-front about the trade-offs. She says reaching a cell-only user costs twice as much as a landline. A major reason: the federal Telemarketing and Consumer Fraud Abuse Act says cell phone calls must be manually dialed — no computer dialers banging out random digits, as with landlines.

And even as you spend more to dial, you’re less likely to reach an eligible voter. Miller notes cell phones aren’t likely to be shared, and a “significant number” are in the hands of minors. That means more re-dialing, and paying more.

“We chose to do two other things that cost money: have a longer survey, which enables us to get more nuance, and bump our sample size [from 500 or so] to 750,” Miller says. “That allows us to make stronger conclusions about smaller percentage differences.”

This might seem a bit torturous, but Miller says the bottom line is that cell phone exclusion doesn’t move the polling needle all that much: “The demographic differences are big, but to be honest, there are very few political differences” that affect the ultimate results.

One reason: cell phone-only users are far less likely to vote than landline-users. Cell users are younger and less white, two groups that tend to vote less and comprise less of the adult population.

An illustration: The Pew survey (actually a series of relatively massive surveys of over 1,500 respondents each conducted between January and April) asked about party identification. After grouping hardcores and leaners, cell-phone-only users tilted Democratic 52-35. The landline sample? 45-41.

The result if you combined the two groups? 47-40. 

It’s a small difference — plus-7 Dem compared to plus-4 Dem. However, Miller says given the large sample size, could be statistically significant. That gives some credence to Democratic criticisms.

The Pew poll also showed Obama job approval/disapproval is 45-45 in the landline sample versus 47-42 when cell phones were added. In the Congressional horserace, Dems were down 6 points among landliners but even when cells were tossed in.

(Most polls show Dem numbers deteriorating since Pew’s spring meta-survey.)

Even though they eschew spending the coin, landline-only pollsters don’t totally ignore cell phone users. Miller says HHH’s model assigns a greater weight to responses from voters who have landlines and cell phones. That’s an attempt to mimic the cell-only part of the population.

However, trade-offs again: While a larger sample size reduces her margin of sampling error, additional weighing bumps it up. The most recent survey’s was 5.3 percentage points, plus or minus.

At least one local media outlet has bought its way out of the problem. In August, the Star Tribune bragged that its Minnesota Poll — which showed Dayton with a 10-point lead — dialed cell phones as well as landlines.

“It did cost us more money,” says Star Tribune assistant managing editor Kate Parry, who oversees political coverage. “There was a general sense that this is a growing thing, it reached a tipping point … [W]e’ve gotten to the point that the cost is worth it.”

The Strib, which effectively emerged from bankruptcy a year ago, has increased spending in some coverage areas. The paper has not reconstituted its in-house polling operation, disbanded due to budget cuts in 2007. Since then, it uses New Jersey-based Princeton Survey Research, which also did the Pew meta-study.

The paper’s Aug. 1 gubernatorial survey received 831 responses, more than the HHH poll, and reported a margin of sampling error of 4.5 percentage points. (Some pollsters only include sample size when calculating margin of sampling error; the Humphrey Institute includes design effects such as weighting.)

For now, the Strib stands nearly alone in including cell-phone-only Minnesota voters.

Survey USA, which partners with KSTP-TV, has traditionally been landline-only; such a survey gave Dayton a 14-point margin over Emmer in early August. SUSA recently included cell phones in Washington and North Carolina contests.

(The Washington margin moved a single percentage point when cell-only respondents were added; in North Carolina, there was no change.)

Rasmussen Reports, which like SUSA uses robo-dialers and automated questions, is also landline-only. Its Aug. 13 poll showed Dayton up by 9 points. Fox9 is Rasmussen’s local media partner.

Decision Resources, Inc., a local firm which last released a poll in June that had Dayton up by 12, does include cell phones in its random digit dialing, a company official says.