I’ve developed an Eric-Black-on-the-10th-Amendment fascination with cell phones and political polling, and have chatted up Minnesota’s gubernatorial pollsters here and here. And estimated 23 percent of adults live in “cell phone only” households — no landlines. Only the Star Tribune and Decision Resources, Inc. include cord-cutters in their surveys.
Now into the dock comes SurveyUSA president Jay Leve, whose company released the most recent gubernatorial poll, for KSTP. DFLer Mark Dayton led Republican Tom Emmer 38 to 36 percent, with Independence candidate Tom Horner getting 18 percent. It nearly mirrored the MPR/Humphrey Institute’s 34-34-13 results days earlier. Neither poll included cell-phone-only respondents, henceforth known as CPOs.
CPO holdouts have two fundamental explanations: cost, and the relatively small electoral impact of CPOs, who tend to be younger and less politically involved. Leve offered both arguments, but with nuance — and a surprising bit of criticism for a fellow CPO-excluding pollster.
Because of various laws and demographics, the Humphrey Institute’s Joanne Miller told me earlier this month that CPOs costs twice as much to poll as landliners. Leve pegged the rise higher: “three- or four-fold.”
He says, “KSTP is to be commended as the only broadcaster in the Twin Cities who pays hard-cash dollars out of its pocket to commission opinion research. If SurveyUSA insisted that a pre-election poll include CPO respondents, there may not be any KSTP polling in Minnesota, as a practical matter.
“Then the question becomes: what is better, a survey of Minnesota without CPO respondents? Or, no survey of Minnesota? I don’t mean that as a rhetorical question. It’s a real question: Thoughtful persons can answer in different ways.”
The must-include-CPOs argument becomes stronger each passing year. In 2008, there were nearly half as many CPO households (13 percent) as there are today. That’s a big reason the Strib added CPOs this cycle.
For his station’s part, KSTP political reporter Tom Hauser says, “It’s something we’re considering in the future when we determine whether there’s a significant difference to justify the added cost. It looks unlikely for this election cycle, but is something we would like consider more strongly in the future as more people become cell phone-only users.”
Ignoring a quarter of America’s households is only remotely justifiable if that cohort doesn’t move the needle much. Leve notes two SUSA surveys showed just that — in Washington State and North Carolina, where the company spent extra dough to include CPOs.
To his credit, Leve does not say this is a fixed principle: “I am not implying, David, that the results of an election poll will always be the same with CPO respondents included, nor even that the results would be the same a majority of the time.”
This brings us back to the moral dilemma of polling 80 percent of the public instead of 95 percent. CPO-exclusion critics have tended to be Democrats, since CPOs have polled more blue than the general population. A pre-primary Strib poll, for example, showed Mark Dayton’s lead jumping from 5 percent to 10 when CPOs were added in.
Here, Leve cautions CPO-exclusion critics from extrapolating his results. “There is no way to know, in advance, whether or how the results of a poll of Minnesota would be different, had a pollster systematically and defensibly included CPO respondents.”
Leve concluded with surprisingly harsh words for the Humphrey Institute’s method of closing the CPO hole. Miller told me her MPR polls add additional weight to respondents who have landlines and cell phones. She explains, “To the extent that people with have a landline AND a cell phone are similar to the people who just have a cellphone … then can help a little bit (how much so, we are not sure) … but it’s not likely to hurt.”
Leve says this sort of hope is wrong-headed. “There is no defensible weighting that can be done to a universe of reachable respondents [landliners] to compensate for a universe of unreachable respondents [CPOs]. There is a 50% chance that weighting, in an attempt to compensate for respondents who had zero chance of being included, will make the data worse, not better. ”
In sum, he says, “SurveyUSA acknowledges that there are limitations to telephone research in 2010.”
By the way, pollster and analyst Nate Silver of FiveThirtyEight.com places SUSA near the top of his June 2010 pollster rankings. The Star Tribune was middle of the pack, while the Humphrey Institute ranked near the bottom. Silver’s rankings use polls back to 1998 — well before cell phones were a factor — but weighs recent polls more heavily.
