Carleton College Political Scientist (and oft-quoted analyst of Minnesota politics) Steven Schier is skeptical about the two latest Minnesota guv race polls that showed DFLer Mark Dayton outside-the-margin-of-error leads over Repub Tom Emmer. In short, he thinks the sample for both pools has too many Democrats.
In an email, Schier writes: “In the 2006 and 2008 Minnesota exit polls of actual voters, Dems had a 4 percent lead over the GOP (39-35 and 40-36, respectively) with the rest independents. In the Humphrey Institute and Strib recent polls, Dems had a 10 and 7 percent lead, respectively, over the GOP.”
To Schier, it is “impossible to believe” that the Minnesota electorate features a substantially larger plurality of Democrats in 2010, which is universally considered to be a year of Republican surge, than it had in 2006 and 2008, both of which were big up years for Democrats (huge Dem pickups to take control of both Houses of Congress in 2006, the election of Obama and still more Dem pickups in both Houses of Congress in 2008).
Schier also finds it unbelieveable that between late August, when the previous MPR/Humphrey poll was taken and late September, the likely partisan makeup of the likely electorate could have undergone such a dramatic change. “You can’t have an electorate that is plus Republicans 10 points one month then plus Democrats 11 poiints the next month. I believe that is grounds for skepticism. I believe the truth is somewhere in between.”
Schier is not accusing the pollsters for either the Humphrey Institute nor the Strib of any bias nor any specific methodological error. But “somehow they’re oversampling Democrats.”
Larry Jacobs, who oversaw the Humphrey poll, understands the skepticism. Jacobs himself expressed some skepticism about the Strib poll, before he had seen the final numbers for his own survey, and he knows that these findings defy the conventional wisdom that this is a Republican year, that Dems were suffering from a big enthusiasm gap and that this would be reflected in who turns out in November.
But he followed the methodology in both polls, Republicans were not complaining about the previous poll, which showed Dayton and Emmer tied (Democrats were complaining about that one), that he double and triple-checked the numbers when he saw the results, knowing that he would be criticized, but that ethically and professionally he couldn’t massage the numbers to come closer to the conventional wisdom so that his poll wouldn’t be conrtroversial. In scrutinizing the results, he concluded — and he has emphasized in his analysis of the poll — that something in the past month has lit a fire under Democrats, at least in Minnesota, and erased the enthusiasm gap, therefore pushing a bunch more Dems into the likely voter category. “There’s something going on here that is surprising,” he said. In other words, he stands behind his poll.
One thing I’ve been meaning to mention in my recent writing about polls, and this is the place to do it. The most common critidcism of a poll is that the sample included too many from one party or the other. This criticism implies that the correct partisan breakdown of the likely electorate is known before the poll and the pollster should make sure that breakdown is reflected in the survey sample. But, think about it, the partisan makeup of the likely electorate changes all the time — that’s one of the main things that causes swings from one election to another and at least tentative swings from one poll to another. A poll is designed to construct a sample that reflects the electorate according to known demographic characteristics like age, race, region, income and education level. But the partisan breakdown of the likely electorate is one of the things that the poll is supposed to discover, not something you program in in advance.
Charles Franklin of the University of Wisconsin, also one of the main guys at Pollster.com, put it this way for me: “As long as you’re not weighting to party ID, which is a very controversial practice within the profession, then the partisan makeup of your sample is a random variable like anything else.”