Nonprofit, nonpartisan journalism. Supported by readers.


Minnesota political polling: who’s been right since 2006?

So with the new Star Tribune poll out showing DFLer Mark Dayton with a 9-point lead over Republican Tom Emmer, it’s the right’s turn to howl over alleged bias. In the spirit of Dems accusing Rasmussen Reports of being a Republican house organ, Mitch Berg at the True North blog dubs the Strib results “The DFL morale-booster”:

I’ll remind you that if the Minnesota poll were accurate, we’d be referring to Governor Humphrey (the poll showed Moe with a strong lead over Coleman, with Ventura well out of the running), Senator Mondale (who had a five point lead in the MN Poll on the eve of the ’02 election), Governor Moe (to whom the MNPoll gave a slim lead, while significantly overpolling IP candidate Tim Penny in ’02), Governor Hatch (yep, slated to win in ’06)…

This got me wondering just how the Strib’s poll has done more recently — and versus other polls that the right regards more highly (Rasmussen, SurveyUSA). As some of you may know, the Strib changed pollsters in 2007, when it let go in-house polling director Rob Daves, replacing him with Princeton Survey Research Associates for the 2008 cycle.

An important caveat about pre-election polling: missing the final margin doesn’t necessarily mean a pollster is wrong. Sentiment can swing in the voting booth, after polling ends. (This is why pollsters refer to their results as a “snapshot in time.”) Also, any poll has margin of sampling error. The trick is to see patterns — the so-called “house effect” toward a particular party, and whether results are consistent outliers.

Let’s begin with Daves’ last cycle, the 2006 election.

Mitch rakishly references “Gov. Hatch.” Here are the three major pollsters’ final November results, via Real Clear Politics’ roundups:
Star Tribune, Hatch +3
Rasmussen, Hatch +2
SurveyUSA, tie

Survey USA won the crown in a race Pawlenty eeked out by 1. But no poll — including the Humphrey Institute, Mason-Dixon, Zogby Interactive, or St. Cloud State — had Pawlenty ahead in November … or October.

By the way, Daves and the Strib finished closest to the pin on that year’s Senate race, when Amy Klobuchar bested Mark Kennedy by 20:
Star Tribune, Klobuchar +21
Rasmussen, Klobuchar +16
SurveyUSA, Klobuchar +14

OK, what about the real litmus test: the 2008 Senate race between Franken and Coleman? I’ll skip the drama and say no one predicted the virtual tie:
Star Tribune, Franken +4
Rasmussen, Coleman +4
SurveyUSA, Coleman +5

Everyone missed by about the same amount.

In this three-race survey, the Strib was the only poll to give the DFLer the lead all three times, lending credence to Republican claims of a pro-Democrat “house effect.” The Dayton result — which for now is an outlier — is another brick in that wall.

However, if you’re arguing a poll equals a prediction (which again, isn’t quite right), the Strib picked two winners, SUSA two (we’ll give ’em the TPaw tie) and Rasmussen only the AKlo blowout.

Even allowing for GOP mewling that Franken stole the 2008 election, it seems clear that the three polls have circled the final result roughly equally. I’d also note that, at least from 2006 on, if you’re comparing the final polls to the eventual outcome, SUSA’s house effect is as Republican as the Strib’s is Democratic.

As I’ve noted in several columns this month, the Strib’s 2010 polling now include cellphone-only voters, a potentially significant methological difference with Rasmussen, SUSA, and the Humphrey Institute/MPR poll. (The Strib and HHH use live operators; SUSA and Rasmussen are so-called “robo-polls” using recorded questions and touch-tone answers.)

A potentially bigger difference: how each pollster screens for likely general-election voters. I’m surveying the major pollsters on their “likely voter screens” and will let you know after I hear back from everyone.

Comments (15)

  1. Submitted by Nick Coleman on 09/27/2010 - 11:42 am.

    A more interesting question than the topic of this column would be to examine the causes and consequences of the unprecedented political pressures brought to bear against the Star Tribune’s decades-old, signature public opinion research effort before and after the 2004 election campaign. Eliminating the Minnesota Poll, and its director, Rob Daves (who was president of the American Association for Public Opinion Research) became a demand of the Minnesota GOP, with the Party Chairman (Ron Ebensteiner) and right-wing bully bloggers who held sidewalk protests outside the newspaper to demand Daves’ head on a platter. To its shame, the Star Tribune subsequently caved in to the demands, and replaced a venerable in-house poll with outside hit-and-run pollsters. The results may still be labeled “the Minnesota Poll,” but they are not as deep, as rooted in years of research or, in my view, as reliable. The only thing that has remained unchanged? The poll still serves as a convenient straw man/punching bag for know-nothing bloggers.

  2. Submitted by Thomas Swift on 09/27/2010 - 12:07 pm.

    “The poll still serves as a convenient straw man/punching bag for know-nothing bloggers.”

    But for that, it still doesn’t pack the same, satisfying zing for unemployed lefty commentary writers who “know stuff” that it did a few years ago.

  3. Submitted by Virginia Martin on 09/27/2010 - 12:20 pm.

    Interesting comment, Nick, and something I did not know. I don’t suppose it was widely reported in the S-T newspapers. Thank you.

  4. Submitted by Brian Simon on 09/27/2010 - 12:26 pm.

    “I’m surveying the major pollsters on their “likely voter screens” and will let you know after I hear back from everyone.”

    I look forward to that followup, though I’m skeptical that any of them will tell what their algorithm is.

  5. Submitted by Jason Walker on 09/27/2010 - 12:27 pm.

    The robocalling isn’t as bad a polling gaffe as omitting cell-phone-only voters, but it’s close. Who is really going to listen to a robocall other than the old and the lonely anyway? Come on, I get a robocall about anything (which I don’t, because I’m cell-phone-only) and I’m hanging up immediately. It’s not a representative slice.
    But then again, when it comes to likely voters, the old and the lonely lead the pack. So what do I know?

  6. Submitted by Jeremy Powers on 09/27/2010 - 12:57 pm.

    Apaprently political polling is like buying socks — the more you get for the money the better. It would seem to me that a really well crafted poll, with more respondents and better questions, that would be a better “snapshot” would be more worthwhile because it would be more accurate. Oh, I forgot. The purpose seems to be to hang a headline on something rather than finding out the truth — much like all (OK, most) media today.

  7. Submitted by John Edwards on 09/27/2010 - 03:04 pm.

    Sometimes history provides a useful perspective.

    Following the 1978 election, an Arizona firm hired to do the actual polling for the Minnesota Poll, wrote a letter to the
    Tribune disassociating itself from the poll because the newspaper manipulated the results the firm had obtained in
    the Al Quie-Rudy Perpich race for governor. The newspaper had altered the numbers in a poll on the Sunday before the election to show a Perpich win. Quie won. The St. Paul Pioneer Press, not the Tribune, made the letter public in a news story about a week after the election. The Minnesota Poll lost credibility and was shelved. When it returned about 1980 it refrained from political polling for several more years.

    And that is not the only example. In 1969 the newspaper quashed a poll just before the election in a three-way race for mayor of Minneapolis. The move became widely known and is indisputable. The poll showed the controversial conservative Charles Stenvig beating DFLer Jerry Hegstrom and Republican Dan Cohen. In this case the poll was accurate.

    I know these examples go back a ways, but they also illustrate the indelible stain left on an organization’s credibility when it manipulates news for political reasons.

    Also interesting is that in all the scandals . . . whether it be manipulating poll results; breaking its word to a source in the Dan Cohen case; or suspending columnist Jim Klobuchar for writing speeches for DFL Gov. Rudy Perpich while still working for the paper; the Star-Tribune always seems to somehow have wound up promoting the liberal side. I am sure it is just an amazing coincidence.

  8. Submitted by Nick Coleman on 09/27/2010 - 04:55 pm.

    Amazing coincidences abound! Like the one that always means a saint, such as Mother Teresa, is always the face that is seen on a cinnamon roll or a piece of strudel — never a sinner!
    It depends on what you’re looking for, I guess.

  9. Submitted by Jeff Michaels on 09/27/2010 - 05:04 pm.

    Star Tribune polling always starts out showing the DFLer with a significant lead. The polls inspire campaign workers and help the candidate raise money.

    Then the poll magically tightens just before the election to give the Star Tribune pollster some degree of cover.

    Dayton’s lead will drop from nine point to two in the final poll. You can bet on it. History does repeat itself.

  10. Submitted by David Brauer on 09/27/2010 - 06:22 pm.

    Jeff – perhaps that’s true (we should check the tale of the tape) but it’s also possible polls tighten as Election Day approaches.

    For what it’s worth, I chose to frame my story the way I did partly because Mitch was using end-of-campaign polls to question a mid-campaign poll. Perhaps he chose an inferior data set.

  11. Submitted by Mitch Berg on 09/27/2010 - 10:16 pm.


    The WaPo has this little bit on the history of the Daves-era MNPoll:

    We’re talking about more than just one election. During the Daves era, the poll underpolled Republicans at all levels (in goober and Senate races) by an average of seven points; DFLer, OTOH, averaged about right.

    That’s the history we’re talking about here.

    By the way, since my admittedly puckish premise – that the MNPoll has been a DFL propaganda tool for a generation – is the subject of your piece, then the 2008 elections are the wrong ones to look at. The DFL didn’t need their morale boosted.

    This year, on the other hand…

  12. Submitted by John Edwards on 09/27/2010 - 10:17 pm.

    The link below is somewhat helpful. The interpretation of the graph showing the Minnesota Poll accuracy is, of course, suspect because I think it was compiled by the Republican Mark Kennedy campaign. But the final polls and the margins are indisputable, so viewers can make their own assessment, even though these numbers do not specifically address Mr. Michaels’ point.

    – 7:02pm
    Rudy Perpich: 49.5%. 52.3% (-6.8). 45.2% (+4.3). 1978 U.S. Senate …. In 1998, the final Minnesota Poll predicted that Skip Humphrey would ……/MinnesotaPoll_kennedycampaign071706.pdf

  13. Submitted by David Brauer on 09/28/2010 - 06:36 am.

    Mitch –

    Let’s just be precise: the document you reference is on the Washington Post *server* but it came from the *Mark Kennedy campaign.* You know Mark Kennedy – the guy whose loss the Strib eventually nailed. Yes, the AKlo race was a blowout, but you can hardly argue the DFL didn’t need motivation that year, given the guv’s race that year.

    I guess we can draw the datasets any way we want, but in one way, this debate is a bit silly because the Strib swapped out pollsters in 2007. I’ve read the rightblogs and know they’re all hatin’ on PSRA too (because it polls for Pew? Really?). But if you look at the 2008 Senate flow –

    … the Strib/PSRA had Coleman up through Sept. 12. Franken did assume the lead, but everyone – with the exception of SUSA and St. Cloud State – had Franken up during this period.

    By mid-October, the Strib poll actually pegged Franken with a smaller margin than Rasmussen. Yes, by the end the Strib was alone with Franken, but as noted above, they didn’t miss the mark any worse than the others, and were a bit closer in fact that Ras or SUSA.

    I don’t object to suspicions about the Strib poll – I have DFL sources who have objected in the past – but I do think it’s worth noting the recent history fully as well as the company change.

    And if you really think Strib management is ga-ga over Dayton (or was over Franken) you really weren’t reading their editorials or talking to people about the leadership over there …

  14. Submitted by Tony Angelo on 09/28/2010 - 09:48 am.

    I disagree that the StarTribune poll is an outlier, in fact the Rasmussen poll from last Friday looks like more of an outlier than the StarTribune poll.

    The top lines for the candidates in the StarTribune poll fall into the same ranges (topline +/- moe) as the other two most recent polls, the MPR/Humphrey poll and the SurveyUSA poll.

    It was the Rasmussen poll that showed a result that no one else has seen, Tom Emmer above 36%.

  15. Submitted by Mitch Berg on 09/28/2010 - 10:02 am.


    Yes, I see that Kennedy was the source of the spreadsheet. The data is still accurate; the Daves era was a study in either bias or statistical incompetence.

    I’m not a conspiracy theory buff, honest. But if you accept that the Minnesota Poll is, through systemic error or editorial bias, slanted toward the DFL (as you don’t, and I see evidence of) and serves (intentionally or not) as a DFL morale-booster this time of every even-numbered year, then the fact that they got the Kennedy race (a depressing blowout) and the 2008 races (a depressing year) more or less correct doesn’t change the conclusion.

    Does Princeton’s regime change things? We’ll see. As you note (and as Ed Morrissey and I discussed on the air this morning), it comes down to how one determines a “likely voter”.

    But accepting this Minnesota Poll requires one to believe that the electorate hasn’t changed since before Tom Emmer started campaigning. It doesn’t pass my sniff test.

    “And if you really think Strib management is ga-ga over Dayton (or was over Franken) you really weren’t reading their editorials”

    No, we’ve had this discussion, David; I don’t disagree. Although I’m gonna guess we disagree on the significance of the Strib’s brass supporting Horner, if that’s the tack they take; you see “Ex-Republican”, I see “DFL Lite”.

    ” or talking to people about the leadership over there …”

    They don’t return *my* calls… 😉

Leave a Reply