Nonprofit, nonpartisan journalism. Supported by readers.


Fact-checking fact-checkers: Politifact’s anti-GOP skew and MPR’s reconsideration

Fact-checking sites have received more attention in recent years.

Fact-checking sites have received more attention in recent years. Today, the verdicts of two — the Pulitzer-winning Florida-based site Politifact, and PoliGraph, Minnesota Public Radio’s collaboration with the University of Minnesota — come under the microscope.

In his “Smart Politics” blog, spreadsheet wizard Eric Ostermeier strongly suggests “selection bias” at Politifact. Between January 2010 and 2011, Politifact rated roughly the same number of statements by GOP and Democratic office-holders. However, Republicans received 76 percent of the worst ratings (“false” or “pants on fire”).

Of Politifact, Ostermeier concludes, “It appears the sport of choice is game hunting — and the game is elephants.”

Ostermeier, a University of Minnesota researcher, doesn’t fact-check the fact-checks, nor does he suggest Democratic statements Poltifact has avoided. Liberals — pointing to the fact-challenged Michele Bachmann and Sarah Palin — might respond that reality has a liberal bias.

Article continues after advertisement

Ostermeier’s data is valuable and thought-provoking, though when he ventures off the tote board, his speculation isn’t always convincing.

For example, he notes that during most of his review period, Republicans were the party out of power, implying that Politifact should have found more lies by the party in control. But does that necessarily follow?

The party out of power, desperate to get back in, could have more incentive to throw wild rhetorical bombs. The 2010 off-year election was all about motivating the base, which Republicans and the Tea Party did far better than the Dems. But more red meat means more chance of spoilage.

I think Ostermeier makes a good point that Politifact isn’t transparent or rigorous enough about its statement-picking process. He doesn’t interview Politifact editor Bill Adair, though he does include older quotes. I’ve sent an email to St. Petersburg and we’ll see what comes back.

Coincidentally, Ostermeier’s U colleague, Catherine Richert, had to reconsider one of her PoliGraph verdicts yesterday.

After originally judging a Gov. Mark Dayton “State of the State” claim on high college tuition “false,” Richert changed it to “misleading.” But I think that verdict applies to her reconsideration more than the governor’s.

Richert usually does a solid job, and MPR is not known as an itchy trigger-finger operation. However, it appears in their haste to fact-check the speech in near-real-time, they didn’t have the administration’s underlying data.

Dayton’ statement: 

“Tuition in our state’s two-year public colleges has risen to the third-highest in the nation; tuitions in our four-year universities are among the top-ten highest.”

Article continues after advertisement

Richert noted — accurately — that no single state college or university ranked that high. That led to her initial “false.” However, it turned out Dayton was using state averages, which confirmed the rankings.

Having hit “send” too soon a few times in my life, I’ve experienced adrenaline-fueled defensiveness, but I think MPR didn’t walk back its item enough. Dayton’s speech clearly refers to groups — “colleges,” “universities.” At worst, he was unclear, not misleading.

Richert prefers to look at individual schools, where no Minnesota college or university is in the Top 10. 

This is a perfectly valid perspective; in fact, she made a similar case a couple of weeks ago when judging ex-Gov. Tim Pawlenty’s claim about public-sector versus private-sector pay.

Like Dayton, Pawlenty used group averages, in this case to assert public workers made more. Richert noted in many instances, job-to-job comparisons showed public workers making less. In the end, though, Pawlenty’s claim was rated “accurate.”

This time around, Richert writes, accurately, “There’s more than one way to look at the cost of the higher education.” She probably should have left it at that.