Nonprofit, nonpartisan journalism. Supported by readers.

MinnPost's education reporting is made possible by a grant from the Bush Foundation.

On Michelle Rhee: Two studies, a dissection of them — and varying conclusions

Michelle Rhee
Michelle Rhee

How many posts about the former schools chief of a city that’s 1,100 miles from here can one blog suffer? I don’t know, but we’re going to find out.

It seems I was a few days premature in writing about the controversies that continue to dog erstwhile D.C. chancellor and bee-eater Michelle Rhee. Her reforms are the subject of two separate, critical national studies, themselves the subject of a new 3,200-word dissection in a high-profile education policy journal.

The first study, undertaken by Alan Ginsburg, a former director of Policy and Program Studies in the U. S. Department of Education, found that contrary to claims by Rhee supporters, D.C. students performed worse during her tenure than they did under her predecessors. See “The Rhee D.C. Record: Math and Reading Gains No Better than Her Predecessors Vance and Janey.” 

The second, by a committee of the highly respected National Academy of Science’s National Research Council (NRC), found that gains in student test scores in D.C. between 2007 and 2009 were no better than in 10 other districts for which comparable data was available. You can read a pre-publication version of “A Plan for Evaluating the District of Columbia’s Public Schools: From Impressions to Evidence.”

The article in Education Next, by Executive Editor Paul E. Peterson, the director of Harvard University’s Program on Education Policy and Governance, asserts that both reports made factual and analytical errors in an attempt to discredit Rhee. His conclusion: Rhee wasn’t in office long enough for anyone to draw firm conclusions about her impact on student performance.

If you are hooked on Rhee-parsing as contact sport, you can call up a footnoted version of the piece.

(Full disclosure: Two years ago I accepted a freelance assignment from Education Next, with which I have no ongoing relationship. I know its scholarly editorial team is widely regarded as conservative. I was not pushed to take any particular slant on my wholly apolitical topic — teacher-run cooperatives — just encouraged to be skeptical.)

I am not going to run any numbers for you, because I have been down this road before and know beyond any doubt it’s above my pay grade. Which is kind of my point: Confronted with the kinds of statistics that are used to compare one set of students to another in terms of achievement, most of us are forced to decide whether we trust the person doing the math enough to swallow what they say the take-away should be.

All three reviews build their cases not on D.C.’s equivalent of the MCAs, but on the more credible, apples-to-apples National Assessment of Educational Progress test results. Often referred to as the nation’s report card, the NAEPs are taken by a representative sample of students.

I wrote about Florida’s NAEP test results in this space two months ago after Star Tribune columnist Katherine Kersten claimed that given that state’s numbers, Minnesota ought to emulate former Gov. Jeb Bush’s reforms. Kersten did in fact supply a set of eye-popping results, but I poked around a little and found that not only had she omitted some case-puncturing ones, there were caveats big enough to drive a fleet of big yellow buses through. 

After that post, I was contacted by a local education policy advocate and self-described statistics geek who spent the better part of two work days slicing NAEP data with me to demonstrate that what the tests really show is that Minnesota kids outpace Florida’s on all but a couple of markers.

You never saw that post for a couple of reasons, chief among them that by the time all of the asterisks and qualifiers had been appended it was every bit as long as Peterson’s piece. Worse, unlike him I could not actually understand the analysis I was contemplating publishing, much less defend it.

(I confess to you in total honesty that I took to my fainting couch, rendered weak by the memory of a humiliating episode where I, a graduate student working in the Los Angeles Times D.C. bureau, regurgitated a federal Labor Department release crediting former President George Bush Sr. with bringing unemployment to an historic low. I did not realize he did so by deciding hundreds of thousands of people were never going back to work and should simply be erased from the rolls.)

So, no numerical arguments from me. I do, however, want to highlight one interesting passage from the EdNext story concerning the National Academy of Sciences panel’s report. The committee — which was made up of people who parse data for a living — concluded that test scores had risen modestly during Rhee’s time and continued to do so. But the scientists would not say whether this happened because of her reforms; in effect, no data linked correlation to causation, they said.

You know what that sounds like to me? Standardized tests don’t necessarily tell us much more than how a particular student, on a given day, does answering a series of questions that are actually quite limited in what they measure. They do not reveal whether a botched equation is the result of a missed breakfast, years’ worth of missing skills or a union-protected dullard of a teacher.

There is one number I think I understand. The National Academy of Sciences raised and spent $650,000 compiling its report, which it describes not as an assessment of Rhee’s reforms but “guidance on how to structure” said evaluation. I make that to be about 76 times the average amount of state funding Minnesota provides for each public school student.

Comments (8)

  1. Submitted by Paul Udstrand on 04/14/2011 - 09:46 am.

    There was a lively discussion about the previous article about Rhee. I was one of the people who continue to question Rhee’s results. Now to be fair, that article was actually about accusations that test answers were actually changed in order to improve results. As far as I can tell that issue hasn’t really, and may never be resolved. There was a higher than normal rate of erasures changed to correct answers, but no one has definitively explained that. The thing that bothered me was Ms. Hawkins apparent premature dismissal of the controversy, and the suggestion that this is about Rhee’s personality not her results.

    I’m not interested in nor do I know anything about Rhee’s personality. I applaud Ms. Hawkins integrity for following up with an article that looks at data. As for the controversy surround Rhee, I don’t see the big mystery, she’s and her acolyte’s have been promoting themselves as the answer to America’s education crises, obviously they attract scrutiny when they do that. Since this is one of the biggest issues/problems of our time I would expect that scrutiny to endure until some reliable observations can be agreed upon. I don’t see this as being about Rhee or her personality.

    As for the Bee Eater, well coming from a background in psychology all I can tell you parents is this: regardless of your parenting style don’t take too much credit for your kids success. The fact is that there are wide variety of parenting styles that produce healthy, educated, and well adjusted children and adults. This is a testament to the adoptive capacity of children, not the brilliance of any particular parenting style. The fact is children have an amazing capacity to thrive regardless of parenting styles (short of outright abuse… and even then…). Every so often a parent writes a book… “do what I did”. These books tell you more about the parents ego than they do the success of their parenting styles.

  2. Submitted by Paul Udstrand on 04/14/2011 - 10:55 am.

    I think we can safely say goodnight Gracie as far as this “revolution” is concerned. Two studies claim no difference, or poor results, and one simply says there no reliable data one way or the other.

  3. Submitted by Dan Hintz on 04/14/2011 - 02:02 pm.

    Excellent article Beth. I know that I (and others) have been far more critical of Rhee than you have, but I give you a lot of credit for bringing up criticism of Rhee and being even handed.

    The most interesting thing here was the conclusion by Rhee defender Paul Peterson:

    “Rhee wasn’t in office long enough for anyone to draw firm conclusions about her impact on student performance.”

    OK, if that is true, why is Rhee being held out as this great reformer? Why is her work in D.C. being touted as a model for other districts? If the best you can say about her was that you can’t draw conclusions about her work, why are so many people drawing the conclusion that she was right?

  4. Submitted by Eric Ferguson on 04/14/2011 - 03:02 pm.

    It seems the issue isn’t that the studies are premature, but that claims were made about big improvements she allegedly made. For her or her supporters to react to negative studies by saying it’s two soon to tell means they made their claims when it was even more too soon to tell. At the best, at the very best, there’s no way to know.

  5. Submitted by Andrew Kearney on 04/14/2011 - 03:28 pm.

    School reform efforts (and reformers) can be usefully classified into two groups. Those reforms that have been shown to increase student achievement and those that are about adult needs. Needless to say most school reformers are in group number two and from across the political/ideological continuum. Michelle Rhee got into trouble because she tried to change practices with adult needs hoping that they would change student achievement. This is the reason most in education roll their eyes at most school reform. It doesn’t have much to do with student achievement and is more about some need of some adults (to bash teachers, to have small class size etc.) I must say that I am not sure how an education reporter can be so if she can not understand student achievment data. Are you sure this is your line of work?

  6. Submitted by Paul Udstrand on 04/14/2011 - 07:09 pm.

    Hmmm, meant to say children are adaptive, not adoptive in #1. Resilient is even better.

  7. Submitted by Bernice Vetsch on 04/15/2011 - 03:52 pm.

    Dan (#3) — Those people who are drawing the conclusion that Rhee was right do so because it aids and abets their anti-public school, anti-unionized teachers and anti-union in general, plus the notion that charter schools always produce better results and should replace public schools wherever possible.

    Diane Ravitch, former undersecretary for research in the Bush administration, demolishes the radical changes being pushed by Rhee and Arne Duncan (who, according to an independent study commissioned by a group of civic-minded Chicago businesspeople, actually did dumb down tests to make it look as though students had made greater “progress” than was true).

    Ravitch left the administration when she decided No Child Left Behind was doing more damage than good for America’s children. Google her name for articles by and about her work since then.

  8. Submitted by Dan Hintz on 04/15/2011 - 10:22 pm.

    Bernice, my question was more rhetorical – I agree with you completely. I know Ravitch’s writings – her review of the fraud that is Waiting for Superman is excellent.

Leave a Reply