MinnPost's education reporting is made possible by a grant from the Bush Foundation.

Teacher merit pay is no ‘magic bullet,’ study indicates

NASHVILLE — Offering middle-school math teachers bonuses of up to $15,000 did not produce gains in student test scores, Vanderbilt University researchers found in what they said was the first scientifically rigorous test of merit pay.

The results could amount to a cautionary flag about paying teachers for the performance of their students, a reform strategy the Obama administration and many states and school districts have favored despite lukewarm support or outright opposition from teachers’ unions.

The U.S. Department of Education, in part through its Race to the Top competition, has pushed states to find ways to reward teachers for their performance in an effort to make teaching careers more attractive and retain those who are most effective. Teachers’ unions have often objected to merit pay on the grounds that they don’t have fair and reliable ways to measure teacher performance. In most school districts, teacher pay is based on years of experience and educational attainment levels.

The report’s authors, of the National Center on Performance Incentives (NCPI) at Vanderbilt University’s Peabody College of Education, stress that their research design was just one of many possible approaches to giving teachers incentives to boost student achievement. The Nashville teachers who hit the mark based on their students’ test scores received a bump in their paychecks for their efforts but no additional mentoring or professional development. Neither their principals nor fellow teachers knew who participated in the experiment or who received bonuses.

Matthew G. Springer, director of the federally funded NCPI, said pay-for-performance is not “the magic bullet that so often the policy world is looking for.”

At least in this experiment, Springer said, “it doesn’t work.”

Research from 2006 to 2009
The study was conducted from 2006 to 2009 in partnership with the nonprofit RAND Corp. A local industrialist and Vanderbilt benefactor, Orrin Ingram, put up the nearly $1.3 million in bonuses.

Some 296 middle-school math teachers — two-thirds of the district’s middle-school math teachers – volunteered to participate in the experiment. Half were placed randomly in a control group, while the rest were eligible for bonuses of $5,000, $10,000 or $15,000 if their pupils scored significantly higher than expected on the statewide exam known as the Tennessee Comprehensive Assessment Program.

One third of the eligible teachers — 51 of 152, or 34 percent — got bonuses at least once. Eighteen teachers received bonuses all three years.

Except for some temporary gains for fifth-graders, though, their students progressed no faster than those in classes taught by the 144 other teachers.

The local teachers’ union in Nashville agreed to the experiment in collective bargaining, according to Erick Huth, president of the Metropolitan Nashville Education Association. He said the results were not at all surprising.

“I’ve believed for a long time that what improves instruction is having an instructional leader who is able to get all players in a school to collaborate,” Huth said.

The bonuses amounted to as much as 30 percent of teachers’ yearly salaries here in the Music City, where the scale runs from $36,000 to $64,000, Huth said. The nation’s 3.2 million public school teachers earned $53,910 on average in 2008-09, according to the National Center for Education Statistics.

The study was released last week at a two-day conference, “Evaluating and Rewarding Educator Effectiveness,” at Vanderbilt’s Peabody College that drew participants from Colorado, Georgia, Tennessee, Washington, D.C., and other places conducting their own experiments with performance pay. Some states have moved to tie teacher and principal evaluations to student test scores.

Education secretary’s faith unshaken
The study did not shake the faith of U.S. Secretary of Education Arne Duncan in merit pay.

“While this is a good study, it only looked at the narrow question of whether more pay motivates teachers to try harder,” said Sandra Abrevaya, a spokeswoman for Duncan. It did not address the Obama administration’s push to “change the culture of teaching by giving all educators the feedback they need to get better.”

The Nashville experiment, known as POINT (Project on Incentives in Teaching), doled out the $15,000 bonuses to those teachers whose students performed “at a level that historically had been reached by only the top 5 percent of middle school math teachers.” Teacher performance was calculated by using a value-added model, which predicts how students will do in a given year based on how they performed in the previous year. The teachers had to hit the 80th and 90th percentiles to pocket the $5,000 and $10,000 bonuses, respectively.

The study’s design, in which teachers were judged against percentile benchmarks rather than their colleagues’ performance, sought to preserve collaboration among teachers.

In surveys about the program, most teachers said they were already effective without the incentive of additional pay. Eight in ten said they didn’t change the way they taught to improve their odds of earning a bonus. Many teachers came close to getting a bonus — so close that they would have qualified if their pupils answered two or three more questions correctly on the 55-question state exam.

The Nashville math teachers, according to the study, “expressed moderately favorable views toward performance pay in general, though less so for POINT in particular.” The experiment ran smoothly, although the teachers became less enthusiastic over the three years. “They did not come away … thinking it had harmed their schools,” the study said. “But by and large, they did not endorse the nation that bonus recipients were better teachers.”

The fact that many fifth-grade teachers teach multiple subjects to the same students may have been a reason for the positive impact of merit pay found in fifth grade, according to the study’s authors. But “the effect did not last. By the end of sixth grade it did not matter whether a student’s fifth-grade math teacher had been in the treatment or control group,” the study said.

It ‘simply did not do much of anything’
The researchers said the Nashville experiment didn’t stir the negative reactions that have attended some other merit pay programs, but it “simply did not do much of anything.”

Springer of NCPI said the study lays a foundation for further experiments on a topic that educators have been debating “for over a century.”

Tennessee Commissioner of Education Timothy Webb said the Nashville study shows, “Money alone is not enough to encourage people to go into challenging schools and teach the most difficult students.” He stressed the importance of improving teachers’ working conditions, not just their pay.

Frederick M. Hess, director of education policy studies at the American Enterprise Institute in Washington, D.C., said he does not believe the study says much of value and worries it will only confuse the issue.

“The fact that teachers don’t respond to cash bonuses like rats do to food-pellets does nothing to diminish my confidence that it’s good for schooling if teacher pay better reflects the contributions that teachers make,” Hess said. “Serious proponents of merit pay believe the point is not any kind of short-term test-score bump but making the profession more attractive to talented candidates.”

William Slotnik, executive director of the Boston-based Institute for Compensation Reform and Student Learning at the Community Training and Assistance Center, has argued that performance-based compensation tied directly to the educational mission of a school district can be a lever to transform schools. But he said it will take more than financial incentives to improve student achievement and that merit pay “is hard to get right. … If all you are doing is focusing on money, there is no track record in that resulting in the kind of changes needed to do this work well.”

Read the report for yourself [PDF]

Christopher Connell is a freelance writer. Liz Willen contributed to this article. Beth Hawkins is away; this piece is provided by The Hechinger Report, a MinnPost partner in education coverage.

Comments (5)

  1. Submitted by NIcole Masika on 09/29/2010 - 01:03 pm.

    this seems so obvious to me, the actual performance comes from the STUDENTS, how does offering to pay the teacher more help a kid who has problems at home or got off to a bad start? Yes, a good teacher may find ways to help these kids and deserve some credit for that, but test scores actually measure student performance, not teacher performance

  2. Submitted by Robert Ryan on 09/29/2010 - 01:07 pm.

    This quote struck me: “I’ve believed for a long time that what improves instruction is having an instructional leader who is able to get all players in a school to collaborate,” Huth said.

    I once saw a study conducted by UNESCO that indicated that the single most important factor in improving student achievement was the Principal. In my own experience I’ve seen that to be true, not only because the principal can get teachers working together, but also that they can get more cooperation and support from parents.

  3. Submitted by Ray Schoch on 09/29/2010 - 04:11 pm.

    Thanks to Nicole (#1).

    I’ve said essentially the same thing in several other posts on educational topics on MinnPost. It’s not a TEACHER achievement gap that’s grabbing the headlines. Teachers have already graduated from college. They’re licensed. They’ve passed their tests. They’ve presented the lessons. It’s the STUDENTS who are performing poorly, whether on statewide, standardized tests to which far too much importance is being attached, or according to more nuanced and comprehensive evaluations from their teachers (and sometimes, their employers).

    It doesn’t make sense to reward teachers for kids who do well. It doesn’t make sense to punish teachers for kids who do poorly. The teacher is an important part of the process, and there’s plenty of research to show that a good teacher makes a real difference, but there are a vast array of factors completely beyond the control of the teacher that influence student achievement, or the lack thereof. Moreover, at present, there are no consequences to students (except, sadly, in the long-term, lifetime sense) for blowing off a test, a course, an academic year, as trivial, unimportant, boring, and there are no consequences, meaning benefits, to students for performing superbly on the same test, in the same course, over the same academic year, except in that same long-term, lifetime, sense.

    We live in a culture where scholars are called “geeks” and “nerds,” and are dismissed and ignored when growing up, while we lavish attention on athletes and entertainers. Then we complain that academic performance is lagging.

    Sigh.

  4. Submitted by Ron Gotzman on 09/30/2010 - 08:29 am.

    “It does not work?” Well let us be willing to try something that may work.

    Instead of investing more and more money into the “largest special interest group in MN,” and then have that money “trickle down” to the kids. Let us invest directly in kids.

    Maybe if we invest in “the children” our kids can have the same kind of education that Mr. Obama enjoyed and his children currently enjoy.

    Giving more and more money to “schools” so they can give more and more money to the DFL has not worked and will not work.

  5. Submitted by Bernice Vetsch on 09/30/2010 - 09:32 am.

    From how far to the right did the idea that teaching and learning are competitive endeavors rather than cooperative?

Leave a Reply