Teacher merit pay is no ‘magic bullet,’ study indicates

NASHVILLE — Offering middle-school math teachers bonuses of up to $15,000 did not produce gains in student test scores, Vanderbilt University researchers found in what they said was the first scientifically rigorous test of merit pay.

The results could amount to a cautionary flag about paying teachers for the performance of their students, a reform strategy the Obama administration and many states and school districts have favored despite lukewarm support or outright opposition from teachers’ unions.

The U.S. Department of Education, in part through its Race to the Top competition, has pushed states to find ways to reward teachers for their performance in an effort to make teaching careers more attractive and retain those who are most effective. Teachers’ unions have often objected to merit pay on the grounds that they don’t have fair and reliable ways to measure teacher performance. In most school districts, teacher pay is based on years of experience and educational attainment levels.

The report’s authors, of the National Center on Performance Incentives (NCPI) at Vanderbilt University’s Peabody College of Education, stress that their research design was just one of many possible approaches to giving teachers incentives to boost student achievement. The Nashville teachers who hit the mark based on their students’ test scores received a bump in their paychecks for their efforts but no additional mentoring or professional development. Neither their principals nor fellow teachers knew who participated in the experiment or who received bonuses.

Matthew G. Springer, director of the federally funded NCPI, said pay-for-performance is not “the magic bullet that so often the policy world is looking for.”

At least in this experiment, Springer said, “it doesn’t work.”

Research from 2006 to 2009
The study was conducted from 2006 to 2009 in partnership with the nonprofit RAND Corp. A local industrialist and Vanderbilt benefactor, Orrin Ingram, put up the nearly $1.3 million in bonuses.

Some 296 middle-school math teachers — two-thirds of the district’s middle-school math teachers – volunteered to participate in the experiment. Half were placed randomly in a control group, while the rest were eligible for bonuses of $5,000, $10,000 or $15,000 if their pupils scored significantly higher than expected on the statewide exam known as the Tennessee Comprehensive Assessment Program.

One third of the eligible teachers — 51 of 152, or 34 percent — got bonuses at least once. Eighteen teachers received bonuses all three years.

Except for some temporary gains for fifth-graders, though, their students progressed no faster than those in classes taught by the 144 other teachers.

The local teachers’ union in Nashville agreed to the experiment in collective bargaining, according to Erick Huth, president of the Metropolitan Nashville Education Association. He said the results were not at all surprising.

“I’ve believed for a long time that what improves instruction is having an instructional leader who is able to get all players in a school to collaborate,” Huth said.

The bonuses amounted to as much as 30 percent of teachers’ yearly salaries here in the Music City, where the scale runs from $36,000 to $64,000, Huth said. The nation’s 3.2 million public school teachers earned $53,910 on average in 2008-09, according to the National Center for Education Statistics.

The study was released last week at a two-day conference, “Evaluating and Rewarding Educator Effectiveness,” at Vanderbilt’s Peabody College that drew participants from Colorado, Georgia, Tennessee, Washington, D.C., and other places conducting their own experiments with performance pay. Some states have moved to tie teacher and principal evaluations to student test scores.

Education secretary’s faith unshaken
The study did not shake the faith of U.S. Secretary of Education Arne Duncan in merit pay.

“While this is a good study, it only looked at the narrow question of whether more pay motivates teachers to try harder,” said Sandra Abrevaya, a spokeswoman for Duncan. It did not address the Obama administration’s push to “change the culture of teaching by giving all educators the feedback they need to get better.”

The Nashville experiment, known as POINT (Project on Incentives in Teaching), doled out the $15,000 bonuses to those teachers whose students performed “at a level that historically had been reached by only the top 5 percent of middle school math teachers.” Teacher performance was calculated by using a value-added model, which predicts how students will do in a given year based on how they performed in the previous year. The teachers had to hit the 80th and 90th percentiles to pocket the $5,000 and $10,000 bonuses, respectively.

The study’s design, in which teachers were judged against percentile benchmarks rather than their colleagues’ performance, sought to preserve collaboration among teachers.

In surveys about the program, most teachers said they were already effective without the incentive of additional pay. Eight in ten said they didn’t change the way they taught to improve their odds of earning a bonus. Many teachers came close to getting a bonus — so close that they would have qualified if their pupils answered two or three more questions correctly on the 55-question state exam.

The Nashville math teachers, according to the study, “expressed moderately favorable views toward performance pay in general, though less so for POINT in particular.” The experiment ran smoothly, although the teachers became less enthusiastic over the three years. “They did not come away … thinking it had harmed their schools,” the study said. “But by and large, they did not endorse the nation that bonus recipients were better teachers.”

The fact that many fifth-grade teachers teach multiple subjects to the same students may have been a reason for the positive impact of merit pay found in fifth grade, according to the study’s authors. But “the effect did not last. By the end of sixth grade it did not matter whether a student’s fifth-grade math teacher had been in the treatment or control group,” the study said.

It ‘simply did not do much of anything’
The researchers said the Nashville experiment didn’t stir the negative reactions that have attended some other merit pay programs, but it “simply did not do much of anything.”

Springer of NCPI said the study lays a foundation for further experiments on a topic that educators have been debating “for over a century.”

Tennessee Commissioner of Education Timothy Webb said the Nashville study shows, “Money alone is not enough to encourage people to go into challenging schools and teach the most difficult students.” He stressed the importance of improving teachers’ working conditions, not just their pay.

Frederick M. Hess, director of education policy studies at the American Enterprise Institute in Washington, D.C., said he does not believe the study says much of value and worries it will only confuse the issue.

“The fact that teachers don’t respond to cash bonuses like rats do to food-pellets does nothing to diminish my confidence that it’s good for schooling if teacher pay better reflects the contributions that teachers make,” Hess said. “Serious proponents of merit pay believe the point is not any kind of short-term test-score bump but making the profession more attractive to talented candidates.”

William Slotnik, executive director of the Boston-based Institute for Compensation Reform and Student Learning at the Community Training and Assistance Center, has argued that performance-based compensation tied directly to the educational mission of a school district can be a lever to transform schools. But he said it will take more than financial incentives to improve student achievement and that merit pay “is hard to get right. … If all you are doing is focusing on money, there is no track record in that resulting in the kind of changes needed to do this work well.”

Read the report for yourself [PDF]

Christopher Connell is a freelance writer. Liz Willen contributed to this article. Beth Hawkins is away; this piece is provided by The Hechinger Report, a MinnPost partner in education coverage.