The case against merit pay is mounting with the release today of a much-anticipated study out of Nashville that shows no impact on student performance from teacher bonuses.
Produced by the National Center on Performance Incentives at Vanderbilt University, the study examined the test scores of 300 middle school math teachers who agreed to participate in the Project on Incentives in Teaching, a three-year randomized experiment that tested the assumption — an assumption that under girds Race to the Top — that teachers will work harder and produce greater student gains if they are rewarded for it.
Not so, according to the findings, which follow the recent policy brief by the nation’s top ed researchers challenging the effectiveness of merit pay.
“We tested the most basic and foundational question related to performance incentives — Does bonus pay alone improve student outcomes? – and we found that it does not,” said Matthew Springer, executive director of the National Center on Performance Incentives. “These findings should raise the level of the debate to test more nuanced solutions, many of which are being implemented now across the country, to reform teacher compensation and improve student achievement.”
The study found little evidence that merit pay improves student achievement, noting improvement only in 5th grade scores in two of the three years. There was zero impact on the scores of sixth and eighth graders. leading the researchers to conclude that while the merit pay did not create dissent among teachers, “…neither did it yield consistent and lasting gains in test scores. It simply did not do much of anything.”
According to the official statement on the study:
The Project on Incentives in Teaching, called the POINT Experiment, took place over the 2007 – 2009 school years with participation by mathematics teachers in grades 5 through 8 in Metropolitan Nashville Public Schools. Nearly 300 teachers, approximately 70 percent of all middle-school math teachers in Nashville’s public schools, volunteered to participate. The complete study, including setup and analysis, began in 2005 and ended in 2010.
POINT tested no other types of incentives or systems of support for the teachers, such as professional development or guidance on instructional practices – many of which have evolved over the five years since POINT began.
“We designed POINT in this manner not because we believed that an incentive system of this type is the most effective way to improve teaching performance, but because the idea of rewarding teachers on the basis of students’ test scores has gained such currency,” Springer said. “We sought a clean test of the basic proposition: If teachers know they will be rewarded for an increase in their students’ test scores, will test scores go up? We found that the answer to that question is no. That by no means implies that some other incentive plan would not be successful.”
Here’s how the POINT experiment worked:
• Following a year of detailed project design by a multi-disciplinary team from Vanderbilt and RAND, all middle-school math teachers in Nashville were invited to volunteer for the experiment. Approximately 70 percent of all middle-school math teachers in Nashville’s public schools stepped forward to participate.
• Approximately half of the nearly 300 volunteers were randomly assigned to a “treatment” group, in which they were eligible for bonuses of up to $15,000 per year on the basis of their students’ test-score gains on the Tennessee Comprehensive Assessment Program (TCAP).
• The other half were assigned to a “control” group not eligible for these bonuses. Teachers were evaluated based on an historical performance benchmark for MNPS teachers, not on competition with one another. All teachers in the treatment group had the chance to earn bonuses. (The names of participating teachers – and which group they were in – have been kept confidential by the research team.)
• The annual bonus amounts were $5,000, $10,000 or $15,000. Over the course of the experiment, POINT paid out more than $1.27 million in bonuses. Overall, 33.6 percent of the original group received bonuses, with the average bonus being approximately $10,000.
Teacher attrition occurred during the experiment. About half of the 296 teachers who initially volunteered remained through the end of the third year. The teachers who left the study either left the school system, moved to other grades or stopped teaching mathematics. Only one participating teacher specifically asked to be removed from the experiment.
While there was no overall effect on student achievement across the entire treatment group, the researchers found a significant benefit for fifth graders in Year 2 and Year 3 of the experiment: fifth graders taught by teachers who earned bonuses did show gains in test scores. However, the effect did not carry over to sixth grade when students were tested the following year. Springer said this finding raises questions about what is different about fifth grade and what factors –student development, curriculum, teaching and classroom structure – may have played a role.
He also noted that implementation of POINT went smoothly, with no complaints from teachers about the calculation of bonuses, the payment of awards, bonuses they did or did not receive or the fairness of the process. This in itself is a significant finding, Springer said, because historically, teacher associations have opposed performance or merit pay plans, particularly if the pay plan awarded teachers solely on their individual value-added score.