Another blow for merit pay: Long-awaited Tennessee study finds no impact on student achievement

The case against merit pay is mounting with the release today of a much-anticipated study out of Nashville that shows no impact on student performance from teacher bonuses.

Produced by the National Center on Performance Incentives at Vanderbilt University, the study examined the test scores of 300 middle school math teachers who agreed to participate in the Project on Incentives in Teaching, a three-year randomized experiment that tested the assumption — an assumption that under girds Race to the Top — that teachers will work harder and produce greater student gains if they are rewarded for it.

Not so, according to the findings, which follow the recent policy brief by the nation’s top ed researchers challenging the effectiveness of merit pay.

“We tested the most basic and foundational question related to performance incentives — Does bonus pay alone improve student outcomes? – and we found that it does not,” said Matthew Springer, executive director of the National Center on Performance Incentives. “These findings should raise the level of the debate to test more nuanced solutions, many of which are being implemented now across the country, to reform teacher compensation and improve student achievement.”

The study found little evidence that merit pay improves student achievement, noting improvement only in 5th grade scores in two of the three years. There was zero impact on the scores of sixth and eighth graders. leading the researchers to conclude that while the merit pay did not create dissent among teachers, “…neither did it yield consistent and lasting gains in test scores. It simply did not do much of anything.”

According to the official statement on the study:

The Project on Incentives in Teaching, called the POINT Experiment, took place over the 2007 – 2009 school years with participation by mathematics teachers in grades 5 through 8 in Metropolitan Nashville Public Schools. Nearly 300 teachers, approximately 70 percent of all middle-school math teachers in Nashville’s public schools, volunteered to participate. The complete study, including setup and analysis, began in 2005 and ended in 2010.

POINT tested no other types of incentives or systems of support for the teachers, such as professional development or guidance on instructional practices – many of which have evolved over the five years since POINT began.

“We designed POINT in this manner not because we believed that an incentive system of this type is the most effective way to improve teaching performance, but because the idea of rewarding teachers on the basis of students’ test scores has gained such currency,” Springer said. “We sought a clean test of the basic proposition: If teachers know they will be rewarded for an increase in their students’ test scores, will test scores go up? We found that the answer to that question is no. That by no means implies that some other incentive plan would not be successful.”

Here’s how the POINT experiment worked:

Following a year of detailed project design by a multi-disciplinary team from Vanderbilt and RAND, all middle-school math teachers in Nashville were invited to volunteer for the experiment. Approximately 70 percent of all middle-school math teachers in Nashville’s public schools stepped forward to participate.

Approximately half of the nearly 300 volunteers were randomly assigned to a “treatment” group, in which they were eligible for bonuses of up to $15,000 per year on the basis of their students’ test-score gains on the Tennessee Comprehensive Assessment Program (TCAP).

The other half were assigned to a “control” group not eligible for these bonuses. Teachers were evaluated based on an historical performance benchmark for MNPS teachers, not on competition with one another. All teachers in the treatment group had the chance to earn bonuses. (The names of participating teachers – and which group they were in – have been kept confidential by the research team.)

The annual bonus amounts were $5,000, $10,000 or $15,000. Over the course of the experiment, POINT paid out more than $1.27 million in bonuses. Overall, 33.6 percent of the original group received bonuses, with the average bonus being approximately $10,000.

Teacher attrition occurred during the experiment. About half of the 296 teachers who initially volunteered remained through the end of the third year. The teachers who left the study either left the school system, moved to other grades or stopped teaching mathematics. Only one participating teacher specifically asked to be removed from the experiment.

While there was no overall effect on student achievement across the entire treatment group, the researchers found a significant benefit for fifth graders in Year 2 and Year 3 of the experiment: fifth graders taught by teachers who earned bonuses did show gains in test scores. However, the effect did not carry over to sixth grade when students were tested the following year. Springer said this finding raises questions about what is different about fifth grade and what factors –student development, curriculum, teaching and classroom structure – may have played a role.

He also noted that implementation of POINT went smoothly, with no complaints from teachers about the calculation of bonuses, the payment of awards, bonuses they did or did not receive or the fairness of the process. This in itself is a significant finding, Springer said, because historically, teacher associations have opposed performance or merit pay plans, particularly if the pay plan awarded teachers solely on their individual value-added score.

78 comments Add your comment


September 21st, 2010
5:37 pm

Regardless of what the research says, the politicians and business leaders will continue to push for this kind of radical change to teacher pay scales. Politicians don’t really care what the research says. They only care about the perceptions they are able to generate.

Sixth grade teacher

September 21st, 2010
6:33 pm

@Tony — I agree. I suspect that those outside of education who are screaming for merit pay are doing so under the (subconscious?) belief that “Many other professions set salaries based on performance, so why should teachers be any different?” As with so many aspects of education, that’s reasonable in theory but very different in practice.

My main concern with merit pay is that most of the proposals simply cannot be implemented in a truly fair and equitable way for all teachers. I teach in a metro Atlanta county with a vast range of income and CRCT performance levels, from schools in the top ten statewide to those on their fifth or sixth year of not meeting AYP. My low-income school did finally make AYP for enough consecutive years to take us off the “low performing” list. (Yay!) We’ve made significant percentage point gains every year, and many of my colleagues would probably qualify for merit pay under the most common proposals. What about teachers in high-income schools that boast 99% pass rates every year — will they not qualify because they don’t show “gains”? (And given how their students come to them with high scores in the previous year, do those pass rates mean they’re genuinely *better* teachers than those in low-performing schools who raise students’ scores?)

Then there’s the issue of data measurement. AYP is currently based on this year’s eighth graders vs. LAST year’s eighth graders — two completely different groups of human beings. It simply isn’t fair to judge a teacher’s or school’s performance on apples vs. oranges. If I were to enter a merit pay program, I’d want the criteria to be based on the scores of my current students compared to how the same kids did in the previous year. But gathering that data requires a heck of a lot more work than simply looking at overall pass rates… which is why it will never happen.

(And, of course, let’s not even get into the massive problems with the proposals based on highly subjective teacher evaluation instruments!)

Like I said above, I think that many teachers ARE in favor of merit pay if we could be assured that it was done in a fair and reasonable manner, but there are just too many ways that it could backfire and punish deserving teachers while rewarding those who are good-but-not-great.


September 21st, 2010
6:42 pm

Once again, the “whining” teachers are proved correct by research. Maybe some people, including people who post on this blog, should realize that we really do know what we’re talking about.


September 21st, 2010
6:44 pm

Well, I was just about to send you the link, and you’ve already seen the writeup of the study! Very good!

And it suggests one of the things we have been saying. Doesn’t work. (Unless you want to use the Atlanta model, which involves changing answers, of course).

Old Timer Educator

September 21st, 2010
6:45 pm

My biggest problem with the idea of merit pay is the underlying supposition: let’s pay teachers to do a better job because obviously they’re not doing their best. It wouldn’t matter if they gave me a bonus of $1000/kid for each student who passes the End of Course Test because I’m already doing the best job I can. They don’t have to pay me more to get “more teaching” because I figure they’re already paying me to do the job. I just find the whole concept insulting.


September 21st, 2010
6:55 pm

Sixth grade teacher–also a massive problem of what our CRCT does NOT do. It isn’t a valid short-term test, much less one that can be used for longitudinal comparisons!

What the results suggest to me is that promising teachers more money for higher scores–that teachers are already putting in full effort to help students; that the limitations in gains are NOT due to teacher effort!


September 21st, 2010
6:57 pm

But of course any student failure is really failure by the teacher, right?

History Teacher

September 21st, 2010
7:00 pm

The real reason that merit pay is being pushed is because the state legislature can decide every year not to fund the bonus because of budget shortfalls. they dont have that control over step raises.My real experiences with how the state legislature has honored their agreement with me for my national board certification supplement and my bonus for jumping through the hoops to acheive master teacher status show that everything that comes out of the state legislature in regard to education is completely worthless. I have more faith in my daily horoscope.


September 21st, 2010
7:20 pm

I want teacher free agency. I am holding out next fall for a new contract. Since I am the best science teacher, my students are 92% passs rate, that is .900 to you baseball folks. I should recieve a signing bonus, a press conference and a 5-10 year deal worth $8-10 million a year. I know I will blow the schools salary cap, but that is not my problem, I am the best!!

You can always cut me in camp or trade me if you are not happy with my preformance or we don’t make AYP, you can bench me for a few days, until I straighten up.


September 21st, 2010
7:47 pm

I am not a teacher or even have a horse in the race. I just think that teacher pay and student performance are not necessarily connected. I think like most people teachers do their jobs and take their paychecks for doing just that. Teaching takes not only a willing teacher but a willing student. Some students will be great and all any teacher can do is get in their way. Others will be just so-so and can be helped with some encouragement and many others will be total wastes of time no matter how good the teacher or curriculum is. Way too many variables to tie student scores to teacher salaries.

Jordan Kohanim

September 21st, 2010
7:58 pm

At this point, keep the money–just shrink my class sizes. Please–for the kids’ sakes.

Maureen Downey

September 21st, 2010
8:18 pm

@Jordan, Glad to see you back. I just read your note and plan to respond. You are always a voice of sanity and civility on this blog and we need more of that here. (For any nutcases who are taking offense at my comment, it was intended.)


September 21st, 2010
8:27 pm

Georgia is way too backwards to pay any attention to this kind of research. We’ll just keep throwing our money away on useless tests while the kids’ brains waste away.


September 21st, 2010
8:46 pm

Amen to History Teacher. So true. So many snakes in the gold dome.


September 21st, 2010
8:51 pm

Merit pay. Can someone explain how this is going to help get those “great, high achieving teachers” to teach classes in low performing schools, SPED, or collaborative classes? Not many people want those classes/students (LOTS of work!!!) as it is, why would someone take even LESS money to deal with all the extra crap associated with them?

not my usual name

September 21st, 2010
8:53 pm

I’m not using my usual name, since my next sentence would give away my school. Sonny’s coming to my school on Thursday to roll out the state’s new fitness program. If I run into him (fat chance), should I ask him about the new study? Or maybe one of Alvin’s people (since Alvin’s not coming)? What do you think they will say?


September 21st, 2010
9:01 pm

I agree Jordan….shrink the class size and PLEASE send me money to buy lab supplies.

Here's a thought...

September 21st, 2010
9:11 pm

Newsflash! Teachers aren’t in the profession for the pay. We work hard everyday and want our students to be successful. If we wanted to make more money, we’d be in corporate. So don’t dangle a carrot in front of us thinking that we will run any faster. We are already going full speed.


September 21st, 2010
9:12 pm

I still like Robert J. Samuelson’s article from last week’s Newsweek, “Why School ‘Reform’ Fails“.

Public Teacher

September 21st, 2010
9:14 pm

Study after study after study reports the same results. The State of Florida crashed horribly after their attempt at merit pay. Now Tennessee clearly shows that it has zero benefit.

So why doesn’t GA try it? Well, to start think of the cost involved to accomplish this “merit pay” stuff. Collecting the data and report it not by school system, not by school, not by grade, but by TEACHER. That alone has a huge cost associated with it.

Then, someone has to come up with the scale for the merit pay – and justify it. There will be court battles and lengthy politican battles over this one.

Then, a group of administrators paid for by the State will have to manage this mess. Again, more cost involved.

And all of this for what? To force a change to Georgia education that has been PROVED time and again that it will not help.

But hey, we Georgians continue to vote for republicans here at home and our State is in the crapper anyway, so why not?


September 21st, 2010
9:30 pm

Does Georgia ever adopt a reform before it’s been shown not to work in half the other states?

Maureen Downey

September 21st, 2010
9:32 pm

@ScienceTeacher, We were passing the 66 percent rule on school spending just about when all the studies were coming out to say it didn’t make a bit a difference. Didn’t faze the legislators one bit.


September 21st, 2010
9:44 pm

How many states will waste their Race to the Top grant money (which runs out eventually) on merit pay in education? It failed in Florida and the latest research out of Tennessee shows it has “no merit” on test scores and it will never come to pass in states that recognize unions because the unions will not bargain for it (states in the Northeast and Midwest). Unlike the business world, educators do not turn out a product that is an inanimate object like a car, food products, cleaning products, housewares, furniture, etc. We turn out “educated human beings” that unlike inanimate objects, have free will and can choose to study, can choose to pay attention, can choose to come to school, can choose to behave in class, etc. We also deal with factors beyond our control that affect our “final product” such as the lack of parental involvement and socioeconomic factors to name a few. I don’t believe doctors are paid by the number of lives they save, lawyers by the number of cases they win nor dentists by the number of human mouths free of cavities and root canals. Each of these professions deal with human products (in some sort of capacity) as well as teachers. Why should teachers be paid based on test scores of human products when these other highly paid professionals dealing with human products are not paid based on some type of “merit”?


September 21st, 2010
9:52 pm

“Why should teachers be paid based on test scores of human products when these other highly paid professionals dealing with human products are not paid based on some type of “merit”?”…because we are paid with tax dollars and politicians are in charge of that. (and they are a pretty slow group)

V for Vendetta

September 21st, 2010
9:55 pm

Here’s a thought . . . ,

I have a family to feed, so I AM in it for the money. Take your faux nobility somewhere else. However, I do NOT think merit pay can be equitably distributed, so it is really a moot point. This study only confirms what all of the teachers already knew. Now, if an entire school paid teachers more in order to attract the best and the brightest–with the knowledge that it could fire at will any teachers who did not perform their duties to the best of their abilities–that would be a different story.

(And don’t pretend that if someone suddenly gave you the power to fire five teachers from your school, you couldn’t think of five losers to fire. We all know who they are. You don’t need statistics or test scores to know who the bad teachers are. It’s obvious. Just think of the message such an action would send . . . .)


September 21st, 2010
11:38 pm

Wow. Almost uniform agreement that merit pay doesn’t work. I have to agree that I’m in the choir on this one.

The question that I’ve been asking myself lately, is how do you tell the difference between a teacher that entertains their students and one that educates them? Note: I’m not saying entertainment and education are mutually exclusive. But you can walk into a classroom where all the kids are excited and engaged, and that doesn’t necessarily mean they are all learning.

What if any types of assessment and accountability models do teachers believe to be fair and reasonably accurate? Where are these models useful and how do you prevent them from being abused?

Ann Duffy

September 22nd, 2010
1:33 am

Setting aside the issue of whether or not merit pay increases student achievement, another interesting finding from this study is that this merit pay program was implemented smoothly. That is very unusual for merit pay programs, which tend to be littered with complaints about fair implementation or lack of funding. So, this one study seems to find that a merit pay program can be implemented well, without disrupting teaching or undercutting the professionalism of teachers. Of course, there doesn’t seem to be any reason to do it based on student achievement after 5th grade. Great addition to the debate….


September 22nd, 2010
2:48 am

Standing in the hallway tonight I was talking with two other teachers about the topic of raises. We were still at school at 8:00 pm for a family reading night. We had arrived at work at 6:45, taught all day, and came back to school for this PTO event for 2 hours in the evening. Despite being tired and feeling under appreciated, we all agreed – Keep our raises, just please reduce these class sizes! THAT is the single biggest change that will influence student achievement.


September 22nd, 2010
6:13 am

Maureen, there’s also block scheduling, math 123, and several other things I’ll probably be able to think of after another cup of coffee…

Math Teacher

September 22nd, 2010
7:07 am

I don’t think the fact that none of the teachers complained is as significant as they make it sound. If you’re going to give me a bonus based on test scores, that’s one thing. But, if you are possibly going to reduce my salary based on how my students do, that’s a different story.


September 22nd, 2010
8:42 am

@st671-The Newsweek article is a good one.

I would like to see GA keep the current pay-scale that rewards teachers for advanced degrees and annual COLA raises. At the same time, GA could offer a BONUS to teachers with strong test scores and/or evidence of growth using the Class Keys. I also like the idea of offering bonuses to the entire school if they reach certain benchmarks. Schools must work together as teams. Every member of the team….special education teacher, librarian, fine arts instructor, counselor, vocational teacher, academic teacher…..plays an important role.

I understand the argument that some teachers have turned to the infamous “diploma mills” for the advanced degrees. Instead of taking away the pay raise that comes with an advanced degree, just close the door to the “diploma mills.” Encourage more state colleges/universities to offer cohorts in areas of state that have limited access to a college or university (I live in an area where the closest college is 45 miles away…down a winding mountain road). Professional development is key to building a strong teaching force.

[...] Read more: Another blow for merit pay: Long-awaited Tennessee study finds no … [...]


September 22nd, 2010
8:50 am

So if merit pay doesn’t increase performance, how about automatic raises for longevity and degrees? Wouldn’t we save a ton of money we set a minimum salary and let teachers get raises the same as everyone else – sucking up to their bosses?


September 22nd, 2010
9:15 am

A good teacher is a good teacher, a great teacher is a great teacher. All the pay in the world will not make a bad teacher a great teacher. USe the money for professional development. For certain don’t base the pay on CRCT scores. Crazy!

Hey Teacher

September 22nd, 2010
11:09 am

@ Jordan — amen sister. I’d also like a duty-free lunch, a bathroom break, paper towels in the restroom (budget cuts — and the air blower never works) , tissues for my students (I have to buy them myself), and a stapler that works. Also on my wish list would be some planning time, but if I’d have to choose, I’d take the lower class sizes, even over the paper towels.

Richard Woods

September 22nd, 2010
11:10 am

The news on this topic is that there has never been news to report on it. We have more research and more knowledge about educating students than ever before. Merit pay has never been presented or documented to make a substantial or lasting impact on student and teacher performance. Unfortunately in Georgia, we continue to ignore what research has shown to make substantial and lasting gains in learning.

One should perhaps follow the money trail. We either continue to pursue the federal breadline or owe our allegiance to those who stand to make a buck from education. I wonder how much money testing and remedial service organizations spend in Georgia or donate to political candidates. We also continue to elect the same individuals or same types of individuals to make our educational leadership decisions. If you keep doing what you have been doing, you will keep getting what you have been getting.

For those schools who have poor teachers, either the administration is not or is not being allowed to do their job. It is like illegal immigration in this country, we have laws and regulations; we just need to enforce them. However, it does appear Georgia will adopt the merit pay concept under RttT. Perhaps the original intent of the concept was not to improve education but a backdoor attempt to reduce state cost for education.


September 22nd, 2010
11:23 am

Maureen, why not do something that makes sense about true educational reform? I taught my first class in 1973 and just retired last year. I have seen them come and seen them go. Everyone is afraid to make bold statements. I knew merit pay did not work. If I got a 3% pay raise I did not work 3% harder. I always did everything I could as hard as I could. Everyone blames the teacher. Here is one concept, why not blame the system? It would be so easy to change but everyone is scared to change it into something that is very simple. This country, as best as I can tell, is the only country that tries to make every student into a well rounded everything person. We are always compared to every other country so why not follow their lead? They let students pursue their areas of interest. I never understood why I had to take art and music. I hated art and music and did everything I could to get out of thise classes including disrupting class. I wanted to study the sciences, specifically biology and physics/physical science. I could not do that until i was well into my college life. I took every class in those 2 areas I could get my hands on, not for a degree but just to learn, understand and expose me to different ways of looking at things. I hated math but took every math course I could just to study what I loved. Kids today are no different. So, why do we have a canned curriculum that everyone MUST take? Student motivation is key to learning and we are destroying that. Do we not change because it is so easy?


September 22nd, 2010
11:24 am

Please excuse my typos. My grandson is sitting in my lap helping me type.

Batgirl, a.k.a. Nutcase

September 22nd, 2010
12:21 pm

@teacher & mom, thank you for pointing out that we all play an important part. I am a school librarian, and if it were up to some of the teachers to pick five people they would get rid of as someone else suggested, I would be one of them because they think a monkey can do my job. Plus, I was a very good classroom teacher, but this is the direction I wanted to take my career.

As for the diploma mills, the state has stopped accepting degrees from some of them, but it just seems that more of them pop up.

What's Best for Kids?

September 22nd, 2010
2:17 pm

Casey Cagle suggested Merit Pay in Georgia as a way to save money. I think that says it all. I agree with Richard Woods. Someone is trying to find a way to take money from teachers, not to introduce incentives to good teachers.


September 22nd, 2010
2:18 pm

I agree with catlady: “teachers are already putting in full effort to help students; that the limitations in gains are NOT due to teacher effort!”

Fruthermore, I have my time wasted less frequently (so, in a sense, I work “less”) at my current school, which has a higher local supplement than my former school, because here my students are better behaved in general and my administration does not give me busy work or ask me to help do its job (eg. dealing with tardies on my planning period.)


September 22nd, 2010
2:20 pm

As much as I want to say “na-na, teachers were right,” about this study, I do have a few quibbles:
-the teachers self-selected to participate.
-only math teachers participated.
-only middle school teachers participated.

The social scientist in me knows that we should hesitate before generalizing these findings to other subjects and other grade levels.


September 22nd, 2010
2:49 pm

If a merit pay system doesn’t work, perhaps lower salaries won’t reduce students’ performance, either. We can save a lot of tax money by cutting teacher salary by 30%.

I am a teacher

September 22nd, 2010
4:12 pm

What about the ability of merit pay, or some kind of common-sense approach to deciding who gets paid what, to draw a higher level of applicant into the field? A lot of high achieving college students won’t even consider teaching because they don’t want to put in 10-15 years before they can think of making a decent living. If the profession were a meritocracy, the best and brightest might feel differently.

Step raises and rewarding degrees that rarely help in the classroom are not the way to keep going…

Clayco Parent

September 22nd, 2010
5:24 pm

Do we really need a study to tell us that teachers can’t crank up student performance just b/c they can get a few extra bucks for it? Compare teachers to healthcare workers with patients in a hospital. The nurses (teachers) take care of patients (students) and facilitate the healing process (learning) through various medical interventions (teaching methods, curriculum, etc). Would you expect a nurse to take better care of his/her patients if they got a bonus if the patient didn’t kick the bucket within a year? Patients are ALOT like students….they all have unique issues, problems, family support (or not) and attitudes about their health. The problem patients continue smoking, drinking, overeating and not keeping their doctor’s appointments. I imagine problem students have their own set of issues that keep them from achieving to their potential. I consider myself fairly satisfied with my profession, but if some brilliant SUIT decided my pay would be determined based on my patients outcomes, I’d go to one of these cracker jack trade schools and learn a trade that doesn’t depend on the input or participation of anyone else in order for the job to get done right….Yeah I can’t think of anything that fits that description either. Dang


September 22nd, 2010
5:46 pm

Clayco Parent, marvelous analogy! :)


September 22nd, 2010
5:52 pm

Good luck finding qualified teachers who would work for 30% less than what teachers already make, especially once the economy improves. And most especially, in high-need areas like high school math and science, and all special ed. At one school in which I worked, it took them over a year to find a qualified math teacher… and this was in one of the largest urban areas in Georgia!

Your education=business thought process does very little but show your ignorance of the difference between the two.

We can, however, make a lovely analogy with nurses and teachers in this instance, too. Remember what happened when nurses’ salaries were in the basement? We had a huge, dangerous shortage of nurses which, in some specialties and locations, continues today. It turns out that you can’t pay people peanuts for doing a tough job, or they will find something else to do, even if they love that job. I’m sure that’s some sort of economic theory or principle, but I’ll let you tell us which one, LLL, since I know you are clearly so well-versed in economic theory.

Ole Guy

September 22nd, 2010
7:29 pm

So it took a “study” of this nature to “officialize” that which we have known forever. Problems cannot/will not be solved merely by tossing money in the general direction of the issue. Simply brilliant what these high-powered studies reveal!


September 22nd, 2010
7:34 pm


LLL has an interesting idea – though I’m not sure how serious s/he is. Politicians assume merit pay will increase students’ achievements, but research seems to show otherwise. You assume low pay will decrease the number of “qualified” teachers – and therefore lower students’ achievements. But, is your assumption really correct? If it is wrong – no difference in students’ achievement? If the pay doesn’t matter, and if we want students’ achievements as a product of schooling, then, why not lower teacher pay? If you are really concerned about getting qualified teachers to high need schools, then give teachers at “high needs” schools 150% or even 200% of salaries of teachers at non high-needs schools. Make those high-needs schools the only place where teachers can make really good money – and the salaries tied to the schools.

Jordan Kohanim

September 22nd, 2010
8:30 pm

Clayco– to borrow your analogy and extend it further–our “patients’ often don’t even know there is anything wrong with them. They can’t understand why they should change a history of poor academic behavior if it got them this far. So we have to diagnose, treat, and battle with resistant and unhealthy habits. Again–keep the money; just shrink my classes please.