Report: Use student performance to rate teachers

A new report says teacher evaluations should include objective measures of student achievement; drive additional support for teachers; and be used to make hiring, retention and tenure decisions.

Policy 2.0: Using Open Innovation to Reform Teacher Evaluation Systems” spells out new criteria for teacher evaluations in the k-12 . The report was developed by the Hope Street Group, a nonprofit, nonpartisan organization in collaboration with a team of 22 educators, six private sector professionals, and eight participants from the civil society sector.

The report has a lengthy list of student achievement measures:

Value-added data from standardized tests (where available);
Student work, including performance criteria and evidence of student growth;
Teacher-generated information about student goals and growth;
Formative assessments;
Objective performance-based assessments;
Assessments of affective engagement and self-efficacy

Among the comments from teachers involved in the report:

“There is much to do in evaluating
teachers. In my school most teachers
are evaluated every two years and
nothing is done if the teacher is less
than stellar. I truly believe in mentoring,
but the administrative teams, the
unions, the state have to come
together to make it a true path to
better teaching.”

I know that many teachers are suspicious of using student achievement and doubt the reliability of value-added measures.

But this train seems to have left the station. I am going to hear Arne Duncan this morning. (He is in Atlanta to speak to the National Black Child Development Institute.) Duncan and the White House want teacher pay systems to consider student achievement.

In Denver, the local school district has developed an acclaimed performance-pay system that rewards teachers based on 10 weights, including pursuing professional development, working in a high-needs school, receiving a glowing evaluation and raising test scores.

One point that I have heard Matthew Springer, director of the National Center on Performance Incentives at Vanderbilt University, make again and again is that merit pay plans can’t succeed without teacher buy-in.

What would it take to get you to embrace a performance pay plan?

52 comments Add your comment

SallyB

October 26th, 2009
11:52 am

I might consider it IF the student achievement portion were measured by pre-test on material to be taught, post-test on material taught…. AND included an attendance requirement AND allowed special consideration for students in the class who are learning disabled or have behavior disorders , etc. Also, it seems to me that if 50% of a class show vast improvement in knowledge of the material, 20% show adequate improvement, 10-15%show minor improvement, and 10-15% show little or no improvement….the teacher should be considered successful.

V for Vendetta

October 26th, 2009
12:02 pm

Sally B,

I completely agree; however, you and I both know that hoping for such a system is akin to a snipe hunt: It will never happen. This is the government we’re talking about here, and all they want is a quick and easy solution to a nearly indefinable problem. They’ll take a look on one set of scores, possibly a pre test and post test, and use that to make assumptions about a teacher’s qualifications. As you pointed out, it can’t be done.

Though a teacher has much to do with student achievement, so much of student achievement is out of a teacher’s hands it is difficult to measure it objectively. If some studies show that a student’s academic success can be affected by early childhood learning prior to age five, well before the school system gets its hands on him or her, then how do we somehow rate a teacher for that student’s gains or losses later in life? What about students who don’t come to school or have deplorable home environments? What about discipline problems? What about sped kids?

These are questions Mr. Duncan has failed to consider because they can’t easily be legislated into submission like the rest of education. If a system like the one Sally B were proposed, I’d be very willing to buy into it. I can almost promise you that whatver Arne Duncan proposes will be nothing of the sort.

This is what happens when people sit around and let their government run rampant over what used to be considered their freedom. Until someone stands up to them, expect more of the same.

catlady

October 26th, 2009
12:06 pm

And what about the other kids in the class with the mainstreamed or unidentified or unmedicated or undisciplined student? Their scores will generally suffer, too.

Badger

October 26th, 2009
12:08 pm

This type of evaluation will only work when the school systems learns to get rid of the kids who don’t want learn and don’t want to allow anyone else to learn. When the school system gets rid of these kids only then can a teacher effectively teach.

you said it

October 26th, 2009
12:21 pm

SallyB for President. We also must account for students who will not complete homework, English Language Learners, etc. If we were somehow able to measeure student achievement when students actually TRY, then I’m all for it. Unfortunately, V is probably right. The higher ups will look for an easy (yet completely inadequate) way of measuring student achievement. I should be all for this since I teach AP students, but I would feel as if I were stealing money.

V for Vendetta

October 26th, 2009
12:39 pm

you said it,

I know what you mean. I’m getting my Gifted cert. for this coming year. Not a moment too soon, eh?

GOB

October 26th, 2009
12:53 pm

As someone who teaches small group classes, I would need to see some major conditions put in place to make this even remotely workable. Either that or I need to work on my AP certification…

Mac

October 26th, 2009
12:55 pm

Still based on standardized test scores=garbage

Meme

October 26th, 2009
12:57 pm

And I am getting my retirement papers in order. After 35 years things have changed way too much.

FultonTeacher

October 26th, 2009
1:00 pm

And how would non-core teachers be evaluated? This is unrealistic and anyone that teaches knows it.

DeKalb Conservative

October 26th, 2009
1:13 pm

Why even propose this when there is a massive cheating scandal going on? Consider the criteria and their subjectivity:

- Value-added data from standardized tests (where available);
- Student work, including performance criteria and evidence of student growth;
- Teacher-generated information about student goals and growth;
- Formative assessments;
- Objective performance-based assessments;
- Assessments of affective engagement and self-efficacy

How do you quantify these things?

Batgirl

October 26th, 2009
1:27 pm

You people are all right on the money. Our middle school houses only 7th and 8th graders. Our 7th grade teachers are getting kids who read on a first or second grade level and don’t know their multiplication tables. Will those teachers be expected to get them on grade level in order to get merit pay or will they just be required to bring them forward somewhat? I’m afraid those folks high up in government and big business just don’t know the kinds of kids and families that educators know. They only know their own kids and their friends and they’re bright, intelligent, well fed and rested, so I suppose they think all kids must be.

You said it and V, I am in the opposite situation. I am the media specialist, and even though I run two reading programs to help our students, I rarely receive any notice for what I do, and I’m sure I won’t get a pay raise.

oldtimer

October 26th, 2009
2:25 pm

I am so glad I’m done!!

What Would Mr. Ott Say?

October 26th, 2009
2:39 pm

I’m guessing that Mr. Ott will have no problem with such a system – if you wait till all the conditions to be perfect before you change anything, then nothing gets changed. There should be NO question that students achievement is tied to teacher performance, and teacher performance should be tied to the teacher education programs they went through. We need a much better data system to track what is happening in schools.

Old School

October 26th, 2009
2:55 pm

Let’s create a grade 8 1/2 and put in it all students who cannot add, subtract, multiply, divide, read near grade level, write a complete sentence that is also legible, or pass a standardized test. That keeps them out of the middle school population and out of the graduation numbers. So what if they are there until they are 21, they’ll get with the program, learn and move on, or they’ll drop out.

As long as we keep reroofing the house with the sorry, crumbling foundation, we’ll keep investing money where it can do little good and the crumbling will start bringing down the walls.

Warrior Woman

October 26th, 2009
3:01 pm

Why do you need teacher buy-in to implement a performance standard? Professional staff in other industries don’t expect veto power of their management’s performance measurement systems. Student improvement should be the primary driver of teacher evaluations. Objective measures (including pre- and post-tests) combined with student criteria such as attendance and capabilities – the bar should certainly be higher for teachers of gifted and honors classes – should be used along with standardized tests to rate teacher performance. Objective criteria should ALWAYS be preferred over subjective criteria. The goals must be clear, measurable, and reasonable.Teachers that don’t meet minimum requirements in student achievement should not be retained. Teachers that do well in improving their students’ performance should be rewarded.

Just a teacher

October 26th, 2009
3:16 pm

I used to be opposed to using student acheivement to evaluate teachers but not any longer. The biggest difficulty in my field, the arts, is evaluating student performance. I have graduated a number of students through my theatre program who have gone on to earn degrees in performance at outstanding universities and colleges. To me, those would seem like success stories. However, facts like that don’t show up on standardized tests. Does that mean that I am a poor teacher if my students don’t do well on tests that fall outside of my subject area?

d

October 26th, 2009
3:51 pm

My major concern is that without some student accountability, standardized tests aren’t a good measure of the teacher. I’ve had students sleep through the majority of EOCT (despite me trying to wake them up on multiple occasions) just to have them christmas tree the scantron when I announced 5 minutes left. The students know that test is only 15% of their grade and tend to take them less than seriously, especially when they realize they could get a zero and pass the class if they have an 82 going into it. And yes, I have seen this on more than one occasion. If the student knew he or she would have to retake the class if the EOCT was not passed, I think he or she would take the test a bit more seriously.

On the other hand, we could set up an assessment that truly measured student learning.

AlreadySheared

October 26th, 2009
3:56 pm

Given the CRCT, start with a total count of Needs Improvement, Meets, and Exceeds students.
I.e., Of the 100 kids left at the end of the year who were taught for the entire year by Mr. Jones, students came into his 5th grade math classes with 20 Needs Improvement, 60 Meets, and 20 Exceeds from the end of their 4th grade year. After a year with Mr. Jones, this cohort tested out at the end of 5th grade with 10 Needs Improvement, 65 Meets, and 25 Exceeds.

Getting this information for all the 5th grade math teachers in a school system would give you a pretty good idea of who is doing a really good job, and who is doing a really poor one, relatively speaking.

you said it

October 26th, 2009
4:29 pm

AlreadySheared

That only works if the standards build on one another. If the courses are unrelated, then a student could ‘Exceed’ at the end of one year and end up ‘Needing Imporvement’ the next even if the teacher is the same. Take for example middle school social studies. The sixth and seventh grade courses are world geography/history courses. The eighth grade course is Georgia Studies. How does learning about the history of conflicts in the Middle East prepare a student to learn about the geography of Georgia? The curriculum is unrelated. A pre and post test would be more appropriate for that situation. Your math example works only because concepts in math are cumulative. Or am I wrong? Math teachers-would that work?

you said it

October 26th, 2009
4:30 pm

I forgot to add that there are not standardized tests at every grade level, so what do you use then?

Badger

October 26th, 2009
4:42 pm

I volunteer at my daughters high school here in Cobb. for the most part I help out in AP classes but I have one normal class that I work in. Its amazing at the difference in the students. The AP students want to learn, they study hard and they interact during class. Even after class and on weekends they get together in study groups. These kids want to learn, want to be a productive part of society. Now lets look at the normal classes.

For the most part the majority of students in class are smart intelligent students. Except for a select few they want to succeed, graduate and have a good life. Those select few ruin everything for everyone. They are disruptive when they actually show up in class, almost never complete homework and other than being disruptive or smart !@#’es they have no useful reason to be there. Unfortunately unless they commit a mortal sin the teacher and the class are stuck with them. Get these hoodlums out of the classroom. Yes I said hoodlums, pants sagging to the back of their knees no recognizable language or than maybe what some call ebonics, what useful purpose is it to have them in class. Unfortunately they are black which in turn give the black students a bad reputation or adds to the one they already have.

Marie

October 26th, 2009
4:45 pm

For those of you who are not teachers, consider teaching a full load of high school math classes which are not college prep classes, only one parent per class bothers to show up for the PTSA meet the teacher night, and the majority of the students will not do any work outside of class and many won’t even listen in class and are disruptive. You call parents of students who disrupt class and are often told “Good luck, I can’t do anything with them at home either.” Some place in my education past, I recall that a student’s grade in a math class is 50% of their math background, 25% effort, and 25% the classroom teacher. How in the world do we come up with a fair way to make “pay for student performance” work?

Drew

October 26th, 2009
5:43 pm

I will never buy in. If it’s touched by a standardized test, I’m out.

GA Teacher

October 26th, 2009
6:03 pm

I would go for it if it were legitimate. The problem is that it will be developed by a bunch of ultra-liberal education professors who live in a world where all children are inherently interested in performing to their greatest capabilities. Anyone who has actually spent time in an urban school and has not just watched Dangerous Minds knows that some children (the # goes up as the children become more ghetto) CHOOSE to waste their time and there is nothing that anyone can do about it. That is the true problem–there are no REAL consequences for those who refuse to perform in school OR life.

ATEACHER

October 26th, 2009
6:52 pm

What if the teacher has kids moving in and out of the class all year- how do you evaluate that teacher? One year in my 2nd grade class of 19 I had 17 move in and out. My room was akin to a store- stop by, learn a little, and move on. Some kids have TERRIBLE home lives- how can I help those children succeed on standarized tests?

catlady

October 26th, 2009
6:54 pm

In an fair, objective world it would work. However, kids are not widgets. Not only are they not tuned to a fine variance, but many are actually OPPOSED to learning anything. And they don’t just mess up for themselves; they screw up the whole class.

Complicate that with tests that are not vaild and reliable. (see CRCT)

Notice that in second grade most of the CRCT is read to the student, but in third grade they are on their own. So comparing a fair cohort won’t work with that.

Add to it some schools have 70% turnover from year to year. You gonna decide about a teacher based on 8 students (who may not have moved because they lack the resources to do so),

I’d be all for pay for performance if I could get a class whose success depends mostly on my work, if they could be fairly/competently evaluated, and if the class situation could be completely controled by me: discipline and ways to meet the GPS, instead of having those decisions taken out of my hands and made by those who have been out of the classroom for decades.

catlady

October 26th, 2009
6:56 pm

Clogged filter!

Lee

October 26th, 2009
7:17 pm

Let’s see, the same bunch who passed illiterates from grade to grade prompting the passage of the hideous NCLB, now wants to work it’s magic on the teachers.

The same administrators who hide behind zero tolerance policies because they are incapable of making a decision merely want a metric that they can plug some numbers into and announce whether or not a teacher is worth a crap.

News flash. The other teachers, parents, and students already know this.

AlreadySheared

October 26th, 2009
7:19 pm

you said it:

If you used the aggregated information to compare teachers ACROSS the same grade, you could get useful information.

For example, suppose “Improved” meant going up in performance, say from “Needs Improvement” to “Meets”, or “Meets” to “Exceeds”, and “degraded” meant going down, say from “”Exceeds” to “Meets”, or “Meets” to “Needs Improvement” .

Suppose also that we decided to represent teacher performance as a pie chart, where “improved” is colored green, “degraded” is colored red, and outcomes where the student’s status doesn’t change are grey.

THEN, you could look at the results of all the teachers of, say, Georgia geography. If all the pies look about the same then it is what it is. But if you look at a collection of pies and some are a lot redder than the average Georgia geography teachers’ pie, that would tell you something useful. Same for pies that were a lot greener than average.

It would give you pretty good information on who is doing a good job, who is doing a bad job, and who is average. And yes, if the same teacher is “Mr. Red Pie” for two or three years running, that would seem to me to be a STRONG indication that either
1) Mr. Red Pie is in serious need of some additional training, or failing that,
2) a career change is in order, either voluntary or involuntary.

ScienceTeacher671

October 26th, 2009
7:23 pm

In 9th grade physical science, whether or not students pass the EOCT depends as much on how they did on their 8th grade reading and math CRCT tests as it does on who the teacher might be. They have to be able to read the test, and they need basic math skills.

AlreadySheared

October 26th, 2009
7:46 pm

Then I guess for 9th grade science, “improved” and “degraded” would be a function of 8th grade math and reading crcts.

True story:
Back in the day when I was taking sophomore physics at Georgia Tech, the physics department had made the decision that all physics classes would take the same standardized quiz every Friday. I believe this came about because before then the various physics profs would prepare exams for their classes which varied wildly in difficulty (for the same class).

This exam structure, along with the fact that the classes were taught in large auditoriums, led to the following delightful outcome. Some classes ended up being standing room only, while the less competent teachers tended to give their lectures to half-empty auditoriums. No point putting up with Dr. Smith’s useless, incoherent lecture (even if you had scheduled his class) if you had an open period at the right time to catch Dr. Jones’ helpful instruction. Just show up at the right time on Friday to take the quiz!

V for Vendetta

October 26th, 2009
8:26 pm

Already Sheared,

Some important things to note:

1. Teachers do not choose their students. I have had wildly different bunches from year to year or period to period. I would LOVE to be evaluated based on some, but I would CRINGE at being evaluated based on others. How do you account for such discrepancies?

2. Of course college students would react that way. At Georgia Tech, you’re talking about students who were probably in the top 5% of their graduating classes. They desire education, so, when faced with a decision that impacts their educations, they choose what is best for them. Some of my students couldn’t point to Georgia on a map. Do you think they even CARE what’s in their best interests? (Assuming, of course, they are competent enough to know what’s in their best interests in the first place.)

Let me put it this way: You can’t quantify the job of a teacher for the same reason you can’t quantify the job of a doctor. A doctor might receive some patients who are relatively healthy, some who have serious ailments, and some with life-threatening diseases. Of course, if he were to get a disproportionately large number of critical patients over a certain period of time, one would not automatically conclude that he is a bad doctor if a higher percentage of them died.

Unfortunately, in our PC government-run school system, a child cannot be labeled as a “loser” even if he has shown all the symptoms of it. He’s just a student.

And no matter what he does, if he fails, the teacher takes the blame.

mike

October 26th, 2009
8:28 pm

Enter your comments here

mike

October 26th, 2009
8:33 pm

Good evening.

I have read the published comments above. Many are thoughtful. I teach school, and from what I can tell, and I am told I am pretty successful at it. I still learn daily. I am not sure how to evaluate teachers. Many schemes have been proposed. Some, I believe have greater merit than others. I think I can recognize good teaching. What bothers me is something beyond the evaluation. What is the purpose of the American education system? Lots of ideas have been floated, and some appear valid, but I think until we decide what it is we expect the education system to do, then evaluation of teachers, and much other evaluation is problematic.

AlreadySheared

October 26th, 2009
8:44 pm

V:
1) Then I would imagine that your not-so-good bunches would offer LOTS of chances for “green” (and very little chance of “red”), whereas color-wise, you might not move the needle much in your strong classes.

2) Reading back, I was unclear. I wasn’t talking about the students, but rather the professors. They didn’t let on much, but I really think the jilted profs teaching to half-empty halls sort of didn’t like it. I myself don’t like being measured and judged, but that doesn’t mean it shouldn’t happen.

Maureen Downey

October 26th, 2009
8:48 pm

Catlady,
Sorry. You’re out.
Maureen

plc

October 26th, 2009
8:51 pm

As a teacher in a leadership position, I would personally love to see student performance tied to a SMALL portion of teacher pay or at least the teacher evaluation. There are too many teachers who are horrible and give us a bad name, either through not caring or not knowing just how badly they are teaching. The data would speak for itself. Even through a bad test, you could at least show the teacher that their kids scored 10 points below the other averages at the school. It would be a great way to get rid of these teachers that we all complain about! As much as I love job security, this is one area where I think we need to be more like the corporate world – our success and pay should be impacted by performance and productivity.

ScienceTeacher671

October 26th, 2009
8:55 pm

Sheared, V makes some excellent points about high school students.

All things being equal, I wouldn’t mind being judged by the test scores of my students – but all things are rarely equal. Most likely my advanced class will all “exceed expectations”, and I expect my “average” class to perform better than the state average…but when it comes to the class full of students who are on IEPs and/or were socially promoted to high school because they were turning 16 this year, who knows?

The student who hasn’t passed a single section of the CRCT since maybe 4th grade (if then) probably won’t be able to pass the EOCT either, so I guess that student won’t be “improved” or “degraded”…s/he might be able to successfully complete and orally explain the lab project, but that doesn’t necessarily translate to performance on the written test…

AlreadySheared

October 26th, 2009
8:55 pm

ATEACHER,
I beat you to it – reread the below:
“Of the 100 kids left at the end of the year who were taught for the entire year by Mr. Jones,”

AlreadySheared

October 26th, 2009
9:13 pm

“who were taught for the ENTIRE YEAR by Mr. Jones,”
(emphasis added)

[...] intend added hold for teachers; and be utilised to make. Here is the example post:  Report: Use enrollee action to evaluate teachers | Get Schooled Posted in Tea, Uncategorized | Tags: law-school-, new-report-, objective-measures, [...]

SET

October 26th, 2009
10:14 pm

What a stupid notion.

Any attempt to rate teachers on student performance will result in a direct negative premium in being assigned Negro (love that word!) students. One can beat around the bush about just trying to avoid dull students in favor of bright ones, but to speak plainly as I tend to do – you’d want Jewish kids in your assigned classrooms, not blacks.

Because all people are not created equal. Some people tend to – in groups – test higher.

It’s hard enough to get teachers willing to work with black students as it is.

So there’s your pay them by performance issue. Who in their right mind would work under such nonsense?

MathMan

October 27th, 2009
6:34 am

Mr. Ott and Warrior Woman obviously don’t know the story about the difference between public education and how Blue Bell Blueberry Ice Cream is made.

What Would Mr. Ott Say?

October 27th, 2009
8:23 am

Well, all you teachers are just incredibly creative with coming up excuses. If teachers can’t be evaluated based on students’ achievement, how else should they be evaluated? How they dress? How much they weigh? How loud they can scream? If you can’t improve students’ achievement from the beginning of the year to the end of the year, you don’t deserve to be in classrooms. Get out!!!

Who ran the AJC into the ground?

October 27th, 2009
10:36 am

By the logic of this report, shouldn’t newspaper editors and journalists be accountable for declines in circulation?

Rural Education

October 27th, 2009
11:37 am

Really should be simple. Give a pre-test sometime during the first week and then give the exact test at the end of the year. Since we are already being trained to teach to the test this seems to be fair. The thought of letting adminstrators being responsible for some sort of pay evaluation is scary. You only need three years of classrooom experience to become an adminstrator in Ga. and we see how well that has worked.

Maureen Downey

October 27th, 2009
11:56 am

Who ran the AJC, I can tell you that reporters/bloggers are held accountable for what can be measured directly, including hits on the Web. We all have access to a service that measures online readership day by day.
To your specific comment: The decline in print readers is more than offset by the massive surge in online readership, but newspapers have yet to figure out how to turn the growth in online readership into a profit-making enterprise.
If readership counts were based on AJC.com traffic, we’d all have raises this year.
Maureen

Twenty Year Veteran

October 28th, 2009
9:58 am

Well, What would Mr. Ott Say?, since you have all the answers, when are you coming into the classroom? Since you know best what we should do and how we should be evaluated, come on down! Show us all how it should be done! I just love experts who’ve never taught but still know best how we should do our jobs, don’t ya’ll?? Reminds me of the old saying: just because I have an [anus] doesn’t make me a proctologist…just because you went to school doesn’t make you an educator!

brumley70

October 28th, 2009
1:43 pm

I teach in a very impoverished area of KY. I would LOVE to see success of a student tied to the welfare checks of parents. I can teach, use all the theory in the world, curriculum map myself to death, go to all of the PD that I can go to, but if a child and their parents do not view education as a priority, I cannot change that. Have parents get their welfare checks cut down according to the success of their child and you would see parents jumping to help their children succeed. They would beg us as teachers to help and even possibily see all the work we as teachers put in to see their child succeed.