Here is the Governor’s Teacher Survey results, per your requests

Much thanks to my AJC colleague Jim Galloway – read his Political Insider blog – who helped me with a PDF that Bert Brantley of the Governor’s Office sent me in response to your requests for the teacher survey results that the state says suggest support for performance pay. (And thanks to Bert to sending it so quickly.)

Here is the message from Bert with the link to the survey.

Maureen, I promised you the results of the Teacher Survey, both with the questions, results and some idea of where the respondents came from.

Let us know if you have questions.

Bert

Bert Brantley, Director of Communications, Office of Governor Sonny Perdue

33 comments Add your comment

Mark1

January 25th, 2010
12:51 pm

Where are the questions on teacher’s pay being influencsed by test scores-It’s not there. This survey is terrible. It has nothing to do with merit pay.

Confused

January 25th, 2010
1:17 pm

It’s pretty clear that the governor’s statements last week were an exaggeration at best on what teachers support. I’m looking through it and I don’t see any result that shows me that 80% of teachers want a pay system based on test scores.

lyncoln

January 25th, 2010
1:23 pm

The best question about test scores influencing pay would be the one about should salary increases be based on effective teacher ratings. And more than 50% didn’t agree with that statement. It seems that teachers do believe that effective teaching is a good thing to measure, but not a good thing to base their salaries on.

I thought the conclusion about a state-wide plan for teacher evaluation was also poor.

The question was “do you think a state-wide evaluation system would
make it easier for everyone to be evaluated the same way?”

The conclusion is that most teachers support having such a state-wide system. I don’t think the conclusion is well supported by the question. I can agree that everyone using the same evaluation standard would standardize the measurement, but I might still disagree that we should use the same evaluation standard.

I also notice that the responses were made mostly by teachers who work in their core subject, which is the inverse of most teachers in the state. I wonder if that had any influence on the way the answers were provided. Is there some sort of distinct difference in the respones of a teacher in their core subject vs. a teacher in a non-core subject?

Mark1

January 25th, 2010
1:35 pm

If Georgia does receive the RACE TO THE TOP (the reason for the fake survey) does that help out Georgia Education Budget at all?–What was the possible sum — 300-400 million? Also the budget for 2011 does not include teacher furloughs but it also does not include funding to prevent teachers from being furloughed- Is there is legislator talking at all about any revenue source? All I hear is no-tax because it is an election year. Is there any legislature discussing 1/2 cent sales tax. In other words is there any hope the General Assembly supporting education at all?

CH

January 25th, 2010
1:38 pm

As the education budget cut crisis looms over the state, the department still finds ways to spend tons of $ on pointless surveys. Most teachers are forced to do these surveys by the administration while they have 1000 other things going on. I usually wind up just bubbling “strongly agree” with everything so I don’t waste a half hour analyzing each question. Therefore the validity of any conclusion drawn from these surveys may be questionable. Let’s think about this proposal though…if I know that I going to get paid more based on my students performance on some stupid standardized test, then I’m (along with every other teacher) going to fight to get those “elite” students in my class. Guess what students get left out???

Pete

January 25th, 2010
2:15 pm

lyncoln,

I personally thought it was strange to ask if teachers “think a state-wide evaluation system would make it easier for everyone to be evaluated the same way?” Georgia DOES have a state wide teacher evaluation system- The Georgia Teacher Observation Instrument (GTOI) and the Georgia Teacher Duties and Responsibilies Instrument (GTDRI). Each teacher is observed using the GTOI at least once a year, and each teachers “effectiveness” as indicated by the State End of Course Test (EOCT), CRCT, or other applicable test results, is detailed in the GTDRI each year. How can something be “easier” than itself?

Mac

January 25th, 2010
2:22 pm

Man, that survey instrument is full of fail. The percentage of educators that actually responded is much too small to make any claims one way or the other. To say the least, the format and wording is biased and manipulative. The only question that even approached the pay for performance issue showed only 47% of that pitiful sample number in favor.

Idea – Lets send out another, better designed survey now that the attention is there and we can most probably get a lot more participation and see where the numbers fall.

If the people behind this survey refuse to do so then I would say that entirely proves there was intentional manipulation going on here.

Public School Parent

January 25th, 2010
2:35 pm

As a parent of two children in our public schools I was dismayed to see that while the majority of the teachers feel that their effectiveness is being judged fairly well, that 63% of participants say that ineffective teachers are not consistently being removed from the classroom.

My take away is that the state and individual school systems don’t need a new teacher evaluation/pay system nearly as much as the principals and administrators need to remove chronically poor teachers from the classroom. My experience is that 95% of my children’s teachers are good to very good (some are good teachers but have horrible organizational skills). But about 5% have been awful and this can ruin a school year. And these are not new teachers but are teachers who have been in the classroom for 10 or 20 years.

Does the Race To the Top application include a mandatory requirement that poor teachers be let go?

look at the numbers

January 25th, 2010
2:48 pm

The teachers who responded to the survery represented 13% of teachers state-wide. How is that anywhere near the blanket statement of 80% of teachers want merit pay?

clueless

January 25th, 2010
3:28 pm

Fewer than 15% of the teachers statewide responded, and those teachers aren’t a representative sample…look at core vs. non-core students. ‘Nuff said?

Teacherforlife

January 25th, 2010
5:04 pm

Whoever analyzed the results of this survey must not be able to pass the 5th grade CRCT! My 5th graders could tell you that this survey is not valid based only upon the small number of responses.

Get Right

January 25th, 2010
5:52 pm

Are sure these were teachers that responded? Let’s do a TRUE survey of TRUE teachers and share it TRULY! This is a worthless survey with waisted time and money all over it.

just browsing

January 25th, 2010
7:05 pm

This is such a flawed instrument. The worse part is that it was sold to the general public as having teacher support. No wonder Georgia is in the shape it is in. Totally unethical and running afoul of what teachers REALLY think about merit pay. Just terrible.

ScienceTeacher671

January 25th, 2010
7:21 pm

Our district is listed as among those participating, but I did my own informal survey of approximately 13% of the teachers in our school, and none could remember ever seeing this survey instrument before.

Disgusted

January 25th, 2010
7:33 pm

Someone should send this to Washington sling with the Race to the Top application. I guess the cheating and misrepresentation we see in our schools surrounding high stakes testing is just a mirror of this kind of the same from our top leaders. Really disturbing.

Disgusted

January 25th, 2010
7:36 pm

sling = along.

Benny

January 25th, 2010
8:16 pm

Mac – great idea but we are dealing with the state and idiocy of a government official. Do you really believe they could come up with a valid instrument? Sonny?

jim d

January 25th, 2010
8:31 pm

I cant help but wonder if they checked for excessive erasures?

ScienceTeacher671

January 25th, 2010
8:52 pm

jim d, that’s funny, I don’t care who you are! ;-)

Veteran teacher, 2

January 25th, 2010
9:11 pm

And we would believe anything else the Governor or his cronies say after looking at this?!?!? My gosh, they really do think we are all stupid!!!!

V for Vendetta

January 25th, 2010
9:27 pm

LOL, jim d!

Let me echo your statement on the other blog: LEGALIZE AND TAX MARAJUANA! Boom. Education funded.

catlady

January 25th, 2010
10:20 pm

Vet2–we haven’t shown them much better to be convinced that we are awake and sentient. We act like dumb sheep.

d

January 25th, 2010
10:44 pm

Well this is an interesting read….. so, basically the evaluation process I’ve been a part of in two counties (GTEP, GTDRI, etc) are not state-wide? 47% is an overwhelming majority? That must come from the new math curriculum. Neither teacher organization was a part of the process which is going to cost Georgia points…. hopefully enough to stop this application before it comes to fruition. Seriously, principals coming into education with no prior education experience — and only a bachelor’s degree?

metro atlanta teacher

January 25th, 2010
11:06 pm

I took this survey a few weeks ago–it was emailed out by our tech guy or one of our APs, and they said we needed to fill it out. I totally agree with the earlier comment that said most teachers just probably filled out “strongly agree”. So for there to be a dissenting opinion on certain questions says that those questions more than likely should have been even more negative than they were. And I voted “NO” on the pay for scores, because I sure wouldn’t want to get paid on the basis of my 9th graders’ Math I scores………

M G

January 25th, 2010
11:10 pm

I remember this survey. In fact, I completed it twice, once as a teacher and once as a community member less than 5 minutes later. I question the validity of the results since it was completed with such a lack of security and the obviously biased questions only reinforce the lack of credibility.

Now that teachers know the survey is out there, why don’t they open it up again. I’m sure the results would be much different.

lyncoln

January 26th, 2010
8:53 am

I have to comment about the number of teachers that responded. 15% is a perfectly valid amount of responses for a survey to determine the feelings of the whole group.

Look at the Gallup polls used for determining the national feelings of politicians, etc. They are considered valid and well done, and they consist of a 3-day rolling average of 1000 responses per day.
I’m pretty sure that 3000 responses is a lot less than 15% of the voting population. Therefore, the 15% of teacher responses is more than enough to attempt to extrapolate the feelings of the overall teaching population. I’m not worried about the fact that only 15% of the group responded, that’s a plenty large number of responses.

The difference between core and non-core teachers is a potential problem. It’s very possible that core and non-core teachers might respond with different biases and the difference in make-up of the sample vs. the whole might skew the results of the survey.

Mac

January 26th, 2010
10:06 am

A poll and a survey are somewhat different instruments in terms of use and design typically. – Public opinion polls are useful in tracing people’s views on important social issues. Polls are used to assess people’s preferences in political races, and the results are used to predict election results. Surveys are a study of what people think or believe about a topic or question and are often employed in marketing and advertising research to measure and predict consumer’s reaction to products.

Gallup Pols are limited in sample size by design – the survey being discussed was supposedly made available to all teachers and educators in the state, so it was an all encompassing sample.

Another issue is that respondents didn’t necessarily know what the were responding to in context. With a Gallup poll call a large part of the potential sample knows about whatever political item is being questioned. In a marketing survey it is explained what products or service is being looked at.
The survey I took said nothing about Race to the Top or moving to teacher merit pay as I remember.
Depends somewhat on the context, but for survey data to be reliable you should be aiming at 50% or more of the target population to be confident that your statistics are correct.

“Surveys rely on a random sample of the target population and if the response rate is too low then the sample is probably not random.
For instance in a telephone survey of the US and you normally get 45% response rate (45% of people called complete the survey questions) but you run a poll in the middle of the superbowl or 2am in the morning and the response rate drops to 10%, then the sample is skewed – those 10% who complete are different from the other 90% of the telephone owning population in some way.”

Apparently there was no clear directive to Superintendents as to who to forward the survey links to. Each individual Superintendent made that choice, some may have sent it to all schools with a directive for all teachers to participate and some may only have sent it to people in the central office. Too many variables.

“Response rates are a potential source of bias. The results from a survey with a large non-response rate could be misleading and only representative of those who replied. French [12] reported that non-responders to patient satisfaction surveys are less likely to be satisfied than people who reply. It is unwise to define a level above which a response rate is acceptable, as this depends on many local factors; however, an achievable and acceptable rate is ~75% for interviews and 65% for self-completion postal questionnaires [9,13]. In any study, the final response rate should be reported with the results; potential differences between the respondents and non-respondents should be explicitly explored and their implications discussed.”

Another question is found in the order of questions – were the questions locked in order or provided to the people surveyed in random assignments. Wording and order of questions is well known to influence the results of a survey and interject response bias.

Also, if not password-protected, online surveys are easy to manipulate by completing multiple times to skew results and tend to skew results to younger and/or more technologically literate respondents.

Another issue may be self-selection bias, which is possible whenever the group of people being studied has any form of control over whether to participate. Participants’ decision to participate may be correlated with traits that affect the study, making the participants a non-representative sample. For example, people who have strong opinions or substantial knowledge may be more willing to spend time answering a survey than those who do not. Another example is online and phone-in polls, which are biased samples because the respondents are self-selected. Those individuals who are highly motivated to respond, typically individuals who have strong opinions, are overrepresented, and individuals that are indifferent or apathetic are less likely to respond. This often leads to a polarization of responses with extreme perspectives being given a disproportionate weight in the summary. As a result, these types of polls are regarded as unscientific.

These are just a few flaws my admittedly slow brain can see.
It needs to be done again now that there is sufficient focus on the issue and the results need to be compared.

Mac

January 26th, 2010
10:14 am

“Look at the Gallup polls used for determining the national feelings of politicians, etc. They are considered valid and well done, and they consist of a 3-day rolling average of 1000 responses per day.
I’m pretty sure that 3000 responses is a lot less than 15% of the voting population. Therefore, the 15% of teacher responses is more than enough to attempt to extrapolate the feelings of the overall teaching population. I’m not worried about the fact that only 15% of the group responded, that’s a plenty large number of responses.”

You have to look at the percentage of the group polled – therefore i would say the percentage of the chosen Gallup number of 1000’s response rate would have to be much higher that 15% to be considered valid.

Apples and oranges

Gallup poll selected sample =1000
This survey’s selected sample = all GA educators

For this to be true the educator survey would have had to have been designed to only go out to a representative sample of educators (say 1000 out of all of them in the state) and then those results reported and analyzed. That isn’t what happened here, the entire population of educators was supposedly surveyed.

The response size is an issue here.

samlee

January 26th, 2010
12:15 pm

As an educator, I also took the “survey”. I find these annoying and a waste of time, but we were told to participate. Consequently, my answers are almost consistently “strongly disagree”. Most of questions are not valid and when I hear that the governor thinks the majority of teachers want pay for performance, I know that it is not valid. I am a special education teacher. So, in other words, I would not be paid because my students don’t pass standardized tests the majority of the time.

Roger Hines, Candidate for State School Supt.

January 26th, 2010
12:21 pm

Once again, teachers are being blamed for GA’s hovering at 46th in education. Linking a standardized test score to an individual teacher denies the fact that no singular teacher is ever responsible for a standardized test score. The truth is that many teachers are responsible, as well as many classes and educational experiences. Will teacher pay fluctuate as the standardized scores fluctuate, which they will surely do? Teachers are already being evaluated to a fare-you-well by their principals with their own evaulative visits and written instruments, not to mention the media and politicians’ cries for “accountability.” In 42 years of being in education, I have known maybe two or three teachers whom I would not want my children to have. Teachers are not our problem. Sometimes students just won’t study. Students are getting off the hook in our conversation on accountability and performance pay. (Parents, too.) Teaching is not like selling pants where you know at the end of the week how many pants you have sold. As for the survey, it doesn’t look like it directly asked teachers if they favored performance pay. I am sure most do not because they have taught school long enough to understand that pay based on the habits of children and teens is never going to be reliable.

[...] to address the unique calculus that her boss employed to conclude that 80 percent of teachers who took his December survey supported merit [...]

gaseaslover

March 29th, 2010
10:30 am

It’s interesting that I work in THE largest school in my district, my district IS LISTED as participating in this survey, and yet not a SINGLE teacher I’ve talked to (those with higher degrees, National Board Certification, and who are teacher-leaders within the school) participated in this survey.
It would seem that, in this day and age of technology, the state COULD have sent out a link to Survey Monkey OR Zoomerang to ALL district offices to be forwarded to ALL teachers within EVERY SINGLE SCHOOL DISTRICT. Rather than “sampling” teachers – why not find out what EACH AND EVERY TEACHER IN GEORGIA THINKS ABOUT SONNY’S PROPOSAL???? Hmmm….could it be that the state did NOT, in fact, sample “80%” of teachers….yeah, I would say “NOT!”
It would be even more interesting to place a performance pay option onto ALL of our state legislators and Kathy Cox & Sonny Perdue – I think they would not like what the people in the state of Georgia would tell them – they do not deserve a dime of their pay for what they are doing to education in Georgia. Perdue could care less – his pork plans are more important the education of children in this state. The entire nation is laughing at our audacity to disregard National Board Cerfication as producing effective classroom teachers and how Sonny’s “Master Teacher” program is better. It’s not. It’s simply a numbers game. I can be a master teacher – “hey kids – christmas tree your answers on the pre-test” – then they WILL show growth throughout the year as compared to their pre-test – THAT’S what the “Master Teacher” plan promotes – getting kids to pass a test….sad….they’ll pass the tests, but with no idea how to learn, or read, or find information, or think analytically, or ask questions….SKILLS needed to survive after high school. Perdue – you’re an idiot. Cox – you’re not much better. And Kathy – please continue to do nothing – very soon you are up for re-election and guess what? you’ve made it so that there won’t BE a teaching job available to you once you are kicked out of office…. good luck with that….

anon

April 27th, 2010
2:46 pm

Oh awesome. DCSS blocks the results website.