Do unionized teachers really produce better results?

At the outset of the public-union protests in Wisconsin, a number of pundits and at least one commenter on this blog pointed to the SAT/ACT scores of Wisconsin students versus those in states that prohibit collective bargaining by teachers.

The reported statistics showed the five non-unionized states ranking 44th in the country (Virginia), 47th (Texas), 48th (Georgia), 49th (North Carolina) and 50th (South Carolina), compared to second in the country for Wisconsin. It was suggested that these stats and others demonstrated the superiority of unionized educators, or at least the inferiority of the nonunion (and low-tax) model.

The normally brutally satirical Iowahawk has broken out of character to provide a straight explanation of why these stats are the wrong ones to examine — and to pass along some that tell a more accurate story:

As a son of Iowa, I’m no stranger to bragging about my home state’s ranking on various standardized test. Like Wisconsin we Iowans usually rank near the top of the heap on average ACT/SAT scores. We are usually joined there by Minnesota, Nebraska, and the various Dakotas; Vermont, Maine, New Hampshire…

… beginning to see a pattern? Perhaps because a state’s “average ACT/SAT” is, for all intents and purposes, a proxy for the percent of white people who live there. In fact, the lion’s share of state-to-state variance in test scores is accounted for by differences in ethnic composition. Minority students — regardless of state residence — tend to score lower than white students on standardized test, and the higher the proportion of minority students in a state the lower its overall test scores tend to be.

Please note: this has nothing to do with innate ability or aptitude. Quite to the contrary, I believe the test gap between minority students and white students can be attributed to differences in socioeconomic status. And poverty. And yes, racism. And yes, family structure. Whatever combination of reasons, the gap exists, and it’s mathematical sophistry to compare the combined average test scores in a state like Wisconsin (4% black, 4% Hispanic) with a state like Texas (12% black, 30% Hispanic).

So how to compare educational achievement between two states with such dissimilar populations? In data analysis this is usually done by treating ethnicity as a “covariate.” A very simple way to do this is by comparing educational achievement between states within the same ethnic group. In other words, do black students perform better in Wisconsin than Texas? Do Hispanic students perform better in Wisconsin or Texas? White students? If Wisconsin’s kids consistently beat their Texas counterparts, after controlling for ethnicity, then there’s a strong case that maybe Texas schools ought to become a union shop.

He then compares scores in math, reading and science for white, black and Hispanic students in the fourth and eighth grades in Texas and Wisconsin on the National Assessment of Educational Progress (NAEP). For some local fun, I’ve added the scores for Georgia. I’ve also coded the state names and scores for some quick, easy comparison: boldface text means the state performed statistically better than the national average, plain text means there was no statistical difference, and italicized text represents a statistically worse score. (NAEP itself determined which scores were statistically meaningful.)

Without further ado:

2009 4th Grade Math

White students: Texas 254, Wisconsin 250, Georgia 247 (national average 248)
Black students: Texas 231, Georgia 221, Wisconsin 217 (national 222)
Hispanic students: Texas 233, Georgia 231, Wisconsin 228 (national 227)

2009 8th Grade Math

White students: Texas 301, Wisconsin 294, Georgia 289 (national 294)
Black students: Texas 272, Georgia 262, Wisconsin 254 (national 260)
Hispanic students: Texas 277, Georgia 270, Wisconsin 268 (national 266)

2009 4th Grade Reading

White students: Texas 232, Georgia 229, Wisconsin 227 (national 229)
Black students: Texas 213, Georgia 204, Wisconsin 192 (national 204)
Hispanic students: Texas 210, Georgia 208, Wisconsin 202 (national 204)

2009 8th Grade Reading

White students: Texas 273, Wisconsin 271, Georgia 268 (national 271)
Black students: Georgia 249, Texas 249, Wisconsin 238 (national 245)
Hispanic students: Georgia 254, Texas 251, Wisconsin 250 (national 248)

2009 4th Grade Science

White students: Texas 168, Wisconsin 164, Georgia 159 (national 162)
Black students: Texas 139, Georgia 126, Wisconsin 121 (national 127)
Hispanic students: Wisconsin 138, Texas 136, Georgia 133 (national 130)

2009 8th Grade Science

White students: Texas 167, Wisconsin 165, Georgia 161 (national 161)
Black students: Texas 133, Georgia 129, Wisconsin 120 (national 125)
Hispanic students: Texas 141, Georgia 137, Wisconsin 134 (national 131)

Here are some totals:

  • Across all 18 of the above categories, Texas students were better than average in 15, the same in three and below average in none. Georgia and Wisconsin were each better than average on four tests, but Georgia was below average just once, while Wisconsin was below average in four. (In fact, Texas was so far ahead, across the board, that I’ll compare only Georgia and Wisconsin below.)
  • Wisconsin was significantly ahead of Georgia among white students. But Georgia was significantly better than Wisconsin among black students (two above average and four average, compared to two average and four below average) and about the same among Hispanics.
  • Georgia was slightly better than Wisconsin in math and slightly worse in science. But it was markedly better in reading (one above average and five average, versus four average and two below average).
  • Overall, on tests where there was a statistically meaningful difference between Georgia and Wisconsin, Georgia was better on six tests and Wisconsin better on four.

Granted, these are one year’s worth of results, albeit from the most recent year reported. If anyone wants to go back and compile previous years’ results, knock yourself out.

And granted, these aren’t the only educational statistics that mean anything. High-school dropout rates are crucial, and Wisconsin performs better than Georgia on that count — although not among black and Hispanic students. (I don’t know why Georgia’s white students compare so poorly to white students elsewhere, but that’s a topic for another day.)

But it’s impossible to look at these results and argue that Wisconsin’s unionized teachers produce measurably and unquestionably better results than Georgia’s non-unionized educators do. It doesn’t prove the reverse, either, but it does suggest that allowing teachers to bargain collectively isn’t really “for the kids.”

– By Kyle Wingfield

Find me on Facebook or follow me on Twitter

Or tweet this blog post:

77 comments Add your comment

get out much?

March 3rd, 2011
2:26 pm

I am still waiting to hear you or anyone else explain how you expect to attract and retain quality teachers by paying them less.

Kyle Wingfield

March 3rd, 2011
2:30 pm

If you did “get out much,” you would know that Georgia pays teachers quite well compared to our neighbors and the national average. I’ve been over that on a previous comment thread, too.

Kyle Wingfield

March 3rd, 2011
2:35 pm

White Power

March 3rd, 2011
2:43 pm

Let me get this straight. The numbers indicate a gap between ACT/SAT test scores of minority and white students which can be attributed to differences in socioeconomic status, poverty, “and yes”, racism. Yet, our Governor has just proposed modifying the HOPE Scholarship program so that to be eligible for a full (100%) tuition reimbursement, students must have a minimum SAT or ACT score?

Sounds like a proposal than only a white person could love.

arnold

March 3rd, 2011
2:50 pm

You are correct that Georgia pays better than surrounding states. Georgia also doesn’t have a teachers union. Georgia also doesn’t have tenure. So what is the big deal about being concerned with tenure, unions and pay. Those are nothing but red herrings.

The real problem stems from lack of parental involvement. It may not be PC, but the white population has more parental involvement. it’s a fact of life.

GATeacher

March 3rd, 2011
2:51 pm

Blah Blah Blah.

get out much?

March 3rd, 2011
2:53 pm

Kyle, I have seen the statistics you (and others) have posted regarding teacher salaries both here in Georgia and other states, so I will rephrase my question for you. How do you expect to attract and retain quality teachers by cutting their salaries and/or benefits (the cuts in benefits are one of the things the union in Wisconsin has agreed to).

Linda

March 3rd, 2011
3:04 pm

Judge Vinson in Fla. just ruled again on the health care bill. The Justice Dept. had gone back to him to “clarify” his original opinion that the entire law was void. He wrote, “While I believe that my order was as clear & unambiguous as it could be, it is possible that the defendants may have perhaps been confused or misunderstood its import.” “It was not expected that they would effectively ignore the order & declaratory judgment for two and one-half weeks, continue to implement the Act, & only then file a belated motion to ‘clarify.’”

That’s not all. He gave the O adm. seven days to appeal his ruling.

http://blogs.abcnews.com/politicalpunch/2011/03/judge-vinson-clarifies-florida-ruling-maintains-that-entire-health-care-law-is-unconstitutional.html

Buck Jones

March 3rd, 2011
3:13 pm

Poverty…socioeconomic status huh…so I suppose poor white kids test the same as black kids. And I suppose white scores during periods like the great depression tanked.(and other things caused by poverty like crime skyrocketed).

Not.

Want to try again?

Bob

March 3rd, 2011
3:14 pm

Get out much – you pay less by paying for performance, not just for showing up. Why do teachers, or any union member, deserve to get paid more each year just because they go to work and don’t kill anyone. In non-union shops, you get a raise based on your performance, not because you are part of a mob.

Let good teacher’s get larger raises, and bad teachers get lower (or no) raises. This will make the good teachers happy, and make the bad teachers 1) improve or 2) leave.

That’s the way raises work in the real world, which I’m guessing either 1) you’re not part of, or 2) you don’t do very well on your yearly performance review.

Legend of Len Barker

March 3rd, 2011
3:16 pm

Georgia pays pretty well. Well, they pay teachers well. Support staff generally barely make a living wage.

But there is a point at which a good many teachers have found themselves at. Is this aggravation worth the money? For many, it’s ‘no.’ You lose a ton of teachers in the first 3-5 years because either they’re not good or the administration is terrible and non-supportive. In this day and age, it’s about 50/50 on that.

I think good administration is more valuable than a union. Good administration can make up for bad teachers (and usually weeds them out). Good teachers have issues overcoming bad administration as it tends to breed bad teachers and a clique environment.

If you want to know why Georgia’s white students have a higher dropout rate … I can give a very long spiel on that one. My county annually brings those statistics down. Our graduation rate is annually about 60-70%. Woo-hoo! Rural Georgia!

Kyle Wingfield

March 3rd, 2011
3:21 pm

White Power: The research I’ve seen ties SAT performance, but not necessarily ACT performance, most closely to income. But if not the SAT/ACT, what do you use to a) verify the validity of the GPA, and b) control for grade inflation?

In any case, an ACT score of 26 would put a student in the top 16 percent of test-takers nationally and somewhere in the middle of GT and UGA enrollees. A 3.7 GPA would put a student in the top 12 percent of all Georgia HS graduates (according to 2009 figures). Is that really too high a bar to clear for a 100% tuition scholarship? Sure, it’s higher than before…but we don’t have the money to keep doing what we we’ve been doing.

wallbanger

March 3rd, 2011
3:26 pm

You can talk all you want about statistics and averages, but the sad fact is that most states are deeply in the red, much like this entire country, and can”t afford a privileged class of teachers who can retire after 30 years and collect more in benefits after retirement than most have contributed in their worklife. At the same time the private industry taxpayer, who saves for his own retirement sees taxes go up to fund these same teachers, and his lifestyle decrease. Not fair at all.

StJ

March 3rd, 2011
3:26 pm

“it’s impossible to look at these results and argue that Wisconsin’s unionized teachers produce measurably and unquestionably better results than Georgia’s non-unionized educators do”

Guess Bookman and Tucker missed that memo. :)

Mr_B

March 3rd, 2011
3:34 pm

Wallbanger: if you had a financial advisor who recommended a retirement plan in which you only collected what you put in, you could replace him or her with a box under the mattress. Pension contributions are invested, not buried. And yes, we teachers do contribute to our own retirement. And while we can retire after thirty years, many of us stay on.

Mr_B

March 3rd, 2011
3:41 pm

Kyle: the unionized teachers versus test results thing is a strawman arguement. It doesn’t make any difference if unionized teachers don’t produce “more educated” students as long as they don’t produce “less educated” students. The purpose of collective bargaining is not to affect the outcopmes in the classroom; it is designed to give teachers or other employees some leverage in interactions with their employers, who may not necessarily have the best interests of the employee in mind. While it is true that teachers and other public employees work for the taxpayers, they don’t get to bargain with the taxpayers; they have to bargain with elected school boards and appointed superintendents.

Linda

March 3rd, 2011
3:42 pm

get out, Markets vary. In real estate, there are buyers’ & sellers’ markets. In the economy, there are employers’ & employees’ markets. Currently, there is an employers’ market. In the private sector, there is no need for employers to offer extravagant benefits to attract the best employees.
Part of the reason for the trend to reduce collective bargaining for teachers is to be able to fire poor teachers & retain the best teachers with merit pay rather than tenure.

Tychus Findlay

March 3rd, 2011
3:44 pm

Teachers have the right to unionize so long as the State has the right to negotiate with other potential employees.

TrickleDownStupid

March 3rd, 2011
3:49 pm

It is really dumb to use the correlation of test scores (by states) to question whether or not unionized teachers produce better results; it could be that their curriculumns are producing the bad results.

I would also go out on a limb to say that comparing Wisconsin to (Texas and Georgia )is like comparing apples and oranges. What are the similarities of the three states?

CDog

March 3rd, 2011
3:50 pm

African-American students score less than white students not because of socioeconomic status or racism but because of cultural values. Black culture, especially in the south, tends to be one of anti-academic achievement as that is viewed as acting white. I teach at a majority black high school and I observe this daily. Instead of saying the test is biased against black students, we should say that black culture is biased against the test. The companies that administer the national standardized tests have committees that review questions for cultural bias. Asians are a minority with a different culture than whites, yet they score higher than whites on average. Why does the alleged bias not affect them?

Another reason for state disparity in SAT scores is % of students tested. In many states only the top %iles of students take the SAT, whereas in GA about 70% of students take it.

chef pierre

March 3rd, 2011
3:51 pm

As if Ga needed another reason to be called ignorant ,lets run off the best teachers and that will fix things…

Kyle Wingfield

March 3rd, 2011
3:52 pm

Mr_B: If it’s a straw man argument, it’s one that originated with the Wisconsin unions and their sympathizers, as I described at the beginning of the OP. Ditto for the “for the kids” line, which has appeared on plenty of the striking unionists’ signs.

chef pierre

March 3rd, 2011
3:54 pm

Unions are the reason most on this blog earn a decent wage , get paid o.t. and have any benefits.
get real.

Linda

March 3rd, 2011
4:00 pm

Public employees have the right to join unions. Public employee unions have the right to demonstrate grievances to the govt.
Do state employees have more or less rights than federal employees to collectively bargain for wages &/or benefits? Do state employees have more or less rights than federal employees to be required to join unions?

Linda

March 3rd, 2011
4:15 pm

Should Wisconsin students be taught history—of the labor union movement & collective bargaining, as required by a law passed by the progressive Democrats?

http://dailycaller.com/2011/02/24/wisconsin%E2%80%99s-teachers-required-to-teach-kids-labor-union-and-collective-bargaining-history/

Pdentgrou

March 3rd, 2011
4:18 pm

But how are the “good teachers” going to be rewarded? It should be like the private sector, and bonuses should be given to the good teachers/best teachers. If there is a shortage of funds, how will that happen? Everybody shouts merit pay, but where will that money come from?

ByteMe

March 3rd, 2011
4:48 pm

Mr_B: If it’s a straw man argument, it’s one that originated with the Wisconsin unions and their sympathizers, as I described at the beginning of the OP

Except that that’s not their argument. Their argument isn’t that unions produce better results, but that unions are desired by the teachers to help negotiate pay and benefits as a collective instead of as individuals.

But you knew that.

that's goofy

March 3rd, 2011
4:49 pm

The argument of union teachers = better scores is a result of talk radio’s claim union teachers = lower scores.

Union vs non union has zero to do with it. Students willingness and parental support have a huge impact on performance – as much as the teacher.

GA has an association not a union – same as FL

GA pays better than FL – that’s why I moved here. Then my position was eliminated (last in first out in my county).

Paying teachers on merit works if the teachers are on an even playing field. For the pro business types: imagine your pay was tied to the performance of your employees – but you had zero say in hiring those employees and they could not be fired for refusing to work. Sound fair?

The scores at a low socioeconomic school will not be as high as more affluent areas. The better teachers will move to the better area schools to receive more money. I taught at both – there is a difference.

Paying teachers a bonus to work at poor schools is an idea that “associations” shoot down.

Reward teachers if the students improve from the beginning to the end of the year. There should be differential pay between subjects and grade levels. All teachers and subjects are not the same.

Last stat I read: 50% of teachers do not make 5 years.

Linda

March 3rd, 2011
4:54 pm

It looks like the fed. govt. won’t be shut down, at least for another couple of weeks, but what about the NFL, that segment of our economy that most watchers are watching? We’ve got 7 more hours, folks!
Wish the fed. govt. had only $9 B to worry about!
O weighed in today & did not send his Pay Czar.
This is about greed & selfishness! It is not fair! It is inequality! This is about social justice/socialism!
How dare the NFL be able to score 6 points when the NBA can only score 3 at the most!

GA Jim

March 3rd, 2011
4:54 pm

Great article Kyle. Kudos on two points: 1) Being a fellow Iowahawk reader 2) Being courageous enough to bring up a topic involving any mention of race on your blog

I Report (-: You Whine )-: Credible, Compelling, Complete....Bwahahahaha, just sayin...

March 3rd, 2011
5:01 pm

(I don’t know why Georgia’s white students compare so poorly to white students elsewhere, but that’s a topic for another day.)

Probably the dimwit inner city guilty white liberal Atlanta children, just sayin…

Always Skeptical

March 3rd, 2011
5:17 pm

Kyle, isn’t throwing the GA statistics into the fray ( especially for minority students) rather meaningless given the fact that they may very well be propped up with bogus test scores from some of the state’s largest school districts? Wisconsin is still on top.

Logical Dude

March 3rd, 2011
5:21 pm

Hi Kyle,
Do you also realize that more Georgia students take the SAT in order to qualify for HOPE, so there are A GREAT MANY MORE students taking the SAT than other states (in general). Most other states, ONLY college bound students take the SAT. Here in Georgia, MOST students take the SAT to qualify for HOPE, which will see IF they can go to college.

I believe that in Georgia, this is a much greater drag on the average SAT score than just socio-economic or race excuses.

killerj

March 3rd, 2011
5:26 pm

Lets compare private schools to this assumption and see what you think then.

Always Skeptical

March 3rd, 2011
5:30 pm

And for a little more food for thought…Parental involvement is certainly THE determining factor regarding a child’s success. But I dare say, whether you’re while black or purple, if you’re poor and a working single parent, or if you’re poor, married and have two working parent’s, it’s a much tougher road than if you’re married and can afford to have a stay at home mom to keep things in check at school. Yes poor white kids and poor black kids both perform under par. If you’ve never been poor, coming from undereducated generations of poor ancestors, black or white, you can never comprehend the barriers that poverty places in to paths of impoverished children that never asked to be born in the first place.

Linda

March 3rd, 2011
5:37 pm

In Wisconsin, collective bargaining is not just about teachers’ rights. It’s about union & taxpayers’ rights. It has not been explained by the Corrupt Media or even by Gov. Walker that collective bargaining is a fiscal piece of the budget. The teacher’s union was able to “bargain” for health care, the most expensive insurance on the market, costing $68 M more than necessary. Now, that teachers will HAVE NEW RIGHTS & teeth in their bottom line, they might elect to reduce their cost & that of the taxpayers.

http://doingadvancework.blogspot.com/2011/03/wisconsin-schools-could-save-68-million.html

Kyle Wingfield

March 3rd, 2011
5:39 pm

Thanks, GA Jim. Iowahawk is great — although, oftentimes, a bit too salty for me to quote verbatim on a family-friendly website.

Always Skeptical @ 5:17: I haven’t heard anyone explain how APS could have cheated on the NAEP, which, I understand, is administered by NAEP’s own people. So, I think those scores should be legitimate.

Logical Dude: I am aware of that, although I have no idea whether students in Wisconsin also take the SAT in disproportionate numbers. But you’re right that this factor produces a drag on Georgia’s average SAT score.

Kyle Wingfield

March 3rd, 2011
5:41 pm

And a belated thanks to you, Linda, for your kind words the other day about my GPB appearance. I taped another show (episode?) this week on the budget and deficit, but I don’t know yet when it will appear.

catlady

March 3rd, 2011
5:57 pm

It is possible to statistically control for things like income, race, average teacher salary, family type, even “ability” as measured by test scores. It is called multivariate analysis. Let’s see those. In the past, income has been shown, when controlling statistically for other generally accepted variables, to be the biggest predictor of achievement as represented by test scores.

Jefferson

March 3rd, 2011
6:02 pm

Its ok to be in the bottom if you are the State of GA, but you have have to be in the top to get HOPE $$$. Why such low standards for the state gov’t?

Linda

March 3rd, 2011
6:11 pm

Kyle@5:41, You are welcome! I’m sure your family was duorp, proud &
d
u
o
r
p
That’s proud spelled backwards, forward & upside down.

Old Physics Teacher

March 3rd, 2011
6:12 pm

Kyle, once again, figures don’t lie, but when you compare a 4-year degreed individual against the “average worker,” you’re not comparing apples to apples. Pull out all the workers on that page with less than a 4-year degree and your numbers …uh…don’t add up. If you add in the ones with the required advanced degrees that we have to get, it gets worse. When you compare the science degrees, chemistry, physics, geology, etc, there is no comparison. Oh, and be sure to add in the retail store managers whose decision-making skills are exactly what we must use in the classroom daily (I spent 10 years as a retail store manager – I know exactly how much money I lost when I started teaching) We are greatly underpaid when the comparison is made apples-to-apples.

Now since that’s out of the way, thank you for admitting the teachers in Georgia are doing quite well when you correct for the discrepancies across state line. I’m sure you’ll be bragging on us when our SAT scores overall are used against us by the mainstream Republicans trying to blame us for all the ills in society and wanting to give money to private schools who’s SAT scores are over-inflated, right?

I’m sure you’ll do the right thing, right?

jconservative

March 3rd, 2011
6:16 pm

Does this mean our minorities are smarter than Wisconsin’s minorities?

The big problem in Georgia seems to be the White Students who can do no better than third on the results. The minorities are pulling second place routinely.

Pierce Randall

March 3rd, 2011
6:28 pm

Wingfield et al get bored and decide to race bait to impress the white supremacists who frequent the AJC blogs.

If the this blogger is arguing in good faith that racial disparities reduce to economic disparities and racist mistreatment, then we could merely study poverty and racial attitudes to test that hypothesis. He proceeds not to do so.

What’s really clear regarding the union debate is that these statistics are clearly cherry-picked. Why Wisconsin, Texas, and Georgia (the under-performer that gets thrown into the mix)? Why not unionized states versus non-unionized states across the board?

My hypothesis will be that collective bargaining doesn’t have a direct affect on student education enough to make a difference, and that instead it’s an important right for workers.

Linda

March 3rd, 2011
6:34 pm

Pierce@6:28, What workers? Federal? State? Why?

JW

March 3rd, 2011
6:49 pm

I’ve just seen a music video on Youtube that is incredibly relevant in this situation.
It could, in fact, be an anthem for all the student protests around the world.
Look for “Take That – Kidz”.
Btw, Take That are currently the biggest band in the UK, so I’m not plugging some unknown lame band here.

Tom

March 3rd, 2011
6:57 pm

Saw an article today in paper which implyed that Wisconsin(unionized) scored more higher than Georgia(non-union, right-to-work)was near the bottom % in SAT and ACT scores. That is not the REAL WORLD, folks! Ok, let’s get real for a moment and stop this BS, Georgia has approx 450,000 illegal immigrants and Wisconsin has what maybe 20,000 illegals? This is sad to say but we also have a very high % of black children in single-family homes compared to Wisconsin! With our large number of black and brown(illegal and legal) children, it is not fair to compare us to Wisconsin(which is mostly white) in educational issues! It is not the caliber of teachers, not whether they are unionized or not, it is the real world of society that we live in!

Ho Hum

March 3rd, 2011
7:01 pm

When you cherry pick the stats, and for only one year, of course you can find support for your position. If I ask only Georgia Republicans whether Obama is a nazi, socialist, jihadist who was born in Kenya and hates America, my poll will show that 75% of the persons polled think that Obama is a nazi, socialist, jihadist who was born in Kenya and hates America.

duh.

F. Sinkwich

March 3rd, 2011
7:08 pm

Kyle,

You seem like a nice guy but you are failing as the “AJC Conservative Columnist.” You focus on the minuscule which sacrifices the bigger picture.

The issue is not whether union teachers are better or worse than non-union teachers. The issue is the unsustainable compensation unions demand from politicians who derive their power from the unions that elect them. How is the taxpayer represented in this corruption?

You need to broaden your horizons. Stop posting minutia.

Kyle Wingfield

March 3rd, 2011
7:13 pm

F. Sinkwich: This is the fifth blog post I’ve written about the union protests. I covered the unions-versus-taxpayer angle in three of the previous four.