At the outset of the public-union protests in Wisconsin, a number of pundits and at least one commenter on this blog pointed to the SAT/ACT scores of Wisconsin students versus those in states that prohibit collective bargaining by teachers.
The reported statistics showed the five non-unionized states ranking 44th in the country (Virginia), 47th (Texas), 48th (Georgia), 49th (North Carolina) and 50th (South Carolina), compared to second in the country for Wisconsin. It was suggested that these stats and others demonstrated the superiority of unionized educators, or at least the inferiority of the nonunion (and low-tax) model.
The normally brutally satirical Iowahawk has broken out of character to provide a straight explanation of why these stats are the wrong ones to examine — and to pass along some that tell a more accurate story:
As a son of Iowa, I’m no stranger to bragging about my home state’s ranking on various standardized test. Like Wisconsin we Iowans usually rank near the top of the heap on average ACT/SAT scores. We are usually joined there by Minnesota, Nebraska, and the various Dakotas; Vermont, Maine, New Hampshire…
… beginning to see a pattern? Perhaps because a state’s “average ACT/SAT” is, for all intents and purposes, a proxy for the percent of white people who live there. In fact, the lion’s share of state-to-state variance in test scores is accounted for by differences in ethnic composition. Minority students — regardless of state residence — tend to score lower than white students on standardized test, and the higher the proportion of minority students in a state the lower its overall test scores tend to be.
Please note: this has nothing to do with innate ability or aptitude. Quite to the contrary, I believe the test gap between minority students and white students can be attributed to differences in socioeconomic status. And poverty. And yes, racism. And yes, family structure. Whatever combination of reasons, the gap exists, and it’s mathematical sophistry to compare the combined average test scores in a state like Wisconsin (4% black, 4% Hispanic) with a state like Texas (12% black, 30% Hispanic).
So how to compare educational achievement between two states with such dissimilar populations? In data analysis this is usually done by treating ethnicity as a “covariate.” A very simple way to do this is by comparing educational achievement between states within the same ethnic group. In other words, do black students perform better in Wisconsin than Texas? Do Hispanic students perform better in Wisconsin or Texas? White students? If Wisconsin’s kids consistently beat their Texas counterparts, after controlling for ethnicity, then there’s a strong case that maybe Texas schools ought to become a union shop.
He then compares scores in math, reading and science for white, black and Hispanic students in the fourth and eighth grades in Texas and Wisconsin on the National Assessment of Educational Progress (NAEP). For some local fun, I’ve added the scores for Georgia. I’ve also coded the state names and scores for some quick, easy comparison: boldface text means the state performed statistically better than the national average, plain text means there was no statistical difference, and italicized text represents a statistically worse score. (NAEP itself determined which scores were statistically meaningful.)
Without further ado:
2009 4th Grade Math
White students: Texas 254, Wisconsin 250, Georgia 247 (national average 248)
Black students: Texas 231, Georgia 221, Wisconsin 217 (national 222)
Hispanic students: Texas 233, Georgia 231, Wisconsin 228 (national 227)
2009 8th Grade Math
White students: Texas 301, Wisconsin 294, Georgia 289 (national 294)
Black students: Texas 272, Georgia 262, Wisconsin 254 (national 260)
Hispanic students: Texas 277, Georgia 270, Wisconsin 268 (national 266)
2009 4th Grade Reading
White students: Texas 232, Georgia 229, Wisconsin 227 (national 229)
Black students: Texas 213, Georgia 204, Wisconsin 192 (national 204)
Hispanic students: Texas 210, Georgia 208, Wisconsin 202 (national 204)
2009 8th Grade Reading
White students: Texas 273, Wisconsin 271, Georgia 268 (national 271)
Black students: Georgia 249, Texas 249, Wisconsin 238 (national 245)
Hispanic students: Georgia 254, Texas 251, Wisconsin 250 (national 248)
2009 4th Grade Science
White students: Texas 168, Wisconsin 164, Georgia 159 (national 162)
Black students: Texas 139, Georgia 126, Wisconsin 121 (national 127)
Hispanic students: Wisconsin 138, Texas 136, Georgia 133 (national 130)
2009 8th Grade Science
White students: Texas 167, Wisconsin 165, Georgia 161 (national 161)
Black students: Texas 133, Georgia 129, Wisconsin 120 (national 125)
Hispanic students: Texas 141, Georgia 137, Wisconsin 134 (national 131)
Here are some totals:
Granted, these are one year’s worth of results, albeit from the most recent year reported. If anyone wants to go back and compile previous years’ results, knock yourself out.
And granted, these aren’t the only educational statistics that mean anything. High-school dropout rates are crucial, and Wisconsin performs better than Georgia on that count — although not among black and Hispanic students. (I don’t know why Georgia’s white students compare so poorly to white students elsewhere, but that’s a topic for another day.)
But it’s impossible to look at these results and argue that Wisconsin’s unionized teachers produce measurably and unquestionably better results than Georgia’s non-unionized educators do. It doesn’t prove the reverse, either, but it does suggest that allowing teachers to bargain collectively isn’t really “for the kids.”
– By Kyle Wingfield
Or tweet this blog post: Tweet