Higher CRCT failure rates should erase doubts about whether test sheet tampering occurred last year
By using open records to obtain CRCT scores from the major metro districts and diving into the data, the AJC investigative team examined how schools under suspicion for test tampering in 2009 fared this year when testing protocols were tightened and answer sheets were guarded like gold bullion. (You can see the results for yourselves here as we have just posted them.)
Not very well, it turns out.
The newspaper’s analysis by AJC reporters John Perry and Heather Vogell found that score drops in the 39 “severe” Atlanta schools drove up the district’s overall failure rate in reading, English and math.
For all of you who have insisted that this was a witch hunt, consider this: Most schools that did not have unusual erasure patterns showed a slight gain in their CRCT scores. Is this also random?
Those schools with suspicious erasure rates last year and jump in failures this year ought to be doing some soul searching. And Superintendent Beverly Hall ought to searching for the reasons for the plummet in scores. And for the people responsible for any tampering.
Yes, there are fluctuations in test scores year to year. But abnormal fluctuations can’t be ignored in light of the state erasure analysis.
While the state won’t release final school results until mid July, the AJC used the state’s Open Records Act to obtain preliminary scores for nearly 500 schools in the six biggest metro districts. In general, the scores showed that students at schools where state officials said cheating concerns were “severe” suffered average score drops of as much as 11 points, while students at schools considered “clear” made slight gains.
Responding to evidence of cheating uncovered by an AJC investigation in 2008, the state reviewed every 2009 CRCT answer sheet to measure how often kids changed wrong answers to right by virtue of erasures on the sheets. Because every test sheet was checked, the state was able to develop a reliable index of how often test answers were changed from wrong to right and flag schools that had inordinate occurrences of answer changes, right down to the classroom level.
The worst incidents occurred in Atlanta.
For some APS schools, the drop from last year was stark. At Gideons Elementary, 92 percent of fifth-graders passed math in 2009. This year, 39 percent did. And at Dunbar Elementary, about 87 percent of fourth-graders passed math last year, compared to 49 percent this spring.
In a statement prior to the AJC’s story, Hall acknowledged the scores dropped but maintained the results overall demonstrated the district’s “continued academic progress.”
I am assuming that she will come out with a stronger statement soon and that it will not gloss over the very real evidence of test tampering.
According to the AJC:
The scores showed that overall, students at schools where state officials found cheating concerns were “severe” suffered average score drops of as much as 11 points, while students at schools considered “clear” made slight gains.
Passing rates mirrored the sagging scores. In severe schools, for instance, the percent of students failing math rose roughly 12 percentage points. In cleared schools, the percent of failing students declined by 1 percentage point this year.
School districts should view stark score drops at severe schools as potential evidence of tampering, said Kathleen Mathers, executive director of the Governor’s Office of Student Achievement.
“Generally, in schools where there proves to be a dramatic drop, we have concerns about what would cause that drop,” she said. “If it happens to be a school that had a high number of answers changed last year from wrong to right, that could be an indication that there had been intentional wrongdoing.”
Tell us what you think about the site
Customer Care »
Vacation stops, manage subscriptions and more
AJC Services
Sections
Other Editions
Visitor Agreement | Privacy Statement
© 2013 The Atlanta Journal-Constitution
130 comments Add your comment
APS students need Maureen
June 10th, 2010
11:51 am
Thank you Maureen and keep investigating. I know there’s more. Beverly Hall is going to try to use her persuasive jargon to justify why the scores plummeted. She is far too “intelligent” (arrogant) to allow her reputation to be challenged by mere commoners (she thinks!!!). Beverly Hall has swindled the APS students out of their right to have the highest quality education possible! CLL’s motto is, “It’s about the children.” It appears they are talking about their own and not the ones in APS. This time she has written an enormous check her posterior CANNOT cash. All of her convincing lingo will not keep it from bouncing!! She must pay the return check fee!!
catlady
June 10th, 2010
11:53 am
I certainly agree on giving more credence to ITBS scores. But, you say, it doesn’t measure what we are doing here in Georgia. Boy, you got that right! We spend countless hours teaching the state bird, flower, etc rather than teaching things that are important in the big picture, such as identifying the topic of a paragraph. And, as you say, Georgia politicians are not awarding contracts to their friends for the ITBS nor can they manipulate the data (only suppress the data when they don’t like it.) I also give more props to the norming process, rather than the “cut scores” we change from year to year.
Could someone find out what the ITBS would cost per year vs the cost of the CRCT?
The buck stops here
June 10th, 2010
12:01 pm
Will the editorial board have the courage to call for Hall to resign?
CPT
June 10th, 2010
12:01 pm
Were there any schools that we ‘cleared’ of cheating as a result of this year’s results? Any schools that actually succeeded in raising their scores and sustaining those gains with a new set of students?
Springdale Park Elementary Parent
June 10th, 2010
12:09 pm
I’m really disappointed in most of the commenters here. It’s as though they’re making the argument, well, sure there was a whole lot of cheatin’ going on, but…but…but….
No buts. There was clearly cheating, and those of you who think there’s a threshold for “serious” (vs. less serious?) test-administration cheating are wrong. It must NEVER be tolerated in ANY amount. It is STEALING the future from a child to let that child accept credit for lessons unlearned.
There is MASSIVE test-administration cheating inside the APS; the upcoming numbers from Utah will prove that conclusively even without today’s news. So start orienting yourselves to the task at hand: scrubbing clean the APS bureaucracy, right down to the school level (but starting with Bev Hall). No superintendent of schools can in good conscience fail to resign when her claim of “nine years of steady improvement” is so suddenly and thoroughly debunked. But since she doesn’t have a conscience, we’ll have to get rid of her. Sonny, this is your moment. Put down that barbecue and use those greasy, stubby fingers of yours to do the dialin’ that needs doin!’
Maureen Downey
June 10th, 2010
12:12 pm
To all from John Perry on the GOSA methodology:
Here’s the shortened link: http://bit.ly/b26jPP
Or the full link
http://public.doe.k12.ga.us/DMGetDocument.aspx/GOSA%20CRCT%20Analysis%20Report_3.1.2010.pdf?p=6CC6799F8C1371F6E8EF7082EB992B23516C27A28F7D821EBC2A0AAB1DA796EA&Type=D
John Perry
Database Specialist
The Atlanta Journal-Constitution
Now Maureen?
June 10th, 2010
12:16 pm
Now that we see the monumental drops that prove beyond any reasonable doubt that cheating occurred, will the AJC finally try to find out what did Beverly Hall know, and when did she know it?
Eraserhead
June 10th, 2010
12:28 pm
Anyone asking for “more information” before they are convinced is delusional. It doesn’t matter that these are different students considering it’s across every school. It’s not like kids born in a certain year are particularly stupid or smart. It averages out.
catlady
June 10th, 2010
12:33 pm
I expect that more excuses will be made. But, maybe it is time for the groups that “awarded” APS, and in particular the supt., with hefty financial rewards, to sue to recover those and give them to the truly deserving. Also, a public apology from those groups would be appropriate, with a public stripping of the awards.
THEN, the removal of Ms. Hall and her underlings. Perhaps the taxpayers can get a refund also. No one should keep ill-gotten gains.
ASHLEY
June 10th, 2010
12:58 pm
Dingy, I to am a graduate of the public school system in Alabama , Huntsville to be exact (1976). I am apalled by the Georgia school systems and the excuses they make for their incomptence. You can blame the parents, you can blame the teachers you can even blame the administration., but the only people who suffer are the students. we live in this 21st century bubble of high-tech, yet education is worse now than it was thirty years ago. Houston we have a big problem.
catlady
June 10th, 2010
1:02 pm
Since there is such disparity among failure rates, could your database guru include SES (as measured by Free/Reduced Lunch rates)?
E. Cobb Parent
June 10th, 2010
1:23 pm
Did they note that they lowered the cut scores this year?
Dunwoody Mom
June 10th, 2010
1:33 pm
I don’t think the cut scores were lowered.
To Springdale parent
June 10th, 2010
1:40 pm
Springdale parent, your principal Yolanda Brown came from CW Hill which was also on the Severe list.
mypassion
June 10th, 2010
1:52 pm
The secret to a successful school system are the parents. If all parents would practice the four basic ingredients for academic success with their children, every school and child would be successful. Check out the four (4) basic ingredients at http://www.readtomeamerica.com
APS Teacher
June 10th, 2010
2:06 pm
APS is Corrupt hit the nail on the head. The system is broken beyond repair. The only solution is a complete central office and administration house cleaning. These people are all in one another’s back pockets, taking tax payer money for doing a completely and thoroughly incompetent job, all the while harassing and intimidating the employees who know it.
Angela
June 10th, 2010
2:07 pm
@ Maureen,
You still have yet to address my question about GACE. You continue to discuss the CRCT all of it is about cheating and wrong doings. Is there something we are missing or hiding? We continue to discuss good and qualified teachers why can’t we seem to discuss the other end of the cheating? Or, should I twitter this information, twitters seem to create more of an uproar?
APS Teacher
June 10th, 2010
2:08 pm
@ To Springdale Parent- If Springdale Parent is who I suspect he is, he already knows this, and knows that Y. Brown is well-protected by the powers that be in APS and, if anything, will likely come out of this mess with a promotion.
RobertNAtl
June 10th, 2010
2:09 pm
The cheating scandal at APS will never be adequately investigated in the absence of a criminal proceeding. The “commission” appointed to “investigate” the matter is just a huge whitewash mechanism. It is the DA who needs to take charge of this matter, not the school system.
Educator for Life
June 10th, 2010
2:15 pm
I see some people are still trying to find loopholes in the erasure scandal. It is obvious that cheating occurred. The funny thing is that I keep hearing how you can’t compare this to that, pressure on the students, pressure on teachers, etc. If a 6th grader has been a high achiever every year except this year, there could be outside factors, but there is a chance that his/her answers were changed. One grade going from 92% to 39%? There is no way to explain that, especially since the same teachers remained at the school (I know that for sure). Look, bottom line is that we should be teaching to master standards and the students will pass any test if we do teach that way.
In regards to monitors all over the place in APS, I was confused as to why APS felt the need to send 3 people from the district to monitor the classrooms. THe GOvernor’s Office of Student Achievement sent a state monitor to watch for the lock-up of materials and to peek in on classrooms. Those district folks were getting in the way and were trying to do the Testing Coordinator’s job. It was crazy.
double take
June 10th, 2010
2:23 pm
last year when all these “scores” came in B. Hall got a big fat bonus; she blamed teachers for cheating, while 2 admin pleaded guility; now scores have dropped and she will blame teachers again. Hall and the admin are after the bonus money and could give a “____________” about you or your kids. She takes up pay from 5-6 classroom teachers; expense account, car, etc… Time to let her go to DC with K Cox so they will both be out of GA
double take
June 10th, 2010
2:26 pm
@catlady 12:33
i like it
Springdale Park Elementary Parent
June 10th, 2010
3:03 pm
@APS Teacher–you are right, I did know about Springdale Park principal Yolonda Brown–and have blogged extensively about this (do a search for vahiblog + greenwood avenue) The cheating accusations have put her in a very difficult position. How are parents to react when they’ve been sold a new principal by Bev Hall on the strength of that principal’s school’s CRCT scores, which later turn out to be fraudulent? Sort of like this question: how do you continue to trust a superintendent whose “gains” are shown to be illusory (the result of gentrification and test-score cheating) and who, in 9 years, still hasn’t managed to clear out the crooks in technology procurement?
An advocate for public education change & choice
June 10th, 2010
3:44 pm
There is alot of room for debate over teh method of test score comparsion, however to see the percentage drop between this year and last is a clear signal to me that something has happened and is happening.
We you combine this with the ERATE scandal, I beginning to wonder to myself how much longer before the fingers start to point squarely at Dr. Hall’s office. Could it be that the Empress has no clothes??
Teacher4APS
June 10th, 2010
4:19 pm
This is my first year as a teacher in APS, and I found it odd that students on all grade levels were told that they had to pass the CRCT to be promoted. I was in awe as 4th graders came up to me saying that they passed and would be promoted. Is this being told to students everywhere?
I’ve heard a lot of rumors about my school which is on the severe list and other schools, but I don’t know what is true. What I do know is that the reform programs are a waste of money and hurt more than help. SFA is a total joke. But if you’re caught off script you will be reprimanded. It will interesting to see how this plays out.
Time to ask
June 10th, 2010
4:20 pm
Maureen in light of the possibility that criminal activity has been committed again at APS, will you ask DA Paul Howard why he hasn’t launched a criminal investigation?
And since you seem to communicate with Bert Brantley on a regular basis, will you ask absent of a DA investigation, will Sonny ask someone from GBI to look into the possible illegal activity at APS?
Attentive Parent
June 10th, 2010
4:41 pm
Maureen,
I haven’t seen a story on Obama nominating Beverly Hall to be a member of the prestigious IES Board where she would be in a tremendous position to help set national ed policy, especially in the area of what is effective research and practices..
Here’s the link to this week’s story:
http://blogs.edweek.org/edweek/inside-school-research/2010/06/this_one_almost_got_by.html
Given the extent of the evidence in this CRCT scandal and the fact that the IES position is subject to Senate confirmation, does this seem to be an appropriate appointment?
Angela
June 10th, 2010
4:49 pm
@Time to ask
June 10th, 2010
4:20 pm
Maureen in light of the possibility that criminal activity has been committed again at APS, will you ask DA Paul Howard why he hasn’t launched a criminal investigation?
And since you seem to communicate with Bert Brantley on a regular basis, will you ask absent of a DA investigation, will Sonny ask someone from GBI to look into the possible illegal activity at APS?
******************************************************************************************************
It would be nice to see if you get an answer. It seems to me that when questions get a little too direct or toe steppin – the questions become ignored.
Fericita
June 10th, 2010
4:53 pm
I’m sure cheating went on, but I agree with Atlanta Mom. This would be a much more impressive comparison if students’ scores this year were compared to the same students’s scores from the previous year. I think that would give a shockingly clear picture of what these schools did. Did students pass with flying colors last year and then fail this year? I bet they did, and that would be the nail in the coffin.
Maureen Downey
June 10th, 2010
5:00 pm
@Angela, Literally just arrived in Savannah for Georgia School Boards Association where candidates for governor and school chief will speak. I can ask Howard or the gov’s office, but cheating goes to the PSC first.
Maureen
Ros Dalton
June 10th, 2010
5:01 pm
Does anyone still want to debate whether or not this is an institutional cheating issue as opposed to an individual one? We’re not talking about a couple of kids here and there, we’re talking about entire classrooms, entire schools, where the results overwhemlingly show a broad scale of cheating.
The question now is whether or not the administrators and officials who approved and oversaw the cheating effort have enough pull to escape the taint. I personally suspect that in our top heavy education beauracracy there are plenty of empty hats standing around acting as insulation, ready to be fired to seal off those responsible from their mess… which they’ll be perpetrating again just as soon as we get distracted by the next scandal.
teacherforlife
June 10th, 2010
5:03 pm
@DunwoodyMom – My AP (I teach in DCSS) said that cut scores in 3rd and 5th Math were lower than last year. I have looked for confirmation on the GDOE website, but have not yet been able to find it.
Angela
June 10th, 2010
5:09 pm
@Maureen,
Thanks, If that is the case should I send my findings about GACE to the PSC too?
Maureen Downey
June 10th, 2010
5:17 pm
@Angela, I think it would be a good place to start. You could sent it directly to the head of the PSC Kelly Henson.
Maureen
Maureen Downey
June 10th, 2010
5:23 pm
@Fericita, Did you see the note I posted from John Perry, our database expert, responding to this issue?
I am repeating his note here in case folks missed it earlier in the postings:
Beverly Hall's "New" Record Deserves National Attention
June 10th, 2010
5:39 pm
Probably missed it, but has anyone asked what a normal fluctuation is from year to year for other school systems? Students vary year to year, but not by “that” much for a whole system. Where the teachers are concerned year to year on a class level, there can be greater differences in students abilities year to year and that’s why you can’t implement a fair evaluation system based on a fairy tale test.
Looks like everyone’s keeping an eagle eye on the teacher’s giving the test when this is probably not when the cheating occurs. Otherwise there would be less of a need for erasures. It’s the administrators with access to the test later and have time to change the answers. Unless changes were made when the teachers got together to “clean” up the extraneous marks on the tests after the fact. But extra monitoring during the test is overkill and mostly for show.
@APS IS CORRUPT
June 10th, 2010
5:44 pm
I remember the convocations APS would hold at the dome and I do remember the 2007 targets. We were handed little trinkets like rubber keychain holders that read “We will meet %100 in 2007″. I felt like the dome was a huge church day trying to pump us up or something. Someone figured out that planning day was a waste of money and mandatory convocation ended.
pierre
June 10th, 2010
6:40 pm
I have taught in Georgia for ten years and have therefore dealt with standardized testing in one form or another throughout my career in this state. Although I certainly intuit differences in the aggregate talent/ability of my students from year to year, the comparative mean and median test scores never change that much. Never more that a couple of percentage points. So the complaints of a number of head-in-the-sand apologists on this blog about how the AJC has crunched the data have no merit whatsoever. Admit it. There has been no miraculous upward march in APS system test scores. While I doubt that there was organized collusion, there is no question that some deep, systemic cheating went on until it was no longer possible for it to continue. What the AJC has unearthed is a fair, revealing, and damning indictment of an entire culture of intimidation on the part of Beverly Hall (get me results or else; by all means make me look good but never, ever make me look bad) and flat out cheating for survival on the part of certain teachers and administrators in the APS system. The fact that the underlying test-driven philosophy of No Child Left Behind is the root cause of this panic-stricken need to cheat in order to survive is no excuse for undermining the integrity of the teaching profession. I want heads to roll, starting with that self-aggrandizing piece of work that goes by the title of APS superintendent…
East Cobb Parent
June 10th, 2010
7:05 pm
To Dunwoody Mom, yes the cut scores were lowered this year.
Dunwoody Mom
June 10th, 2010
7:06 pm
Any idea what those new cut scores are? I looked at the 2009 and 2008 cut scores for elementary schools and they were the same.
On the front page of the NYT right now....
June 10th, 2010
7:13 pm
Front page story right now in NYT on the increasing incidence of standardized-test cheating, and Georgia and APS are right in the crosshairs. So who do you think the NYT went to to deliver the key defensive quote, the “it’s not as bad as you think” quote? That’s right: Bev Hall, who desperately, pathetically wants it to be thus.
Find sand. Plant head. Repeat.
Bev: retirement beckons.
Fericita
June 10th, 2010
7:14 pm
Thanks for clarifying that Maureen; I somehow missed that earlier post about comparing grade cohorts.
This is is sad for the students. Individual schools and the state as a whole make instructional decisions based on the CRCT. Kids qualify (or don’t) for the Early Intervention Program based on “bubble” scores (low passing – just failed) and get extra, small group help. ESOL kids can be exited out of the ESOL program if their score is high enough on the ACCESS test for ELLs and the CRCT. Kids are even grouped into extra sessions without official titles if they are on that bubble, and benefit from individualized attention. So, as a result of this cheating, kids were denied services for at least a year that could have really helped them. Not to mention the inflated opinion parents and the kids had about their abilities. So shameful.
I wonder what made the administrators or teachers, or whoever, do it. I’d rather my borderline, struggling students fail than pass. Why should they pass if they’re not on grade level? That’s their reality, and the scores are used so often to group students that I want a score that will get them extra attention.
catlady
June 10th, 2010
7:22 pm
How do scores improve? First, the students could genuinely know more and answer the questions correctly. OR the cut scores can be manipulated. OR the test questions can be written with easier vocabulary and questions. OR you can cheat.
Fericita
June 10th, 2010
7:22 pm
Gwinnett Parent – The only thing scarier than a 39% passing rate is the fact that in order to pass, you only have to get 45% of the questions correct!! That might have changed this year, but I was told that was last year’s cut-off score.
Could be worse
June 10th, 2010
7:26 pm
How bad would it have been for APS if they hadn’t made the test scores easier and hadn’t lowered the cut scores?
Maybe someone needs to pull some email and phone logs from Kathy Cox and her “good friend” Beverly Hall.
Fericita
June 10th, 2010
7:32 pm
On the front page of the NYT right now…thanks for posting that NYT has an article too. I just read it – the angle they take is that with high-stakes (high-punishment) testing, cheating increases. I wonder if this will change any opinions at the federal or state level about tying merit pay to test scores.
"Under Pressure, Teachers Tamper With Test Scores"
June 10th, 2010
7:39 pm
“Under Pressure, Teachers Tamper With Test Scores”
http://www.nytimes.com/2010/06/11/education/11cheat.html?hp
double take
June 10th, 2010
7:50 pm
@pierre
very well said
Que Vee
June 10th, 2010
8:13 pm
@ Maureen….. in light of this investigation, it would behoove the tax payers to know that APS has violated federal guidelines with regards to E-rate and other programs such as Title One. Further investigations will reveal friviolous spending of federal dollars.
David S
June 10th, 2010
8:15 pm
3 kinds of lies – lies, damn lies, and statistics. (Benjamin Disraeli)
Government school failure will just result in the call for more money, never in the call to end government schooling. Private businesses would fail with these kinds of results. Government schools continue to flourish. Judge that for yourselves.