Was the Blue Ribbon commission’s clearance of so many APS schools in APS CRCT cheating probe authentic?
An AJC story today suggests expediency played a larger role than evidence in the commission’s focus on 12 schools with the worse profiles.
One of the problems now is that the commission received no admissions of cheating, although it found overwhelming and irrefutable statistical evidence of widespread cheating at 12 schools. (Even at those schools, there were some classrooms with no cheating, which makes me wonder about the dynamic at those schools when the honest teachers kept wondering why their colleagues down the hall had such great scores and yet their students didn’t seem any better prepared. I have talked to one such teacher and she said she could never understand how her peers got such results.)
I will be interested to see how APS handles the teachers and administrators — and there were many more teachers referred by the commission for sanctions than administrators, suggesting this was a classroom to classroom practice rather than a schoolwide practice. If nobody is owning up to the test sheet cheating, what happens now?
In other states with cheating probes, few people lose their jobs for this reason: Knowing cheating went on is one thing; Pinpointing who cheated is another and far more difficult to prove.
Seventeen schools suspected of some of the most widespread cheating were barely investigated and, consequently, avoided recommendations for sanctions. Another 14 schools where state officials voiced a moderate concern about cheating received similar treatment. The investigators disregarded testing irregularities in hundreds of Atlanta classrooms.
A commission that studied questionable gains on the 2009 Criterion-Referenced Competency Test, or CRCT, reported this week that alleged cheating seemed to permeate just 12 Atlanta schools, rather than the 58 — more than half the schools in the district — state officials flagged last winter.
The commission recommended possible disciplinary action against 109 Atlanta Public Schools employees.
Commission members defend their work. They say they focused mostly on schools highlighted in a statistical analysis performed by a consulting firm they hired.
But a review of the commission’s report and interviews with education officials and testing experts suggest that the investigation fell far short of unearthing the scope of a cheating scandal that calls into doubt a decade of higher test scores and other academic progress by Atlanta students.
“Our expectation was all 58 [schools] would be rigorously and thoroughly investigated,” said Kathleen Mathers, executive director of the Governor’s Office of Student Achievement, which is overseeing the state’s review. The investigation should have continued, Mathers said, “until you’ve cleaned up the problem.”
In particular, the Atlanta commission did little to investigate 17 of the 43 schools where state officials had found excessive erasures on test papers in 25 percent to 51 percent of classrooms.
At Deerwood Academy, for instance, the state flagged almost half of 90 classrooms. The Atlanta commission’s investigators noted unusual numbers of erasures and 100 percent pass rates on two tests given by one teacher. But the investigators interviewed just four people at Deerwood, cleared the entire staff and submitted a report that omitted the fact that an earlier investigation found strong evidence of cheating on a CRCT retest there in the summer of 2008.
At other schools, investigators spoke to as few as two staff members. In the case of one recently closed school, they spoke to none.
Even strong statistical suggestions of cheating and specific allegations did not prompt additional inquiries.
At Humphries Elementary, for example, where the state had flagged nearly half the classrooms for examination, a staff member told investigators that CRCT administrators “cleaned up” test papers in the school auditorium. Yet investigators spoke to only three people at Humphries and reported no educators for possible transgressions.
Scrutiny of Atlanta’s scores on the CRCT, a key indicator of whether schools meet federal and state standards, began after The Atlanta Journal-Constitution published analyses in 2008 and 2009 showing improbable increases in test scores at many schools in Atlanta and in other Georgia districts. In February, the state ordered Atlanta and 34 other districts to investigate schools found in its analysis to have excessive numbers of erasures on test papers.
The commission was appointed by the Atlanta Board of Education and populated by people with financial and civic ties to the district. By relying heavily on the work of its consultant, Caveon Test Security, the panel took an approach that seems to have conflicted with instructions from the state.
Instead of rigorously investigating all 58 schools, the commission gave a lower priority to 31 schools after Caveon reported that its own data analysis turned up fewer concerns than the state had reported.
Many of the 31, however, displayed the hallmarks of altered scores: classrooms or entire grades with suspicious erasures across multiple subject tests; classes with hundreds of erasures when only a few dozen might be expected; double-digit drops in 2010 test scores, under heavy scrutiny; and tips from other educators who suspected wrongdoing.
State officials are reviewing the report. So far, they aren’t happy with the results.
“I would not say I take everything Caveon says as the way it is,” Mathers said. “The state did an analysis, and we are confident in that analysis.”