New AJC investigation casts doubts on integrity of testing nationwide. Is there a whole lot of cheating going on?

In the cheating hall of fame, Atlanta may stand out, but it may not stand alone.

Nearly 200 school districts across the country have such suspicious test score patterns that the odds of them occurring by chance are worse than 1 in 1,000.  And in 33 of those districts, the odds are worse than one in a million.

In a powerhouse investigation in Sunday’s Atlanta Journal-Constitution, the investigative reporting team that uncovered test disparities in Atlanta Public Schools reveals the findings of a seven-month analysis of 1.6 million records from 70,000 public schools nationwide.

Here is a link that will get you to the entire package, but plan to spend some time as it has multiple elements.

The AJC used freedom of information laws to collect test scores from 49 states — 14,743 districts and 70,000 tests –  to look for the sort of patterns that signaled cheating.

Along with our own database reporters, the AJC consulted outside experts to assess our analysis. (Please pick up a Sunday AJC as it will outline all the detailed work that went into this investigation and all the care to check and recheck the findings.)

To be clear, the new AJC national analysis doesn’t establish that cheating occurred. But it points to the same troubling  pattern later verified in Atlanta schools to be test tampering after a probe by an outraged Gov. Sonny Perdue.

The student performance rises and dips in many Atlanta schools turned out to be a seismograph of shame.

The findings also point to a universal truth: Hold people accountable to standards, benchmarks or quotas that they feel are unrelenting, unrealistic and unfair and some will cheat.

“We are putting way too much pressure on people to raise scores at a very large clip without holding them accountable for how they are doing it,” Daniel Koretz, a Harvard Graduate School of Education testing expert, told the AJC.

The AJC’s findings also raise questions about whether anyone knows yet how to succeed in schools with high concentrations of poor students; most of the districts with troubling test score swings were rural and urban districts steeped in poverty.

Some immediate questions come to mind as you read the in-depth investigation by AJC staffers Alan Judd, Heather Vogell, John Perry, M.B. Pell and Dayton Daily News database specialist Ken McCall.

Are we expecting too much of teachers instructing the toughest students?

By basing school evaluations on student test scores, are we using too narrow a lens to see what is truly happening in our schools, perhaps overlooking positive developments that are not reflected in a single score?

Are we escalating the pressure on educators by linking their reviews and salaries to student scores, creating even greater motivation to doctor test results?

As the story states:

“These findings are concerning,” U.S. Secretary of Education Arne Duncan said in an emailed statement after being briefed on the AJC’s analysis. He added: “states, districts, schools and testing companies should have sensible safeguards in place to ensure tests accurately reflect student learning.”

In nine districts , scores careened so unpredictably that the odds of such dramatic shifts occurring without an intervention such as tampering were worse than 1 in a billion .

In Houston, for instance, test results for entire grades of students jumped two, three or more times the amount expected in one year, the analysis shows. When children moved to a new grade the next year, their scores plummeted — a finding that suggests the gains were not due to learning. {See response from Houston school chief here.}

Overall, 196 of the nation’s 3,125 largest school districts had enough suspect tests that the odds of the results occurring by chance alone were worse than 1 in 1,000. For 33 of those districts, the odds were worse than one in a million .

A few of the districts already face accusations of cheating. But in most, no one has challenged the scores in a broad, public way. The analysis shows that in 2010 alone, the grade-wide reading scores of 24,618 children nationwide — enough to populate a midsized school district — swung so improbably that the odds of it happening by chance were less than 1 in 10,000.

In Georgia, it fell to the governor’s investigators to prove cheating occurred. Led by two former top prosecutors, the Perdue investigation entailed 2,100 interviews and 800,000 documents and led to more than 80 confessions of cheating. State investigators accused a total of 38 principals with participating in test-tampering. Cheating was confirmed in 44 of 56 schools examined.

The findings toppled the much-heralded regime of Dr. Beverly Hall, and led to extensive upheaval in the leadership of the Atlanta schools.

The findings also sparked a national debate over whether schools teaching the least advantaged and most challenging students are being held to unattainable standards and whether test scores are a fair way to judge success.

The new AJC investigation is bound to reignite that debate.

Among the discoveries by the AJC team:

•Improbable scores were twice as likely to appear in charter schools as regular schools. Charters, which receive public money, can face intense pressure as supposed laboratories of innovation that, in theory, live or die by their academic performance.

•The newspaper found changes in test scores that were statistically improbable in nearly 20 cities, with swings in scores that were virtually impossible in about a half dozen. Human intervention is the most likely explanation  In some cities, we found so many dramatic shifts in scores that the odds of that happening by chance are one in 10 billion.

•In some cities, the results for entire grades of students jumped two, three or more times the amount expected in one year. The next year, when children moved to a new grade, their scores plummeted.

•Though high-poverty city schools were more likely to have suspicious tests, improbable scores also showed up in an exclusive public school for the gifted on the Upper West Side of Manhattan. And they appeared in a rural district roughly 70 miles south of Chicago with one school, dirt roads and a women’s prison.

•The findings call into question the approach that dominated federal education policy over the past decade: Set a continuously rising bar and leave schools and districts essentially alone to figure out how to reach it.

–From Maureen Downey, for the AJC Get Schooled blog