Look ma — No cheating! Dougherty review finds no evidence of tampering. What does state do now?
A consultant hired by the Dougherty County School System criticized the Governor’s Office of Student Achievement’s erasure analysis on the 2009 Criterion-Referenced Competency Tests, which had raised questions about many classrooms in the southwest Georgia county.
In what I suspect will be the findings of most systems asked by the state to review their CRCT practices, Dougherty’s consultant James Wilson maintains that the higher number of erasures from wrong to right reflects the test-taking strategies taught to students.
I do agree with Wilson’s comment that an easy fix to this issue would be online testing.
I am not sure how the state will react to these clean of bills of health that counties are giving themselves. I do think that the position of Dougherty — and likely other systems — still falls short of addressing a critical point. Why would students in Dougherty have such different test-taking approaches than their peers around the state?
In other words, are these counties with high wrong to right erasures full of kids who take tests differently than their peers in Gwinnett or Cobb, which both had very few classrooms with erasure anomalies?
I would love a plausible explanation as I think this is the sticking point.
According to good piece in today’s Albany Herald:
Wilson, who has worked with the board for 11 months and is on contract, said the state relied too heavily on one indicator — wrong-to-right answers — to be able to substantiate its case that Dougherty County schools had an exceptionally high number of test sections with a large number of wrong-to-right
“It’s really kind of sad to be honest,” he said of the GOSA’s efforts. “I honestly think the Office of Student Achievement got what they were looking for. They were looking for a trigger, and that’s their term, to create controversy without doing a full investigation. They only used one indicator going from wrong-to-right and they could’ve used multiple indicators.”
Wilson even noted that test vendor, “CTB-McGraw Hill clearly suggests that these types of analyses must be supported by additional, collateral information.”
As a 35-year educator who has owned Education Planners of Marietta for six years, Wilson spearheaded a three-person team that represented Dougherty County when it viewed the CRCT test documents in CTB-McGraw Hill’s secure warehouse in Indianapolis on March 22. The team of Wilson, Gerald Eads, the coordinator of evaluation and research for the Georgia Professional Standards Commission, and DCSS Test Coordinator Renee Bridges studied the actual scoring sheets of third- to eighth-grade students.
Sheets for the first grade and second grade were not available since some of the test questions could be used again the next two years. Of Dougherty’s flagged test sections, 63 percent of those in the eight elementary schools in the Severe Concern category and five elementary schools in the Moderate category were in first or second grade test sections.
Four percent of state schools made the Severe Concern list by having a school with 25 percent or more of its test sections flagged for wrong-to-right answers. Six percent of state schools were listed at the Moderate concern level with 11 percent to 24 percent of test sections flagged. Eighty percent of schools in the state were placed into the Clear of Concern category, which is less than 6 percent of the test sections within a given school were flagged.
“The erasure analysis cannot tell who, when, or why erasures were made, only that a high number of erasures of wrong to right answers,” Wilson states in his 13-page DCSS CRCT Erasure Study. “Viewing the actual test answer sheets, we were able to validate that with almost every student there were erasures from wrong to right and erasures from right to wrong and wrong to wrong.
“Many classroom scoring sheets were reviewed,” he continued. “In one fifth-grade class, English/Language Arts, with 22 students, we reviewed every erasure mark. The erasure marks of the individual 22 students on this particular test: 9, 8, 7, 7, 6, 4, 4, 3, 3, 3, 2, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0. This class was flagged.”
Wilson said that some questions caused students to mark all the answer options and then erase all but one choice. Wilson said this may have been caused by teachers encouraging students, especially first- and second-graders, to answer all questions and then going back later to check for possible mistakes. He also said some teachers suggested making marks on questions you’re not sure about, Wilson said of conversations he had with teachers.
Dougherty Supt. Sally Whatley said students were taught to use a process of elimination, which caused a lot of erasures. As a result of the state’s findings, Whatley said students were instructed when they took this year’s test to make as few marks as possible on the answer sheets and that excessive erasures were discouraged.
“In such a case, if the erasure of multiple items were of the same depth, CTB clearly stated that the scanner could have picked these up as wrong to right changes,” Wilson stated in his report. “There were several lines where students had made small tick marks with their pencil. CTB stated that in such cases, the scanner could have picked these up as wrong to right changes. There were instances where the student erased so hard that it almost wore through the paper. Several students tried to draw the circles and letters back. CTB stated that in such cases, the scanner could have picked these up as wrong to right changes.”
After giving his presentation, board member Michael Windom provided his assessment of the state’s erasure analysis investigation. “The results weren’t valid because you need other forms of judgment,” he said. “I would just conclude the Office of Student Achievement just isn’t very accountable. If we’re going to be accused of something, at least prove it.”
Whatley, who had to file an Open Records request to get some information from the state, said that if the first and second grade erasure marks were taken out of 12 of the 13 schools, the percentage of test sections flagged decreases significantly. At the end of his presentation, Wilson said adapting an online testing system would eliminate entirely the use of erasure analysis investigations.
“The state ought to let the students take the tests online,” he said. “You take it, you finish it, you’re done with it. That’s where we ought to be today is working with technology instead of accusing second graders of cheating because of their erasure marks.”
Our AJC database reporter John Perry – who was part of the AJC team that first reported on the improbable CRCT score swings that sparked the state audit — sent me two notes about the Dougherty review that I am sharing:
Apart from the erasure analysis, our analysis of test score change points to at least three Dougherty County elementary schools with unusual gains in 09: Wet Town, MLK and Jackson Heights. They all had at least one test that was at least 3 standard deviations above their predicted score. Jackson Heights had 3 tests with gains above 2 standard deviations above the predicted score, and MLK and West Town each had 2.
All three schools were in GOSA’s severe concern category because of erasures. And even with Whatley’s “corrected” count of flagged classes, West Town is still in the severe category. Jackson Height and MLK both dropped to the moderate concern category.
Also, your blog might be a good place to ask if those test-taking tips are really different from what teachers statewide tell their students. If those techniques are common, then they don’t explain why one school has more erasures than another
Tell us what you think about the site
Customer Care »
Vacation stops, manage subscriptions and more
AJC Services
Sections
Other Editions
Visitor Agreement | Privacy Statement
© 2013 The Atlanta Journal-Constitution
62 comments Add your comment
V for Vendetta
April 29th, 2010
12:39 pm
Here we go . . . .
Maureen, you are absolutely correct. It is PAINFULLY obvious that these counties and schools cheated. I fear this will be yet another example of how low we are sinking as a community, state, country, etc.
It’s always someone else’s fault. There’s always some sort of excuse . . . .
Allen
April 29th, 2010
12:50 pm
I’m not so sure in this particular case V.
If I were taught to tick A,B,C,D on a question on which I was unsure of the correct response and to go on so I didn’t run out of time, then came back to that question and figured it out, erasing A,C,D and leaving B, which happened to be correct–which appears to be how thses kids were taught to take the test and what happened in some cases–by the criteria OSA used, which counted only the times I went back and chose B (correct) and not the times I went back and chose D (incorrect) then it would appear cheating had occurred, because the measure of cheating was not valid.
Which is kind of ironic, given that the CRCT itself is an entirely invalid measure of anything.
GA Teach
April 29th, 2010
12:56 pm
This is a process that has happened in our testing culture. Why teach the content if you can teach how to test?
It’s called teaching to the test…….
Joy in Teaching
April 29th, 2010
1:24 pm
Let them take these tests online? My school has 2 computer labs with 25 stations in each one. My school also has over 1100 students. I’m not going to do the math, but it would take WEEKS to get the testing completed for the entire school.
By the way…I’m sick, sick, SICK of having to sit there and twiddle my thumbs while watching my students test this week. We start around 7:45 and don’t get finished until after 10:00. It is boring, stressful, and did I say boring? I can’t do ANYTHING whatsoever other than stare at my students and hope they don’t use their eraser. We’ve been told that if we see a student erasing more than once, then we need to record it on the “Reportable Incidents” form.
Sigh.
GA Teach
April 29th, 2010
1:38 pm
These schools did not cheat.
GA Teach
April 29th, 2010
1:39 pm
They are told to mark the test with the plausible answer the go back and erase the one they believe is not correct.
Allen
April 29th, 2010
1:44 pm
@GA Teach–
Exactly. And if you count only the times they erase and replace with the correct answer and not the times they erase either right or wrong answers and replace with wrong, it will look odd–because you are only looking at part of the data.
catlady
April 29th, 2010
1:59 pm
As a teacher of nearly 4 decades, I have given many, many standardized tests. Although I give my students many test-taking hints and strategies during the months before the CRCT, I virtually NEVER see ANY going back and erasing ANY answers! The few I have had that did several erasures were all sped kids, who would randomly mark, and seemingly randomly erase, to pass the time.
In the testing period just concluded, I tested 5 students. Not a one erased a single answer, that I noticed (and I was circulating constantly).
I cannot figure out why the difference.
catlady
April 29th, 2010
2:00 pm
Let’s see if these miracle wrong to right changes happen this year!
catlady
April 29th, 2010
2:04 pm
Maybe instead of telling kids to “mark an answer” then go back later and review it, they should tell them to “take your time and mark the most correct” answer the first time, then go back and review any you are not certain about.
d2
April 29th, 2010
2:07 pm
If you have ever proctor you will notice that some students will be off a number and do’t realize it until it has ahppen for a whole role. Other students will just erase half the answers because when they started to take the test they didn’t really care and finished early then begin to get bored with all the time and started to try to answer the questions. It amazes me that the State Government has question the ethics of teachers cheating while they are busy getting a 100,000 tax write off, sleeping with lobbyist, passing bills that benefit thier on company such as a web-base grocery chain, demote their own kind for having a difference of opinion, can’t even pass a half way descent ethics bill, sneak in other bills as an amenedment because if any citizens found out they would be mad, lie about not raising taxes–before the General Assembly and the Governor throw stones, they should look at their on ethical problems!
catlady
April 29th, 2010
2:09 pm
In addition, the kids can mark in their books. Why teach them to mark their answer sheets incorrectly at all? Have them leave blank the ones they are unsure of and come back to them after they get to the end. Surely the kids are not unsure of every question! You do have to teach them to be sure their question number and answer number match, however. If, in fact, this explains their excessive erasures, perhaps someone associated with this county needs to teach better test-taking strategies!
Allen
April 29th, 2010
2:31 pm
Catlady–
I agree with your idea–it’s how I took tests and what I tell my kids to do. I’m just saying that–if the story is accurate–it does not appear to be what these kids in Dougherty were told to do. And–again, if the story is accurate–in any such analysis to count only the erasures that end up correct without comparing to the number of erasures that end up incorrect, well that’s just not valid.
V for Vendetta
April 29th, 2010
2:42 pm
Allen,
I’m with catlady. I think you might be giving them too much credit, and, as Maureen pointed out, why don’t we see similar patterns in more successful conuties such as Cobb and Gwinnett?
It is statistically improbable that the counties in question did not tamper with the tests.
RobertNAtl
April 29th, 2010
2:44 pm
My 8-year-old was writing a note to her teacher last night (for “Teacher Appreciation Week”), and she wrote, erased, and re-wrote just about everything on there two or three times…she misspelled something and wanted to fix it….she didn’t write in a neat line because the note card was unlined….just nothing she did was good enough for her. I think a LOT more evidence is needed to conclude that there was cheating than simply the number of wrong-to-right answers in isolation.
d2
April 29th, 2010
2:45 pm
The problem with using computers is that there is not enough computers to use for students. I mean school systems can’t even have working pencil sharpeners in every room.
who ever heard?
April 29th, 2010
2:47 pm
I have never heard of marking in all of the answers and then erasing – not in my years as a student or a teacher. Still smells fishy to me…
d2
April 29th, 2010
2:47 pm
that should be can’t afford to have working pencil sharpeners in every room
Kira Willis
April 29th, 2010
2:47 pm
I have a question about the whole testing process, any way. What would happen to me if my child went to school and told her teacher that I made her sit in a seat for 200 minutes without talking or moving? I know what would happen; DFACS would be called, and I would be investigated as a parent. Yet we do this to our children.
Allen
April 29th, 2010
2:52 pm
V–
Actually there’s no way to know about the statistical probability if OSA didn’t look at all the statistics: right to wrong, wrong to wrong, and wrong to right.
I’m not saying no cheating happened, I’m saying relying on the GA DOE’s to do a good job of analysis may not be the soundest strategy for determining what did happen.
catlady
April 29th, 2010
2:53 pm
Robert, writing a note (free response) is quite a bit different from taking a multiple choice test!
If the folks in Dougherty (or wherever) don’t know any better than to coach their kids on test-taking like this proports to be, they should be considered for termination–is that how they took tests for their degrees?
I am guessing that the follow-up will tell the tale. Did the kids do worse, better, or the same this year? How did the erasures look? Hopefully the state has set aside money for the follow-ups?
d2
April 29th, 2010
2:53 pm
I am sorry about all the typos in my first post at 2:07–Scan trons are so easy to get off a number–you can have a whole roll off before you realize it–I am not saying that is the case in some of these incidents–I kinda agree about seeing if the erasure were right to wrong as well as wrong to wrong and wrong to right. I am just speaking in general terms with all the flags flown up in several of the schools–not just down in Albany.
Lilard
April 29th, 2010
3:04 pm
Who can shake up education in Georgia? These candidates are going to change education by issuing a bunch of platitudes on their generic websites? Come on. Please! The more things change, the more they remain the same. I too am disgusted with all of the standardized tests like the CRCT. Dougherty County hired a consultant and was exonerated. Give us a break. The cartoon character Noggins could come up with something better than this!
one eyed jack
April 29th, 2010
3:06 pm
If the state provided every teacher with a detailed lesson plan and text book for each subject that is taught in the state and was uniform from system to system, then the standardized tests might have some meaning. However, each school system picks its material differently and it may or may not reflect the state standards that are being tested. This is a glaring example of why standards based testing will never has any real credibility.
Lilard
April 29th, 2010
3:15 pm
That’s a fact, Jack!
catlady
April 29th, 2010
3:19 pm
Allen, you can’t count on the DOE for much of anything!
Jack, unfortunately what you may be advocating does not take into account that in your fifth grade class you might have only 30% able to do “grade level” work. Teachers are supposed to take that into account, provide “needs-based” accomodations and try to make up 2 or more years of basic instruction while still teaching the GPS.
Most systems have a map that tells teachers when they should be on which standard, BTW. So you’d better be doing two digit division, even if your students cannot add or subtract accurately!
GoodforKids
April 29th, 2010
3:55 pm
In the above article, the reviewer talks about looking at the total number of erasures in a fifth grade classroom (not just wrong to right erasures). Why all the references here to only considering wrong to right? Is that what the OSA originally did? I cannot remember.
I agree with catlady in terms of typical test-taking behaviors. I am a psychologist who has always worked with young children and in schools. Also a former school counselor. I have not given lots of bubble tests, but I have given lots of tests and observed lots of kids in twenty years. The description given of students being encouraged to mark all the answers on a bubble sheet and then going back later to erase all but the right one is nothing I have ever heard before. I am not saying it didn’t happen, but it is certainly an unusual test-taking strategy. These systems need better training for teachers if this is what they are doing.
also don’t think young children change their mind about answers that often. They generally give it their best shot and leave it behind. Even if they check their work, I do not think they are as likely as older children or adults to reconsider. A small percentage of them will, but I would wager that most don’t. Has to do with their cognitive development and also their likely level of investment in the task. Even if their school held a pep rally and their teacher is doing back flips for the CRCT, it is still, as a teacher blogged above, a BORING task that is VERY DIFFICULT to maintain engagement in. Just my opinion. It is certainly possible that teacher instruction caused a very different behavior but…
I also agree that the child who was writing to her teacher and erased a lot is different in some ways.
Rob
April 29th, 2010
4:02 pm
I am a skeptical of both the school system and the company that did the analysis. They both have interests in a particular, though different, outcome. The schools need to show AYP. If the company doesn’t find evidence of cheating, then states will be less likely to pay for their services.
If students were taught to approach problems they aren’t sure about by crossing out answers on the scantron that they know are wrong and to go back at the end and either work on these problems some more or guess, then that could produce a high volume of wrong to right erasures. (Doing this on the form and not in the answer book is a good method to make sure you don’t skip a line on the scantron form.)
I agree that the analysis seems incomplete and should have included wrong to wrong and right to wrong erasures.
V for Vendetta
April 29th, 2010
4:17 pm
Allen,
HAHA, you’ve got a point there! I don’t want to rely on the DOE for anything. Look where it has gotten us!
oldtimer
April 29th, 2010
4:35 pm
In all my many years of teaching and (yuck) testing…I have never-ever heard anyone tell anyone to mark all the answers..ever. There are many stratigies for skipping an answer so you do not make a mistake..but this was too weird.
You try giving the test!
April 29th, 2010
4:41 pm
As a specialist who gives tests to small groups of students, I decided to count the number of times I saw children erase and tally it. These were erasures only; I did not read to see correctness. Because I read the test to them, I had to check each child carefully after each item to make sure they were on the right item. Alarmed by the large number of erasures in a ELL/Special Ed class, I reported it to the test director and began logging it in specifically, eg. student x put the answer for no.2 as number 6, student erased and moved answer. I was running up to 9 or more erasures for 3 students, 7 for 2 more. If children with specific needs get off line routinely, wouldn’t younger children (grades 1 and 2) also track in test booklets wrongly,then change the answer when the item was repeated. The fact that the charge was made that children in inner city Atlanta and DeKalb Co. had larger number of erasures may be explained similarly. Children there get less resources to practice bubbling and tracking and may have learning difficulties similar to ELL and Sp.ED. Try doing research before accusing. All research is flawed by the bias of the researcher. They make sure they look for what they want to find by designing the study to prove it easily; why not look at other factors, record them and do a factor analysis to see if other things are contributing to the findings. Valid and reliable research consider findings as either supporting or failing to support the hypothesis. This sounds more like a witch hunt than a study of what is happening and why!
Tweets that mention Look ma — No cheating! Dougherty review finds no evidence of tampering. What does state do now? | Get Schooled -- Topsy.com
April 29th, 2010
4:42 pm
[...] This post was mentioned on Twitter by Maureen Downey. Maureen Downey said: Look ma — No cheating! Dougherty review finds no evidence of tampering. What does state do now? http://bit.ly/9ucq0i [...]
You Asked
April 29th, 2010
4:44 pm
So class… what did we learn from this?
Better yet what are the kids learning? Anyone? Beuller?
Fericita
April 29th, 2010
5:13 pm
If I remember correctly, the instructions teachers read aloud before the students take the CRCT include something along the lines of “If you don’t know an answer, skip it and go back to it.”
GoodforKids
April 29th, 2010
5:48 pm
Anyone? War of 1812?
Jack
April 29th, 2010
5:51 pm
This happened to me this year 8th grade class. I read the instructions in bold print exactly as I was supposed to do. On Thursday the students were to take the science test. The science block for answers is on the same side of the answer sheet as the social studies section. I said “you may begin” and the put the start time on the board. I walked around and had 3 students that were marking in the social studies section. Were they really focused? Were they daydreaming? I do not know but I pointed out to each that they were marking answers in the wrong section. One was on #8, one on #5 and the last was on #6. BAM, I will now have my class kick out as cheating? I wrote it down and after the test had each student sign along with the proctor. This was a higher level class and this is the first time this has ever happened to any class I have taught. On the social studies test day (Friday) I had one student that absent mindedly (?) tapping his pencil and it happened to be in the answer block for science. How sensitive is the scanner? Will it pick these up as a erasure? I also have all my students put a blank sheet of typing paper in the middle section (reading, LA and math section) and leave it there for the entire testing period of 5 days. On the 5th day I had students take out this sheet of paper and turn it in. The procter and I looked at these sheets and there were many graphite transfers from the answer sections to the typing paper. Could these smear as students test over the 5 day period and cause the scanner to pick up as multiple answers? Way to many variables and not enought ways to ensure this does not happen. Did my students cheat? Did I? No and I made sure my proctor was there every step of the way and I documented. What about putting a scanner in every school and having the test coordinator for each school scan in the tests every day. The school would not have to get any scores but it would eliminate next day answer changes.
Testing makes us paranoid
April 29th, 2010
7:10 pm
I cringed everytime I saw a student erase. Usually with the little ones, they erase the correct answer and then bubble in the wrong answer. LOL.
HStchr
April 29th, 2010
7:32 pm
I’ll be administering EOCTs next week. It is amazing to me that we have to tell students to be careful about erasures and document anything that MIGHT be flagged as suspicious. It makes me very nervous and uncertain about agreeing to administer these tests. I think if this is an issue this year, then as teachers we need to refuse to administer the tests or at least require an administrator or BOE member to be there to witness what we do.
I agree though, that not many kids in all the years I’ve taught have erased and changed answers. Certainly not enough to “flag” any group I’ve tested. In high school, they just want to get done so they can sleep until the test is over. The entire testing situation is hard on them and scores rarely match what kids can demonstrate knowing throughout the year. With all the issues we’ve had concerning reliability and validity of these assessments, I doubt the usefulness of such a measure of learning anyway.
Share GA Teach share
April 29th, 2010
7:44 pm
Share with us GA Teach, the wonderful strategies that were used to make gains that outside, independent statisticians with NO ax to grind said were a ONE IN A BILLION chance of happening.
After that, share with us those wonderful team building strategies that led to you your church league buddies beat Kobe Bryant and the Los Angeles Lakers at the annual church picnic.
APSparent
April 29th, 2010
7:54 pm
The Albany Herald: “Wilson spearheaded a three-person team that represented Dougherty County when it viewed the CRCT test documents in CTB-McGraw Hill’s secure warehouse…”
So Wilson was “representing” Dougherty County Schools, with whom he has a contract? What does that mean? Does that mean he was looking to draw an objective conclusion, or does that mean he was looking for plausible deniability just as surely as he accuses the State of “looking for a trigger.” It sure sounds to me like he was hired to find another explanation besides cheating. However, as the AJC database reporter points out, even with all the mitigation Wilson can dream up and all the misdirection he can muster, some of the county’s schools still clearly cheated.
Some of you desperately want to believe the GA CRCT cheating scandal is an invention of the Governor and public education critics; that these poor kids were just doing what they were taught–”keep re-evaluating your work as long as possible and change answers when necessary…”
But to believe that the erase-happy schools did not cheat, you also have to believe the kids in unsuspicious, cheat-free counties like Gwinnett simply never make errors or are taught in a vastly different way, or have socioeconomic advantages that preclude the need for any erasures.
And you must further believe that the OSA was either stupid or incompetent when it attached labels like “severe” to some of these schools.
You doubters are standing atop a tall, rickety scaffolding built with your specious arguments. I’d hold onto something if I were you, because some strong gusts are coming (from the direction of Utah). If you’ve been reading this blog, you know what I mean.
catlady
April 29th, 2010
8:25 pm
Rob, I believe Ms. Downey advised me that you could figure the number of right to wrong by subtracting the wrong to right from the total erased. Do I remember correctly?
Fericitia, the directions do NOT say to skip. But a wise teacher instructs the children how to handle questions they are unsure of long before the test actually happens.
Jack, you really need to worry about those 8th graders! The science section is clearly BEFORE the SS section, and next in line after math, right? Why would they skip a section? On the grade I tested, science is at the top of the page, SS, the last section is the last one on the test booklet.
catlady
April 29th, 2010
8:29 pm
And, Jack, I think you giving those kids that piece of paper constitutes a TESTING IRREGULARITY. The only day the kids can have extra paper is the math day (3) and the paper is to be provided by the school and counted when turned in.
d2
April 29th, 2010
8:33 pm
It is sad that it has come to this–testing, testing, testing, and more testing.
Maureen Downey
April 29th, 2010
9:04 pm
To all, On the GOSA site are charts that show all erasures, as well as wrong to right ones. So, you can look at those charts to figure out the right to wrong.
I will post a link to those charts shortly.
Maureen
Reality
April 29th, 2010
9:33 pm
I have a sister that has taught in the Doughtery County School System for over 25 years.
She has told me point blank that cheating on the CRCTs are not only acceptable practice, but ENCOURAGED by the administration. The teachers are told to do “whatever is necessary” to be certain that the students pass.
Some teachers walk around the room and point the correct answers to students. Some teachers read the questions out loud to the students and when they get to the correct answer, they raise their voice loudly. Some teachers just let the students quickly fill out the form so that they have time to collect them and CHANGE THE ANSWERS to the correct ones.
Maybe someone at the State level should interview the teachers under the condition of job safety or something like that. Some teachers are embarrassed by the cheating and really do want to come clean.
bootney farnsworth
April 29th, 2010
10:16 pm
who actually did the review?
Sonny Perdue?
Lee
April 30th, 2010
7:29 am
Did anyone expect any different results? Investigate yourself? Really??
The reality of this situation is that there is probably no way to confirm cheating by merely reviewing test sheets. You’re going to have to get someone to admit to cheating.
It will be interesting to see the results from these same schools for this year’s CRCT. My guess, some of those schools who saw miraculous gains last year will fall back to their historical norm.
Would that indicate cheating? If it walks like a duck, quacks like a duck…….
RJ
April 30th, 2010
8:08 am
These are the results that I expected. These companies are employed by the school system. What would be their motivation in finding different results? I’ve never heard of teaching students to answer all questions and then going back to figure out the answer. This smells pretty fishy.
The truth will come out when test scores are released for this school year. I wonder what the excuse will be then.
catlady
April 30th, 2010
9:56 am
When does the APS finding of innocence come out?
Tweets that mention Look ma — No cheating! Dougherty review finds no evidence of tampering. What does state do now? | Get Schooled -- Topsy.com
April 30th, 2010
10:16 am
[...] This post was mentioned on Twitter by Christopher Herring. Christopher Herring said: "Look ma No cheating! Dougherty review finds no evidence …" http://tinyurl.com/28×8r9a #MLK #ihaveadream San Antonio [...]
Allen
April 30th, 2010
12:25 pm
For the record–I DO think some cheating has occurred; I have doubts about the way it was assessed.
It saddens me to think that we’ve come to this: we’re arguing over the validity of a review of alleged cheating on a test that is itself an invalid measure, of anything, but that so threatens some systems & teachers they feel compelled to take unethical measures to “succeed” on it. In other words, since the test is meaningless, the fact that some systems cheated while other systems were “cheat-free” (or, more accurately, “cheat-lite”) is also meaningless, in terms of educational quality.
If we had a DOE, a governor, and a legislature who actually cared about education, and not just about passing measures to make it look like they care maybe education in GA could get somewhere.
Ed Johnson
April 30th, 2010
12:45 pm
catlady, expect for APS a finding of “indeterminate.”
john konop
April 30th, 2010
4:10 pm
James Wilson,
In all due respect nothing in his background demonstrates Mr. Wilson has knowledge in research methods. I would love to look at the paid by the accused school district methodology used by Mr. Wilson.
@ APS parent
May 3rd, 2010
5:27 am
APS Parent,
Let me tell you that Gwinnett county does cheat on the tests! In my school, the classes flagged for abnormal erasures were 7th grade ESOL LA, 8th grade ESOL LA, and both 7th and 8th grade “intervention” teams. You see, Gwinnett just lumps all the students who need “help” into like groups and the AP over testing corrects the answers over the weekend! GCPS manipulates the testing environments to facilitate cheating! I have seen this done, but when I spoke up, the negative evals began!
Laurie
May 4th, 2010
11:02 pm
“On the GOSA site are charts that show all erasures, as well as wrong to right ones. So, you can look at those charts to figure out the right to wrong [sic - actually, from that info you can get the TOTAL of WTR PLUS WTW answers, not the breakdown between those two, but anyway...]. I will post a link to those charts shortly.”
Yes….?
Maureen Downey
May 5th, 2010
8:03 am
Laurie, If you go to http://www.gaosa.org/, you will see a list of the reports in a prominent box on the lower right of the screen. The reports have the erasure charts in them.
Teachasa2ndCareer
May 5th, 2010
11:40 pm
If some of you pay attention to kids’ conversations, you will also learn that they are listening to the news as well and are asking a ton of questions. Do you think they cheated? Should I erase on my test? If I do, will I be considered as cheating? These are fifth grade students who were so bombarded with the news that they brought the conversation into the classroom. Just like I tell my own children, I said you have every right to erase. If you feel you got it wrong after review and especially if you realize after looking for your answer in a reading section, please erase. Check your answers! By the way, the teacher manual for test is on the GA DOE site. Teachers are instructed especially this year to read it word for word. One of the parts is for us to tell the students (& I am paraphrasing) to go ahead an answer if you are not sure to answer it with what you think it is and go back and change it. I asked the same questions at our first teacher’s meeting about any analysis performed that shows the percentage of wrong to wrong and right to wrong. No one had this answer either. So, I will be definitely looking at the above link.
Also, the schools in the northern part of the state that are low may have more exposure and a higher percentage of parents who are high school and college educated even today. Furthermore, statements from parents and students who attend there say they soley teach to the test and not do those wonderful performance tasks that the state say do. Finally, has anyone looked at maybe certain people have access to the test prior to the CRCT…that is a form of cheating as well. Please don’t assume just because a school system is in the low are of erasures this means that they don’t cheat. We are still talking about human nature. If you are saying that it happens in one school district, then it happens in all of them.
For me, I have a conscious. I don’t expect my children to cheat or be helped on a standardized test and neither do I. I teach, assist, support, reteach in every possible way I can and work my butt off at home and in the classroom. They know what they know when the test arrive and I try to not make their days anymore stressful than they already are.
I agree, look at all the analyses first and other factors before making the ridiculous statement that erasures alone is an indication of cheating. Apparently, you don’t have children, you haven’t taught at schools with diverse populations, or you haven’t taught at various grade levels for a more in-depth knowledge base.
Laurie
May 6th, 2010
7:40 am
Maureen, there are a number of reports (summaries, etc.) and a number of charts within. Could you please let us know specifically the name or number of the chart (and in which report) contains the number of total erasures classroom-by-classroom? Thanks.
Laurie
May 11th, 2010
12:59 pm
Allen wrote: “Actually there’s no way to know about the statistical probability if OSA didn’t look at all the statistics: right to wrong, wrong to wrong, and wrong to right.
I’m not saying no cheating happened, I’m saying relying on the GA DOE’s to do a good job of analysis may not be the soundest strategy for determining what did happen.”
This is EXACTLY right.
Maureen Downey has posted: “To all, On the GOSA site are charts that show all erasures, as well as wrong to right ones. So, you can look at those charts to figure out the right to wrong.”
This, as it turns out, is incorrect.
I have asked on this blog about 5 times for a pointer to a chart that shows total erasures by classroom (or, alternatively, RTW and WTW erasures, as well as the WTR erasures on which the state relies to flag schools). Neither Maureen Downey nor anyone else has pointed me to them. I have read and reread and reread and reread the reports and the charts therein, looking for the information that is supposed to be there.
Finally, I emailed AJC database reporter John Perry to ask him. He has confirmed in an email to me that the state has NOT released that data, to his knowledge. The state HAS the data (the reports themselves so state), but it hasn’t released the data to the public. (Hopefully, it has released that data to APS and the other school systems it has charged with investigating the matter – although apparently Dougherty had to get some of the data it needed to conduct its investigation with an Open Records request!!) The publicly available charts on the GOSA site show the data on the total erasures STATEWIDE. But there is simply no way to determine from the data the state has released whether, say, in a class that was flagged because the average child had 3 WTR (wrong to right) answers, the average child also had 4 RTW (right to wrong) answers, and 4 WTW (wrong to wrong) answers, for a total of 11 erasures per test. I’ll repeat: there is NO WAY to determine from the data publicly released whether in fact Atlanta kids (or kids in any of the other school systems, or in any particular classroom) in fact do simply erase more frequently than the state average. And there is NO WAY to determine, for any particular flagged classroom, whether the erasures that occurred raised the students’ scores or LOWERED them. Don’t you think that information is, at the least, quite relevant to whether cheating occurred? Don’t you think, if you’re going to rely solely on the high number of erasures in a classroom as proof of cheating, you’d also want to know whether the erasures raised or lowered a classroom’s CRCT scores?
Like Allen, I’m not saying that cheating didn’t occur. (In fact, I think it’s highly unlikely that any school district in the whole state is 100% free from cheating – in a huge group of apples, there are going to be some bad apples.) I’m not even saying that cheating didn’t occur on a widespread basis in APS or any other school district.
I’m saying that looking at only the number of WTR erasures, while refusing to look at the total number of erasures and the relative proportions of WTR, RTW, and WTW answers means that you can’t reasonably draw inferences about the likelihood of cheating from erasure data. Period.
Some, including Ms. Downey, have speculated about the plausibility that Atlanta students, or those in other flagged schools, erase far more than the state average. Well, this isn’t something we should have to be speculating about, because that information is already available to the state. It may be true. It may be false. If true, it may seem strange. If true, it may indicate, as Catlady says, a less than ideal test coaching strategy (or not – I am neither an administrator nor a teacher – I’m a parent who took standardized tests as a child and was never instructed to cross out answers in my answer book either). But IF true, if high WTR erasures is just part of a pattern of high erasures of all kinds (whether in a particular classroom, or in a school district), that DRASTICALLY decreases the chances that the high rate of WTR erasures indicates cheating, and increases the chances that there is some other explanation that some students are erasing more than others.
My question is: why are we speculating about what is “plausible”, when the data has already been collected?
And that’s what is really strange (fishy? or maybe just indicative of very, very sloppy work): the state HAS this data. It says so in the reports themselves. Why is it not releasing this data? Why, even though it’s now been months since spokespeople and others from APS started suggesting that its students were instructed differently and/or may have erased more frequently than some others, why is this data still not public? Those who keep saying that it’s terrible that APS is taking so long to investigate the situation (and apparently just want, what, for Bev Hall and every principal and teacher at every APS school to be summarily fired without any actual investigation?), I ask you to remember that the state itself had almost a year to compile this data and prepare a report – and then ask yourselves, why, many months after the state was on notice (if it had not already been on notice, as I think it should have been) that analysis of all types of erasures was extremely relevant to determining whether cheating occurred, why has it still not released data it has in its possession, much less undertaken to analyze the data?
And – why doesn’t the AJC care?
Maureen Downey
May 11th, 2010
2:18 pm
@laurie, I didn’t get that you were searching for total erasures by classroom. I am hoping that you did see the state breakdown of total erasures by grade, which are online.
John Perry and I just discussed your comment and he has sent a note to GOSA.
Maureen
Laurie
May 11th, 2010
5:56 pm
“I am hoping that you did see the state breakdown of total erasures by grade, which are online.”
I have read all of the GOSA reports and charts, carefully, several times. And, yes, I have seen the breakdown of total erasures in GEORGIA by grade, but I’m not sure why you are referencing them. Those charts are completely irrelevant to the question Dougherty is raising, to the question Allen and Catlady and some others are discussing in the comments to this blog, and to the question that I have repeatedly raised. The issue is not erasures of third graders versus the erasures of fifth graders. The issue is how often APS students (and those in other vilified districts) erase versus how often Georgia students in general erase. We already know that they likely change their answers from wrong to right more often than Georgia students in general do. But do they also make more erasures in general (not just wrong to right ones erasures, which one might expect to be the case if teachers or administrators are doing the erasing in hopes of raising CRCT scores)? And if so, to what degree is this the case? Charts with Georgia data provide only half the answer. To compare APS with Georgia as a whole, you have to have data about APS. (Similarly, to compare an individual classroom with Georgia as a whole, you have to have data about an individual classroom.) You can’t compare two sets of data if you have only one of them! And breaking down the Georgia by grade level, as is done in the charts you reference, doesn’t change that. (I’m not sure why that grade-by-grade chart is even in that report, since the data it contains is statewide only, and thus, in the absence of breakdown by district, school, or classroom, irrelevant to whether any district, school, or classroom is cheating. Perhaps it’s just intended by the report drafters as a distractor, in which case, I can see that it has unfortunately fulfilled its purpose.)
“I didn’t get that you were searching for total erasures by classroom”
Given that AJC.com previously had a “check your child’s classroom and see if the teacher cheated” chart online, I’d love to see data by individual classroom, so that individual teachers have a chance of exonerating themselves publicly. (Yes, I know it didn’t identify the classrooms by name, but in some cases, it was possible to determine anyway, e.g., when all 4 First Grade classes in a particular school were flagged.) And given that GOSA’s WTR analysis was a classroom-by-classroom analysis, ultimately, that’s the kind of data that will be needed to correct their analysis.
But for now, I’ll point out that there is not even any publicly available ***district-by-district*** data on total erasures, so there is no basis for claiming that Atlanta students DON’T erase more than Georgia students as a whole. And no basis for asserting that APS students “def[ied]the odds that kids will sometimes change right answers to wrong ones,” as you stated in a previous blog post.
“John Perry and I just discussed your comment and he has sent a note to GOSA.”
Yay. Thank you.
Maureen Downey
May 11th, 2010
6:01 pm
Laurie, In response to the issues you raised, John sent GOSA a note today. GOSA has sent over additional data and John is going through it today and tomorrow. I asked John if he could write a summary once he takes a look at it. (John is our database expert.)
Maureen