Do you trust the CRCT?

Welcome back from Spring Break. I hope you all had a restful week and a happy holiday.

It’s test season in metro Atlanta. Some kids will start taking the CRCT tomorrow, while other districts will begin giving the test over the next two weeks.

The controversial test became more so last spring after early results showed about 70 percent of the sixth- and seventh-graders failed the social studies exams and about 40 percent of eighth-graders failed the math.

Parents and teachers were outraged, with many accusing the state of developing faulty tests.

State schools Superintendent Kathy Cox ultimately threw out the social studies results, citing a breakdown between the test questions, the new curriculum and what teachers taught. But she let the math results stand.

Since then the state developed new social studies standards and new exams. Teachers across the state received training on the new curriculum. Also, in math, the state and local school district provided teachers with more training and students with more extra help.

Is this enough to prevent a repeat of what happened last year?

How much faith do you have in the exams?

91 comments Add your comment

Gwinnett Educator

April 13th, 2009
9:09 am

I have ZERO faith in the CRCT! That said, I am NOT looking forward to this mess.

ScienceTeacher671

April 13th, 2009
9:22 am

Does it matter? Most of the students will be promoted whether or not they pass the CRCT. What difference does it make?

Steven Samuels

April 13th, 2009
9:27 am

Whether or not a test is good depends on its reliability and validity. If the CRCT is reliable and valid, then it doesn’t matter whether we (parents/teachers/administrators/etc.) like it or not. I would hope that the DOE asked the company that produces the CRCT to provide the data to show the reliability and validity of the tests. I further hope that those data are publicly accessible. That’s the only way to determine whether or not the test is good.

BB

April 13th, 2009
9:33 am

As a former teacher, I just have to say the CRCT is useless! I taught third grade, and I can’t begin to describe the stress level for both students and teachers. The whole process has become ridiculous. I wasted so much time doing test prep when I could have been teaching. I also spent time covering every inch of walls, bulletin boards, every “word” in my classroom so someone wouldn’t gain an “unfair advantage” over someone else. I also spent time counting out tests, checking them in and out of the office. I’m surprised there weren’t armed guards around the school “protecting” the precious test. The whole thing is a waste of time and money.

flipper

April 13th, 2009
9:46 am

I don’t pay the CRCT any attention at all. It’s a bogus test dreamed up by a bogus president and a bogus senator. It’s not even worth my kids’ time. I just tell them to relax, take it easy and enjoy the extra break time. Too bad they can’t be learning something this week.

jim d

April 13th, 2009
9:48 am

Laura,

Funny you should use the term “FAITH” since faith is something not based on any relevant facts or evidence.

William Casey

April 13th, 2009
9:51 am

Who would trust ANY test mandated by the state?

Tony

April 13th, 2009
9:58 am

There is no reason that the people of Georgia should place confidence in the CRCT as the chief indicator of student learning or school success. I will enumerate just a few of the reasons.

1. The CRCT contracts are issued on a low-bid basis. There are some quality control measures in place, but by and large there are not strong enough measures that assure validity of the tests. This becomes evident when teachers notice irregularities in the materials and are blasted with ethics violations for trying to report the problems. Teachers are not supposed to look at the tests and are thus prevented from “knowing” that there are errors in the materials.

2. Multiple choice items can not measure the important things kids should learn in school. Students should learn to think and reason more than anything else while in school. A multiple choice test, especially one developed under a low-bid contract, can only measure the lowest levels of learning – simple recall. Some will argue with me on this point by saying things like “if the kid can’t perform at that level, then they don’t deserve…” or “modern testing theory allows the development of advanced questioning techniques that infer reasoning…” My response to the child’s performance on the test is that these tests do not measure what is intended. For instance, the math and science tests are more about reading than about the important concepts. The response to the testing theory is that we don’t invest enough money in the tests to allow sufficient field testing of items or continuous development of items.

3. The tests are not released to the public. In the original plan for statewide testing under CRCT there was a requirement that the tests be released annually. What better way would the be to assure the quality of the assessment program for the state. One where parents could actually scrutinize the materials. It is a shame that the materials are not routinely released for public scrutiny and it takes away greatly from the credibility of the tests.

4. The emphasis on testing has robbed our children of meaningful educational opportunities. Field trips are limited to curriculum based only. Lessons in classes are limited to what is on the test. No enrichment. Our top students are forced to endure repeated practice drills to prepare for the test. The real effect of such a testing mentality is an overall dumbing down of the curriculum in schools.

It is time for the people of Georgia to take back their schools, end such extensive and unreliable tests, and promote strong values for learning. It is in this third item where we as a state have really fallen down on the job. We have blogged many times before about the values or lack thereof that affect student learning.

HS Teacher, Too

April 13th, 2009
10:12 am

“A multiple choice test, especially one developed under a low-bid contract, can only measure the lowest levels of learning – simple recall.”

Not true; multiple-choice questions CAN get beyond simple recall. That being said, good questions are very difficult to write and to validate, and both of those processes require time and money to do it properly.

Do I trust the CRCT? HECK NO! How can we place any confidence in a test rendered invalid by the way classroom teachers are mandated to drill and kill in preparation for the very test? It merely tests how much drill and kill the teacher did, and not what the students know or should know.

jim d

April 13th, 2009
10:17 am

Tony,

While the ggod people of Georgia have the pitch forks out perhpas they could take back not only the schools but their local, state and federal governments as well.

Any thoughts?

jim d

April 13th, 2009
10:18 am

Good (I really need to work on my stutter problem) LOL

Bogus question + lazy reporting = AJC

April 13th, 2009
10:43 am

Yet another example of a mail it it topic, when the AJC has utterly failed. Why is this even a topic, when the AJC has reported a huge, multi-system cheating scandal on summer school CRCT exams then, despite all the evidence, refuses to do another story about it?

The question should be, do you trust the AJC and the obvious answer, considering reports they are losing a million dollars a week, is a resounding no.

Northern Visitor

April 13th, 2009
10:57 am

You slack-jawed yokels still trying to understand standardized testing??? I’m dreading next week when I have to come to georgia on business!

ScienceTeacher671

April 13th, 2009
12:00 pm

Tony, IMO, the AJC *ought* to be writing stories addressing exactly the points you raise in your 9:58 a.m. post.

We were told several years back that the GHSGT science test had a significant reading comprehension component, because it is meant to be a “gateway test” — i.e., it’s more palatable for the general public to believe that students don’t understand science than it would be for a significant number to fail the ELA section of the test, and for the public to believe they can’t read.

Reality2

April 13th, 2009
12:07 pm

DB: I don’t understand why you just don’t teach what we expect students to know (GPS). Shouldn’t that “prepare” students for the CRCT? How can you treat “preparation” separately from “teaching”?

Tony: AS HS Teacher said, multiple choice questions CAN test things that go beyond simple recall. Also, as HS Teacher noted, it is expensive to create good multiple choice questions. And, perhaps that has something to do with the fact that the CRCT problems aren’t released publicly. Although I don’t necessarily disagree with the usefulness of releasing test questions publicly, I don’t see that contributing to the validity of the tests. Reliability and validity should NOT be determined what the public (or teachers) think. It is psychometric that can/should be established away from any emotions/beliefs/ideologies/etc.

Tony

April 13th, 2009
12:58 pm

Let me add a few more details about advanced questions, there development and use on multiple choice tests. While some questions may go beyond basic recall of facts, definitions, or simple answers to calculations, the use of multiple choice tests lock students in to a limited number of choices for the response. By trying to place levels of thinking into discreet chunks like the four choices give, students are still limited in their thinking. There is very little opportunity to develop items that require evaluation and synthesis of ideas. These kinds of responses require students to formulate ideas and present them. Essays or constructed response items are too expensive to grade. If critical thinking, problem solving, and higher level skills are desired, the use of multiple choice items still imposes severe limits.

Costs are definitely a factor for not releasing test items. The aspects of reliability and validity that are enhanced come because others are able to scrutinize the tests. This process would require higher quality tests. We are already subjected to emotions, beliefs, politics, and ideologies that have impact on our student achievement.

And, yes, jim d, I’m ready to reclaim local, state and federal governments. When do we meet?

Tony

April 13th, 2009
12:59 pm

oops – “there” on first line should have been “their”

Ernest

April 13th, 2009
1:00 pm

As I indicated in the 3/25 blog, Fewer tests to $ave money, I lack confidence in the CRCT. Several of the problems we’ve had with the administration and results magnify the concerns that many in the state have regarding this test. I believe an instrument like the ITBS can provide more insight with regards to measuring academic standing and growth.

GC Parent

April 13th, 2009
2:07 pm

Two reports worth your time:
Accountability Report: Thomas B. Fordham Institute & Education Watch: The Education Trust

Both discuss deeply the CRCT tests in Georgia versus National Norm Test Results – and both should provide a bit of a jolt of reality to parents in Georgia.

jim d

April 13th, 2009
2:10 pm

Tony,

I believe there is a meeting planned at the Gold Dome on this Wed. to do just that. I think i heard someone by the name of Hannity was going to be there?

Reality2

April 13th, 2009
2:16 pm

GC Parent,

We discussed this issue before, but nationally normed tests do not serve as a valid instrument to measure whether or not students are learning what the state standards are expecting them to learn. If multiple choice tests are limiting, the ITBS is a prime example of such a test. What we want to measure is not just generic “academic growth” (though that may be very valuable) but whether or not students are learning the specific ideas in the standards (thus making sure teachers are indeed teaching those ideas).

Any form of testing has its own limitation. Most people agree (I think) that no single test should be used to make a high-stake decision [although I have heard many people complain here about students being promoted even though their the CRCT scores weren't high enough]. I think schools put too much emphasis on the CRCT. Just teach what’s in the standards in the best possible ways. Then, if students aren’t learning, then modify what you are doing. Having “CRCT prep” time seems to be such a ridiculous idea.

VOICE

April 13th, 2009
2:36 pm

We need something to determine if “any learning” is taking place in our schools. The CRCT is not perfect, but it does provide a measure for some degree of ACCOUNTABILITY. I don’t like it, nor do I agree with it. But, all of you complainers are not suggesting anything to hold teachers and students ACCOUNTABLE. Until y’all can offer a better mouse trap, it’s the CRCT.

Northern Visitor, your point is well taken. We are SOOOOO BEHIND.

And, oh yeah, what’s this about reclaiming our governments??? Who took ‘em? :)

jim d

April 13th, 2009
3:39 pm

VOICE,

Here’s your new mouse trap.

How about along with some of this testing each state were to establish a statewide review board that would function independently of the state’s department of education.

The review board would send small teams of professionals to make periodic visits with each visit preceded by an internal self-study.

The review panel would then write a report on each school that, along with the school’s response, would be made public.

This would evaluate the school on a far broader set of outcomes than student test scores alone.

Whatta think?

jim d

April 13th, 2009
3:41 pm

Oh and Voice,

Lifetime Politicans have stolen our government.

jim d

April 13th, 2009
3:45 pm

Voice,

and if you really want accountability.

Provide school choice!! everything else is merely lip service.

ScienceTeacher671

April 13th, 2009
3:57 pm

jim d, sounds sort of like SACS.

jim d

April 13th, 2009
4:34 pm

Science teacher,

Sorta, but if SACS makes public disclosure or not of their findings I’m not sure and aren’t they on something like a 5 year cycle?

Public School Parent

April 13th, 2009
4:44 pm

I am a parent of two high achieving students. The CRCT is worthless. Giving it wastes valuable instructional time and I feel sorry for all the talented teachers who must spend the year “teaching” to the test rather than offering exciting and creative classroom content.

ALL parents want the ITBS or another nationally normed test to know how our students are doing in K-8. Then we look at the ACT, the PSAT and SAT in high school. We don’t care how our children rank within Georgia or how they compare to another school or county. Frankly, parents suspect that the CRCT and the other standardized tests are manipulated by some school systems and the DOE. Look at what Cox has already done with the Math I EOCT! The results are awful so she won’t count them or even report them. If the results were good, she would be on national TV talking them up.

I fully understand the NCLB requirements for measuring yearly progress but the state should dump the CRCT and use the ITBS.

Dr. Craig Spinks /Evans

April 13th, 2009
4:51 pm

ESSENTIAL QUESTION: Are the people who devise, administer, score and disseminate CRCT results trustworthy? By the way, Public School Parent’s last sentence posted immediately above merits widespread and thoughtful consideration at the highest levels of Peach State public policymakers.

Nikole

April 13th, 2009
4:52 pm

A test cannot be valid or reliable when a 2nd grade student passes the reading section, but they cannot read.

BB

April 13th, 2009
5:05 pm

I agree with Public School Parent. The ITBS is a much better way to measure student knowledge, since data is compared to students in the rest of the country.
Reality2- I did teach to the GPS in a standards-based classroom. GPS doesn’t cover everything on the CRCT. Questions are worded differently, not to mention the fact that teachers are supposed to teach more than multiple choice concepts. As far as test prep, it is a difficult task for some children to transfer from a test booklet to an answer sheet. Third graders are also required, for the first time in their school career, to read the entire test themselves, as opposed to having portions read to them. There is a lot of test prep that doesn’t have anything to do with GPS.

ScienceTeacher671

April 13th, 2009
5:40 pm

Public School Parent is exactly correct that the DOE manipulates the CRCT (as well as the EOCT and GHSGT) scores.

BB

April 13th, 2009
5:53 pm

Nikole, A second grader can pass the reading portion of the test because it is read to them! What is the sense of that?

catlady

April 13th, 2009
6:03 pm

My disdain for the CRCT is legendary (at least in my own mind.) It is neither reliable nor valid. It serves as a sop for some state people to make money devising and REvising the test each year. Like Reading First, it is a way to reward some politically connected folks.

As someone above said, it bases some scores on other factor than the test: for example, the math tests reading comprehension BEFORE math skills.

I also believe the DOT manipulates the scores, setting varying cut off scores (which are always low anyway). There is NO WAY kids who make 720 can, after 3 weeks of 2 1/2 hour a day instruction, make over 800. (Kids who cannot read/comprehend a lick).

The CRCT is a big moneymaker for some folks (think of all the test prep materials, too) and a joke among educators with any training in test construction. Think of what better use that money could be put toward!

On a similar subject, we are supposed to get the results back within a week or so so that we can reassign children for “summer school” the last 2 weeks of school (to continue the first 2 weeks fo summer vacation). Whose idiotic idea is that? And does ANYONE think the test results will be back??

The whole thing is a sorry, sad little joke on the taxpayers of Georgia.

Reality 2

April 13th, 2009
6:18 pm

BB,

Your statement, “GPS doesn’t cover everything on the CRCT” is absurd. The CRCT is supposed to test whether or not students learned what the state standards, i.e., GPS, expect.

As far as wording is different and those thins are concerned, well, perhaps in spite of what Tony claimed earlier, the CRCT is asking students to do more than just regurgitate memorized facts. If teachers can’t see the alignment of concepts regardless of wording, then there is a problem – either with the test or with the teachers’ content knowledge.

As for using a nationally normed test to compare students nationally, that is meaningless until we have a national standard. As long as each state has its own standards, comparing scores is useless since students are learning concepts at different times within their curricula. Perhaps the only time it does make sense is at the end of high school (such as the SAT and the ACT).

I certainly think whether or not to have national standards is definitely a discussion worth having. But, until then, using the ITBS or any other nationally normed tests will not compare our students against other students in a meaningful manner.

Esential question is essential

April 13th, 2009
6:31 pm

Dr. Craig Spinks essential question, on whether we can trust those in charge of the test, is the correct one. How can we trust them if a cheating scandal, involving multiple school systems, has been uncovered and Kathy Cox and her minions have done nothing to address it?

And while we are at it, how can we trust the AJC when they bury the story, after initially reporting it?

GC Parent

April 13th, 2009
6:59 pm

Reality 2 -
Did you go back and read the papers I suggested ? If a student only has to get 50% of the CRCT questions correct to meet expectations, then do you really think that a student is learning the state standards ? And do you think that the state is setting a false expectation of achievement when a student “meets expectations” under those guidelines ? And do we really think that the Georgia CRCT is a challenging test ? I am interested in your opinion, please go read the reports and let me know what you think.

Gwinnett Educator

April 13th, 2009
7:17 pm

In the Reading Comprehension sections for grades 1 and 2..the passages are read by the students and the answer choices are read by the teacher. I understand what Nikole is saying because many students do not attempt to read that passage on their own. Some of the questions involve vocabulary, grammar, etc.

When I taught 2nd grade in Dekalb County, all too often, I would have students glance at the passage and then back up at me (as directed by the test booklet instructions) and wait for me to read the questions/answer choices. Then they would begin the bubbling of their choices..usually making little designs and patterns.

catlady

April 13th, 2009
7:27 pm

Laura, Why didn’t my insightful, sucinct observation on the dismal failure that is the CRCT get put up on the blog?

What is the status of the investigation into those schools showing incredible gains and lack of comparison with the ITBS results?

And are you going to write a piece about the plan to have students begin “summer school” the last 2 weeks of school (talk about a total disruption during a disrupted time)? And how the state has said it will get the test results back in a week (yeah, right, we have been lied to before, year after year. Many years we have gotten the results back the day before school is out, and this was when we took the test over a month earlier than this year.)

luvs2teach

April 13th, 2009
7:43 pm

In answer to the question, “Do you trust the CRCT?” Why, no, I don’t. Between the controversies raised over question validity and cut scores the last couple years, and the fact that I have students that nearly exceed standards on the CRCT and yet are below the 50th %ile on the ITBS, no, I have trouble trusting the test – I’m glad my personal children rode the crest of this particular wave, and weren’t drowned by its undertow.

To the complaints about the DOE not releasing the tests – according to the DOE website, our access to the OAS (Online Assessment System) is supposed to replace the releasing of the tests. That’s all well and good, but I know I’ve seen questions for my subject, 8th grade physical science, that aren’t well-written (there have been improvements, but…).

To those who want to replace the CRCT with the ITBS, one thing that no one has explained is that they are not the same type of test. The CRCT is a criterion referenced test – the student’s performance is being compared to a a standard body of knowledge. A well-written criterion-referenced test can actually be very valuable in determining if a student has learned the curriculum. It can’t be used to rank students like the ITBS, nor can it be used for admission into gifted programs.

The ITBS, on the other hand, is a norm-referenced test – the students’ performance on a given test is compared to a norm-group’s performance on the same test – it’s not technically compared to how all students do on the test in any given year – it’s how all students nationwide do compared to the norm-group. The norm group is generally made of a variety of age and abilities. For example, a wide range of students will take the 8th grade science test, and then they will extrapolate a line to determine grade equivalencies and whatnot. When a student’s grade equivalent score is given on the 8th grade test – let’s say 10th year, 1st month, or 10.1, it doesn’t mean that the student should be in the 10th grade – it means that the student did as well on the test as a 10th grader would on the same 8th grade test! I think the ITBS is a valuable diagnostic tool – it shows where students rank in broad knowledge against nation-wide norms. It can be used for identifying gifted students.

I like to think of criterion-referenced and norm-referenced as being like X-rays and MRIs – they both show you a different picture of what’s going on. Until our state gets its act together on the CRCT and puts out a test that parents and teachers trust and students care enough about to take seriously, then the ITBS might be an ok choice, but people need to be prepared for low scores on areas like science (the ITBS is broad subject, instead of subject specific) and social studies (similar reason as science).

BlondeHoney

April 13th, 2009
9:47 pm

My experience is based on the FCAT (Florida Comprehensive Assessment Test) and it isn’t my experience but both my son’s; they both told me that a kid would have to be beyond remedial NOT to pass the FCAT. So if the CRCT is anything like the FCAT, and kids are not passing but graduating anyway, that does not bode well for the future.

Lee

April 14th, 2009
6:07 am

“Do you trust the CRCT?”

The reason for the CRCT and all these other high stakes tests (graduation tests, end of course tests, et al) is that we couldn’t TRUST the school systems to provide a simple, accurate assessment of a student’s academic accomplishments (or lack thereof).

Grade inflation, passing a student from grade to grade even though they could not do the work, graduating illiterates…. sound familiar?

It should – because schools are STILL doing it.

jim d

April 14th, 2009
7:04 am

Bottom line?

I’ve yet to see any empirical evidence that testing has any propitious effect on learning.

VOICE

April 14th, 2009
8:19 am

jim d, I think Science teacher is right. Your suggestion sounds like SACS, only with annual evals. Nothing new there, but I love your idea about school choice. In fact, I think we will eventually be forced to allow it state-wide.

And, I understand and agree with your view on life-time/career politicians. One problem, though. They didn’t “steal” our governments, we “gave” our governments to them, Republicans and Democrats.

Reality2

April 14th, 2009
8:54 am

GC Parent,

I said this before but what you raise (and I assume that’s what those papers you reference talk about – no I have not read them) is a different issue from whether or not a test is reliable or valid. The question of validity is about whether or not the score of 50% tells us that the student mastered about a half of what is expected (in the GPS). Whether to consider that as a enough to “pass” is an arbitrary and policy decision, nothing to do with the reliablity and validity of the test. The same issue remains no matter what test, e.g., ITBS, we use. The cut scores are policy decision.

Reality2

April 14th, 2009
8:56 am

jim d,

Who are being evaluated, the state DOE? individual school district? individual school? The cost will grow very rapidly as the number of entities that need to be visited and evaluated. Also, the entity who will be evaluated will have to produce a lot of “reports” for their self-studies – some more work for teachers and a few more [needless???] positions in the central office.

teach1

April 14th, 2009
9:45 am

A test is a test and hopefully something is learned from the results. But confidence in that the CRCT really tells us how our students are doing? NAH! W

Why are we testing 6 year olds for a solid 2 1/2 hours 3 days in a row, when they only have an attention span that lasts 15 minutes? How much do you think the test really tells us about what they know?

ScienceTeacher671

April 14th, 2009
11:12 am

Reality 2, I’m beginning to think that you work for the state DOE.

I agree with GC Parent – the CRCT could be the most reliable and valid test in the whole wide world, BUT –

If students “meet expectations” by answering approximately 50% of the questions, there is no indication of mastery of the GPS, and students, parents, and the general public get a false impression of the level of achievement in our schools — which means that, as given, the CRCT cannot be trusted.

ScienceTeacher671

April 14th, 2009
11:24 am

The science and math tests require quite a bit of reading. I talked to the state testing division about this, and they assured me that the reading on the science tests was “appropriate for the subject matter and grade level being tested.”

They couldn’t tell me anything more definite than that, and claimed that a Lexile couldn’t be determined on the science test because there weren’t enough long reading passages on the test to make that determination.

Here’s the problem – a 9th grade science textbook, which would generally be considered “appropriate for the grade level and subject matter,” is written at about a 1040 Lexile. A student who “meets expectations” on the 8th grade reading CRCT has a Lexile score of 805 – WHICH IS NOT HIGH ENOUGH TO READ AND COMPREHEND A NINTH GRADE TEXTBOOK!

Can that student read and comprehend well enough to pass a 9th grade EOCT? Who knows? The state DOE either does not or will not say.

VOICE

April 14th, 2009
11:33 am

Teach1, you got a point. ScienceTeacher671, so do you. But, what else do we have? Maybe I’m wrong, but I think we need “something” to determine if teachers are teaching and students are learning. And, that “something” should apply across the board, state-wide.

Jim d suggested a plan for accountability, which actually makes sense. It’s called CHOICE. However, it has certain shortcomings, too.

Who, then, can lead us on our quest for the “perfect mousetrap”? We should probably just try to improve the one that we have already.