DOE responds to NAEP/CRCT score gap

Here is what DOE spokesman Matt Cardoza wrote in response to my concern over the gap in proficiency between the Georgia CRCT and the federal NAEP:

The first thing to point out is that this study is based on 2007 information, not our most recent assessments.  The math assessments of this study were based on QCC, not GPS. We should not forget that this study doesn’t say anything to dispute our progress on the NAEP itself
(2007 & 2009).

This past year, Georgia was one of only 15 states to show “significant”
progress in 8th grade math.  So, regardless of the comparison of our state assessment and the mapping they do, we are making tremendous progress when compared to other students across the country.

Also, there are a few other things about this study to note:

-  It does not account for differences in what each state assessment measures and what NAEP measures, nor does the study account for differences in the definitions of proficiency.
- Georgia students performed as well on NAEP as students in other states with “higher” standards on their state assessments.  In 2007, Georgia students in grade 4 scored one point below the national average.  In grade 8, Georgia students scored two points below the national average.

Certainly NAEP is an important indicator to gauge the success of our student achievement.  This mapping study is one piece of information but the results on NAEP are the most important indicator. If you look at the
2007 and the most recent (2009) NAEP results you see that Georgia is making steady gains and preparing students well to compete against other students from across the country.  We continue to see increased student achievement on both the NAEP and CRCT, including decreases in the achievement gap.

The study simply compares percentages of students on state assessments and NAEP.  The study does not take into account the differences between state assessments.  If NAEP is the better apples-to-apples comparison of how our students are prepared in comparison to students from other states, then the NAEP results show Georgia is definitely improving (1 of 15 states to show significant improvement).

While the Reading assessment “mapped” lower, it is important to remember that the GPS curriculum is much more rigorous so the material (assessment) is more rigorous.

You may recall the 8th grade math CRCT results in 2008?  The percentage of students “passing” went down dramatically compared to 2007.  That’s because the curriculum and the assessment (GPS) were more difficult.

20 comments Add your comment

Dubious@best

October 29th, 2009
6:05 pm

Saying the state shows “significant improvement” is not necessarily saying much. Give us facts on where we are, not only that we are doing better. If we are at 20% proficiency in a subject with a national average of 70%, but we suddenly show a huge 50% improvement, we are still at 30%! So?
As with most of the education administrators today, local and state, it’s all about CYA and what can I do today to justify my existence.
Teachers are being burnt out with being over-administrated, and students are caught in middle.

Lee

October 29th, 2009
7:42 pm

Bottom line, even with reams of data, the best the Ga DOE can do is say “Trust us. We’re doing a good job. Reaaaallllyyy.”

ScienceTeacher671

October 29th, 2009
7:55 pm

…and after Matt released that statement, he had to sit down, because he was dizzy from all the spinning.

Old School

October 29th, 2009
8:10 pm

ScienceTeacher 671, that has got to be the most accurate (and hilarious) statement any blogger ever made on this site! Good one!

Dr. Craig Spinks /Augusta

October 30th, 2009
3:16 am

ScienceTeacher671, if you think Matt got dizzy while “spinning” NAEP/CRCT state-level disparities, wait until he’s challenged to explain the disparities between CRCT/ITBS results earned by Georgia’s local systems and individual schools. Of course, Matt won’t have to worry about “spinning” state-level comparisons between GA ITBS scores and national norms because the GDOE doesn’t collect and report such comparisons. Does anyone wonder why?

ScienceTeacher671

October 30th, 2009
6:43 am

@Old School, thanks! @Dr. Craig Spinks, “ain’t gonna happen”! Comparing the scores of students who barely pass the 8th grade CRCT – in both reading and math – with the grade equivalents on their ITBS scores shows that those students are working between a 4th and 5th grade level. And that is with the new math curriculum….

I don’t know how they would possibly spin it, but I don’t think the state wants to broadcast that.

catlady

October 30th, 2009
7:13 am

You are right, Dr. Spinks. However, what the DOE always says is, “Test X doesn’t measure what we teach in Georgia.” My question is, “Why don’t we teach basic grammar, math, or reading skills? Exactly how is/should Georgia’s curriculum (other than Georgia history and (cough, cough) biology different from the rest of the country’s, and why?” If our curriculum is not similar, why are we so out of step?

Old School

October 30th, 2009
10:10 am

AMEN, catlady! Why DON’T we teach basic grammar, math, or reading skills? Why not back off the technology in elementary schools and make sure we build a strong foundation in reading, writing, and arithmetic (with a helping of music, art, and physical education & free play). It will provide the middle grades with a rock solid base upon which to build and enhance those skills. Most of our students would then enter high school truly prepared. Pull out those students who aren’t keeping up and put them in basic classes without all the frills. At least they won’t keep the others mired in a slough of underachievement.

As long as the powers that be insist on fixing education from the top down, we’ll continue to reroof a dilapidated house built on a crumbling foundation. (pardon any spelling errors!)

Micheal Dib

October 30th, 2009
1:16 pm

I wish the state would really examine the facts. In Fulton County the parents have been analyzing the data on ITBS and publishing for all to see. You see as parents we actually care about education and we look at the hard facts. Since 2004 Fulton County Schools has seen a year over year drop in 8th grade ITBS scores. In fact the drop has been 18% over this time frame for Math. Since 2005 we have seen a 40% drop in the number of students passing Algebra and Geometry end of course tests. We have seen no increase in SAT scores since 2004 — they have been flat. Last year many of the 9th graders took the PSAT but the state has refused to release the scores. Based on anecdotal information the new GPS may be more rigorous for Ms. Cox and here team but for our kids it has been a failure. The PSAT scores were abysmal. How much longer does the state have to wait to end the GPS program and as the writers have said get us back to the basics os that our children can succeed. At a recent forum held at Georgia Tech one of the major threats outlined to the university was the fact that in the future Georgia students will not have the academic background and knowledge to attend. That is a sad comment for the state. Simply said Tech will have to look for students else where as Georgia students just will not make the cut. So much for preparing our kids to compete in a global economy.

Singing to the Choir

October 30th, 2009
3:36 pm

And this is where SBG comes in, lets make it look as if the kids are really learning. Be sure to check out the article in the Wall Street Journal, http://online.wsj.com/article/SB125682837972516125.html

http://www.edexcellence.net/flypaper/index.php/2009/10/illusion-of-proficiency-continues-to-shed-its-veil/

ScienceTeacher671

October 30th, 2009
6:54 pm

Maureen, looks like you have some new questions to ask Matt….

Maureen Downey

October 30th, 2009
7:05 pm

ScienceTeacher, I have asked Matt about ITBS scores and he did send me a response, but I don’t fully understand it so I need to talk to him on Monday. I think he took your great “spinning” comment and some of the others made here with good humor as he still seems willing to respond.
Maureen

ScienceTeacher671

October 30th, 2009
9:46 pm

Matt isn’t responsible for the policies; he just has to put the “best face” possible on those policies…a difficult and thankless job, I’m sure.

Micheal Dib

October 31st, 2009
2:53 am

Dear Ms. Downey, When the state provides the data please ask for the data since 2004. We have found that the school system tends to compare single year data so that they can show minimal increases as a sign of success. We prefer to look at multi-year trends and we do this not only by district or county but we do this by school. Also ask that instead of providing the data in hard copy or pdf that they provide it in Excel or even better give you the database. This will allow you and or anyone else the ability to do an analysis. Unfortunately for parents we are only give the data in hard copy so that we have to then enter the data by grade and by school for all of the subjects to do the analysis, This is labor intensive but it does allow us to get the facts.

A fact in Fulton County is that on the ITBS scores while almost all schools have dropped in every area the decrease is not uniform. Some have dropped more than others and only 1 school, Fulton Science Academy (FSA), has shown year-over-year increases in all subjects. This school is now the top performer in Fulton County and since 2004 their ITBS scores in 8th grade math have increased 20%. It is also a fact that our students ITBS scores drop in Math between 5th grade and 8th grade. This unexplained drop is over 15 points. So our children go from being in the 80th percentile nationally into the 60’s.or lower.

Our schools and the DOE need to face facts. They are failing our kids at a time when knowledge is most needed to compete nationally and internationally. They are failing all of our kids and in Fulton County that means 90,000 children. I mean all of our kids. Not everyone needs or wants to go to college. But every child needs to be able to read, write, and do basic math. These are basic life skills our children must have to succeed at any job and in life. We need to move beyond spin and wishful thinking, face the facts, and do what is right by our children and that is provide a good solid education for all.

Carla

October 31st, 2009
10:39 am

If scores are droppng, we need to drop the CRCT. Nationally normed tests are the best indicators of student success. Remember that Georgia is not ranked highly when compared to the rest of the states. O

ScienceTeacher671

October 31st, 2009
10:55 am

Michael Dib has some great suggestions for questions.

I’m with catlady & Carla – the ITBS is the Iowa Test of BASIC SKILLS – which basic skills are we not teaching in Georgia? Why not?

And if a child passes the CRCT, shouldn’t parents be assured that their child has mastered at least the BASIC skills appropriate for his/her grade level? Anything else is deceitful, IMO.

CLS

November 2nd, 2009
10:15 am

To Michael Dib — can you tell me which GA Tech forum that was said at? I’m a graduate of Tech and completely horrified that GT’s CEISMC (Center for Education Integrating Science, Mathematics, and Computing) is involved in this math debacle.

ScienceTeacher671

November 2nd, 2009
8:01 pm

@*#$&# blog monster.

Maureen Downey

November 2nd, 2009
9:00 pm

ScienceTeacher, Not sure it was us this time as there is nothing by you in the filter.
Maureen

ScienceTeacher671

November 2nd, 2009
9:07 pm

Maureen, I was wondering if you had talked to Matt today? Maybe I asked on the wrong thread?