Real math story: Students passing End of Course Tests rose from 2008 to 2010

Here is a detailed analysis of the state math performance by a former Georgian and one of the writers of the Georgia Performance Standards in Mathematics. Brad Findell now works at the Ohio Department of Education leading that state’s mathematics initiatives, and also serves as president of the Association of State Supervisors of Mathematics.

For those of you interested in this, please check out the PDF of his Excel file, which I have placed in Google docs and which you should be able to access. (Let me know if you can’t)

While reading a recent AJC article, I was struck by the fact that the statement about 80,000 failures in the Spring 2010 EOCTs was made without any context.  For comparison, there were about 71,000 failures in the Spring 2008 EOCTs in Algebra and Geometry.  But the real crime is that the AJC did not mention that the number of students passing the EOCTs rose from about 99,000 in 2008 to about 116,000 in 2010.  With a little comparison, it is hard to call the Spring 2010 results a crisis.

By digging a little deeper, the data reveal a more interesting story:  All of the increased failure was in Math II as compared to Geometry, which is not surprising, given that Math II was in its first year of implementation.  About 40,000 students failed each of Math I in 2010 and Algebra in 2008.  But about 73,000 students passed Math I in 2010, as compared to 49,000 passing Algebra in 2008.  This is a tremendous increase that deserves to be celebrated.

With the idea that a graphical representation can tell this story more compellingly, and using annual report card data (rather than Spring only), I have prepared the attached Excel file, which shows the statewide data graphically, along with data for a number of districts.  I chose districts that had stories to tell along with a number of districts in the Atlanta metro area as well as Richmond County.

The graphical display for the statewide results shows that Math I is a huge success and that the 2009-10 Math II results are slightly worse than, but not qualitatively different from, the Geometry results from 2007-08.

At the level of districts, you will see that Atlanta, Cobb, DeKalb, Fulton, Gwinnett, and Richmond have seen significant improvement in Math I as compared to Algebra, but they are struggling a bit at the level of Math II, partly because of lower enrollment.  (I am interested in why the enrollment is lower.)  In Cobb and Gwinnett the losses are only slight.

Brooks, Carrollton City, Forsyth, Peach, Ware, and White counties have seen success in both Math I and Math II, and the results in Forsyth and White counties are astounding!

What would be the best way to tell these stories?  Among the counties I have chosen, which stories most need to be told?

–From Maureen Downey, for the AJC Get Schooled blog

132 comments Add your comment

Ernest

March 2nd, 2011
5:04 pm

This kind of analysis in interesting. Assuming that Algebra maps to Math 1 and Geometry maps to Math 2, the numbers of Math 1 are disappointing because there was a larger pool of students Z(about 20-25% more) that took that class, resulting in higher number of failures. Math 2 numbers were interesting as there was a smaller pool of students taking the class (about 5% fewer) yet a sharp increase in the passing rate. I would ask about the factors that caused/influenced the decrease before an accurate assessment could be made.

Dr. Craig Spinks/ Augusta

March 2nd, 2011
5:40 pm

Statewide failure rates of 36% and 46% for Math I and Math II, respectively, are problematic.

Hopefully, Math I and II passing rates will increase, be sustained, and reflect improvements in Algebra and Geometry skill-levels.

ima

March 2nd, 2011
6:14 pm

Students scoring pass plus has dropped significantly overall, even in counties where the numbers failing didn’t change much between the two years.

Teach2Learn

March 2nd, 2011
6:19 pm

Which data to use – % or number of students? When looking at number of students, the increase in passing from Algebra 1 @ 49,000 in 2008 to Math 1 (integrates Alg, Geo, Stats) @ 73,000 is impressive. Change as a %? Not so much. Was Technical Math (sic) still an option or had it been completely phased out? Can this explain the much lower number of passing students because so many less took the EOCT for Alg 1 in 2008?

Glad our students are studying statistics in each Integrated Math course. It’s never too early for our youth to understand the manipulation of data and how it can be used by business people, politicians, etc. to “make a point”.

Scott Allen

March 2nd, 2011
6:25 pm

It’s hard to make any meaningful comparisons here. At the risk of oversimplifying, Math 1 is a combination of Algebra 1 and Geometry. Math 2 is a combination of Algebra 2 and Geometry. The more direct similarity between the tests is that most students now take Math 1 and then Math 2, whereas on the old curriculum they took Algebra, then Geometry. But the mapping of content is somewhat convoluted. Math 2, with 2 Geometry units and 3 Algebra units, is more like Algebra II than Geometry. However, Algebra II did not have an EOTC.

Making it even harder to compare the results, every EOCT has its own set of cut scores that determine what students pass and which fail. In other words, every EOCT is curved, and every subject is curved a little differently. So comparing scores across tests is somewhat like comparing apples to oranges.

However, Mr. Findell does bring up a valid point that brings joy to any stats teacher’s heart. You should never compare “total number of failures” on any tests without comparing the “total number of test takers.” When the number of test takers differs, to make any kind of fair comparison you must instead look at “failure percentage.”

After all, to use an Atlanta Braves’ baseball example, I am willing to forgive Martin Prado for failing to reach base a team-leading 375 times last year, since his on base percentage of .350 ranked 7th on the team (among those with more than 1 at-bat). In fact, he was one of the best hitters on the team. With more opportunities (a team-leading 599 at-bats) naturally come more failures, even at a commendable success rate.

another comment

March 2nd, 2011
6:39 pm

Here is the bigger problem, with this Math 123, the students are not prepared for the PSAT in October of 10th grade. This Math is not sequenced to what the rest of the country is learning. It is being taught out of order and with little depth. Who really cares about the Georgia EOCT’s. Just like the CRCT’s they are meaningless. They have been devalued by this state that sets the pass rate so low.

My daughter has been public, private and now unfortunately due to finances back to public for High School. She has always tested in the 92% to 96% on the Math portion of the IOWA tests. Has always been an A student in Math. Took the SSAT ( Private School Admission test) which predicted she would continue to score in the top 90+ percent of the Nation on Math.

Unfortunately, we don’t have $16,000 to $20,000 extra a year for her to go to a private high school. So we had to go to our local public School and Math 123. Although my daughter has been one of the lucky ones in her class and eeked out a 92 and 94 in Math 1 and 2 on the EOTC and ended up with B’s both years in the class. The Majority or her classmates who are the students taking AP, IB and Honors classes are ending up with much lower grades. The real problem is she ended up only scoring in the 50 percentile on the PSAT in Math. I asked her what gives, did she not take it seriously. She told me no, she tried, but they had not covered the material on the PSAT Math sections in October. This is very troubling.

My daughter is lucky, that she has an educated parent that gives a crap. I will now hire the SAT tutor that my friends who have their kids at the Private schools are using this summer. I have to make sure that she is covering what is on the SAT.

We need to stop screwing the top students and the above average students in this state. The ones at the bottom are the ones we keep teaching to and bringing everyone else down. We need to seperate out the students early on.

ScienceTeacher671

March 2nd, 2011
6:40 pm

The decrease in the percentage attaining “Pass Plus” is a bit worrisome. Quite a few more students taking Math I than Algebra I. Where’d we get them all?

Scott Allen makes a good point that cut scores differ from course to course, and even from year to year. I’m pretty sure that for Math I they only have to get 40-45% of the questions correct to pass.

justin

March 2nd, 2011
6:44 pm

@ another comment,

So, do you have the PSAT score data? Or is it just your opinion? This guy is using data – at least respond in kind.

teacher&mom

March 2nd, 2011
7:11 pm

@Scott Allen – Don’t forget that Math 1 and 2 also cover statistics. I don’t teach math so I proctored our Math 1 and Math 2 EOCTs. I was struck by the number of geometry and statistics questions and the lack of algebra problems….especially on the Math 1 test.

MathTeacher

March 2nd, 2011
7:14 pm

@ScienceTeacher671 – the Algebra EOCT cut off score was also very low. I’m not exactly sure of the cut score for either exam, but both were around a 50%

ima

March 2nd, 2011
7:30 pm

@ScienceTeacher671 – The decrease in the percentage attaining “Pass Plus” is a bit worrisome. Quite a few more students taking Math I than Algebra I. Where’d we get them all?

In a nutshell… repeaters.

ScienceTeacher671

March 2nd, 2011
7:39 pm

MathTeacher, they may both be the same, I don’t know.

This year, although the percentage of Physical Science students passing the EOCT dropped, the percentage making Pass Plus increased a notch. The state also seems to have lowered the percentage required to get Pass Plus slightly, down to 60% of the answers correct, but I’m sure it’s just a coincidence.

ScienceTeacher671

March 2nd, 2011
7:40 pm

ima, not surprised. I’m not sure some of ours will ever get out. When you still have to use a calculator for basic single digit multiplication and division…

Scott Allen

March 2nd, 2011
7:44 pm

@teacher&mom – yes, in my vast oversimplification I mistakenly left out the stats factor. For some reason I fail to understand, each EOCT is 33% stats… which works well in Math 2 (2 units out of 6 are stats units) but is silly in Math 1 (1 unit out of 6 is stats).

I do appreciate the mainstreaming of statistics into high school math classes. But, this makes it even less appropriate to compare the EOCT scores.

One last observation… the new curriculum is more conceptual and less skill-driven than the traditional Algebra & Geometry. For good or ill.

Dekalbite

March 2nd, 2011
7:50 pm

Scott Allen is correct. If your number of students tested varies significantly, you have to go with percentage tested. If I have 100,000 tested and 60,000 pass, that is very different from 50,000 tested and 45,000 passed. Although 60,000 pass looks more impressive, 40% did not pass. On the other hand, 45,000 out of 50,000 pass rate means that only 10% did not pass.

It’s very difficult to compare groups of test takers – e.g. the 9th graders – when total number of students tested varies a lot. It begs the question of why the variance is there in total number of students tested. I saw that DeKalb had a wide variance of students tested. I agree with Ernest that more information is required before any conclusion is reached.

S

March 2nd, 2011
7:51 pm

I am 26 years out of high school and am a succesful person with a college degree, good job, nice house, nice things, yada yada yada. I always got F’s in math in high school. In fact, i did poorly grade wise in most subjects and even worse on tests. ACT…forget about it….SAT…..forget about it even more. Bad grades and bad test scores, bad in math? Who cares? It did nothing to inhibit my success in life. Take that all you education freaks.

JRHD

March 2nd, 2011
8:14 pm

@ Maureen – you ask why enrollment rates are lower in the Metro area. Enrollments are lower because people have lost their jobs and moved back home. This was shared w/ me earlier this year by my child’s teacher. She told me our elementary school had lost over 400 students, mostly Hispanic, because of the economy. I live in Gwinnett County, but since the Metro area has long relied so heavily on the construction industry for jobs I would guess that trend holds across the region. This is not the hard and fast answer across the board, but it is definitely a contributing factor to the sudden decrease.

Dekalbite@S

March 2nd, 2011
8:19 pm

That’s really more of a commentary on U.S. cultural values. We don’t value math and science expertise like other countries. Maybe that’s why we have such a high unemployment rate compared to China and Germany for example. I’m glad you’re doing so well. I didn’t perform well in math in high school or college, but when I got a job that valued math (sales) in terms of my paycheck I quickly became very proficient that subject. My improved skills in math proved exceptionally lucrative. Money makes the world go round, and money is nothing but math.

HStchr

March 2nd, 2011
8:41 pm

I’m chuckling. Reading this reminds me why I haven’t started my PhD yet…I cannot stand statistics, and have found them to be laborious and subject to WAY too much interpretation. I think by the time we include all the possible variables for the aforementioned situation, we’d all have headaches and a bevvy of opinions for what it all means!

The issue I have with all of this is that “passing” scores are moved up and down and are still set too low very often. As more research reveals how irrelevant standardized test scores are in determining if a child has learned, as more tests are proven invalid or unreliable, why can’t we seem to get past them and find something that really works? It’s easier, I guess, to keep using the bad tool than it is to design a new one. Ask any teacher, and he/she will tell you that test scores often conflict with every indicator of progress the teacher/school/system uses besides the score. I’ll let this go as I’m too tired to climb up on my soapbox tonight, so let my objection to the whole standardized test score mess stand as it is…. you data interpreters have fun!!! :-)

Atlanta mom

March 2nd, 2011
8:44 pm

@another comment
Who cares what the scores on the 10th grade PSAT are? Especially if the matter will be taught later in the school year. The 10th grade PSAT scores mean nothing.

Dekalbite@Maureen

March 2nd, 2011
9:29 pm

If you want a really interesting post, ask the Georgia DOE how they set the “cut” scores in Georgia. This is why we need national testing that allows Georgia to be compared with every other state. That of course will drive a national curriculum. That’s not so bad. Almost all 4th graders in every state are taught factoring, addition, subtraction, multiplication and division of fractions, double digit division with a remainder, double digit multiplication with a remainder, etc. If not, they should be taught these skills.

AJinCobb

March 2nd, 2011
9:54 pm

@another comment,

It’s clear you’re very upset about your daughter’s school and her declining success in math. Please realize, however, that your daughter is just one student. As the analysis discussed in this thread makes clear, plenty of students have actually been succeeding at math, under the new curriculum.

HS Math Teacher

March 2nd, 2011
10:11 pm

When scores and pass rates were as dismal as they were then, “improvement” hardly seems a real feat, but rather an expected outcome.

I’m not against the integrated math, just the requirement that all kids have to take 4 years of college prep math. For the folks who think differently, I don’t think they’ve spent enough time teaching lower level classes in an inner-city school, or a poor, rural system. I don’t think they truly know how hard it is to push that boxcar up the tracks.

Maybe one day we’ll get to that utopia where all kids will be prepared for high school, and all will be able to learn high level math; however, we’re not there yet – not by a long shot.

hello.life

March 2nd, 2011
10:47 pm

@another comment

Yes, the new math jumps around a bit but it still covers the basics of the math found in the PSAT. So there is a good chance that your daughter might not have learned the material however its also likely she has learned it but just isnt aware on how to apply it properly. The college board is out to trick your student. They ask questions in ways that make the question seem more difficult then it should be. The math found in the PSAT doesn’t exceed a certain level of algebra. There is no pre-calc or calculus just some algebra and geometry. If schools tried to teach directly for the SAT or PSAT students would learn the same stuff over and over because these tests just cover the same material over and over just worded differently. A tutor specifically for these tests, like you have, is best I believe in order for preparing. Well, I wish your daughter luck.

southern

March 2nd, 2011
11:10 pm

I support the change back to the traditional Algebra i, Geometry, and Algebra II.

hello.life

March 2nd, 2011
11:23 pm

@S
Haha. I’m glad everything has worked out for you. :) But that attitude can’t fit everyone, especially those who want to enter top prestigious colleges, or job fields that require a math background (engineering…). Am I an education freak? I guess you could say that. :P

john konop

March 3rd, 2011
6:29 am

This comparison would get an F in a basic research methods class in college. As many pointed out you cannot compare unlike variables like this and get any conclusion. You have different classes and different end of the year test, it is scary for people who understand statistics that any conclusion about this data could even had been taken seriously. If this is the best and brightest studying this issue GOD help us.

A real comparison would be testing the student results by year verse an SAT and or PSAT. And even than you would have to put in a slight variance because even though the SAT and PSAT are the same type of test it is not the exact same test taken years ago. Not to be tough but seriously ask anyone who has taken research methods and statistics should know the above concepts are fairly basic.

2 cents

March 3rd, 2011
6:48 am

sorry just skimming the blog;

dont think anyone has talked about the change in the CUT score. remember it was lowered so more could/would pass.

Twain……..”lies, dam lies, and statistics”

Dr. Barge has already set out to change the mess that K. Cox and Findell have created. Looks like Findell is trying to bail out a sinking boat.

If we really wanted an intergrated math; math with pyshics should have been done; instead we got this “obscure” mess going on. okay folks when is the last time you have had to prove a function being even or odd in a business setting; found the orthocenter: OR have had to measure, cut, and then make sure the item was within tolerance to be able to use???????????

truth hurts

March 3rd, 2011
7:48 am

Maureen,

Don’t fall for this line of bull&^*( about EOCT scores. The state cooks the books by scaling and massaging the data. What about a true revealing of the number of questions answered correctly out of the total number.

The results would be shocking.

Adults who condone this by ed officials are complicit to fraud against children.

Pluto

March 3rd, 2011
7:50 am

@ teacher&mom I could be mistaken but I have been under the impression that eoct proctors were not supposed to look at the questions; unless of course you work in the APS?

Maureen Downey

March 3rd, 2011
7:51 am

@truth, If the state EOCT scores are “massaged,” then we are in trouble as I don’t think the scores are very good statewide.
Maureen

justin

March 3rd, 2011
7:51 am

People on both sides, john konop and brad findell, use statistics to support their positions. Neither one is a saint, and they just “lie” with statistics. However, it is still better to have an argument based on data.

@ 2 cents,

Although math is used in sciences, at the fundamental level, the ways of reasoning in math and science are very different. Mathematics is deductive while science is much more inductive. Neither way of reasoning is better than the other, but we must pay attention to the fundamental difference if we want to “integrate” math and physics (or any other science).

Furthermore, mathematics IS integrated – although there are branches of mathematics like analysis, algebra, topology, etc., they do interact closely and you can’t be an expert in one branch without having some foundation knowledge of others.

2 cents

March 3rd, 2011
9:20 am

@justin

what is physics?

Jackie T.

March 3rd, 2011
9:33 am

@ 2 cents,

I’m curious to know what your answer to your own question is. Also, what would you say mathematics is?

Vickie@UGA

March 3rd, 2011
9:48 am

How did the Cut scores move during that same period of time????

NG

March 3rd, 2011
9:51 am

You really can’t make a statistical comparison from one to the other since they are different courses entirely. This is just another attempt by an educrat to justify the program. I agree with the poster above that we should compare SAT scores, since all of the content will have been covered by both those in integrated math and traditional math in other states by the time most students take the test.

Nice try to make this whole math fiasco okay, but I’m not buying it. The answer wasn’t to change the curriculum to integrated, but to find better teachers and textbooks to teach the traditional math subjects. A friend of mine recently observed an honors freshman science class, and was horrified that the activity they were doing was something suited to upper elementary school or possibly middle school. If they can’t challenge honors students, then what makes us think that the average kids are getting anywhere near what they need to succeed in life?

Random Thought

March 3rd, 2011
10:09 am

@AtlantaMom

another comment should care about her child PSAT scores are because those scores are what is used to determine the National Merit Scholars

Random Thought

March 3rd, 2011
10:17 am

@AtlantaMom

Another comment should care about her child PSAT scores because those scores are used to determine the National Merit Scholars

Zeno999

March 3rd, 2011
10:26 am

There were some comments above about the integrated math curriculum not preparing students for the PSAT. In actuality, last year’s PSAT average in math for georgia (which was taken by students who have had Math I and II) was the highest it has been in the last ten years!

Another view

March 3rd, 2011
10:47 am

It does not bode well that this fellow is a mathematician, much less the head of the math association. Granted, I have only a lowly statistics Ph.D., but the data as presented in the charts only demonstrate further that the “rollout” of the new curriculum was abysmally poorly done, and seem to suggest that huge numbers of students had “disappeared” (i.e., dropped out?) from ‘08 to ‘10, perhaps in part due to ‘failure’ on the math tests. Barge is likely correct in his attempt to save the state from itself. While Findell’s “new math” WHEN TAUGHT PROPERLY might be acceptable and even desirable for kids heading for UGA, Tech and Princeton (etc.), the concern for ‘normal’ average kids and particularly special needs students remains evident. One of the things I’ve noticed over the years is that many individuals who leave teaching for state departments either do so because of their lack of concern for students OR, perhaps, in some cases, they lose it along the way, so that they actually feel proud of dismal results such as these – “Doesn’t matter how many kids we ruin as long as the ones we want to succeed do so.”

Jackie T.

March 3rd, 2011
11:08 am

@ NG,

Yet those who are critical about the new program often argue that the huge failure rates as an indicator – without really specifying what the failure rates are and how they compare with what we used to have before the new program.

erasures

March 3rd, 2011
11:08 am

“At the level of districts, you will see that Atlanta, Cobb, DeKalb, Fulton, Gwinnett, and Richmond have seen significant improvement…”

How many people actually believe that? In the aftermath of the cheating in APS and elsewhere these kinds of test score improvements will, alas, always be suspect.

john konop

March 3rd, 2011
11:17 am

Justin,

…..People on both sides, john konop and brad findell, use statistics to support their positions. Neither one is a saint, and they just “lie” with statistics. However, it is still better to have an argument based on data….

This once again demonstrates you have no clue what you are talking about when it comes to math. I challenge you to stop slandering me and actually show how I misused any statistics. And anyone who knows anything about statistics and research methods would give Findell an F for his work. This is not a subjective comment it is based on the discipline Mr. Findell should of know in doing a study.

Old Time Educator

March 3rd, 2011
11:56 am

As an educator, I don’t believe that earning a 40% on a test should equate to a 70% or passing. Last year’s scores did just that. Students who scored right at the 40% mark were bumped up to 70%. If the state has to cut the scores to that extreme to get the kids to pass, then something has to change.

OTE

English Teacher

March 3rd, 2011
2:10 pm

“Mr. Findell should of know in doing a study.”

Sigh….

Jackie T.

March 3rd, 2011
2:11 pm

@ Zeno999,

Do you have the actual data for your comment?

Dr. Craig Spinks/ Augusta

March 3rd, 2011
3:37 pm

When many Wall Street financiers wanted favorable ratings for their toxic securities, they found ratings agencies which understood that future business with these financiers was contingent upon the agencies’ providing positive ratings.

Might testing agencies experience pressure similar to that felt by bond rating agencies? Might testing agencies expect future business with any given state DOE to be contingent upon that state’s students performing at or above “standards” on the agencies’ measures?

Two appropriate sayings?

“Tell ‘em (the educrats and the Public) what they want to hear.”

“What they (the Public) don’t know won’t hurt us (the educrats and the test vendors).”

john konop

March 3rd, 2011
3:42 pm

ET,

sorry

known not know but hey it is about math!

truth hurts

March 3rd, 2011
4:04 pm

Maureen and Old time educator,

Why does the media allow the educators to cover up the real performance by miraculously “scaling” the EOCT scores?

The AJC should do the children a service and expose the fraud and abuse occurring by adjusting the CRCT and EOCT numbers to a higher figure. The results are then paraded around and the schools say see how good we are doing and see how smart the students are now.

The whole thing smells of rotten garbage. Maureen and AJC, please demand from the state doe, a revealing of the actual answers correct out of the total number of questions. Then we can have a conversation with some intelligence on this topic.

Or please at least educate us as to why we should just continue to accept the fraudulent scores from adults. I feel the actual results statewide would bury the APS scandal for good.

SR

March 3rd, 2011
4:29 pm

I am a high school AIG (Accelerating Integrated Geometry) teacher. I’m shocked every year at the scores on the EOTC. Students can answer more than half of the questions incorrectly and still “pass”, thanks to huge curves. It’s a completely screwed up system.