College Entrance Test Shown To Be Flawed

Education has always been considered an important factor in gauging a nation’s strength. Thomas Jefferson understood that an uneducated populace was doomed never to be free; and virtually every modern president proclaims “education” a top priority.

Determining how well a citizenry is in fact educated, however, always has been problematic. Standardized testing has long been employed as part of this evaluation.

However, in a rush to simplify the processes whereby colleges and universities judge the potential for success of their schools’ applicants, many educators rely on – and legislators in North Carolina recently mandated reliance on — the use of a standardized test now found to be significantly lacking in predictive capability.

The test in question is the ACT; which has been gaining in popularity among high school students as an alternative to the SAT.

A study released in May by the independent National Bureau of Economic Research (NBER), casts serious doubt on the ability of the ACT to predict whether or not high school graduates applying for college will actually succeed in successfully finishing their college career – notwithstanding that such predictability is the primary purpose for such testing. In fact, the ACT’s own publications highlight the test’s purpose as measuring “college readiness.”

The problem – as detailed in the NBER’s report, “Improving College Performance and Retention the Easy Way: Unpacking the ACT Exam” – lies in the fact that only two of the four tests comprising the ACT (English and math) “can effectively predict outcomes in college.” Neither the reading component, nor the science portion of the ACT, was found to have any value in predicting a student’s success in college. The ACT’s science test, for example, largely tests one’s ability to read and interpret graphs and charts; almost entirely ignoring actual science like physics, chemistry and biology.

If college admissions offices simply considered the scores received by applicants in each of the four component subjects – math, English, reading and science – the process would be far more likely to reveal whether those students were prepared for, and likely to succeed in, their post-secondary education experience.

However, for some reason, the vast majority of colleges that use the ACT consider only a “composite score in their admissions process.” This means that the students’ overall unpreparedness is effectively masked.

The result of this problem, as noted by the NBER, is that students taking the ACT may be matched to schools either too difficult or too easy for their capabilities. This in turn exacerbates the alarmingly high dropout rate we see in American colleges and universities. As found by the NBER researchers, more than one third of students who began post-secondary education, had not received a degree or were no longer enrolled in any institution of higher learning after six years.

Like most states, Georgia includes as an “important” component of its criteria for admission to the University system, an applicant’s standardized test scores; but does not mandate specifically either the SAT or the ACT. (Unlike the ACT, the SAT provides colleges with individual scores from its three component tests, reading, mathematics and writing.) Georgia does require a minimum score on the SAT in order for students to receive a full HOPE scholarship.

Thanks to a law recently enacted in North Carolina, on the other hand, all high school juniors in the Tar Heel State, will be forced to take the ACT. Unfortunately, this law was passed shortly before the NBER study revealed the serious shortcomings in the ACT.

Hopefully, Georgia or any other states considering a move similar to North Carolina’s will require legislators and governors to first read the NBER study. This homework hopefully will short-circuit any such legislative move, and also force colleges and universities to reevaluate the manner in which the ACT is used in the first place.

by Bob Barr — The Barr code

43 comments Add your comment

nancymurrell

July 4th, 2011
5:28 am

Thanks to a law recently enacted in North Carolina, on the other hand, all high school juniors in the Tar Heel State, will be forced to take the ACT. At Georgia State University, undergraduate tuition for this fall will be $3,641 per semester, a $106 increase from the previous fall, it is really high and that is the reason people like universities on the “High Speed Universities” list

Eric

July 4th, 2011
7:06 am

I’m glad to know about the shortcomings of the ACT. I think standardized tests are overrated in general. Isn’t it curious how the bar on the SAT (passing levels) continues to be stealthily raised over time by educational pundits. What was a good score on the SAT in 1980 (e.g., 1000), would be insufficient today. So if students are scoring much higher to enter college, I’d say our society is better educated today than in the past. Yet, people keep bashing the quality of public schools. I’d say our education system is doing great!

Beck

July 4th, 2011
7:48 am

The ACT measures how much students have learned in high school vs. other American high school students. It has never marketed itself as comparable to the SAT.

Let’s remember that both of these tests come from a for-profit company, not an educational organization and keep that in perspective.

Carlosgvv

July 4th, 2011
8:14 am

For politically correct resons, politicians have convinced the public that college is a right that every student should have. The “alarmingly high dropout rate we see” shows the fallicy of this kind of thinking. Unfortunately, for political reasons, no politician will dare tell the public the truth about this. So, look for even more social engineering in this area to occur now and into the foreseeable future.

Mishap

July 4th, 2011
8:23 am

Eric,
The SAT was recentered in ‘95. Kids weren’t getting smarter, they were getting dumber so they moved scores up about 100 points. The 1000 in 1980 is closer to an 1100 today. In the early ’90’s an 800 would get you the median at UGA. HOPE + recentering has moved that number up dramatically.

Bright Shining Star

July 4th, 2011
8:29 am

I’d say our education system is doing great.

I’d say you have blinders on. Our education system is one of the worst in the developed world.

Matt

July 4th, 2011
8:31 am

The ACT is an easier test, but I can’t imagine it is that far out of line from the SAT on the margins. Anything below a 20 is poor, anything above a 30 is exceptional. As long as the subject scores are in that range, what is the problem?

Vince

July 4th, 2011
8:42 am

It always amazes me how people speak about education as if they are experts in the field. I would never pretend to do such in medicine, law or business.

@Mishap – The median SAT score at UGA in 1990 was 1045.

@Eric and Mishap – The average SAT score in the 60’s, 70’s, and 80’s was 1000. Think of the SAT as an IQ test but move the decimal one point over to the left. Thus, an SAT score of 1000 then and now would equate roughly to a 100 IQ score….both right at average.

@ Bright Shining Star…. I agree with the poster who stated our public schools do an amazing job. Name one other developed country that attempts to educate everyone, including a large percentage of its people who don’t even speak the language.

southpaw

July 4th, 2011
9:09 am

“More than one third of students who began post-secondary education, had not received a degree or were no longer enrolled in any institution of higher learning after six years.” Include me in that one third–specifically the second category. I was no longer enrolled in any institution of higher learning after six years, because I graduated before then.

Concerned Teacher

July 4th, 2011
9:18 am

The ACT and SAT are tests by for-profit companies. When a student sits for this test, it still is one test on one day and the stressed student’s future depends on it. I have had students take the SAT as many as six times. Others have taken the SAT and ACT both more than once. At nearly fifty dollars for each test (not including the possibility of late fees or changed dates fees because of life happenings), just how much does each of the companies make in a year? Are these companies really about education? Have educational institutions “bought in” to their bottom line?

Roll Tide with Pride!!!

July 4th, 2011
9:31 am

The problem with the ACT is that it does not test for actual intelligence. It is intended to get the inner city kids into real colleges. But then these same kids find that they are not as smart as the other college students, so they drop out.
Universities should focus on the SAT instead of the ACT.

Just Lucky?

July 4th, 2011
10:06 am

I’m at the top of a very large science field internationally. I’ve been lauded by the National Institutes of Health and the Nobel Foundation. But my SAT, ACT and GRE (graduate record exam scores) suggest that I should sweeping the floor at the AJC, not writing a comment about standardized tests. What is missing in these tests is the ability to measure desire, dedication and creativity. Apparently that’s where I must shine. The world still sends their best and brightest here for graduate work (doctorates). Why? Because our post secondary education system is still the best, warts and all. Despite that, with some exceptions that are increasing, we are far ahead of the world in most of our disciplines. Why? Because our trump card is creativity. Our country was founded on thinking out of the box, when it came to religious freedom, and that has extended into the very fiber of today’s America. Unfortunately, with the persisting and extreme cuts in support of colleges and universities, our greatest national asset is being destroyed, bit by bit. Couple that with the compromised focus on standardized tests in the high schools and you have the loss of what made American science, technology and business what it is and/or was today/yesterday. The lack of support for the arts also is misplaced bean counting. Creativity is the key and it pervades the sciences, arts, business and technology. Applicants to our graduate programs from around the world dwarf those from the US when it comes to scores on the GREs, for example, yet they are not, by and large, the best graduate students in our programs. Why? Because they mostly have been raised in overcrowded countries where independent thinking on all fronts cannot be tolerated due to the enormous masses of people. When these student get over here (and they do and have for decades now), some flourish and rise to the top because here, we tolerate and have, at least in the past, encouraged creativity, hard work and scholarship. We now are handing that over, slowly at first, rapidly now, to the rest of the world. So keep cutting education at all levels and watch the US continue its slide academically. Focus on these standardized tests, cut the budgets every year, underpay teachers and devalue education and you will reap what you sow.

catlady

July 4th, 2011
10:08 am

southpaw: It did not count you as the one third. When you finished, you were considered an on-time completer. Look for the little word OR.

Roll Tide: SAT does not test for intelligence, either. It is a mastery of skills test. For a kid with weaker advanced math skills, ACT is the test because it is so much more reading-dependent. For my son and elder daughter, the ACT produced higher scores. For my younger daughter, the math whiz, the SAT was the best because it is so heavily math. I allowed my own kids to do as I did, take each one once. If they couldn’t show basic skills with one try, they didn’t need to be thinking about college. Do the producing companies make a mint? Sure, but with the variation in grades, you need some sort of baseline. Of course, SAT and ACT scores, like grades, tend to be in direct relationship to parental SES (money and education).

In most states, success in college can be predicted by high school grades at a higher r level. However, HOPE has so skewed grades in Georgia especially that to get good predictive value you need to add in SAT/ACT. (U of A, BS, class of 1973)

As part of my master’s work at a large, popular, OOS school, I had the unpleasant chore of generating denial letters. In 1989, this school was denying kids with SATs of 1200 and GPAs of 3.8, and admitting kids with SATs of 800 and GPAs of 3.8. Let you guess what the difference was.
But my point is, how a kid can have a 3.8 and an SAT of 800 is beyond me. But this happened time after time, and that is why the need for a standardized score.

Jason

July 4th, 2011
10:16 am

Roll Tide, inner city kids? The ACT has historically been favored in the southern states, which as a whole aren’t that urban or that focused on the success of “inner city kids”. Sounds like you have a standard answer to everything to gets twisted to whatever is the topic of the moment.

As for the third that don’t complete a degree in six years, sometimes life happens. I had to drop out of school after one year due to family issues. I ended up working low income jobs for years but eventually I was able to earn an engineering degree from a major university. It look ten years of working and taking classes part time but I made it. It’s unfortunate that because it took me more than the traditional number of years to work my way through college that my degree is considered worthless by Mr. Barr. Luckily my employer doesn’t feel that way.

Carlosgvv

July 4th, 2011
10:28 am

Jason

Well done!!!

realist

July 4th, 2011
10:35 am

“require legislators and governors to first read the NBER study. This homework hopefully will short-circuit any such legislative move”

ha ha ha ha ha

the only thing the state officials in Georgia can understand is what the lobbyists tell them at dinner! Or on vacations in Europe. Or at the beach for a business conference. Or in the boardroom of a big law firm with government contracts.

Reality Check

July 4th, 2011
10:54 am

The truth of the matter is most careers don’t really require a 4 year degree. Hope funding should instead be prioritized on Georgia’s Technical Colleges and Associate Degree programs. Unfortunately, UGA, GT, the GSU’s, and KSU will NEVER permit that.

If we want to see that our college students finish what they started, then the schools and degree programs should be appropriately matched to the career interests of the student.

Tony

July 4th, 2011
11:19 am

The real FLAW with testing is the people who put too much emphasis on them.

Moderate Line

July 4th, 2011
11:49 am

Standardize testing has one purpose. It gives universities a reason to exclude someone. It does not give them a good reason. I am somebody who typically does well on standardize test but on the job neither standardize test or grades can predict how well you work with other people. They do provide insight to certain abilities but they also give people a sense of entitlement. I have seen people who have graduated from college with close to 4.0 but they have a hard time because they can’t work with other people. Most people have to learn this once their on the job because the culture and educational system don’t teach them any people skills. People need a balance between people skill’s and individualism.

areyoukiddingme?

July 4th, 2011
11:55 am

Roll Tide, when I was in school in Alabama in the 70’s, the test of choice, as it is today , was the ACT. Both Alabama and Auburn and all the other schools in Alabama utilized ACT test scores in their admittance decisions. Are you telling me Alabama and Auburn were trying to enroll “inner city kids? I don’t thnk so.

One more thing. Bob Barr, sounds like you’re on the SAT’s payroll.

the original and still the best John Galt

July 4th, 2011
6:29 pm

The “standardized tests” have been greatly dumbed down in recent years. This is the reason for the higher scores required to get into colleges that actually retain some semblance of standards.

The high IQ societies used to use SAT scores as one of the possible ways to gain admission to their ranks, but no more. MENSA, which requires an IQ in the 98th percentile, would allow a test taker with a score on the SAT in the 98th percentile or greater to belong without further testing, but now even a perfect score will not qualify.

The tests also used to be quite statistically valid and reliable. In other words, high scores on the standardized tests used to predict college success very well. Today however both college and the tests have been dumbed down.

old grad

July 4th, 2011
6:47 pm

Reality Check

July 4th, 2011
10:54 am
The truth of the matter is most careers don’t really require a 4 year degree.

That ^^^. We do kids no favors when we steer them en masse toward college. Many aren’t college material but they have some aptitude that will make them successful in a skilled trade. When my A/C dies in the middle of July I need a guy/gal that can repair it in a hurry, I don’t give a rat’s hiney if he/she never spent a day in a college. classroom.

Susie Watts

July 5th, 2011
12:39 am

As a private college counselor and test prep coach, I think the SAT and ACT need to be kept in perspective. Neither test is an accurate prediction of how students will do once they are in college. The SAT and ACT are factors, like many others, that should be kept in context and not given any more importance than they deserve. The grades students get in college prep courses is far more important than any set of test scores.

College Direction
http://www.collegedirection.org

BillyRob

July 5th, 2011
7:35 am

By all means, let’s get rid of standardized tests that actually measure achievement and learning and instead….
Well… we can have committee who meet in secret to use subjective criteria to decide who gets in and who does not. That way there will be no trail, no accountability to the educators at all. They can pick the kids they like and nail the kids they don’t.
Who said “predictiveness” was the key determinant of the value of the test. In most of life, success at one level is a prerequsite for progressing to the next level. If you do a poor job in college, should graduate schools allow you to enroll becasue you are “predicted” to be capable. How about med school? Should we let doctors who fail their courses and can’t demostrate a command of the subject practice medicine because they are predicted to be a good doctor?

J. Wellington Wimpy

July 5th, 2011
8:17 am

Best Comment of the Day:

Unfortunately, with the persisting and extreme cuts in support of colleges and universities, our greatest national asset is being destroyed, bit by bit. Couple that with the compromised focus on standardized tests in the high schools and you have the loss of what made American science, technology and business what it is and/or was today/yesterday. The lack of support for the arts also is misplaced bean counting. Creativity is the key and it pervades the sciences, arts, business and technology. …………………………because here, we tolerate and have, at least in the past, encouraged creativity, hard work and scholarship. We now are handing that over, slowly at first, rapidly now, to the rest of the world. So keep cutting education at all levels and watch the US continue its slide academically. Focus on these standardized tests, cut the budgets every year, underpay teachers and devalue education and you will reap what you sow.

Great job!

J. Wellington Wimpy

July 5th, 2011
8:18 am

Roll Pride: The problem with the ACT is that it does not test for actual intelligence. It is intended to get the inner city kids into real colleges.

I’m amazed that you can equate intelligence with “Roll Pride”. Killed off any trees in Auburn lately?

the watchdog

July 5th, 2011
8:29 am

The U.S. is falling woefully behind China and other far eastern countries in the field of education. I took a walk around Cornell, an Ivy League University in Ithaca, N.Y. Everyone I saw walking on the campus was oriental. Most all the restaurants are Chinese. That says a lot. The natives[U.S.] are having to attend lower quality educational institutions.
However, a little known fact, the upper classes in this country of equal opportunity are going to fertility experts and learning how to have offspring with high IQs. This is the new status symbol, “designer babies”. Now, do not rush into this new phenomenion, it is not always perfect, occasionaly, “anomalies” pop up. There are no guarantees. However, if you want to crow around the office about your super smart children, get off to the right start at “The Baby Store.”

[...] College Entrance Test Shown To Be Flawed (Atlanta Journal-Constitution). Bob Barr discusses a study produced by the National Bureau of Economic Research which casts doubt  on the ACT’s ability to predict a student’s success fin college. [...]

Just Lucky?

July 5th, 2011
9:23 am

@Watchdog
The Asian students are dominating what you consider ‘lower quality education institutions’ such as UGA, GaTech, GSU and every university I know of across the US, not just Ivy League-type schools (Emory as well). Again, this is because we have the best post secondary education in the world, although many of the PhDs that have been trained in the US are going back home and raising the level of the institutions there… thus, soon we will lose our edge on this front as well. Those staying in the US are taking the jobs away from the US citizens (many of these people are on H1 VISAs and convert them into Green Cards and then citizenship). Therefore, they will dominate here as well. That’s fair, as we are all about the best and brightest, no matter where people are born or what religion or race etc. This trend will continue. At my institution, the last 5 hires in the sciences have been Chinese individuals trained in the US. Nothing wrong with that, that’s just the way it is and will continue to be. I know this is the same at other institutions including our medical schools, such as the Georgia Health Sciences University (formerly known as MCG). It may be too late to reverse that trend, even if someone were interested in doing so (not by stopping those applicants but by creating competitive US applicants). Learn Manderin, Korean, and Japanese now if you want to more effectively communicate with the current and forthcoming generations of academics here in the US. Just keep cutting education funds in GA and elsewhere and this is what you will get. Companies come to cities because there are educated people to hire locally because it’s expensive to transfer people from other locations including the place they are moving from to come to Georgia. Why move here? We don’t have the people to perform those jobs. Not only do we have a pathetic number of college graduates, we have a large number of non-high school graduates. Let’s have another round of academic cuts this year too. Let’s have a fifth year of no raises for State employees including college and university professors (who are leaving in droves for other states who value education). Let’s keep it up and then sit back and wonder what happened to the GA economy. Let’s keep the undocumented out of the state as well so there are people to harvest the crops (because the high school drop outs and those laid off will not do this). Then we can either slide into the economic status of Mississippi or Alabama etc. Watch your property values drop and wonder why. That’s the road we are waaaaaaaaaaay down and moving fast. Really!

CDog

July 5th, 2011
10:27 am

The “higher education for all” philosophy and the ethnic diversity of the US is why our standardized test scores are often lower than other countries’ scores in various areas. Countries that are demographically homogeneous and that only allow the top percentiles of students to pursue higher education are always going to have better scores. The same is true for state to state comparisons within the US. Think about it.

Leavenworth, Kansas

July 5th, 2011
11:20 am

Let’s all hope criminal charges are brought against the greedy ignorant crooks
of the Atlanta Public School System.

J Moore

July 5th, 2011
11:52 am

All I know is that is white kids can be tested–so can black ones. Just because you don’t like the results–too bad!

Alabama Communist

July 5th, 2011
1:24 pm

No doubt Bob forgot about the Bachmann and Palin On-Line school of knowledge and history to get around this legal loophole to raise the IQ of the Republican Tea Party….

bert

July 5th, 2011
1:34 pm

“However, for some reason, the vast majority of colleges that use the ACT consider only a “composite score in their admissions process.” This means that the students’ overall unpreparedness is effectively masked.”

This means that it is also true for the opposite. Someone who scored poorly on a section may not be considered because of a composite score.

bert

July 5th, 2011
1:41 pm

I love threads. Everyone has an opinion, no one has an answer,.

ND

July 5th, 2011
2:18 pm

The SAT doesn’t exactly do a great job predicting future college performance either.

SaveOurRepublic

July 5th, 2011
2:26 pm

IMHO, scrap the ACT & raise the bar on the SAT…then make GPA + SAT score the key prerequisites for college admission.

Common Sense

July 5th, 2011
2:48 pm

In 1980 I made a 1020 on the SAT. I was hungover and had stayed out all night. I had taken the PSAT once and never had any sort of prep course. I thought I had ‘failed’ the test because I didn’t know you added the Math and Verbal to get your total score.

I now have a BS and 2 Master Degrees. After 30 years in industry I am taking a post as a college professor. Standardized tests show one thing…they show who takes standardized tests well. People that study and do their assignments do well in college. No test predicts that.

ChristieS

July 5th, 2011
3:02 pm

From the ACT faq page (http://www.actstudent.org/faq/answers/actsat.html):

*****************
“What is the difference between the ACT and SAT?

The ACT is an achievement test, measuring what a student has learned in school. The SAT is more of an aptitude test, testing reasoning and verbal abilities.

The ACT has up to 5 components: English, Mathematics, Reading, Science, and an optional Writing Test. The SAT has only 3 components: Critical Reasoning, Mathematics, and a required Writing Test. ”
****************

The ACT is a criterion-referenced test. Think of it as more of a CRCT-type test on steroids. It tests what the kids learned in high school. The SAT is an aptitude test. It tests how well you can reason things out (an intellectual processing skill), not how much you have already learned. As it is currently configured, the SAT Reasoning Test is a “rough measure of intellectual preparedness for college.” (http://abcnews.go.com/Technology/WhosCounting/story?id=98373&page=1).

The fact that colleges use these tests as part of the requirements for admission is okay, but as predictors of future college success neither test does such a bang-up job. Even the SAT has only been shown to be an adequate predictor of students making it through their freshman year, not their entire college career.

What seems to have an equal or greater impact on whether college students complete their degrees is what type of student that college freshman becomes. How much work is the student willing to put into his or her classes? Sometimes this has a great deal more to do with the maturity level of the student rather than his or her academic bona fides.

For more of the critique on whether to use ACT or SAT for college admissions, you can read this research paper here: http://digitalcommons.uri.edu/cgi/viewcontent.cgi?article=1022&context=srhonorsprog&sei-redir=1#search=%22What%20reliability%20does%20SAT%20have%20predicting%20college%20success%3F%22

Rik Roberts

July 5th, 2011
3:15 pm

The SAT and ACT are not meant to be predictors of performance. They are a way for college admissions to compare students from dissimilar high schools. The SAT has always been scored on a bell curve and the re-centering in 1995 of the bell curve to 500 was simply a reflection of the scores of the moment. One probable reason was that with more students taking the SAT a more accurate mid point was observed. It is a well known fact that a high SAT score does not necessarily mean academic success, and this is why GPA, essays and achievements are taken into account. Having taught SAT test prep for many years, I can tell you that the only predictor of success on the SAT is family income.

the original and still the best John Galt

July 5th, 2011
8:07 pm

The fact is that the ACT and the SAT were reliable predictors of success in college before they were dumbed down. Don’t take my word for it. Do the research yourself.

The tests were not the only reliable predictors of college success, but they were accurate predictors. Now they don’t mean much, just like a Bachelor’s degree doesn’t mean much.

bias

July 6th, 2011
9:42 am

the SEC and ACT are biased against African-Americans with a lot words and problems they’ve never been exposed to. both tests should not be required for minorities–I’m tired of this.

PaulL

July 9th, 2011
5:43 pm

The College Board, “owner” of the SAT test, is a non-profit organization, with about 6500 high schools and colleges in its membership. I’m not sure about ACT, but think it’s a non-profit as well. Even if they were for-profit, the couldn’t be nearly as corrupt as the public service employees in the Atlanta school district, administrators and teachers, who have been systematically changing student test scores for years.