New study says U.S. peers are not lagging international peers. Data “misleading and exaggerated.”

I am listening to a media call on this new report by the Stanford Graduate School of Education and the Economic Policy Institute.

On the call, Richard Rothstein, a research associate at the Economic Policy Institute, calls the use of international test data to vilify U.S. student performance has been “misleading and exaggerated.”

He and Martin Carnoy, a professor of education at Stanford, disaggregated the data used in the international comparison by social class and came up with a far more nuanced picture of U.S. performance.

But here is the official summation:

Socioeconomic inequality among U.S. students skews international comparisons of test scores, finds a new report released today by the Stanford Graduate School of Education and the Economic Policy Institute. When differences in countries’ social class compositions are adequately taken into account, the performance of U.S. students in relation to students in other countries improves markedly.

The report, “What do international tests really show about U.S. student performance?”  also details how errors in selecting sample populations of test-takers and arbitrary choices regarding test content contribute to results that appear to show U.S. students lagging.

In conducting the research, report co-authors Martin Carnoy, a professor of education at Stanford, and Richard Rothstein, a research associate at the Economic Policy Institute, examined adolescent reading and mathematics results from four test series over the last decade, sorting scores by social class for the Program on International Student Assessment (PISA), the Third International Mathematics and Science Study (TIMSS), and two forms of the domestic National Assessment of Educational Progress (NAEP).

Based on their analysis, the co-authors found that average U.S. scores in reading and math on the PISA are low partly because a disproportionately greater share of U.S. students comes from disadvantaged social class groups, whose performance is relatively low in every country.

As part of the study, Carnoy and Rothstein calculated how international rankings on the most recent PISA might change if the United States had a social class composition similar to that of top-ranking nations: U.S. rankings would rise to fourth from 14th in reading and to 10th from 25th in math. The gap between U.S. students and those from the highest-achieving countries would be cut in half in reading and by at least a third in math.

“You can’t compare nations’ test scores without looking at the social class characteristics of students who take the test in different countries,” said Carnoy. “Nations with more lower social class students will have lower overall scores, because these students don’t perform as well academically, even in good schools. Policymakers should understand how our lower and higher social class students perform in comparison to similar students in other countries before recommending sweeping school reforms.”

The report also found:

There is an achievement gap between more and less disadvantaged students in every country; surprisingly, that gap is smaller in the United States than in similar post-industrial countries, and not much larger than in the very highest scoring countries.

Achievement of U.S. disadvantaged students has been rising rapidly over time, while achievement of disadvantaged students in countries to which the United States is frequently unfavorably compared – Canada, Finland and Korea, for example – has been falling rapidly.

But the highest social class students in United States do worse than their peers in other nations, and this gap widened from 2000 to 2009 on the PISA.

U.S. PISA scores are depressed partly because of a sampling flaw resulting in a disproportionate number of students from high-poverty schools among the test-takers. About 40 percent of the PISA sample in the United States was drawn from schools where half or more of the students are eligible for the free lunch program, though only 23 percent of students nationwide attend such schools.

With each release of international test scores, many education leaders assert that American students are unprepared to compete in the new global economy, largely because of U.S. schools’ shortcomings in educating disadvantaged students.

“Such conclusions are oversimplified, frequently exaggerated and misleading,” said Rothstein, who is also senior fellow at the Chief Justice Earl Warren Institute of Law and Social Policy at the University of California – Berkeley School of Law. “They ignore the complexity of test results and may lead policymakers to pursue inappropriate and even harmful reforms.”

Carnoy and Rothstein examined test results in detail from the United States and six other nations: three of the highest scorers (Canada, Finland and South Korea) and three economically comparable nations (France, Germany and the United Kingdom). In cases where U.S. states voluntarily participated in the TIMSS for 8th grade mathematics, the researchers compared trends in these states’ scores with trends in 8th grade mathematics on the NAEP.

The researchers show that score trends on these different tests can be very inconsistent, suggesting need for greater caution in interpreting any single test. For example, declining trends in U.S. average PISA math scores do not track with trends in TIMSS and NAEP, which show substantial math improvements for all U.S. social classes

Carnoy and Rothstein say that the differences in average scores on these tests reflect arbitrary decisions about content by the designers of the tests. For example, although it has been widely reported that U.S. 15 year-olds perform worse on average than students in Finland in mathematics, U.S. students perform better than students in Finland in algebra but worse in number properties (e.g., fractions). If algebra had greater weight in tests, and numbers less weight, test scores could show that U.S. overall performance was superior to that of Finland.

The report comes as the administrators of TIMSS are preparing to make public more detailed data underlying 2011 test results, following last month’s release of average national scores. PISA plans to release detailed data on its 2012 test in December 2013. Carnoy said that he and Rothstein will then be able to supplement their present report by comparing the most recent TIMSS and PISA results by social class across countries. He invites other researchers to conduct similar analyses, to see if their findings confirm those of the present report.

–from Maureen Downey, for the AJC Get Schooled blog

47 comments Add your comment

Dawn

January 15th, 2013
12:31 pm

So, “we’re doing a worse job of maintaining economic parity and educating poorer students” is our justification for doing poorly? Great! Proud to be an American! I’m very skeptical. It seems denying these results is justification by educrats to maintain the status quo.

guest

January 15th, 2013
12:35 pm

Just another article to try to defend why American students are so dumb. Blame it lack of money, phones, cars, clothes, etc. Basically anything and everything is at fault except the student.

Catlady

January 15th, 2013
12:43 pm

guest, I don’t think it is “blaming” other things besides student effort. Everyone knows some poor folks who work very hard to OVERCOME the effects of poverty and lack of parental education. But, for many students, the effects of their parents’ (you fill in the blank laziness, lack of education, poor decision-making, etc) is a difficult task to climb over. More difficult, say, than showing up rested, well-fed, well-provided-for, read-to, having tons of parental time and effort invested since before birth.

mountain man

January 15th, 2013
12:49 pm

One of the “precursor” traits to serious sociopaths is the torture and cruelty to pets and small animals. They get pleasure from this. Of course, when you design a video game where you can see your opponents brains blown out of his skull, who needs to do real cruelty to an animal. You get the same jolt. Did the video game CAUSE the insanity – NO, but it compounds the problem.

Do I think these games should be banned – I honestly don’t know. But I DO think it is hypocritical to ban child pornography because it is a danger to society and still leave those FPS killing games. To me there is no difference.

indigo

January 15th, 2013
12:50 pm

“these students don’t perform as well academically, even in good schools”

This is a job for MORE SOCIAL EXPERIMENTS man!!!!!

guest

January 15th, 2013
12:50 pm

Catlady,

You can agree that thousands of “poor” kids do well and achieve greatness, right? If so, then ALL “poor” kids can. It’s just a matter of how much they want it, and in today’s society, they are taught not to work hard because they’re poor, a minority, etc.

mountain man

January 15th, 2013
12:50 pm

Sorry, somehow jumped to the wrong blog.

living in an outdated ed system

January 15th, 2013
12:53 pm

I look forward to reviewing the report’s methodology further. To me, you take all these reports with a grain of salt. I just have to look at the composition of students at the elite colleges and universities to know how far behind we have fallen. I also know from the composition of students I have interviewed for my alma mater the past 20 years to see that we are falling behind.

Data or no data, our achievement gap is far too wide and needs to be fixed. What I cannot support is research that folks will use to indicate our public education system is not sub-standard. As a college professor to tell you how much time they spend on remedial work because their freshmen come in unprepared because they did not get the foundation in K-12.

bootney farnsworth

January 15th, 2013
12:59 pm

there is so much wrong here…

1-the basic premise is flawed to support a predetermined (ie, we crap on the poor) outcome.
2-after decades in higher ed working at the most international student body, my observations are international students ARE far and away ahead of ours
3-we attempt to educate everybody – most nations don’t
4-we take a perverse pride in demanding lower standards for our children. most of the rest of the world demands higher standards. familiar with the Asian F concept?
5-BS on our poor can’t do it. I saw it daily for decades. Its got zero do to with income, everything to do with desire.

1guest

January 15th, 2013
1:03 pm

Catlady,

You can agree that thousands of “poor” kids do well and achieve greatness, right? If so, then ALL “poor” kids can. It’s just a matter of how much they want it, and in today’s society, they are taught not to work hard because they’re poor, a minority, etc

living in an outdated ed system

January 15th, 2013
1:08 pm

@Bootney, surprisingly, I largely agree with you : ) I would just add that “desire” can be fixed. A combination of school and community support can fix “desire.” I have seen it first hand with strong enrichment programs that focus on social & emotional learning.

bootney farnsworth

January 15th, 2013
1:17 pm

one of the very most rewarding days of the year at GPC was graduation day. it may not seem like much to most people, graduating from a 2 year school. but for most of the graduates, even in this day/age, it was the first time anyone in their family graduated college.

they carried not only their hopes and dreams, but that of entire families, occasionally communities.
they got it done in large part due to desire and hard work.

and BTW: most of the star students were either direct immigrants or 1st generation US.

bootney farnsworth

January 15th, 2013
1:18 pm

two major quotes which apply here

1-poverty is a mental disease
2-those who say it can’t be done are often passed by those who are doing it

bootney farnsworth

January 15th, 2013
1:20 pm

@ maureen

I didn’t see any mention of the effects of sugar mama students.
did I miss something?

8)

Fan

January 15th, 2013
1:56 pm

Didn’t Bracey try to tell us this?

bigdawg88

January 15th, 2013
2:50 pm

Ummm, am I the only one who has never heard of any of these tests? Who are they giving them to? Please tell me after all these years someone didn’t notice that more poor students from the US were taking these tests! Perhaps it is proof that the US education system really is failing…. SMDH

sneak peak into education

January 15th, 2013
3:27 pm

It has long been shown that socio-economic status is one of the major reasons behind a student’s success, or lack of, in school. This study proves what many of the educational psychologists, analysts, etc… have been saying all along; remove that factor from our test scores and we rank far higher in the international rankings than the reformists (i.e. educational privateers) want you to believe. You see, they want you to believe how dire our educational system is so that it opens the door to those who see an opportunity to make a mint at the expense of our students. Watch the documentary “The Shock Doctrine” to understand how those in power seek to scare the general public with cries of “the sky is falling” so that they will willingly oblige to accept the new, quick-fix remedy (and guess who gains to profit by peddling such a remedy? The very ones who create and stir up the fear of parents, etc…) I am not saying that there are things in our public schools that don’t need to be fixed but instead of investing (and I don’t mean solely in monetary terms), but the currently fashionable method of disinvesting in our schools and putting them into the hands of privateers is not the way to go.

10:10 am

January 15th, 2013
4:24 pm

So the perceptions of universities that entering freshmen can’t function at university level … and of employers that young hires lack basic math and communication skills … are without merit?

Gee, wonder what the teachers’ unions think of all this?

Dr. John Trotter

January 15th, 2013
5:27 pm

Just got through writing a chapter for our book entitled, “Family Income is Predictive of Academic Achievement.” I have been writing articles on this for years. Free and Reduced lunch counts always correlate with academic achievement. When the scores of students from very low performing urban schools (it’s almost always urban schools) are factored in, then the scores of the rest of the country pretty much align with the much-heralded Finland. But, what do we do with the urban schools? Our educrats just keep letting the students act like thugs and abuse the teachers. The educrats actually blame the teachers for the low scores because they don’t have the guts to grab the bull by the horn and establish discipline in these war zone schools.

Atlanta Mom

January 15th, 2013
5:28 pm

“disaggregated the data used in the international comparison by social class ”
I sure would like to know how they did that, but that’s one long report.

Dr. John Trotter

January 15th, 2013
5:29 pm

Why are my comments always “awaiting moderation”? Are they that explosive? Ha!

Atlanta Mom

January 15th, 2013
5:29 pm

guest
I suppose you are one of those people who believe that everyone can be a doctor too.

1guest

January 15th, 2013
5:42 pm

Atlanta Mom,

What’s wrong with all kids trying to achieve the most they can? If they don’t get into medical school they have a nice major and go get their Ph.D. You represent what is wrong with this country’s mindset. You probably believe we shouldn’t push kids too hard bc good ole Obama will help out. Besides, it’s not fair someone else got into medical school and I didn’t, right?

1guest

January 15th, 2013
5:46 pm

Hey sneak peek into education,

If the U.S. also removes the test scores for the lower 50%, we would also score higher.

Private Citizen

January 15th, 2013
6:20 pm

10:10 Why do you keep obsessing about teacher unions? the one is Britain has 300,000 members and been there for 150 years.

Jerry Eads

January 15th, 2013
6:21 pm

Yes, Fan. Departed friend Jerry Bracey DID pull the data apart from past multinational comparisons and point out that when those from the same economic levels are compared, our public schools did quite well indeed. Detractors claiming U.S. education a failure conveniently ignore the fact that we have a FAR higher percentage of kids in poverty than the “competing” countries.

Here’s a recent very short piece by Stephen Krashen citing Jerry’s work:
http://www.schoolsmatter.info/2010/10/problem-is-poverty-evidence-from-gerald.html

The first piece I noticed from Jerry on poverty was in 1999 (but I’d guess he started much earlier). Later, in responding to Reid Lyons, obviously untrained in data analysis or research method, who claimed on the basis of international tests that American schools were failing, Jerry noted:

“No they’re not. Some are, no doubt. But in the Progress in International Reading Literacy Study, American kids in low poverty schools stomped the top-ranked Swedes. Even kids in schools with up to 50% of the students in poverty attained an average score that, had they constituted a nation, would have ranked 4th. Only American students attending schools with 75%+ poverty scored below the international average of the 35 participating countries.”

Bottom line is that according to the data available, many American public schools are far and away the best in the world. Georgia’s legislature, however, seems bound and determined to make sure that’s no longer the case.

10:10 and others like: Please remove your dunce hats. NObody, including those in our NOT unions (you still haven’t been able to learn the incredibly simple fact that this is a right to work state?) argues that we don’t need to do better with our high poverty students who yes indeed have trouble when entering college. One of the major goals of our society has been to break the “poverty cycle.” Too bad we voters don’t elect representatives who get that.

Ah, much too long. Last: BEST piece I’ve seen on poverty and schooling is David Berliner’s in TCR: “Effects of Inequality and Poverty vs. Teachers and Schooling on America’s Youth” http://www.tcrecord.org/Content.asp?ContentID=16889 (Sorry, not a freebie)

Jerry Eads

January 15th, 2013
6:27 pm

AM: Several ways to collect socioeconomic data: sometimes it’s just by self-report (not a good one – kids don’t know how much mom and dad make); frequently the data are aggregated from school records from which the sample students are drawn. None of it’s perfect, but the statistical error averages out over large numbers. Big work like we’re discussing usually asks for school record data.

Lee

January 15th, 2013
7:26 pm

The authors of this article needed to take it one step further. Stratify the population by race AND socioeconomic factors. My hypothesis would be that the kids of the same race and socioeconomics would fare well with each other across nationalities.

What if there is a racial “achievement gap” between the races in other countries. What will the politically correct blame it on then?

Atlanta Mom

January 15th, 2013
8:23 pm

Jerry,
I know I never gave socioeconomic information to my children’s school. I do remember one of my children coming home after some standardized test asking what our household income was (no response to that question).
Are we once again basing this on “free and reduced” lunches?
And…………this information also had to be obtained in the other countries as well.
Maybe I’ll scan the executive summary over the weekend and see if I can spot it.

Private Citizen

January 15th, 2013
9:21 pm

Informative video about history of Prussian schooling circa 1850, i.e. what lead to compliant public ready to be enthusiastic for doctrine of a “slow and steady march to a police state.” http://www.youtube.com/watch?v=okPnDZ1Txlo

Private Citizen

January 15th, 2013
9:29 pm

Whoa! Horace Mann circa 1840 talked about “blind allegiance to arbitrary power. Yow. That sounds way too familiar in the present school management structure.

It is apparent to me that any serious studier of U. S. government schools K-12 education need be learn-ed in the works of Horace Mann, from which the whole thing sprung. From the year 1846, https://learn.usf.edu/bbcswebdav/users/ljohnso2/3604/Readings/Mann_10th.htm

[...] "Socioeconomic inequality among U.S. students skews international comparisons of test scores, finds a new report released today by the Stanford Graduate School of Education and the Economic Policy Institute. When differences in countries’ social class compositions are adequately taken into account, the performance of U.S. students in relation to students in other countries improves markedly." | via Get Schooled  [...]

10:10 am

January 15th, 2013
10:51 pm

@ Jerry Eads: If you’re going to be insulting, at least have someone help you be clever about it.

And do drop by the National Education Association’s website. They seem under the impression they are indeed a union—which makes their dues payers here in Georgia (100% of GAE members) union members, does it not?

And there is much more to unions than local workplace militancy.

Private Citizen

January 16th, 2013
12:13 am

10:10 why you being so one sided? because you do not know. I can see your point about the national political. And slay GAE as they are a fake org like all of them in that no one in Georgia protects worker rights to basically not be abused and bouced around like a ping pong ball. That is what you are missing. Historically, unions are for safety reasons not for grouping as a gang to screw the managers out of money, or be a manipulate co-opted political force. I think that is what you are objecting an too and sounds good and you should be specific, but you have no business to throw out the safety and worker protection part, and that is the part the teachers care about, and in Georigia teachers need worker safeguards because they a lot of administrators who order teachers around and I do not mean in a good way. I mean “Pack up your classroom and move it over here” and three months later “Pack up your classroom again and move it over here.” In other words, WORKER HARASSMENT and you missed that part of it. That’s the real core of unions, is to protect workers from unreasonable rough treatment.

bu2

January 16th, 2013
8:53 am

Don’t think the performance of international students in our colleges is relevant for comparing school systems. Those are their brightest, most ambitious students. And they may be in a state school while our brightest, most amibitious tend to be at Ivy League schools. And there are more than 4 times the number of Chinese and Indians as Americans.

sneak peak into education

January 16th, 2013
9:29 am

@10:10 – the unions that you constantly whine about helped to bring about sweeping changes to the landscape of american but in a good way- child labor laws, women’s rights, shorter working week, health and safety in the workplace, health-care benefits, and leave for new moms.

What kind of person would decry these things as being bad for the good of our nation? You still would like to see children being used for cheap labor in the workforce? You would still like to see workers being forced to work endless hours during the week (without overtime pay) and with no weekends off? You would still like to see unsafe working conditions that lead to loss of life or workers being maimed? You would still like to see a woman’s right to have a small amount of time (and yes, it’s so minimal) of time off after she becomes a new mom?

Sounds like you have shown your true colors.

10:10 am

January 16th, 2013
10:34 am

@sneak:

My gripe with the teachers’ union is that it is at least as much about raising money for the Democrat Party and thus protecting union dues revenues—than about workplace protections. http://goo.gl/bNdPt

And its anti-reform, anti-parental choice stances are unconscionable.

10:10 am

January 16th, 2013
10:39 am

@ sneak:

Here’s the link to the NEA’s political slush fund dollars I was actually looking for: http://goo.gl/rtJIZ

Home-tutoring parent

January 16th, 2013
1:00 pm

S. Downey has provided a good review of Carnoy and Rothstein’s report.

The report disaggregates student “class” into 6 subgroups. Subgroups “1″ (”Lowest”) and “2″ are categorized as “Disadvantaged”, while subgroups “5″ and “6″(”Highest”) are deemed “Advantaged”.

Of the U.S. PISA-taking students 38% were Disadvantaged, compared to 18% in the top-3-scoring nations (aggregate of Canada, Finland and Korea), and 29% in “similar to U.S. post-industrial” countries (aggregate of UK, France and Germany).

Of the U.S. students, 18% were categorized as Advantaged, vs. 28% for the top-3-scoring, and 23% for the “similar to U.S.” aggregate countries.

In all individual countries, Advantaged (Groups and 6) students outscored, substantially, their Disadvantaged schoolmates.

Therefore given the U.S.’s having the highest proportion of Disadvantaged students among compared countries, and the lowest proportion of Advangated students, one would conclude that at least part of the U.S.’s lower-than-other developed nations’ scores can be attributable to socioeconomic class-profile disparities between nations.

How did the authors generate 6 social-class sub-groups? This is a key question.

The OECD collects information on test-takers’ parents’ levels of education completion, occupations,
at-home resources such as a desk for home study, having one’s own bedroom, household quietude, a dictionary, poetry books, works of art at home, #’s of household computers, cell phones, TVs, cars, et al. (See report’s “Part IV”, below Table 13 for a complete list of OECD ’s parameters, how they are relatively weighed is unknown).

The authors chose to use as a “social class proxy”, the amount of shelf-space devoted to book storage at home, reported by test-takers in their test questionnaires, which the authors then “converted” to #s of books at home.

Hence,
Social-Class 1 => 0-10 books in the home, Class 2 11-25, Class 3 26-100, Class 4 101-200, Class 5 201-500, Class 6 >500 books in the home.

The authors admit that students’ estimates of shelf space devoted to book storage may be subject to some inaccuracy, and that the authors’ own conversion of students’ shelf space estimates to numbers of books may be confounded by the “thinness vs. thickness” of home books–for example, Koreans [and other East Asians] may tend to own relatively more paperbacks than “cloth” bound books, than Westerners, so children’s shelf-space estimates, where accurate, may not generate accurate book-number “conversions”. [My own note: pirated Chinese textbooks are usually printed in very thin semi-translucent "rice" paper, as well.]

In summary, Carnoy and Rothstein use, as their primary social-class proxy, family literacy levels, i.e. love (or lack thereof) of learning through reading at home. This may be the report’s “take-home lesson”.

Not to trivialize, and certainly not to pooh-pooh the Stanford-Berkeley scholars’ report, which contains much useful information, but it is pretty common knowledge that homes with many books, parents who consistently read to their younger children at bedtime, take them to the library often (children’s section to start), help them obtain library cards, and encourage them to read instead of watching the “boob tube”, tend to produce advanced-reading ability children.

Atlanta Mom

January 16th, 2013
1:23 pm

Thank you home-tutoring parent.
That was certainly not what I expected for social-class identification, but it’s an interesting way to do it.

sneak peak into education

January 16th, 2013
1:57 pm

@10:10 – your argument doesn’t hold weight when the schools that are unionized generally are the ones that outperform the non-unionized schools. Oh dear, that flies in the face of your argument that unions thwart educational success for children. The fact that they donate to the democratic party is a also moot-look at the money that was thrown at the elections by the oligarchs of our time; koch bros, rupert murdoch, et al, with hopes of pushing their ALEC-ridden agenda to put public education into the hands of the private sector-not because they care about children; it’s all about the money. If unions didn’t exist, the powers to be would have the general workforce working for peanuts in the worst of working conditions so that they can maximize their profits.

Home-tutoring parent

January 16th, 2013
2:13 pm

PISA data have been troubling for many years.

On a somewhat positive note, Carnoy and Rothstein demonstrate that disadvantaged (Class 1 and 2) 15 y/o’s PISA Reading and Math scores have risen since 2000, by about 18-20 points, or approximately 6%.

Our disadvantaged students outperform France’s, and Germany’s, and U.K’s disadvantaged students in Reading. For Math, our scores are equal (to Germany& UK) or slightly higher (than France’s). (See report’s figures B1 and C1, and, for number crunchers’ satisfaction, accompanying tables 9A and 12A)

On a less sanguine note, our disadvantaged students’ Reading and Math scores significantly lag Canada’s, and Finland’s, and Korea’s disadvantaged students’ scores. (See figures B2 and C2.)
In the 2009 test, the U.S. vs. Top-3’s “gaps” were smaller than 2000. Simple lines-extrapolation suggests a potential full-catchup in Reading could occur in the next 10-20 years, with Math catchup taking somewhat longer. However, scores can and have dropped at times, so catching up is not assured. But overall, our disadvantaged students’ scores are trending upward, whereas other countries’ not so much, so gap-elimination is conceivable.

Unfortunately, our schools aren’t doing as well with advantaged social-class students. We need to address them because they are our future innovators, inventors, entrepreneurs, managers and leaders.

The bad news is that our Class 5 and 6 students’ PISA Reading and Math scores are statistically the same in 2000 and 2009. Our Reading scores are the same, or nearly so, relative to the Top 3 countries, but the Math score gaps are significant. Our scores dropped from 2000 to 2003, and from to 2006. They have subsequently “rebounded” to 2000 levels, but will have to keep rising (while the Top 3’s stagnate or fall) to eliminate the “gaps”.

Home-tutoring parent

January 16th, 2013
4:09 pm

The Organisation for Economic Co-operation and Development, which created PISA, is, as you may infer from the spelling, a chiefly European-led endeavor.

Our government’s National Center for Education Statistics has information on the test.

http://nces.ed.gov/surveys/pisa/index.asp

Interested readers can even “take a test sampling” of PISA questions, click “Sample Assessment Questions”.

I did this. There are 12 “Items” (reading passages, some with accompanying figures) with 22 accompanying “Questions”, and 30 specific total response opportunities.

The test uses Systeme Internationale (”SI” aka “metric”) measurement units. This could be disadvantageous for American students, in the context that even if SI units are taught in school, our students don’t use them in ordinary daily life, so they’re not “deeply ingrained” as they are in international students.

Aside from this, the PISA is a refreshing change from our standardized tests, because the questions require thinking. At least that was what I found. For example, a very smart student can’t just eyeball-recognize correct answer choices and whip through 4 questions per minute, or even 1 per minute.

The information given is generally complex. For example in a “Movie” question about three 15 year olds’ consensually selecting a movie to go see, during a vacation week, a table of 6 movies, their weekday and weekend showtimes, and age-ratings is presented. Constraints include the boys’ going to an appropriate age-rated movie (3 of the 6 movies), two boys having to attend other matters on three of the vacation days, one boy having already seen one of the 3 allowed movies and rejecting the prospect of a second watching, parental theater pick-up before 10 PM, and different evening-show times, one of them being too late.

So what can they see, and when can they all go to see it?

It’s a reasonable “real world activities” problem. Only about half the students here and internationally specified the 3 movies the boys “should consider watching” and when presented one specific movie, with 5 day/date choices, 1 out of 3 students here and internationally could not identify the correct choice!

There is a problem involving a travel from point A to point B on a city-train system, shown as a “map”, with three intercrossing lines shown, giving three possible route choices. Each line has multiple stations, with travel cost being charged for each station reached from the starting one (not including the starting station), 1 zed per, and uniform interstation travel time, 2 minutes between stations, specified. Time for changing lines is given, 5 minutes.

The question requires pencilling-marking the fastest and cheapest route, and specifying the cost and time required. All three choices require two train changes, so one only has to count stations shown, and find which route has the fewest, multiply this by 2 min per station travel time, add 10 minutes for train changes, and the fare is # of stations passed/reached in zed.

Only 5.5% of Americans got this fully correct, and only 11.2% of international students did.

This question is arguably “biased towards” kids who live in cities with extensive interconnecting commuter-rail and/or bus systems, who would intuitively “get” the question from their own “real world” experience.

Finally, OECD provides many old-test questions to participating schools, along with guidance on the test’s principle objectives. Teachers are free to show the test questions to students, and have them work on them. It would not surprise me in the least if schools in Finland, Korea, Japan, Shanghai, Canada and elsewhere practiced for the test, and focused on the concepts embedded in the tests. As the test is primarily used to evaluate nations, not individual schools or students, at least by the OECD, “test-prepping” may quite conceivably be a national undertaking in the highest-performing countries.

I think, based on the sample questions on the nces.gov website, and others on oecd.org’s website, the test is reasonable, overall, I would even say superior to the NCLB/NAEP, and SAT/ACT/GRE bubble-fill tests and sample questions I have seen. (Yes, I know SAT has added some “free response” math questions, and SAT and ACT have writing tests, but the test questions remain mostly bubble-fill.)

Whether PISA, or more precisely, a PISA analog, should be adopted here (Australia has a PISA-emulating national exam), and whether our schools’ curricula should be overhauled to teach PISA’s principles, are questions that are worth at least asking and exploring by our education community.

I do not say this because, “We must catch up! The ‘Asian Tigers’, Finns and Canadians are leaving us behind!” but rather because excelling in PISA requires critical thinking, juggling multiple facts and ideas, grappling with cognitive challenges, and experiencing interesting insights.

Home-tutoring parent

January 16th, 2013
4:17 pm

erratum: “principle objectives” should be “principles and objectives”, although “principal objectives” is a reasonable alternative. ;-)

Atlanta Mom

January 16th, 2013
4:24 pm

“Unfortunately, our schools aren’t doing as well with advantaged social-class students”
Is this the natural result of our “No child gets ahead” mentality?

Home-tutoring parent

January 16th, 2013
5:51 pm

There were some disconcerting data in the 2003 PISA’s “Problem Solving” test results. (This was added for one year to the regular Reading Literacy, Mathematics Literacy and Science Literary tests.)

Information can be found at

http://www.oecd.org/edu/preschoolandschool/programmeforinternationalstudentassessmentpisa/34009000.pdf

It appears that OECD excludes scores above 95th and below 5th percentile, in each country, in calculating believing that this “outlier” data may inordinately affect “national mean scores”. This is important because you can do the following:

See the website’s Figure 2.5, on page 44.

The top of each nation’s score-band is its 95th percentile score.

The first black “arrowhead” represents the 90th percentile.

Brown-red arrowhead is the 75th percentile.

Black hyphen within a gray “button” is 50th percentile.

This allows one to exclude the bottom half of a distribution, i.e. the lowest 50% of test takers’ scores.

Now, for the remaining 50% of students (top-half scores) a new scores distribution is established. If the original distribution is Gaussian (normal, “camel’s hump”), the new one is not: it is widest at its new bottom.

However, this population still has a 90th percentile. It is the old distribution’s 95th percentile. To wit, if there were 100 students, there were 5 students above the 95th percentile. Now there are still 5, but the “population” has been reduced to the top 50 students, the top 5 students are “top-tenth”. now.

This means that after dumping the U.S.’s bottom-half performers, the top new tenth cutoff for the U.S. would still be lower than the full-nation top-tenth cutoff for 12 nations (their black arrows).

Now examine Figure 2.3 on page 41.

It is not absolutely clear, but the bars here would appear to represent ALL students, with students scoring at level 2 and 3 proficiency being represented above the “0″ line, and students scoring at level 1 or below level 1 shown below this line.

In Finland, Korea, Hong Kong and Japan approximately 27-35% of their students score “level 3″.
In the U.S., approximately 12% score “level 3″.

Now, eliminating the bottom half of U.S. students (which leaves a small percentage of level 1 students, and no below-level-1 students), our level 3 students would comprise 24% of our remaining population. Still less than Finland and Korea, marginally, and well less than Hong Kong and Japan’s full populations. In essence, or new population would have very few Disadvantaged students, comprising almost certainly a smaller percentage of the new population than the other four nations’ full-population-based Disadvantaged student percentages, yet they would still beat us in terms of percentages of their populations scoring at the highest level. In problem solving.

Given trends in U.S. Advantaged students’ Math scores since 2003, which are up slightly, we might be doing better in problem solving, although the two tests’ scores do not entirely correlate.

But the a priori odds are high that we would still substantially trail the top 4 countries in level 3-difficulty problem solving, if today’s 15 y/o’s took the test, because as of 2009, our Advantaged students made only miniscule gains in essential-to-problem-solving reading-skills, as well as math-skills tests, since 2003 (see Carnoy and Rothstein Figs. B1 and C1).

Home-tutoring parent

January 18th, 2013
5:31 am

Atlanta Mom,

“No child gets ahead” is a landmine. If I may gingerly trapse near it without stepping on it…

After 1st grade, we moved to a new city that had semester-scheduling, and enrollment was age based. Instead of being placed in first-semester 2nd grade, they skipped me to second-semester (in the fall) due to my age date. Then I did first-semester 3rd grade in winter and spring.

We moved again, to a conventional-schedule school. I retook first-semester 3rd grade. Ca. October, they administered a strange test to us. It had long-division problems, and figure-association problems. I must have done well, because the principal called my mom in and said,
“Your child is eligible to be skipped up to 4th grade. We don’t necessarily recommend this, because it might be stressful… you choose.”

Unfortunately, my mom never told me. She had not been a great student. She told the principal to leave me in third grade. She never considered that making me retake first-semester 3rd grade was “remedial education” that I did not need. But then neither did the principal consider this. Nor did he advise (my mom recounted this meeting a couple years after the event) that I could be placed in 4th grade, and if I didn’t adjust well, I could be returned to third grade, to encourage my mom to try it. I was counterproductively held back. I know this because the following year I was placed in a 4th-6th grade mixed class, and did fine. When we moved again the following year, I was placed in 5th grade, second track, and was completely bored. The school never sought my old school’s transcripts or state-assessment test scores. The following year I was placed in the whole-city “gifted” program with kids from five different neighborhood schools.

I have heard of public school students who have been skipped as many as three, or even four years. Most of them seem to have attended public school before WWII, but occasionally I read of a 14-15 year old entering college today, I never witnessed this. It is my understanding that many educators think it may harm kids’ “socialization”.

I have personally talked to people who say their AP classes are open to 11th and 12th graders, but not younger. Yet I have seen College Board AP reports that show small numbers of public school 9th graders taking Calculus AB, and even smaller numbers taking Calc BC (and doing well on the exams). Some schools apparently offer AP World History primarily to 9th graders. I haven’t seen any reports of 9th graders taking AP English Language, but my suspicion is that thousands of students (a small percentage of high schoolers) could do fine in it. Some could do fine in the more difficult English Literature course.

Fundamentally, if younger children were allowed to skip 1 or 2 or 3 grades, then take AP courses early, we could conceivably make hs graduation at age 17, with several AP credits, feasible for our 90th-93rd percentile ability students. For our 94th-97th ability students, graduation at age 16, for 98th-99th percentile students, graduation at age 15 are reasonable. This would save states billions of dollars, and AP credits would shorten students’ college times to 3 years, and for some, even 2 years and save them and their parents a lot of money (and states would save money too for public university attending early graduators). Or kids could stay 4 years and earn double or triple major degrees.

Most PhD programs try to promote 5 year completion, as most standard student-stipend grants are for this period. 7 year stints happen, albeit with faculty and administrative consternation. Four year stints are not rare. Three year stints are very rare, but brilliant extremely hard-working students pull them off from time to time. (They tend to make accelerated tenure, and earn full professorships in their mid-30s.)

In medical schools, once in a blue moon a student will graduate in 3 years. Med schools have mulled the notion of shortening the term, to deal with budget constraints, and 6-year B.S./M.D. programs successfully operate in several states.

So, to your point, schools that offer a large portfolio of AP courses are already successfully enabling students to earn a semester, a full year, or even three semesters of college credit, and early graduation. For a year’s credit and matriculation as a sophomore, students need 7-8 APs, and acceptable test scores (3’s in state universities, 4’s in some elite private ones.)

Most larger communities also offer International Baccalaureate programs. Although IB is designed provide basic college preparation, not high-level acceleration, with a primary mission to “create a more peaceful world by fostering human mutual respect and cooperation”, IB has in recent years developed college credit courses. At the private IB Diploma schools, for example AIS, which serve internationally stationed businesspeople and diplomatic staffs’ families, higher-ability students can take up to 4 Higher Level courses. Most public IB-programme high schools are restricted to offering 3 maximum HL courses, due to large disadvantaged minority-student enrollments, and these students struggle to handle 3 college-credit courses, so 4 is viewed to be “too hard” for them. This is reasonable for the minority students, apparently. It does however hold back very talented but non-affluent white and Asian-American students in communities whose public districts use IB as an integration “magnet” and offer few AP courses in the same or other schools, students who could pass 6, 7, or 8 (or more) AP courses and save thousands of dollars in college-attendance costs, if given the opportunity to enroll in many AP courses in public schools.

In any event, in summary, schools are providing some opportunities for kids to shorten their college stays and lessen expenses. At graduate level, acceleration occurs.

But the public education system is nowhere close to providing invaluable, student-centric optimized academic acceleration opportunities to very gifted and extraordinarily gifted children, opportunities that would be challenging and inspiring to them, opportunities that would save their families and taxpayers huge amounts of money, opportunities that would accelerate talent flow into our economy, and enable these young people to make productive contributions to it.