Those Who Can’t, Test

The SAT is: A) outmoded B) inaccurate C) a huge moneymaker for a company you’ve never heard of. Answers below.

Image: Andrew Lichtenstein/Sygma

For indispensable reporting on the coronavirus crisis, the election, and more, subscribe to the Mother Jones Daily newsletter.

What does ‘ETS’ stand for?” asks the instructor. A beat for comic timing, then the answer: “Evil Test Society.”

It’s not all that funny, really — ETS actually stands for Educational Testing Service — but it breaks the tension gripping this room full of college-bound high school students, and a wave of chuckles washes over the Sunday morning Princeton Review SAT prep class in west Los Angeles. The instructor, a comedian and actress by trade, seems pleased to have won over her audience. Though the kids are here by choice — and their parents are paying $745 for the privilege — most of them seem as loath to answer her prompts as any batch of ennui-haunted teenagers. They are peaceful enough, even drowsy. Still, the handful of kids here on a summer morning is at the front lines of a battleground in the war against standardized testing.

America ruptures along fault lines of ethnicity and class, and the SAT, which is written and administered by ETS, is a potential earthquake. Not surprisingly, California feels the strains of the SAT more than most.

In the wake of the passage of California’s controversial Proposition 209 (which ended the use of ethnicity and gender considerations in college admissions), the University of California system is seeing a substantial decline in African American and Latino admissions to its twin flagships, UC-Los Angeles and UC-Berkeley. Some groups suggest that eliminating SAT scores as an admissions criterion might remedy that disparity, but that proposal is unlikely to get far. A more realistic option is the “4 percent plan”: Any student graduating in the top 4 percent of a class would be automatically eligible for admission to the UC system — though not necessarily to the campus of his or her choice.

What California could be stumbling toward, Texas has already fallen into. In 1996, the 5th U.S. Circuit Court of Appeals’ Hopwood decision meant an end to affirmative action at the University of Texas system as well. This fall, in response to the decision, Texas enacted a 10 percent rule, admitting the top 10 percent of every Texas high school graduating class regardless of SAT scores. Thus, no big changes in minority admissions are expected in Texas’ state universities.

Down in Texas, faltering in California — could the SAT really be on the ropes? Princeton Review devoutly hopes so, despite the fact that last year 35,000 kids paid the nationwide coaching service almost $26 million to learn how to beat the standardized test.

“People ask me if I’m being disingenuous: ‘How can we believe you when you say you think SATs should be abolished?'” says Jay Rosner, executive director of the Princeton Review Foundation, the nonprofit wing of the Princeton Review empire that offers reduced-fee courses to minority students. “Don’t you think doctors who make their living treating disease would be happy if disease disappeared?”

A lawyer, Rosner has been tilting at ETS for more than a decade, serving as an attorney in various lawsuits against the testing giant, mostly representing kids whose scores have been challenged by ETS in court. (If ETS thinks your scores are suspiciously high, it will go to mighty efforts to prove you’re a cheater.)

Princeton Review’s founder and president, John Katzman, says that before starting the company in 1981, he was a fan of the test. “I loved the SAT,” he says. “The SAT had been very, very good to me.” But the SAT didn’t love him back. According to Katzman, in 1985, ETS sent spies to some of Princeton Review’s courses and found 13 questions on its practice exams that ETS thought were suspiciously close to actual SAT questions, which are ETS’s intellectual property. ETS sued Princeton Review and ended up settling out of court for $50,000, though Katzman says his legal fees were at least 10 times that. (ETS says it found at least 100 of its questions on the practice tests, and denies using “spies.”)

“It’s like I was walking through India munching a burger and had no idea what I was doing,” Katzman remembers. “You find yourself in the middle of a holy war and you don’t know what hit you. The whole concept of prepping for the SAT was much more controversial than I realized.”

For a nonprofit corporation, ETS makes a lot of money from its tests (it had $41 million in cash reserves as of June 1997). ETS is responsible not only for the SAT but also for the Graduate Record Exam (GRE), the Advanced Placement (AP) tests, and the Test of English as a Foreign Language (TOEFL), among many others. ETS sells its standardized wares to the College Board, ostensibly responsible for the SAT, but The Chauncey Group, ETS’s for-profit arm, also sells ETS tests to companies such as Microsoft, Oracle, and the American Society of Plumbing Engineers, all for use in testing and placing potential employees and members.

When ETS started in 1947, it began administering an aptitude-measurement test meant to forge order from the chaos of the nation’s youngsters and some not-so-youngsters (World War II soldiers returning to school on the GI Bill) by quantifying their academic capacities through a precise, controlled exam. In the minds of the psychometrically obsessed early ETS bureaucrats, the SAT was to be the ultimate expression of the art of mental testing.

In a 1959 booklet, the company claimed the SAT “tells you how fast and how far you can go.” Since then, the official line has changed. Now, according to ETS spokesman Kevin Gonzalez, the SAT measures a student’s “developed reasoning skills in math and verbal [areas].” The strong correlation between a student’s SAT scores and his or her scores on other standardized tests, however, suggests otherwise: What the SAT really tests is how well you do on tests.

So, in addition to trying to teach the very limited body of knowledge the SAT covers (a smattering of vocabulary, and some ninth-grade algebra and geometry), Princeton Review’s instructors mostly teach their students test-taking strategies and tricks.

“ETS wants you to feel rushed,” explains Victoria Depaoli, the Princeton Review instructor in west Los Angeles. “That’s why the test is timed.” She scrawls a chart on the chalkboard to show the students two simple things about the test that can work wonders for their scores: (1) questions in each section get progressively harder, and (2) you lose no points for questions left blank and lose only one-fourth of a point for a wrong answer. If you rush through the easy questions under the belief that you ought to answer each one, you’re apt to make dumb mistakes on problems you could have gotten right. “Don’t go so fast,” she says. She can’t say it enough.

One kid is hesitant about guessing. The odds seem bad — five choices, and only one is right. But guessers can prosper if they can eliminate even one answer as wrong. Keep in mind, the teacher tells the class, the way the test is scored. Nodding their heads, the students are starting to get it: The SAT isn’t about knowledge; it’s about beating the test.

Some high school teachers criticize Princeton Review because its training favors stunts over skills. Today’s Princeton Review class seems to exemplify this tendency. Depaoli demonstrates to an incredulous class that you needn’t read all of a reading comprehension selection to answer most of the questions — go to the questions first, she says, and they will often guide you right to the line or paragraph you need. And for “PC” authors and subjects — and there is generally at least one of those — the answer is unlikely to be anything that sounds negative.

She tells them that for the algebra questions, if the multiple-choice answers are given in ascending order, as they often are, just start plugging in the number at “C” for all of them — if it’s wrong, she says, you’ve still instantly eliminated three potential answers.

One teenager who’s skilled in algebra complains that plugging in the random numbers takes too long. “I like algebra,” she says. Depaoli tells her: “Algebra is where careless errors come in.”

In the next part of the lesson, Depaoli brings up “Joe Bloggs” — the mythical average student that Princeton Review uses to illustrate how ETS designs the SAT to trick its takers. After all, she says, a test everyone does well on is no good as a screening device. So for questions that are supposed to be hard — ones toward the end of a section, remember — any answer that seems obvious is probably wrong. “We’re being manipulated,” laments the young woman who likes algebra. She’s learning Princeton Review’s lesson well.

For the vocabulary section, the instructor gets the kids laughing with verbal tricks that seem so contorted it’s hard to imagine they’re easier to learn than simply memorizing the meaning of the words themselves. The word under discussion is “cogent.” How to remember it? “Picture Tom Cruise in the movie Jerry Maguire,” Depaoli tells the kids. “Remember that movie? Wasn’t Tom Cruise COnvincing as an aGENT in that movie?” Oh, of course. Someone who’s read a lot won’t need these stunts — and also probably won’t need Princeton Review at all. The very fact that these techniques actually work is a powerful blow to the idea that the SAT is truly a test of “aptitude,” as ETS used to swear.

“Take me through the test, question by question — show me where it tests aptitude,” says Katzman. He riffles through a sample test. “That question is asking you the opposite of the word ‘pedestrian,’ and that one’s a geometry question…. It seems very unlikely that this is testing anything deep and meaningful.” ETS seems to have all but admitted this lack of rigor: SAT used to stand for Scholastic Aptitude Test, but in 1994, ETS changed it to the Scholastic Assessment Test.

Although the Princeton Review is the bad boy of test preparation, its older and more staid competitor, the Kaplan Educational Centers, also stresses in its courses that the SAT is by no means an aptitude test. The SAT, say Kaplan’s instructors, tests acquired skills — ones that their 36-hour program can help inculcate. They’re proud that ETS has been forced to admit over the years that coaching can help. Kaplan’s students average a 120-point gain (out of 1,600), with 28 percent achieving a gain of 170 points or more.

ETS will not be quick to let go of the SAT. Even the move toward eliminating the SAT on the part of big state university systems such as in Texas or California — or the fact that highly desirable, highly competitive liberal arts colleges such as Bowdoin don’t even require it anymore — isn’t any real skin off ETS’s nose. Most prospective students still have to take the test and kick their $22.50 fee to the company. (In 1997, 2.3 million students took the test; ETS grosses close to $50 million a year on this test alone.) And even if UC’s board of regents adopts the 4 percent plan, the state will still insist that students take the test, so it can provide a body of data that UC can use to analyze the academic progress of those who were admitted with lower scores. And flawed as it may be, say admissions officials, they have nothing better to rely on. High school grades can vary wildly, and if college admission depended on grades alone, they say, they worry that grade inflation would leap even higher.

But Katzman sees trouble for ETS: “What’s happening in Texas and California is the biggest threat to the SAT ever. People have been confronted with [their] inability to work around the test’s flaws anymore and are saying, ‘It sucks — let’s just get rid of it.’ ”

In the academic community, those who oppose the SAT don’t want to get rid of national standardized testing. Instead, they suggest switching to a test that’s less flawed, arguing that tests such as the Advanced Placement exams or the SAT-II tests (formerly known as the Achievement Tests) test bodies of knowledge that are both wider and more specific, and might thus encourage high schools to improve their curricula.

In the end, the SAT is still really a problem of elites. All correlations between SAT scores and college grades suffer from the problem of “restricted range” — that is, of being a limited, selective sample. Most people going to college are doing better than the mean. (To understand the concept of restricted range, consider this example: A study of professional basketball players might not find any clear correlation between height and success in basketball — that’s because the players are all pretty tall already.) It’s true that having a college degree is growing in importance in America. In 1980, among 25- to 34-year-old males, those who had a bachelor’s degree averaged 19 percent more income than those without one; by 1996, that same figure had shot up to 54 percent. For black people, a bachelor’s degree can mean an even more astounding difference: a 77 percent rise in expected income. For women the rise is even greater still: 88 percent. Yet only 67 percent of high school graduates enter a two- or four-year college, and only around 50 percent of those earn a bachelor’s degree within five years. (For blacks, the figures are 60 percent and 34 percent, respectively.)

The meritocracy that the SAT supposedly represents has a powerful emotional grip on Americans — we like to believe in essential fairness, thinking that our society gives each person an equal chance. But supporters of affirmative action maintain that everyone isn’t on the same starting line in the United States because the opportunities for advancement, including getting into college, have historically been rigged in favor of whites. And public institutions such as state universities and public schools are inherently creatures of politics, so the issues that concern them, such as the SAT, are doomed to be perpetual political battlegrounds.

The debate about standardized testing is usually argued in numbers; it is really about values. ETS, with its belief that quantification can give clear answers to questions of intellectual stratification, has successfully obscured this point. As college graduation rates begin to hit 50 percent, it’s hard to remember that, until 1940, most Americans didn’t even finish high school. It may be too late to turn back the clock to the days when a bachelor’s degree was a necessary totem for the upper class. (To judge by one of the SAT’s strongest correlations, the ability to reason develops in lockstep with family income: the higher your family’s income, the higher your score.)

But the children of elites know how important the right college can be. In west Los Angeles — and across America — they are going to classes like the Princeton Review’s. Most of the students here have taken the SAT before — some more than once. For them, it isn’t a matter of just getting into college, but of making it into UC-Berkeley or UCLA or MIT. Did the price of their summer mornings and their parents’ money seem prohibitive, I asked?

Answered one young woman, “My parents said, ‘We work to make sure you have the best in life…and you will go.'”

Dear Reader,

This feels like the most important fundraising drive since I've been CEO of Mother Jones, with staggeringly high stakes and so much uncertainty. In "News Is Just Like Waste Management," I try to unpack the reality we all face and how we can rise to the challenge. If you're able to, this is a critical moment to support Mother Jones’ nonprofit journalism: We need to raise $400,000 to help cover the vital reporting projects we have planned, and right now is no time to pull back.

Monika Bauerlein, CEO, Mother Jones

payment methods

Dear Reader,

This feels like the most important fundraising drive since I've been CEO of Mother Jones, with staggeringly high stakes and so much uncertainty. In "News Is Just Like Waste Management," I try to unpack the reality we all face and how we can rise to the challenge. If you're able to, this is a critical moment to support Mother Jones’ nonprofit journalism: We need to raise $400,000 to help cover the vital reporting projects we have planned, and right now is no time to pull back.

Monika Bauerlein, CEO, Mother Jones

payment methods

We Recommend


Sign up for our free newsletter

Subscribe to the Mother Jones Daily to have our top stories delivered directly to your inbox.

Get our award-winning magazine

Save big on a full year of investigations, ideas, and insights.


Support our journalism

Help Mother Jones' reporters dig deep with a tax-deductible donation.