<a href="http://www.shutterstock.com/pic-96825760/stock-photo-person-throwing-a-coin-as-symbol-of-risk-and-luck.html?src=_zIuDBIl92ps5hOTWuUfsw-1-85">Sergey Nivens</a>/Shutterstock
In the 1930s, a Duke University botanist named Joseph Banks Rhine was gaining notoriety for focusing a scientific lens on the concept of extrasensory perception, or ESP. His initial research, which he claimed demonstrated the existence of ESP, consisted of case studies of exceptional individuals who seemed to be able to predict which cards a research associate was holding—even when sitting 250 yards away and separated by physical barriers like a wall—with greater accuracy than simple guessing would yield.
But case studies can only take you so far.
One night, Rhine met with Eugene Francis McDonald Jr., the CEO of the Zenith Radio Company. McDonald offered up his technology for what promised to be the largest and most impressive test of ESP yet: a nationwide experiment showing that telepathy is real.
“The idea was that they would have a bunch of people in a radio studio, and they would try to transmit their thoughts to the nationwide radio audience,” explains science writer William Poundstone, author of the book Rock Breaks Scissors, on this week’s Inquiring Minds podcast. “And then people at home could write down what they think they received and send that in, and scientists would look at it and decide if they had shown ESP or not.” The hope, says Poundstone, was that the participation of millions of radio listeners would produce results that were supposedly “much more statistically valid” than earlier ESP studies.
The first few broadcasts were a dramatic success. Most listeners were correct in their guesses of what the “senders” in a radio station in Chicago were thinking. On one episode, writes Poundstone, the thought-senders attempted to use their brains to transmit a series of five Xs and Os—OXXOX—and a majority of the audience members sent in the right answers. “So this seemed very impressive, and the head of Zenith put out big press releases saying that, you know, there’s no way this could be a coincidence,” says Poundstone.
But while it wasn’t a coincidence, a young psychologist named Louis D. Goodfellow figured out that the experiment wasn’t really measuring telepathy. Rather, it was demonstrating something far more interesting about human nature: our inability to behave randomly. It turned out that Goodfellow, who had been hired by Zenith to work on the show, could predict listeners’ guesses even before they had a chance to make them. He started out with the hypothesis that there is no ESP. In that case, the radio audience had to come up with a random sequence themselves. “And he realized that it’s not so easy for a person to make up a random sequence.” says Poundstone. “When people try to do that they fall into certain unconscious patterns, and these patterns are really very similar for everyone.”
In his own laboratory experiments, Goodfellow found that his subjects preferred certain types of sequences when they’re trying to come up with random ones. When he asked people to make up the results of five imaginary coin tosses, for instance, “he found first of all that the most popular first toss was heads,” says Poundstone. How popular? Seventy-eight percent of the study participants selected “heads” as the first result in their supposedly “random” sequences.
What’s more, explains Poundstone, Goodfellow discovered that “people liked sequences that were very well shuffled.” Indeed, the most common sequence chosen by Zenith audiences was heads, heads, tails, heads, tails (or its equivalent in Os and Xs)—they picked it nearly 30 times more frequently than tails, tails, tails, tails, tails. “It’s not too surprising that the least common ones were just five heads in a row, or five tails in a row,” adds Poundstone. “People figured that just wasn’t random.”
So, mystery solved. When the Zenith program transmitted thoughts that matched sequences that were popular with its listeners, “it suddenly looked like the public had a great deal of ESP,” says Poundstone. “But when the sequences were not so popular, then suddenly the telepaths were off their game.”
More recently, psychologists Amos Tversky and Daniel Kahneman proposed the so-called Law of Small Numbers, a theory that accounts for human misunderstandings of randomness. Specifically, we wrongly expect small samples to behave like very large ones. So if you toss a coin five times, you assume that you’ll get some variation of a pattern that includes two or three heads and two or three tails. If your coin lands on tails five times in a row, you tend to believe that it can’t be a coincidence. But in fact, the odds of five tails in a row are 1 in 32—not especially common, but not terribly rare, either. “So we have all these sort of false positives where we figure there must be something wrong with that coin, or maybe the person’s got some magic hot-hand in tossing coins,” Poundstone says.
Understanding these pitfalls can actually help you predict, with accuracy above chance, what someone else is going to do, even when he or she is trying, purposefully, to act randomly. These predictions are at the core of Poundstone’s book, which offers a practical guide to outguessing and outwitting almost anybody—in activities ranging from Rock, Paper, Scissors (men tend to go with rock, so you can beat them with paper) to investing in stocks.
Naturally, the larger the dataset, the more accurately a person—or a computer—can predict behavior. With access to Big Data, large corporations like Target have developed analytics that can predict our behavior with remarkable accuracy, even when we think we’re making decisions in the moment. Siri, your iPhone’s talking app, learns about you and the behavior of all the other iPhone users and uses that information to predict what you’re going to ask her even as you are evaluating your own needs.
And sometimes, the Big Data machine is more observant than even the people closest to us. In his book, Poundstone cites the story of a Minnesota dad (first reported by the New York Times) who complained to a Target manager that his teenage daughter was being encouraged by the company to engage in unprotected sex. The store, he noted, had sent her a mailer littered with photos of cute babies, baby gear, and maternity clothing. As Poundstone writes, the manager apologized and promised that he’d suss out the source of the error. In doing so, he learned that Target analyzes purchases made online and in stores that are predictive of the behavior of an expectant mother. When he called the angry father once again to apologize, he realized just how powerful these algorithms can be. As it turns out, this time the customer was apologetic: Apparently Big Data noticed his daughter’s pregnancy well before he did.
Poundstone draws a direct line between Goodfellow’s debunking of ESP and modern efforts to predict consumer behavior. “It basically demonstrated that a lot of the little everyday decisions we make are incredibly predictable, provided you’ve got a little bit of data to work from,” he says. “And that’s become a very big business today, needless to say.”
But does this predictability apply to everyone? Poundstone knows of at least one person who defies the odds. Computer scientist Claude Shannon built the first computer to predict human behavior. And of all the people tested, he was also the only one who could beat the machine at its own game. When asked how he managed to do this, “he said that he had a very simple secret,” reveals Poundstone. “He essentially mentally emulated the code of the machine and did the algorithm in his head, so he knew what the machine was going to predict, and then he did the opposite.” But Shannon is a special case. “For almost everyone else, mere humans,” says Poundstone, “I think it is pretty easy to predict, at least a good deal of the time.”
Inquiring Minds is a podcast hosted by neuroscientist and musician Indre Viskontas and best-selling author Chris Mooney. To catch future shows right when they are released, subscribe to Inquiring Minds via iTunes or RSS. We are also available on Stitcher. You can follow the show on Twitter at @inquiringshow and like us on Facebook. Inquiring Minds was also recently singled out as one of the “Best of 2013” on iTunes—you can learn more here.
