How to Mislead With Statistics: GMO Crops Edition

Fight disinformation: Sign up for the free Mother Jones Daily newsletter and follow the news that matters.


A few days ago the New York Times ran a big front-page story about GMO crops. Roughly speaking, there were two takeaways: GMO crops don’t improve yields and don’t cut down on pesticide use. So why bother?

That sounded interesting, but I decided not to write about it. The story was pretty shallow in its use of statistics. It assumed that you can compare different countries without controlling for anything (different soils, climates, crops, etc.). And it seemed to suggest that American farmers must be idiots, because they keep buying GMO seeds even though they’re worthless. Put together, this was a little too much for my spidey sense, and it would have taken a long time to educate myself enough to really figure out what was going on. So I passed.

Yesterday, however, Nathanael Johnson of Grist took a close look at the Times piece and concluded that it was pretty misleading. You can read Johnson’s piece if you want all the gory details, but it contained a pretty interesting example of how to mislead with statistics, which I figured I should pass along. The Times story says that in the US, despite the wide use of GMO seeds, use of insecticides has fallen but the use of herbicides has gone up. In France, by contrast, where they don’t use GMO seeds, “use of insecticides and fungicides has fallen by a far greater percentage — 65 percent — and herbicide use has decreased as well, by 36 percent.” Using data from Andrew Kniss, here’s what this looks like in chart form:

It does indeed look like France is doing pretty well. But what if you look at the raw numbers, rather than percentages? Here’s a chart from Kniss:

In the case of herbicides, France started at a high level, and has only recently caught up to the US. In the case of insecticides, France started out way, way higher than the US and is still way higher despite their recent reductions. Looked at this way, it’s clear that France hasn’t actually done that well. What’s more, I had to cherry pick starting and ending years in the top set of charts in order to make France look as good as it does. Plus there’s this, from Kniss:

It is true that France has been reducing pesticide use, but France still uses more pesticides per arable hectare than we do in the USA. In the case of fungicide & insecticides, a LOT more. But a relatively tiny proportion of these differences are likely due to GMOs; pesticide use depends on climate, pest species, crop species, economics, availability, tillage practices, crop rotations, and countless other factors….Given all of these confounding factors, I wonder why France was singled out by Mr. Hakim as the only comparison to compare pesticide use trends. Pesticide use across Europe varies quite a bit, and trends in most EU countries are increasing, France is the exception in this respect, not the rule. In the early 1990’s, France was using more herbicides compared to almost every other country, so it shouldn’t be too surprising that pesticide use decreased as formation of the EU began to standardize pesticide regulations after 1993.

So what’s the best comparison to make: percentage change or raw volume? In this case, presenting only the percentage change leaves you with a very misleading impression. Presenting raw volumes, however, makes everything pretty clear. You can see that France started out far higher than the US and that its use of herbicides and insecticides has been decreasing. So why present only percentages?

POSTSCRIPT: If you click on the chart pack in the Times story, you will actually find charts showing raw volume of pesticide use in the US and France. However, they’re shown in two different charts, using different units, and broken up into different categories. If you were deliberately trying to make a comparison nearly impossible, this is how you’d do it.

AN IMPORTANT UPDATE

We’re falling behind our online fundraising goals and we can’t sustain coming up short on donations month after month. Perhaps you’ve heard? It is impossibly hard in the news business right now, with layoffs intensifying and fancy new startups and funding going kaput.

The crisis facing journalism and democracy isn’t going away anytime soon. And neither is Mother Jones, our readers, or our unique way of doing in-depth reporting that exists to bring about change.

Which is exactly why, despite the challenges we face, we just took a big gulp and joined forces with The Center for Investigative Reporting, a team of ace journalists who create the amazing podcast and public radio show Reveal.

If you can part with even just a few bucks, please help us pick up the pace of donations. We simply can’t afford to keep falling behind on our fundraising targets month after month.

Editor-in-Chief Clara Jeffery said it well to our team recently, and that team 100 percent includes readers like you who make it all possible: “This is a year to prove that we can pull off this merger, grow our audiences and impact, attract more funding and keep growing. More broadly, it’s a year when the very future of both journalism and democracy is on the line. We have to go for every important story, every reader/listener/viewer, and leave it all on the field. I’m very proud of all the hard work that’s gotten us to this moment, and confident that we can meet it.”

Let’s do this. If you can right now, please support Mother Jones and investigative journalism with an urgently needed donation today.

payment methods

AN IMPORTANT UPDATE

We’re falling behind our online fundraising goals and we can’t sustain coming up short on donations month after month. Perhaps you’ve heard? It is impossibly hard in the news business right now, with layoffs intensifying and fancy new startups and funding going kaput.

The crisis facing journalism and democracy isn’t going away anytime soon. And neither is Mother Jones, our readers, or our unique way of doing in-depth reporting that exists to bring about change.

Which is exactly why, despite the challenges we face, we just took a big gulp and joined forces with The Center for Investigative Reporting, a team of ace journalists who create the amazing podcast and public radio show Reveal.

If you can part with even just a few bucks, please help us pick up the pace of donations. We simply can’t afford to keep falling behind on our fundraising targets month after month.

Editor-in-Chief Clara Jeffery said it well to our team recently, and that team 100 percent includes readers like you who make it all possible: “This is a year to prove that we can pull off this merger, grow our audiences and impact, attract more funding and keep growing. More broadly, it’s a year when the very future of both journalism and democracy is on the line. We have to go for every important story, every reader/listener/viewer, and leave it all on the field. I’m very proud of all the hard work that’s gotten us to this moment, and confident that we can meet it.”

Let’s do this. If you can right now, please support Mother Jones and investigative journalism with an urgently needed donation today.

payment methods

We Recommend

Latest

Sign up for our free newsletter

Subscribe to the Mother Jones Daily to have our top stories delivered directly to your inbox.

Get our award-winning magazine

Save big on a full year of investigations, ideas, and insights.

Subscribe

Support our journalism

Help Mother Jones' reporters dig deep with a tax-deductible donation.

Donate