How to Bullshit With Statistics

As you probably know, President Obama put in place higher mileage standards for new cars before he left office. President Trump, naturally, wants to get rid of them. This requires a report demonstrating that the Obama standards would be bad, so he ordered OMB Director Mick Mulvaney to produce one.¹ Mick saluted and scurried off to carry out his boss’s order with the help of the Department of Transportation, which is conveniently run by Elaine Chao, the wife of Senate majority leader Mitch McConnell.

Today the EPA made public an … um … assessment of the report that it wrote a couple of months ago. The EPA’s assessment “privately challenged the Trump administration’s rationale for freezing Obama-era mileage standards,” according to the Associated Press, and that made me curious to see it. Eventually I did, and now I’m going to share a small part of it with you. Fair warning, though: even for chart nerds, this is pretty tedious. But I promise that it’s worth it, for reasons that I hope will become clear.

Scrappage

First off, you have to estimate the effect of higher mileage standards on the total size of the automobile fleet. But the Trump version makes no sense:

The As-Received² model estimates that the Augural³ standards will reduce the year-over-year annual increase sales of new vehicles by approximately 8,000 vehicles on average between CY2021 and CY2032. However, during the same period, the As-Received model estimates that the used fleet will grow by an average of 512,000 vehicles per year, far exceeding the decrease in new vehicle sales. It’s hard to imagine any real-world scenario under which over 60 additional used vehicles are retained for each new vehicle that the sales model predicts will be unsold as a result of the higher new vehicle prices.

And here’s the chart:

As you can see, there’s really only a small difference between the red line and the dashed black line. Who would even notice that it goes in the wrong direction and is therefore projecting that a higher vehicle cost will produce a bigger fleet? But it does, and this is “not consistent with the basic principle of supply and demand,” as the EPA says blandly.

The Rebound Effect

If gas mileage gets better, cars will use less gasoline. Hooray! However, there’s also something called the rebound effect, which means that since cars are using less gasoline, people will tend to drive a little more because it’s cheaper to do so. The result is that although improving gas mileage has a positive effect by reducing the amount of gas used per mile, it also has a small negative effect because people drive more miles. Sometimes life is like that.

Somehow, though, the OMB/DOT folks got this 180 degrees backward. Here’s the chart:

Notice that the dashed lines are all below zero, which means that OMB is projecting lower VMT (vehicle miles driven) from the Obama standards, even though the Obama standards produce higher gas mileage and therefore should produce a positive rebound. As a consequence, this would also project a negative rebound from repealing the Obama standards, which is totally wrong.

Yet More Rebound

It gets even worse. The rebound effect is fairly small, so what if we just assumed it was zero?

With no rebound, we would not expect to see any change in total VMT….However, even with 0 percent rebound, the As-Received model does produce total VMT values that are influenced by stringency level. See Figure 6, below. We believe that this zero-rebound VMT growth is an artifact of the disconnect between the sales model, scrappage model and mileage accumulation schedules described with Issue #1. And while this problem is not directly related to the model’s calculation of the rebound effect, it points to the importance of carefully considering how the various elements are integrated when making changes or additions to a model.

This time, just to make their point clear, EPA places a gigantic red X through the incorrect numbers:

This should be a straight line, not a curve. If there’s no rebound effect, how can the rebound effect produce anything other than a result of zero all the way across?

What It All Means

There’s more, but I’ll stop here. Even for chart enthusiasts, enough is enough.

In any case, the point here is not that the OMB/DOT charts are wrong. The point is that they’re totally disconnected from reality. Basically, somebody whomped up a spreadsheet and then played around with the numbers until they got ones they liked. The fact that it implied gravity was negative or the sky was green never occurred to them. To them, this wasn’t an exercise in creating a model of the real world, it was an exercise in fiddling with spreadsheet cells until the results were ones the boss would like.

And who cares anyway? It’s not like anyone in the press is ever going to look carefully at this stuff. Even the ones who know better will describe it as a “tense technical battle between experts at two separate government agencies” because they’re not allowed to say that the OMB and DOT just bullshitted some numbers and then called it a day. But that’s what happened, and that’s why it was worth wading through a bit of this. This is how the Trump administration works. It’s how the Republican Party works these days. These guys make Karl Rove look like Einstein.

¹I assume it was Mick Mulvaney, anyway. I suppose it might have been someone else.

²“As Received” means the version EPA received from OMB.

³Don’t even ask. For some reason, the Trump report uses “augural” to refer to its proposed future standards. God only knows why.

AN IMPORTANT UPDATE

We’re falling behind our online fundraising goals and we can’t sustain coming up short on donations month after month. Perhaps you’ve heard? It is impossibly hard in the news business right now, with layoffs intensifying and fancy new startups and funding going kaput.

The crisis facing journalism and democracy isn’t going away anytime soon. And neither is Mother Jones, our readers, or our unique way of doing in-depth reporting that exists to bring about change.

Which is exactly why, despite the challenges we face, we just took a big gulp and joined forces with The Center for Investigative Reporting, a team of ace journalists who create the amazing podcast and public radio show Reveal.

If you can part with even just a few bucks, please help us pick up the pace of donations. We simply can’t afford to keep falling behind on our fundraising targets month after month.

Editor-in-Chief Clara Jeffery said it well to our team recently, and that team 100 percent includes readers like you who make it all possible: “This is a year to prove that we can pull off this merger, grow our audiences and impact, attract more funding and keep growing. More broadly, it’s a year when the very future of both journalism and democracy is on the line. We have to go for every important story, every reader/listener/viewer, and leave it all on the field. I’m very proud of all the hard work that’s gotten us to this moment, and confident that we can meet it.”

Let’s do this. If you can right now, please support Mother Jones and investigative journalism with an urgently needed donation today.

payment methods

AN IMPORTANT UPDATE

We’re falling behind our online fundraising goals and we can’t sustain coming up short on donations month after month. Perhaps you’ve heard? It is impossibly hard in the news business right now, with layoffs intensifying and fancy new startups and funding going kaput.

The crisis facing journalism and democracy isn’t going away anytime soon. And neither is Mother Jones, our readers, or our unique way of doing in-depth reporting that exists to bring about change.

Which is exactly why, despite the challenges we face, we just took a big gulp and joined forces with The Center for Investigative Reporting, a team of ace journalists who create the amazing podcast and public radio show Reveal.

If you can part with even just a few bucks, please help us pick up the pace of donations. We simply can’t afford to keep falling behind on our fundraising targets month after month.

Editor-in-Chief Clara Jeffery said it well to our team recently, and that team 100 percent includes readers like you who make it all possible: “This is a year to prove that we can pull off this merger, grow our audiences and impact, attract more funding and keep growing. More broadly, it’s a year when the very future of both journalism and democracy is on the line. We have to go for every important story, every reader/listener/viewer, and leave it all on the field. I’m very proud of all the hard work that’s gotten us to this moment, and confident that we can meet it.”

Let’s do this. If you can right now, please support Mother Jones and investigative journalism with an urgently needed donation today.

payment methods

We Recommend

Latest

Sign up for our free newsletter

Subscribe to the Mother Jones Daily to have our top stories delivered directly to your inbox.

Get our award-winning magazine

Save big on a full year of investigations, ideas, and insights.

Subscribe

Support our journalism

Help Mother Jones' reporters dig deep with a tax-deductible donation.

Donate