Economic forecast models: the pushback

Nate Silver's 2012 prediction model has produced some interesting responses across the blogosphere. Brendan Nyhan notes that Silver's estimate for the effects of candidate ideology in presidential elections just has to be too big*, and he finds that Silver's model underperforms when compared to others. John Sides responds to Mike Tomasky's dissing of political scientists. Then Alan Abramowitz offers a model that includes an important variable that measures how long the incumbent party has held the White House, and Sean Trende pushes back a bit on that one and on the whole concept of economic forecasting.

I want to respond to what Trende wrote, but I'd first like to point out that it's fantastic that we're even having this debate. Just a few years ago, it seemed like political scientists were tearing their hair out just trying to convince political journalists that elections weren't determined entirely by campaign activities, and that the economy and other fundamental aspects of the political environment might be relevant. Now, the debate among political scientists and journalists appears to be more like, "Come on, the economy doesn't explain everything." So I feel like this discussion has moved in a very good direction in the past few years.

Okay, back to Trende's piece. He ticks off a bunch of things that you have to believe if you're going to accept the validity of an economic forecast model. For example:

First, at a basic level, you have to accept that something as complex as voting can be reduced to a simple, three-variable equation. And you have to accept that this equation is linear.

Well, no, you really don't. Now, we have good evidence that you can explain a very high percentage of what goes on in elections with just two or three variables, but that doesn't mean that everyone's vote choice is a result of just those variables. These models do have error terms. Sometimes other things can affect votes. They just usually don't.

And just because most of these models are linear, that doesn't mean that they have to be. If you get way out in the tails of economic performance, you can see that the effect on votes isn't quite linear. Hoover did better in 1932, and FDR did worse in 1936, than a linear model of economic growth would predict, if for no other reason than that any major party presidential nominee is guaranteed close to 40% of the vote; their hardcore partisan supporters simply won't defect no matter how bad things get. But most elections don't occur under such extreme conditions, and the linear model works quite well for those.

Here's another point Trende makes:

You have to accept that there is no problem predicting the president’s vote share from only 16 data points.

That's silly. Of course there's a problem with that. But that's all the cases we have, and they work pretty well. Indeed, it's pretty amazing we get such robust results from so few cases. And keep in mind that a lot of the truths we cling to in American politics, such as "the president's party loses House seats in midterm elections" or "Democrats lose when they nominate liberal New Englanders," are based on this many cases or fewer.

You have to accept that presidential elections haven’t changed at all over the past 64 years. ... You have to accept that the enfranchisement of African-Americans and poor whites in the South, as well as the enfranchisement of 18-to-21-year-olds nationally, had no effect on the outcome of the later races. A casual glance at the results of the 2008 elections would seem to suggest otherwise.

No one I know is claiming that elections are exactly as they were 64 years ago. But the same basic trends do seem to hold: voters blame the incumbent party when the economy underperforms and reward them when the economy does well, and they tend to turn against parties that have been in power a long time. I don't know why he singles out 2008 as some sort of evidence that these fundamentals have changed. In fact, such forecasts nailed the '08 results within a single percentage point.

You have to accept that anything that happens past the end of the second quarter of an election year matters only at the margin. If the economy absolutely collapses, and a previously popular president goes into Election Day with a 20 percent approval rating amid a full-scale depression, where the economy is contracting by 10 percent a quarter, it wouldn’t matter much. If we are attacked and enter a war, it wouldn’t matter much. If a president becomes mired in scandal, it wouldn’t matter much.

Again, no one argues this. A lot of modelers pick the second quarter as a cut-off because it allows time to make a forecast several months before the election while still capturing much of the economic activity on which the incumbent party will be evaluated. It's very rare that an economy that's humming along at three percent growth for a year or more will suddenly plunge into recession the quarter prior to an election. That certainly can happen (and it kind of did in 2008), but it's a very rare event. Similarly, if the forecast models showed Obama likely to win next year, but he decided to shoot Tom Hanks in the face on live TV on October 31st, yeah, he'd probably lose the election. It's not that last-minute twists don't happen, it's just that they're rare, and the things that happen to the economy in the third quarter of an election year usually look a lot like the things that happened in the first and second quarters.

There are a few other similar points that Trende makes. Some are good caveats for forecasters, and it would generally be good for us to be straightforward about the assumptions of our models. But many of Trende's arguments are simply straw men. Look, we have some models that do a pretty good job explaining elections -- a lot better than claims about campaign quality or candidate optimism or likeability. But past performance does not guarantee future results. Make of these models as you will.


*I hope my post using Silver's numbers to draw out a prediction plot wasn't taken as an endorsement of Silver's model. I simply did it because I thought it would look cool.