Unemployment and presidential elections, reconsidered

Nate Silver notices that the change in non-farm payrolls in an election year does a pretty good job predicting presidential elections. I've used his variable in the chart below to predict incumbent party vote shares:

This variable does almost as well as real disposable income in predicting election results. Of course, payroll growth and RDI growth are closely correlated, but this is nonetheless forces me to reevaluate my earlier claim that unemployment levels didn't really predict presidential election results. This measure also calls for some other reconsiderations if you buy the argument. For example, Gore did just about as well as could be expected, Stevenson massively underperformed in 1952 (thanks to the Korean War, five terms of Democratic control of the White House, and his relative lack of Ike-ness), and Nixon actually had a pretty solid economy in 1972. Oh, and McCain slightly over-performed in 2008, so cut him and Palin some slack. Or call the electorate racist. Whatever.

Silver also comes up with a magic number for Obama: he needs 166,000 new jobs per month (a figure that was well exceeded in December and January).

The nerd war goes on

If you thought the debate over election forecast models was over, you thought wrong. Sean Trende has pushed back against my pushback. Well, fair enough. And he does make some important points in there, chief among which is that none of us need to be debating against straw men. Almost no journalists think the economy irrelevant to elections, just as almost no political scientists think the campaign irrelevant. To that extent, we largely agree.

Nonetheless, Trende goes on to question the value of economic forecast models of elections. He notes several presidential and congressional elections in which political science forecasts were wide of the mark, and suggests that we can't ever really know ahead of time whether we're going into an election in which the classic models will work well or not.

I want to respond a bit to that. First, I don't think it's fair to lump congressional elections into this argument; as any election modeler will concede, those elections turn on local as well as national factors and are much harder to predict. Second, yes, we can certainly cherry-pick some presidential election forecasts that missed, but they still, on the whole, tend to come quite close.

Note this collection of forecasts (PDF) submitted roughly two months prior to the 2008 presidential election. The median of the nine forecasts had McCain getting 48 percent of the two-party vote, just one percent more than he actually got. Seven of the nine forecasts came within three points of the actual vote. The most accurate forecast was made 99 days before the election.

Note also Nyhan and Montgomery's collection of forecast models for elections since 1976. Sure, some individual forecasts miss by quite a bit, but the "ensemble" forecasting model rarely deviates from the outcome by more than a point or two.

Now, my posse and I (yeah, I've got a posse) could keep going back and forth with Trende and his posse like this, but I'm not sure how productive that would be. I think instead it might be more useful to address the question of just what these forecasts bring to our understanding of elections.

Forecasts are certainly entertaining. They can also be lucrative. But their real value for political science is that they allow us to test theories about elections. This is why modelers spend a lot of time "predicting" elections that have already happened, a task that might seem silly to some. We're trying to understand just what drives elections. We have theories about the importance of the economy, even about different measures of the economy. We have theories about ideology, about wars, and other things. We also have theories, as John Sides notes, about how the campaigns take advantage of these features of the political environment and make them matter to voters. When we make a forecast, we're attempting an empirical test of our theories. We're trying to figure out just what matters in elections and how much it matters. And each new election improves our understanding of the fundamentals of elections.

In the comments on my last post on this subject, Jay Cost wrote in to say the following:

I would say that I have learned so much more from the history than from the political science. If somebody asked me "How do I understand the 1968 election?" I'd point them to Ted White's Making of the President before any quantitative study in the APSR.

If we're talking about comparing an in-depth study of campaign actors with a forecast model, well then I totally agree. These models tell us very little about any one given election. However, a thick study like White's will offer quite a few reasons why one candidate won and another lost, from the economy to the Vietnam War to the presence of a divisive third-party candidate. What our forecasting models do is provide us with some explanations of just which aspects of the political environment tend to drive elections and which ones don't.

Okay, enough for now -- I've got to go deal with some Thanksgiving stuff.

Update: Hans Noel has a great post up at the Monkey Cage further explaining how forecast models are used to test theories.

Economic forecast models: the pushback

Nate Silver's 2012 prediction model has produced some interesting responses across the blogosphere. Brendan Nyhan notes that Silver's estimate for the effects of candidate ideology in presidential elections just has to be too big*, and he finds that Silver's model underperforms when compared to others. John Sides responds to Mike Tomasky's dissing of political scientists. Then Alan Abramowitz offers a model that includes an important variable that measures how long the incumbent party has held the White House, and Sean Trende pushes back a bit on that one and on the whole concept of economic forecasting.

I want to respond to what Trende wrote, but I'd first like to point out that it's fantastic that we're even having this debate. Just a few years ago, it seemed like political scientists were tearing their hair out just trying to convince political journalists that elections weren't determined entirely by campaign activities, and that the economy and other fundamental aspects of the political environment might be relevant. Now, the debate among political scientists and journalists appears to be more like, "Come on, the economy doesn't explain everything." So I feel like this discussion has moved in a very good direction in the past few years.

Okay, back to Trende's piece. He ticks off a bunch of things that you have to believe if you're going to accept the validity of an economic forecast model. For example:

First, at a basic level, you have to accept that something as complex as voting can be reduced to a simple, three-variable equation. And you have to accept that this equation is linear.

Well, no, you really don't. Now, we have good evidence that you can explain a very high percentage of what goes on in elections with just two or three variables, but that doesn't mean that everyone's vote choice is a result of just those variables. These models do have error terms. Sometimes other things can affect votes. They just usually don't.

And just because most of these models are linear, that doesn't mean that they have to be. If you get way out in the tails of economic performance, you can see that the effect on votes isn't quite linear. Hoover did better in 1932, and FDR did worse in 1936, than a linear model of economic growth would predict, if for no other reason than that any major party presidential nominee is guaranteed close to 40% of the vote; their hardcore partisan supporters simply won't defect no matter how bad things get. But most elections don't occur under such extreme conditions, and the linear model works quite well for those.

Here's another point Trende makes:

You have to accept that there is no problem predicting the president’s vote share from only 16 data points.

That's silly. Of course there's a problem with that. But that's all the cases we have, and they work pretty well. Indeed, it's pretty amazing we get such robust results from so few cases. And keep in mind that a lot of the truths we cling to in American politics, such as "the president's party loses House seats in midterm elections" or "Democrats lose when they nominate liberal New Englanders," are based on this many cases or fewer.

You have to accept that presidential elections haven’t changed at all over the past 64 years. ... You have to accept that the enfranchisement of African-Americans and poor whites in the South, as well as the enfranchisement of 18-to-21-year-olds nationally, had no effect on the outcome of the later races. A casual glance at the results of the 2008 elections would seem to suggest otherwise.

No one I know is claiming that elections are exactly as they were 64 years ago. But the same basic trends do seem to hold: voters blame the incumbent party when the economy underperforms and reward them when the economy does well, and they tend to turn against parties that have been in power a long time. I don't know why he singles out 2008 as some sort of evidence that these fundamentals have changed. In fact, such forecasts nailed the '08 results within a single percentage point.

You have to accept that anything that happens past the end of the second quarter of an election year matters only at the margin. If the economy absolutely collapses, and a previously popular president goes into Election Day with a 20 percent approval rating amid a full-scale depression, where the economy is contracting by 10 percent a quarter, it wouldn’t matter much. If we are attacked and enter a war, it wouldn’t matter much. If a president becomes mired in scandal, it wouldn’t matter much.

Again, no one argues this. A lot of modelers pick the second quarter as a cut-off because it allows time to make a forecast several months before the election while still capturing much of the economic activity on which the incumbent party will be evaluated. It's very rare that an economy that's humming along at three percent growth for a year or more will suddenly plunge into recession the quarter prior to an election. That certainly can happen (and it kind of did in 2008), but it's a very rare event. Similarly, if the forecast models showed Obama likely to win next year, but he decided to shoot Tom Hanks in the face on live TV on October 31st, yeah, he'd probably lose the election. It's not that last-minute twists don't happen, it's just that they're rare, and the things that happen to the economy in the third quarter of an election year usually look a lot like the things that happened in the first and second quarters.

There are a few other similar points that Trende makes. Some are good caveats for forecasters, and it would generally be good for us to be straightforward about the assumptions of our models. But many of Trende's arguments are simply straw men. Look, we have some models that do a pretty good job explaining elections -- a lot better than claims about campaign quality or candidate optimism or likeability. But past performance does not guarantee future results. Make of these models as you will.


*I hope my post using Silver's numbers to draw out a prediction plot wasn't taken as an endorsement of Silver's model. I simply did it because I thought it would look cool.

Some forecasts

The 2012 presidential election forecaster up at fivethirtyeight.org is great fun, and you could get lost in the numbers for a few hours there. Basically, Nate Silver has made assumptions about the Republican candidates' ideological positions and plugged those into a forecast model, along with growth in gross domestic product and Obama's approval ratings. Just for giggles, I put together the following graph based on the forecasting data used in that model. I have assumed Obama's approval rating remains at 43 percent.

The official forecast for economic growth is 2.7%, by the way, which would put Romney and Obama at a near dead-heat.

Now, here are some reasons not to take these numbers too seriously:

• First, Obama's approval rating is not independent of economic growth. If the economy begins to grow at a quicker clip, his approval rating will probably rise into the 50s. It will likely drop into the 30s or worse if we experience an actual recession.
• Second, even if economic growth stays right where it is today, Obama's approval rating is likely to change somewhat as a function of the campaign. Nearly all Democrats will come to approve of his performance, even if they may have reservations today. And nearly all Republicans will come to disapprove, although they're probably doing that already.
• Third, perceptions of the Republican nominee's ideological stances may well change by next year. It's very hard to make realistic projections of Cain's governing ideology since he's never governed before. Perry would be facing a more liberal electorate than he's ever faced, and Romney would be facing a more conservative one. Plus, given Romney's history, there should be substantially large error bars on either side of his line.

Update: Graph label fixed.

Using economic projections to make vote projections

When I'm asked whether I expect President Obama to get reelected, I often look around to see if there's an economist nearby to tell me how the economy is going to be doing next year. Well, here are some economists now, in the form of USA Today's survey of 39 economists. Unfortunately for my purposes, they don't forecast real disposable income, which is probably the most reliable predictor of presidential elections, but they do use gross domestic product, which is the next best thing. Their average projection of GDP growth from the fourth quarter this year to the third quarter next year is 2.53 percent. How does that stack up with economic growth and electoral performance in previous years?

Well, if we use this metric and the USA Today forecast, Obama will be facing lower economic growth than what Bush faced in '92 and his son faced in '04. That's real nail-biter territory. The regression line suggests Obama getting 50.9 percent of the vote, although, as the graph shows, that line is being pulled upward a bit by Eisenhower's unexpectedly good performance in 1956. If you take out that case, the Obama forecast is 50.03 percent of the vote. In other words, don't expect to get a lot of sleep on election night next year.

All the usual caveats apply here. The forecast for 2012 economic growth is just a prediction -- it could get substantially better or worse. And this is just a bivariate regression model, which doesn't include things like foreign policy or ideological extremism or campaign quality. And, of course, past performance does not necessarily predict the future.

But this could be a really, really close race.

Election 2012: Time to sweat the small stuff

John Sides looks at recent economic growth numbers and polling trends and concludes that "Barack Obama is on the cusp of becoming the underdog in the 2012 election." He's right, although I'll offer a few caveats. First of all, polling done more than a year before an election just doesn't have much predictive value, so I wouldn't place a whole lot of stock in that right now. Second, while the economic growth figures look really bad, Obama will, for the most part, be evaluated on the economic growth that occurs between now and the fall of 2012. That is, Obama will be held responsible for an economy that doesn't yet exist.

What will that economy look like? Well, the forecasts for growth aren't that great, and most of the forecasts we've had over the past few years have turned out to be too optimistic. And the outcome of the debt ceiling crisis will likely be at best neutral for economic growth and quite possibly negative.

All this means that the economy will probably not slip back into a recession in the next year, but economic growth will be anemic. That is, the economy may be slightly better than the one that Jimmy Carter faced during his reelection effort, but not by a whole lot. We could be looking at 1 or 2 percent annual growth.

What does this mean for Obama? Let's look at a scatterplot, shall we?

The horizontal axis above measures growth in real disposable income from the 3rd quarter of the year prior to the election to the 3rd quarter of the election year. (We're currently in the 3rd quarter of the year prior to an election.) For example, real disposable income grew by 1.8 percent prior to the 2004 election, in which George W. Bush received the narrowest reelection margin for a president in U.S. history. So it's possible for an incumbent to win during a time of mediocre economic growth, but the odds aren't great. Incumbents win 2/3rds of the time when they stand for reelection, but in the data shown above, of the six elections where RDI growth was below 2 percent, the incumbent party only won two of those. The record is 2-2 when a sitting president is up for reelection under those circumstances.

Loyal readers of this blog will note that I'm generally unimpressed with the ability of campaigns or candidate personality traits to affect election outcomes. It's not that they have no effect, just that that effect is usually paltry compared to the effects of the economy and wars. But if the economy leads us to predict a 50-50 shot for Obama next year, then smaller effects become all the more important. The ability of Obama's reelection campaign to communicate with potential voters like it did in 2008 could be pivotal, as could the identity of the Republican nominee.

More on unemployment and presidential elections

Carlisle Rainey, a poli sci and statistics graduate student at Florida State, has written a response to my earlier post in which I argued that there was no relationship between unemployment levels and presidential election results. Carlisle's post is definitely worth a read. His main objection to my argument is that there are only 16 elections since WWII on which to draw conclusions. It is possible, he says, that there is an important relationship between unemployment levels and election outcomes, but we just can't detect it because we have so few observations.

This is certainly a valid concern. Unfortunately, those are the data we have. Now, we can look to other elections, as well -- notably, unemployment levels don't seem to affect congressional elections, either -- but if our concern is specifically over presidential elections, we're limited to very few cases.

And perhaps it's incumbent on me to revise, or at least recast, what I said a bit. I wasn't so much trying to argue that unemployment has nothing to do with elections as I was trying to call out journalists who seem to think unemployment has everything to do with elections. If you're going to argue that Obama is in trouble because the unemployment rate will be above 7.5% next year, you really need to deal with the fact that some very easily obtainable data do not support that claim. Reagan won in an historic landslide during a time of very high unemployment, while the Democrats lost control of the White House in 1952 during the lowest unemployment on record.

Interestingly, Carlisle drills down into the data a bit more, finding an important trend if you isolate just those elections in which the president had been previously elected:

Nice catch. And there's a plausible story there, suggesting that voters hold incumbents accountable for unemployment rates but not necessarily parties. (The trend would still hold if you counted LBJ '64 and Ford '76 as incumbents.) My one concern would be that if 16 elections are too few to make good inferences, we should be even more concerned about seven. Also, as Brendan Nyhan points out, we have plenty of measures, like the growth in real disposable income, that explain elections quite well whether or not there's an incumbent running.

All in all, Carlisle has written a thoughtful post about the use of statistics in elections. I look forward to reading more on his blog.

Defending economic forecast models

I wanted to elaborate a bit on my reaction to Nate Silver's recent post on the limits of economic forecasts of elections. Silver's post is a good one. He explains one of the better forecasting models out there, Hibbs' bread-and-peace model, which explains something like 90 percent of the variance in elections just by examining real disposable income growth and troop fatalities in American military engagements. But then he notes that the model doesn't do a great job predicting out-of-sample elections. Silver produces this nice chart showing the difference between the model's forecasts and the actual presidential election results:

It's really not a shock that a model would do somewhat less well predicting out-of-sample results. After all, the model was based on only 10 elections -- if we add the data from more recent elections, we get a better model. And really, the model does just fine in post-sample elections. The one marginally large miss is 2000, for which the model over-estimated Gore's vote share by about five points. And just about all forecast models made similar errors with that election. We could chalk this up to poor campaigning skills by Gore, but really, should it be that much of a shock that the party whose president was impeached would underperform in the election a year later?

Where the model really falls down, though, is in the 1930s and 40s. There are some very good reasons why most economic forecast models of presidential elections begin with 1948:

1. World War II was not really politics as usual in America. Criticism of the incumbent president was rather muted on the most central issue in American life for two election cycles, and troop deaths were atypically high in an atypically popular military engagement. 
2. The elections of the 30s and 40s encompass the Great Depression, during which economic growth vacillated in fantastic and horrifying ways. Real per capita disposable income dropped by 24 percent in 1932! It grew by 26 percent in 1942! It turns out that voters' partisanship is somewhat robust to these shifts; even when Americans have lost a quarter of their income in a single year under Republican leadership, 40 percent of the country is still willing to vote Republican.  We shouldn't abandon a forecast model for falling down in such extreme circumstances.
3. Prior to FDR's presidency, the federal government simply had a smaller role in influencing economic booms and busts. It wouldn't be shocking if voters had different expectations with regards to the government back then and voted accordingly.

I don't want to overly apologize for all errors in a model. These really are based on very few data points, and occasionally they'll get it wrong. But Silver makes a huge jump when he concludes, "It’s the economy, stupid. And everything else too." If you were to construct a forecast model of elections based on charisma, optimism, some the-candidate-you-want-to-have-a-beer-with variable, campaigning skills, or even unemployment, you would be catastrophically wrong a good deal of the time. You'd probably do no better than a basic coin-toss. Income growth, unpopular wars, and moderation/extremism really do affect votes on a large scale, while most other stuff really doesn't, despite the quantity of ink spilled over it.

Unemployment and presidential elections

Binyamin Appelbaum, in today's New York Times:

No American president since Franklin Delano Roosevelt has won a second term in office when the unemployment rate on Election Day topped 7.2 percent. Seventeen months before the next election, it is increasingly clear that President Obama must defy that trend to keep his job.

I feel a scatterplot coming on...

Well, let's for a moment ignore the obvious problem with Appelbaum's claim: Reagan won reelection in a landslide with annual unemployment at 7.5 percent. (I'm assuming Appelbaum is working with monthly or quarterly data or something.) The fact is, as the above scatterplot demonstrates, the unemployment rate does not predict presidential elections at all. The Democrats failed to hold the White House in 1952 during the lowest unemployment on record. Parties have both lost and retained the White House during periods of high unemployment. And the biggest reelection margins have occurred with unemployment between five and six percent -- right around the middle of its historic range.

What does matter a great deal is growth in real disposable personal income. As Harry Joe Enten (via Brendan Nyhan) shows, that's actually not looking particularly good for Obama right now:

According to the BEA's prior April report, RDPI grew at 1.8% in the fourth quarter of 2010 over the preceding quarter and 2.9% in the first quarter of 2011. Last Friday, the BEA re-adjusted those numbers to 1.1% and 0.8%.

Of course, what will matter far more for Obama's reelection prospects is how personal income grows between now and next summer. But the nation's unemployment status by itself is not going to affect Obama's.