Saturday, September 20, 2008

The cellphone bias

There's increasing concern that the massive use of cellphones is biasing polling numbers. Some younger voters, in particular, don't even have land lines, and such voters are sure to be missed by polling organizations that don't make an effort to include cell phone numbers in their samples. In the past few elections, such omissions have been considered negligible. It's not that the samples aren't biased; they just don't seem to be biased in a politically meaningful way. Roughly as many Democrats as Republicans have been omitted.

This may not be the case this year (perhaps because the electorate is suddenly so polarized by age). Nate Silver at Fivethirtyeight did a little study comparing the polling organizations that use cell phone numbers with those that do not. It's a small-N study, but it's still pretty suggestive:
Six of the seven cellphone-friendly pollsters have had a Democratic (Obama) lean, and in several cases it has been substantial. On average, they had a house effect of Obama +2.8. By comparison, the control group had essentially zero house effect, so this would imply that including a cellphone sample improves Obama's numbers by 2.8 points. (Or, framed more properly, failing to include cellphones hurts Obama's numbers by 2-3 points).

The difference is statistically significant at the 95 percent confidence level. Perhaps not coincidentally, Gallup, Pew and ABC/WaPo have each found a cellphone effect of between 1-3 points when they have conducted experiments involving polling with and without a cellphone supplement.
Just to be clear: these findings suggest that those polling firms that do not survey cell phone users are understating Obama's true support by 2-3 points. This is a huge difference in a close election like this one. In Silver's state projections, he finds that accounting for cellphones moves Florida from lean-McCain to tossup, and moves Nevada from tossup to lean-Obama.


Eric Rubin said...

i read all these polling numbers and i continually ask myself: why have i never been polled? is there some special place you have to live to be a part of the polling numbers? Who are these polls polling?

I've been a voter for 14 years and not once have i been asked who i was going to vote for. is it because i never answer the 1-800 numbers? essplainittome...

Seth said...

The odds of any one voter being contacted are pretty remote, actually. Say there are 100 polls in a race. They each contact about 1,000 people. Even if they each contacted totally different people, that's only 100,000 people out of 200,000,000 registered voters: a 1 in 2,000 chance of being contacted. Also, they might have called you, but you weren't home and they wouldn't bother to leave a message. Or maybe you wouldn't pick up a number that you didn't recognize.

If you really want to be polled, be home every evening between 5 and 8pm and answer every call you get on your land line. But even that will only improve your odds slightly.

Eric Rubin said...

thanks. now i know and knowing is half the battle!