Wednesday, January 20, 2010

How did the pollsters do?

I've heard a few folks remarking that one lesson from the Massachusetts Senate election is that pollsters actually did pretty well.  Despite all the difficulties in modeling a special election population, they came pretty close.  Did they?  Here are the major polls since Jan. 4th, as compiled by 538:

These polls predicted the winner would get 50.2 percent of the vote, which isn't far off from what he got.  But look at the variation.  The standard deviation of the estimates of Brown's vote share is 5.16.  Compare this to the final two weeks of polling in the Obama-McCain contest in '08, as compiled by RCP:


The Obama vote predictions there had a standard deviation of 1.46, roughly a fourth of the standard deviation in the MA race.

However, this is an unfair comparison -- there actually was some movement in the MA race, whereas the presidential race was pretty stable in the final weeks.  Actually, if you limit the MA polling to just those surveys conducted since Jan. 15th, the standard deviation of the Brown estimate is 1.54.  So that's actually pretty impressive.

No comments: