Friday, January 22, 2010

They can't both be right


Okay, this is a technical post, but I don't know the answer.  Keith Poole uses the NOMINATE program to turn roll call votes into ideal points.  Simon Jackman uses some sort of Bayesian analysis to perform the same task.  Usually, the two methods yield pretty similar results.

Yet Poole's approach finds that Russ Feingold of Wisconsin is the most liberal Democrat in the U.S. Senate, while Jackman's approach finds that Feingold is the fourth most conservative Democrat, sandwiched between McCaskill and Webb.

What's up with that?

7 comments:

bshor said...

Yah, it does sound strange to call Feingold a conservative Democrat. But, where are you looking for NOMINATE scores? Poole's website only has scores through the 110th. And one more thing: Jackman's score is just for the 111th. So the closest analogue would be W-NOMINATE which is Congress-by-Congress.

Seth Masket said...

Actually, Poole and Rosenthal's website has common space DW-NOMINATE scores going through the 111th now.

Will Bullock said...

I don't think it's a NOMINATE v. Bayesian ideal point (CJR) issue. It seems as though Feingold has actually changed his voting considerably from the 110th to the 111th.

In the 111th Congress he's most likely the 4th most conservative Democrat. In the 110th Congress he was around the 25th most conservative Democrat.

Here's the code in R to duplicate my analysis:

library(pscl)

s111 <- readKH("http://adric.sscnet.ucla.edu/rollcall/static/S111.ord")
id.s111 <- ideal(s111, maxiter=100000, burnin=80020, thin=20, verbose=T)

s110 <- readKH("ftp://voteview.com/sen110kh.ord", desc="110th U.S. Senate")
id.s110 <- ideal(s110, maxiter=100000, burnin=80020, thin=20, verbose=T)

summary(id.s111)
summary(id.s110)

Seth Masket said...

No, near as I can tell, Poole and McCarty are both using the exact same roll call data (111th Congress) and coming up with staggeringly different results for Feingold.

Will Bullock said...

So when I go to http://voteview.ucsd.edu/sen111.htm
and then scroll down, I see Feingold ranked as the 4th most conservative Dem, just as with the CJR estimates.

Where do you see them ranking Feingold as the most liberal Dem?

bshor said...

Leave it to Poole to come up with NOMINATE in the 80s, and have a web site design from then, too!

And there's actually an explanation for Feingold's change in ideology here ...

http://voteview.ucsd.edu/feingold.htm

Seth Masket said...

Good catch, Boris. I found the common space DW-NOMINATE scores here in Stata format, showing him as the most liberal. But those scores are constant across sessions. So apparently Feingold is just a weird one, casting so many unconventional votes that our methods don't do such a hot job describing him.