Not "scientifically valid"

Sounds like some of the folks at the Douglas County (Colorado) School District could use a statistics refresher course. Writes the Denver Post:

The Douglas County School District has deemed its spring survey of parents "inconclusive" — a poll in which a majority of responding parents saw the district's suspended voucher program as "unfavorable" and expressed unhappiness with the district's overall direction.

District officials contend that not enough parents participated in the survey to make it a valid representation. As a result, they stamped every page of survey results posted on the district's website in red letters saying: "Inconclusive due to insufficient response rate."

More than 4,900 parents completed the survey. There are approximately 76,500 parents whose children attend Douglas County schools. 

[...]

District spokesman Randy Barber said the approximately 6 percent of parents who responded to the survey this spring was significantly under the 30 percent the district wanted to make it "scientifically valid."

Yes, people usually have more confidence in a survey if it has a higher response rate. But that doesn't mean that the results of this survey aren't valid, and there's nothing magic or "scientific" about the 30 percent threshold. We can conduct very accurate national surveys based on just 1,000 responses -- roughly a fifth of what they managed in this parent survey. More important than overall numbers is representativeness. That is, does the sample of parents who responded to the survey look roughly like the overall population within the school district? This can probably be figured out using demographic questions (although the only one I can find in the survey is just a race question. Area of residence, income, family size, ideology, etc., would be really helpful here.) 


Of course, maybe the reason that the district rejected the results as "inconclusive" has less to do with sample size than it does with the results, which were less than flattering for the district. If they really want to know what district parents think, they could conduct a survey of a representative (and probably smaller) sample and get solid results. But if they just want certain results, then by all means they should just keep doing surveys until they get the answers they want.

The foie gras lobby

On July 1st, the production and sale of foie gras will become illegal in California, a result of a state law passed back in 2004. Ed Leibowitz at The Atlantic has an entertaining story about some of the lobbying surrounding this new law. According to the article, over 100 chefs are now lobbying for the reversal of the law, and they have some effective talking points:

“Foie gras is low-hanging fruit,” [Chef Josiah] Citrin* says, in the resigned tone of someone explaining the obvious. “You think the foie gras industry has money to fight, like the beef industry?” He points out that a class barrier also keeps voters from rallying in defense of foie. “You go out in the street and ask 25 people ‘What do you think about fattened duck liver?’ and they’ll say ‘Ooh, I don’t like that.’ You don’t have to take a poll.”

Citrin has joined a coalition of more than 100 chefs lobbying for the reversal or suspension of the foie gras ban. (The coalition, which insists that it does not oppose animal rights, says it favors the humane treatment of all livestock, waterfowl included.) In a few days, many of the chefs will travel to Sacramento to lobby on foie’s behalf, and in the weeks ahead, high-end restaurants will hold foie-filled dinners to raise funds for their quixotic fight.

On the other side of the issue is the ever-quotable John Burton, the one-time lion of the state legislature and now chair of the state Democratic party:

The chefs’ coalition has warned about the ban’s potential impact on California’s high-end restaurants in a bad economy, and the state’s diminished standing in the world of haute cuisine. “California will no longer be a food destination?,” Burton said. “In other words, a guy’s sitting around and says ‘Let’s go to California. They’ve got these beautiful views. They’ve got Yosemite, the bridges, Universal City, the redwoods. Oh, shit! They don’t have foie gras! Let’s go to South Dakota.’”

Nor did he buy the argument that a restaurant could go broke without foie gras, unless that restaurant’s specialty was incredibly narrow. “If you had the House of Foie Gras, you’d be fucked,” he said.

Anyway, a great story that manages to humanize both sides of a lobbying battle, over an issue on which the vast majority of people probably don't have strong opinions and will never be affected.

*Disclosure: Citrin is a family friend.

Networks

Light posting here this week, as I'm attending the Political Networks conference in Boulder. You can follow all the action at the Political Networks Twitter feed, or on the hashtag #polnet2012.

Zombie fundraising meme

Last week, Politico Buzzfeed ran a story claiming that Obama is having fundraising problems, noting that many of his 2008 donors haven't given him any money this year. (Here was my response to that.) Today, it followed up with an analysis of where Obama's fundraising shortfall relative to 2008 has been greatest. For the record, I was actually interviewed for this story, and I explained to the reporter (Rebecca Elliott) that I didn't think there was any there there. That is, Obama hasn't received donations this year from a lot of his 2008 donors for one main reason: he didn't face a primary challenger this year, while he was in one of the most competitive presidential nomination races in modern history four years ago. We had a good chat about this, but nothing along these lines made its way into the article.

The new article, meanwhile, starts from the premise that Obama is suffering, and tries to explain why his fundraising shortfall relative to 2008 has been greater in some states than in others. But again, I just don't think there's much here. As Elliott reports, 88% of Obama's 2008 donors nationwide have yet to contribute to him this year, and when you break those figures down by state, the highest shortfall is in Oregon at 91%. Now, if the mean is 88% and the highest value is 91%, it just doesn't sound like we're talking about a whole lot of variation here. The political scientists quoted in the story do a valiant job coming up with some reasons why Western state donors would experience higher dropoff, but it just doesn't sound like there's much of a phenomenon to explain here.

I understand the desire to run stories about how Obama is having a harder time this year than he did in 2008, and that is certainly true in many measurable ways. And who knows -- maybe we'll ultimately find that he did have a hard time raising money. But drawing these comparisons between a campaign with a well-funded opponent and a campaign with no opponent is misleading, and terribly, terribly frustrating.

Update: I inaccurately claimed that the above stories were from Politico. Rather, they were from Buzzfeed.

What the President needs is a good bleeding

I so love 19th century medicine.

The first doctor to reach President Abraham Lincoln after he was shot in a Washington theater rushed to his ceremonial box and found him paralyzed, comatose and leaning against his wife. Dr. Charles Leale ordered brandy and water to be brought immediately.

Brandy -- good for what ails ya! Like massive head trauma!

Farewell to Ed Quillen

The Denver Post's Ed Quillen has passed away at the age of 61. I must admit I was rather a fan of his columns. He came off as kind of a lefty mountain crank, but his columns were cogent, well-written, and well thought out. Note this one, in which he compares modern day Republicans with the Confederates of 1860. I don't necessarily subscribe to everything he wrote, but it's a nice observation about the persistence of ideology, even if party labels may jump around a bit.

Rest in peace, Ed.

Research: It's so important that someone else should pay for it

If you're interested in the ongoing saga over NSF funding for political science, please do not miss Charles Lane's op/ed in yesterday's Washington Post. Lane starts by taking on Christopher Zorn's post at the Monkey Cage, in which Zorn raised concerns about the politicization of NSF funding decisions. Then Lane goes on to make some odd economic claims, such as:

The relevant question... is whether society could have reaped equal or greater benefits through other uses of the money — and how unreasonable it would be to ask the political scientists to rely on non-federal support. [...] 

If this research is as valuable as its proponents say, someone other than the U.S. Treasury will pay for it.

That last sentence is pretty astounding. Swap out the words "this research" for "the study of cancer" or "national defense" just to get a sense of it. Just because something is important does not automatically mean it is popular or well-funded. Strangely, Lane seems to concede as much just two paragraphs later:

The private sector chronically underinvests in basic scientific research; the costs and risks are relatively high, and the benefits relatively hard to commercialize. Government support compensates for this “market failure,” enabling society to reap “positive externalities” — economic, environmental or military.

Um, yeah! That's just what I was saying! But Lane thinks this logic only applies to the "hard" sciences, not the social sciences:

Though quantitative methods may rule economics, political science and psychology, these disciplines can never achieve the objectivity of the natural sciences. Those who study social behavior — or fund studies of it — are inevitably influenced by value judgments, left, right and center. And unlike hypotheses in the hard sciences, hypotheses about society usually can’t be proven or disproven by experimentation. Society is not a laboratory.

Wow. Okay, last point first: Of course we can use experiments to test claims about society! Political psychologists, among others, do this all the time. Political scientists also use natural experiments all the time. Here's one: Nebraska and Kansas have very similar populations but different institutional rules for their state legislatures, and this has important effects on legislative partisanship. Yes, society can be a laboratory.

Now, as for Lane's other points in there, let's just pretend for a moment that those who study the hard sciences are not influenced by value judgments, such as desires to cure cancer, to make fusion energy cheaply available, to prove or disprove human-made climate change, etc. Are social scientists influenced by value judgments? Well, I suppose we'd need to define the word "influenced." Their political beliefs probably cause them to find certain questions interesting and to spend time researching them as opposed to other questions. So I suppose that's a form of influence. But that's probably not what Lane is saying. Rather, he seems to be suggesting that our political judgments cloud our results.

So here's a challenge for Lane: Please browse through the most recent edition of the American Political Science Review, the American Journal of Political Science, the Journal of Politics, or any of the other major political science journals and show me where scholarship has been compromised by the scholar's ideological judgments. If you don't have access to these articles, just let me know and I'll send them to you. Hell, you can find most of my publications here: show me where my findings have been influenced by my value judgments.

The question of whether society should be subsidizing research about politics is an interesting one, and while I certainly have my opinions, I welcome debate on the topic. But the idea that social scientists can't do research without being clouded by political judgments and that this makes our research inferior to that of the other sciences is, frankly, offensive.

(Cross-posted from Mischiefs of Faction)

Obama's suffering because Hillary Clinton didn't challenge him in the primaries

I think that's what Politico is arguing here. It's hard to tell. Ben Smith and Rebecca Elliott have written another in the long line of articles alleging that Obama's slow fundraising pace in 2012 relative to the pace in 2008 is evidence of a serious problem for Obama. And like the other articles, it fails to note that Obama didn't face a primary challenger in 2012. It just offers a whole bunch of other possible narratives -- donors are hurting financially, Obama's big supporters from four years ago are disappointed in him, the thrill is gone, etc. And while all of this may be true, it doesn't change the biggest difference, which I guess I'll italicize yet again: Obama didn't face a primary challenger in 2012. The major reason Obama hasn't raised the kind of money in the spring of 2012 that he raised in 2008 is because he hasn't needed it.

(via Jamelle Bouie)

The Queen's got a long way to go

The new Queen meets with
President Harry Truman

I'm not sure whether it would be appropriate for me, as an American, to bow to Queen Elizabeth II on the occasion of her 60th anniversary jubilee, but I'll at least tip my hat to her. Sixty years in office... that has to be some kind of record, right?

Actually, if you check out Wikipedia's list of longest reigning monarchs, she doesn't even crack the top fifty, and that's just the list of verifiable ones. Sobhuza II of Swaziland served 82 years. The Egyptian pharaoh Pepi II allegedly served 94 years. Now, obviously it helps to have taken the throne as a child, and Elizabeth was already 25 when she became queen. That'll make it tough to crack the top tier without resorting to cryogenics.

New Media and the 2012 Presidential Campaign

On May 11th, the University of Denver hosted a panel on new media and its effects on the 2012 presidential election. The panelists were Brent Blackaby (Trilogy Consulting), Jay Newton-Small (Time), Dorian Warren (Columbia University), and Dave Weigel (Slate/MSNBC). Professors Nancy Wadsworth, Peter Hanson, and I moderated.

The discussion was wide-ranging, covering the use of social media sites by campaigns as well as the impact of such media on journalism, voters, and candidates. I found the discussion fascinating, but don't just take my word for it. You can watch the video here.