Hans Noel and I try to remedy this in a new article (gated or ungated, PDF) in Political Research Quarterly. Our approach is to examine votes on legislative referenda in California. That's when state legislators vote on a bill and then send the exact same text onto the voters -- it's one of the only times that voters and politicians are voting on precisely the same thing. We use these referenda votes as "bridges" between legislators and their districts. Then we can put legislators' roll call votes and districts' votes on initiatives onto the same scale and generate comparable ideal points for districts and legislators.
How do they look? Well, it looks like legislators are a lot more polarized than their districts. There are actually plenty of moderate Assembly districts in California; there are basically no moderate Assembly members. Virtually every Democrat in the Assembly is more liberal than her district; virtually every Republican member is more conservative than her district.
We also find that members of the majority party tend to deviate further from their districts than members of the minority party do. Time out of office, we suggest, causes the minority party to try to moderate to win back the majority.