John is wise to point out that there are probably other important variables to consider here, and also that the relationship described in these plots is descriptive, not necessarily causal. The point is well taken, but let's just suppose for a moment that this relationship is causal, and that greater unionization leads to greater public debt load. Just how shocking should this revelation be?
Not very. The main benefits that union membership conveys are higher pay and improved working conditions, both of which cost the employer (in this case, the public) more. So if a greater percentage of your state's work force is unionized, you're probably going to be paying more for the labor. And since labor costs are usually contractual and not easily cut in the short run, states with greater labor costs will go into debt more quickly when revenues take a dive. This shouldn't be particularly controversial or surprising.
What the evidence above doesn't show is what citizens get by paying more for public labor. With higher pay, you can usually attract better trained workers, turnover will be lower, services will be of higher quality, etc. (By the way, the two states with the lowest public sector unionization rates -- Louisiana and Mississippi -- have the highest corruption rates.)
Sometimes good things -- even when they come from the government -- cost more. A country that devotes 1% of its budget to defense will probably have a lower debt load than one that devotes 20%, and a country that slashes education spending will no doubt see less red on its balance sheets than one that doesn't. But where would you rather raise your kids?