Yesterday’s post on gun ownership and state terror tackled an important issue for anyone interested in restricting firearms access in America: does widespread gun ownership help to protect citizens from tyranny? The simple cross-national evidence that I presented was not very encouraging. The U.S. has by far the most extensive private gun ownership of any country in the world, and scores quite well on most metrics of freedom from state terror. But the U.S. does not score as high as most proponents of the freedom-from-tyranny defense might think, and there are reasons to expect that regime type and economic development (rather than gun ownership) are responsible for our relative freedom.

Today I want to push further on what precisely we can conclude from those results. This is a post about what we should conclude about policy options when we confront a bunch of partial correlations that have no clear causal interpretation.

Those results are not identified

What do I mean by “no clear causal interpretation”? Just this: What I have shown is that there is a positive but statistically insignificant partial correlation between gun ownership and freedom from state terror. But this has not identified a causal effect of gun ownership on state terror. (Unfamiliar with these terms? Check out my quick-and-dirty intro to identification for non-specialists.) For empirical social scientists, this is basic stuff, but it’s worth recalling just what the threats to causal identification are.

1. We want to know if Gun Ownership affects Freedom from State Terror: GO → FST. A correlation between the two might mean that GO → FST, or that FST → GO. That is, countries like Tunisia (until 2011) that like to terrorize their citizens might also restrict gun ownership.
2. I find that democracies are less likely to terrorize their citizens, which I interpret as D  → FST. But they are also probably less likely to restrict gun ownership rights (see above). So might be the case that D → FST and also that D  → GO  → FST. This means that it is extremely difficult to isolate the independent causal effect of GO on FST, even using what an unnamed friend likes to call “super-duper regression.” And, also, to interpret  D  → FST as a causal effect. This is related to the problem of post-treatment bias, one of the hard unsolved problems in the social sciences.
3. I really don’t know why countries restrict gun ownership. I don’t know why GO has the values that it has. Maybe there’s a lurker in there, some other variable W that causes both GO and FST (and maybe D too). Maybe W is “culture.” Maybe culture, on average, explains why some countries suffer from more state tyranny and why they embrace widespread gun ownership. If so, it could be the case that gun ownership does protect citizens from state tyranny, but that the places with the most state tyranny (for cultural reasons) also happen to have higher gun ownership rates (for cultural reasons), so we’re missing the fact that these countries would be even more tyrannical if it weren’t for the guns that they do have.
4. Non-classical measurement error. I bet that the data on private gun ownership in the U.S. are pretty good, and the data for lots of poor dictatorships (where gun ownership is probably a lot lower) are pretty bad. The lack of a partial correlation between GO and FST when controlling for regime type might just reflect that we are really miscounting the guns in the places where ownership is likely to be low and where FST is likely to be high.

There are probably lots of other possible ways to interpret the partial correlation between GO and FST. (Interested readers, leave ’em in the comments!) The point is, my results cannot be viewed as identifying anything like a causal effect.

Give up?

If I were writing an article for a good social science journal, I’d probably stop right here and abandon the project. Thankfully, we have eliminated some of the numerology from quantitative social science in the past two decades, meaning that we cannot wave our magic interpretive wand over a regression table to reach our preferred conclusion. If you want to claim to have identified “the effect of” gun ownership on freedom from state terror, partial correlations will no longer suffice.

But we still learn policy-relevant things from these results even if they do not identify a causal relationship. The first point is to remember that the question of interest is not the average causal effect of gun ownership on state terror (which, for better or for worse, as become the question of interest for quantitative social science research). Instead, our policy question is more squishy: does such widespread gun ownership protect American citizens from tyranny? Here is what we have learned even without an estimate of a causal effect.

1. American citizens aren’t as protected from state terror as we might think.
2. Plenty of countries rate as highly (or more highly) than the U.S. with lower levels of gun ownership.
3. Plenty of countries with lower levels of gun ownership experience far more state terror with lower levels of gun ownership.
4. The partial correlation between gun ownership and state terror disappears when you take regime type and economic development into account.

All of these data are hard to square with the idea that the ubiquity of firearms in the U.S. is protecting Americans from state terror. We can construct a theoretical world in which gun ownership at the levels that we see in the United States today is protecting us from tyranny, but that theoretical world must have a lot of curious features to it to also produce the results from yesterday. The probability that that world is the “true” world is lower conditional on the data and results shown yesterday than it would be if (1) the results showed a consistent partial correlation between gun ownership and freedom from state terror in the multivariate models or (2) there were no countries that had low tyranny scores along with low gun ownership rates.

As a matter of policy, there are lots of reasons why one might oppose tighter gun laws. Defending against state terror is unlikely to be one of them.

Sidebar: probably no quasi-experimental fix

Readers working in North American social science departments are probably thinking that it would be great to have an instrument for gun ownership. But these readers may not realize that even a quasi-experiment would probably not produce the policy-relevant treatment effect for the U.S. case.

Private gun ownership is a continuous variable with no zero, so we have to think hard about what the relevant treatment effect even is. But say that we could imagine something like a dose-response model where we are estimating different treatment effects τ for various treatment intensities. So assume, impossibly, that one of the policy option is to have no private gun ownership at all. We might then hypothesize J treatment states where j = 0, 1, … 88.8. For each j our treatment effect is the difference in state terror with that treatment and without it: τ_j = FST_j − FST₀. We want to make inferences about the difference in the treatment effects for the highest treatment intensity versus some lower treatment intensity: is τ_88.8 greater than, say, τ₅₀? We might estimate a local average treatment effect, but the U.S. case is so far out at the very end of the distribution, so much so that I can see no way to estimate what τ_88.8 is in order to compare it to any other τ. I have a difficult time reconciling any β_I.V. with the statement “if we lower U.S. gun ownership from 88.8 guns per 100 people to j we can anticipate a thus-and-such effect on state terror.” I would be curious to read an alternative interpretation of how a quasi-experiment could be interpreted along these lines.

Even then, the local average treatment effect is not the policy-relevant treatment effect unless our instrument is an implementable policy. And any instrument for cross-national variation in gun ownership is almost certainly not an implementable policy. This just means that even if we had a good quasi-experiment, we’d still need to be careful about listing the assumptions necessary to draw policy conclusions from it.

Conclusion

These two posts have covered a lot of territory. Yesterday’s post was about the data on gun ownership and state terror, and today’s was about how to interpret these data. And even without identifying causal effects, I conclude that there is little reason that policy planners should oppose reforms to America’s very liberal gun laws for fear of unleashing the tyranny of the state. I hope you conclude the same, and if not…that’s what the comments are for!