Category: Current Affairs

Yesterday’s post on gun ownership and state terror tackled an important issue for anyone interested in restricting firearms access in America: does widespread gun ownership help to protect citizens from tyranny? The simple cross-national evidence that I presented was not very encouraging. The U.S. has by far the most extensive private gun ownership of any country in the world, and scores quite well on most metrics of freedom from state terror. But the U.S. does not score as high as most proponents of the freedom-from-tyranny defense might think, and there are reasons to expect that regime type and economic development (rather than gun ownership) are responsible for our relative freedom.

Today I want to push further on what precisely we can conclude from those results. This is a post about what we should conclude about policy options when we confront a bunch of partial correlations that have no clear causal interpretation.

Those results are not identified

What do I mean by “no clear causal interpretation”? Just this: What I have shown is that there is a positive but statistically insignificant partial correlation between gun ownership and freedom from state terror. But this has not identified a causal effect of gun ownership on state terror. (Unfamiliar with these terms? Check out my quick-and-dirty intro to identification for non-specialists.) For empirical social scientists, this is basic stuff, but it’s worth recalling just what the threats to causal identification are.

1. We want to know if Gun Ownership affects Freedom from State Terror: GO → FST. A correlation between the two might mean that GO → FST, or that FST → GO. That is, countries like Tunisia (until 2011) that like to terrorize their citizens might also restrict gun ownership.
2. I find that democracies are less likely to terrorize their citizens, which I interpret as D  → FST. But they are also probably less likely to restrict gun ownership rights (see above). So might be the case that D → FST and also that D  → GO  → FST. This means that it is extremely difficult to isolate the independent causal effect of GO on FST, even using what an unnamed friend likes to call “super-duper regression.” And, also, to interpret  D  → FST as a causal effect. This is related to the problem of post-treatment bias, one of the hard unsolved problems in the social sciences.
3. I really don’t know why countries restrict gun ownership. I don’t know why GO has the values that it has. Maybe there’s a lurker in there, some other variable W that causes both GO and FST (and maybe D too). Maybe W is “culture.” Maybe culture, on average, explains why some countries suffer from more state tyranny and why they embrace widespread gun ownership. If so, it could be the case that gun ownership does protect citizens from state tyranny, but that the places with the most state tyranny (for cultural reasons) also happen to have higher gun ownership rates (for cultural reasons), so we’re missing the fact that these countries would be even more tyrannical if it weren’t for the guns that they do have.
4. Non-classical measurement error. I bet that the data on private gun ownership in the U.S. are pretty good, and the data for lots of poor dictatorships (where gun ownership is probably a lot lower) are pretty bad. The lack of a partial correlation between GO and FST when controlling for regime type might just reflect that we are really miscounting the guns in the places where ownership is likely to be low and where FST is likely to be high.

There are probably lots of other possible ways to interpret the partial correlation between GO and FST. (Interested readers, leave ’em in the comments!) The point is, my results cannot be viewed as identifying anything like a causal effect.

Give up?

If I were writing an article for a good social science journal, I’d probably stop right here and abandon the project. Thankfully, we have eliminated some of the numerology from quantitative social science in the past two decades, meaning that we cannot wave our magic interpretive wand over a regression table to reach our preferred conclusion. If you want to claim to have identified “the effect of” gun ownership on freedom from state terror, partial correlations will no longer suffice.

But we still learn policy-relevant things from these results even if they do not identify a causal relationship. The first point is to remember that the question of interest is not the average causal effect of gun ownership on state terror (which, for better or for worse, as become the question of interest for quantitative social science research). Instead, our policy question is more squishy: does such widespread gun ownership protect American citizens from tyranny? Here is what we have learned even without an estimate of a causal effect.

1. American citizens aren’t as protected from state terror as we might think.
2. Plenty of countries rate as highly (or more highly) than the U.S. with lower levels of gun ownership.
3. Plenty of countries with lower levels of gun ownership experience far more state terror with lower levels of gun ownership.
4. The partial correlation between gun ownership and state terror disappears when you take regime type and economic development into account.

All of these data are hard to square with the idea that the ubiquity of firearms in the U.S. is protecting Americans from state terror. We can construct a theoretical world in which gun ownership at the levels that we see in the United States today is protecting us from tyranny, but that theoretical world must have a lot of curious features to it to also produce the results from yesterday. The probability that that world is the “true” world is lower conditional on the data and results shown yesterday than it would be if (1) the results showed a consistent partial correlation between gun ownership and freedom from state terror in the multivariate models or (2) there were no countries that had low tyranny scores along with low gun ownership rates.

As a matter of policy, there are lots of reasons why one might oppose tighter gun laws. Defending against state terror is unlikely to be one of them.

Sidebar: probably no quasi-experimental fix

Readers working in North American social science departments are probably thinking that it would be great to have an instrument for gun ownership. But these readers may not realize that even a quasi-experiment would probably not produce the policy-relevant treatment effect for the U.S. case.

Private gun ownership is a continuous variable with no zero, so we have to think hard about what the relevant treatment effect even is. But say that we could imagine something like a dose-response model where we are estimating different treatment effects τ for various treatment intensities. So assume, impossibly, that one of the policy option is to have no private gun ownership at all. We might then hypothesize J treatment states where j = 0, 1, … 88.8. For each j our treatment effect is the difference in state terror with that treatment and without it: τ_j = FST_j − FST₀. We want to make inferences about the difference in the treatment effects for the highest treatment intensity versus some lower treatment intensity: is τ_88.8 greater than, say, τ₅₀? We might estimate a local average treatment effect, but the U.S. case is so far out at the very end of the distribution, so much so that I can see no way to estimate what τ_88.8 is in order to compare it to any other τ. I have a difficult time reconciling any β_I.V. with the statement “if we lower U.S. gun ownership from 88.8 guns per 100 people to j we can anticipate a thus-and-such effect on state terror.” I would be curious to read an alternative interpretation of how a quasi-experiment could be interpreted along these lines.

Even then, the local average treatment effect is not the policy-relevant treatment effect unless our instrument is an implementable policy. And any instrument for cross-national variation in gun ownership is almost certainly not an implementable policy. This just means that even if we had a good quasi-experiment, we’d still need to be careful about listing the assumptions necessary to draw policy conclusions from it.

Conclusion

These two posts have covered a lot of territory. Yesterday’s post was about the data on gun ownership and state terror, and today’s was about how to interpret these data. And even without identifying causal effects, I conclude that there is little reason that policy planners should oppose reforms to America’s very liberal gun laws for fear of unleashing the tyranny of the state. I hope you conclude the same, and if not…that’s what the comments are for!

Like many Americans, I am saddened and sickened by the recent shootings in Aurora and Oak Creek. Like many Americans, I have also been discussing the prevalence of firearms in the United States. There are more guns in America per capita than in another other country in the world. And we also live in a very violent society, especially relative to our level of socioeconomic development. It is natural to wonder if guns are partially responsible for this violence, and to think about what the associated costs of gun control might be.

This is a two-part post in which I will tackle one common defense of America’s liberal firearms laws: popular defense against state terror. A broader methodological point will be to show that even without causal identification, it is easy to produce findings that should shape our understanding of public policy, which is contrary to the idea that only causal identification produces policy relevance. (More on that in the next post.)

The Argument

My good friend Matt Glassman is one very articulate defender of (some forms of) gun ownership rights. One argument that Matt finds particular compelling is that an armed society can protect itself against tyranny or state terror. Matt puts it best:

 I swear, you mention the idea that gun ownership protects people from state tyranny, and pretty much every liberal immediately makes some joke about Joe Sixpack stopping a U.S. military tank with his handgun. But I think this precisely misreads the situation on two levels. The first is the factual: anyone who has ever watched the youtube videos from Iran knows that the tyrannical state doesn’t maintain its grip by employing tanks everywhere; there’s simply not enough money to do that. Instead, it relies on thugs and other ill-trained militia to terrorize citizens with knives, clubs, and small arms. You watch the video of a man being assaulted on his own front lawns by one or two state thugs with knives and clubs, and it become self-evident that such practices simply would not be possible in a society with as many homeowner firearms as the United States.

It just so happens that I have lived under authoritarian rule—the soft authoritarianism of contemporary Malaysia, where the state isn’t tyrannical but where state terror can, and has, cropped up from time to time. Let’s not pretend that I faced anything like a threat to my physical safety from the Malaysian state. But some people in Malaysia have. My sense is that arming Malaysian civilians would not protect them against the terror that they face. But I could be wrong.

The Data

One way to check is to collect a broad range of data on state terror and gun ownership. Is it true that countries with lots of guns tend to have less state terror? First let’s look at the data on private gun ownership. I gathered this from the Small Arms Survey, using 2007 data.

The U.S. really does have a lot more privately-held firearms than any other country, relative to our population.

But is this an essential part of our defense against state tyranny? If so, we’d expect that overall, countries with lower levels of gun ownership are more subject to the tyranny of the state than than countries with high levels of gun ownership. Let’s compare the (natural log of) privately held arms per 100 people with some standard indices of state terror. Take first some rankings developed from Amnesty International. These are 2000-07 averages of a yearly index that ranges from 1 – 5, with 1 the “worst” for state terror and 5 the “best.” (Data are from the Political Terror Scale.) To illustrate another important distinction, I’ve graphed the democracies in blue, and the dictatorships in red, using the Cheibub and Gandhi binary regime classification.

The overall trend line (in black) shows that there’s a positive relationship overall between gun ownership and freedom from state terror. But these data also don’t make the U.S. look very good in the comparative context. We aren’t as free from terror as we think we are. So let’s use data collected by the State Department itself about political terror worldwide.

Again, a modest positive relationship between gun ownership and freedom from state terror. We can repeat this further, if we like, for an index of civil liberties.

The same pattern continues to hold. Now looking at these three figures, we can see a couple of conclusions.

1. There is a modest positive relationship between gun ownership and freedom from state terror. This is important.
2. There is also clear evidence that democracies terrorize less than dictatorships. (The blue countries tend to be further to the right than the red ones, on average.)
3. The United State is a major outlier. There are plenty of examples of countries that are equally as free from state terror but which do not have nearly the same level of private gun ownership.

Taken together, this makes me skeptical that broad ownership of guns by citizens is much of a defense against state terror. It is hard to conclude that the level of gun ownership that the United States sees today is necessary to defend against tyranny. We can even repeat the exercise for other more precise indicators of state terror.

In fact, you look at these indicators and you see that the U.S. doesn’t even really do that well, even with all of our guns.

A Closer Look

But does gun ownership help, on the margins? Let’s run some simple regressions.

	Pol Terror (Amnesty)		Pol Terror (State Dep.)		Civ. Lib. (F. House)
Ln(guns)	0.166**	-0.0525	0.250***	-0.0514	0.431***	0.0439
	(3.22)	(-0.95)	(4.50)	(-0.93)	(4.45)	(0.60)
Democracy		0.417**		0.399**		2.366***
		(3.27)		(3.22)		(14.41)
ln(GDP per capita)		0.335***		0.455***		0.357***
		(5.80)		(8.11)		(4.80)
constant	3.118***	0.419	3.061***	-0.526	3.790***	0.116
	(29.83)	(0.97)	(27.09)	(-1.24)	(19.08)	(0.21)
N	158	152	167	161	168	162
t-statistics in parentheses. * p < 0.05, ** p < 0.01, *** p < 0.001.

Consistent with the graphical results, there is an unconditional correlation between gun ownership and freedom from political terror. This correlation is statistically significant. But it disappears once we control for two other standard determinants of rights: democracy (measured as before) and economic development (measured as the log of GDP per capita, 2007-09, from the World Development Indicators).

The conclusion that this suggests is that the United States is relatively free from state terror because it is a developed democracy, not because we allow our citizens broad freedoms to arm themselves.

Of course, these are not estimates of the causal effect of gun ownership (or anything else) on state terror. These are conditional correlations, and there are plenty of reasons why we might believe that the causal relations here are more complicated than what this discussion has implied. More in the next post on what those threats to causal inference are, and how we ought to go about drawn policy recommendations from messy conditional correlations such as these.

Replicate!

Finally, to recreate these figures (and more) and to recreate these analyses (and more), you can download my Stata .do file here. Just put it in a folder with the path c:dataguns_and_terror, make sure you’re connected to the internet, and run it.