Why Comparing Gun Ownership Rates Across States Shouldn’t Inform Policy
There’s a lot of gun data in the news, and much of it isn’t right for the job
Welcome to Range Widely, where I hope to help you think new thoughts.
If you just subscribed, you can check out the intro post here. If you’d like to subscribe, you can do that below.
Some of the conversations below posts have been really interesting lately, so please feel free to join the conversation. Here’s a direct link to this post.
The graphics in a Fox News article about guns caught my eye recently.
The headline declares that “higher rates of gun ownership do not correlate with more gun murders, data show.”
I’ve been interested in this topic lately, and trying to check out the data presented by publications of all political persuasions. Below is one of the graphics that stopped me.
Before diving into the data, I want to point out why this graphic is difficult to understand. The red and blue bars are using the y-axis on the left, while the yellow line is using the y-axis on the right, but the axes aren’t labeled, so you’re left to figure this out using your own sense of what these numbers should be. (Fermi estimation FTW, as usual.) Plus, for the red bar, the left y-axis is counting the number of murders, whereas for the blue bar that very same y-axis is serving as a percentage. Human brains weren’t built for data presentation like this.
Finally, the assignment of the axes is either thoughtless, or intentionally misleading. Take Maryland, for example. On the chart, Maryland has about 8 gun murders per 100,000 citizens (the red bar), and about 9 murders overall per 100,000 citizens (the yellow line). And yet, that “9” is about five times higher on the chart than the “8”. The cynical take here is that the creators of this chart wanted to make gun murders look like a small portion of overall murders, not the large portion that they actually are. But that aside…
Why That Chart Isn’t Using the Right Data for the Job (“Reverse” and “Forward” Causal Questions)
Fox used FBI data essentially to test the hypothesis that more gun ownership leads to more gun deaths. The chart shows that there is no clear relationship between the portion of households that own at least one gun in a given state and that state’s 2019 rate of gun-deaths. Thus, they concluded, gun ownership has no effect on gun deaths.
My guess is that many liberals looking at this data would argue that these state-level associations are confounded — i.e. there are many other factors between states that influence gun violence and gun ownership. Perhaps they would point to the data that clearly shows an association between countries of gun ownership and gun violence. And then, conservatives would probably say that those country-level associations are confounded — because there are many other factors between countries that influence gun violence. In conclusion, we get nowhere.
The state or country-level gun-ownership data would only give us a solid conclusion if gun ownership were the only variable that caused gun deaths. If, instead, there are multiple variables that affect gun deaths — as both sides of the aisle argue depending on what data are presented — then the way to try to isolate the influence of one variable is, well, to try to isolate it.
The FBI has actually issued short explainers in the past cautioning people who use its crime stats about this. Included among the “factors known to affect the volume and type of crime occurring from place to place,” according to the FBI:
Population density.
Variations in composition of the population, particularly youth concentration.
Stability of the population with respect to residents’ mobility, commuting patterns, and transient factors.
Modes of transportation and highway system.
Economic conditions, including median income, poverty level, and job availability.
Cultural factors and educational, recreational, and religious characteristics.
Family conditions with respect to divorce and family cohesiveness.
That’s a partial list. Suffice it to say, it’s complicated.
It’s not that the data Fox presented aren’t worth looking at, but to the extent that the article is implying a policy argument, that’s not the right data for the job. The Fox article is using the data to address a “reverse causal question”: Why do some states have more homicides than others? That article argues it isn’t because of gun ownership rates. But the data suggest something more subtle: that gun ownership rates aren’t a single overwhelming factor determining homicides the way that, say, latitude determines average annual temperature. But that should be kind of obvious. Humans are messy and social science (i.e. the study of human behavior) is messy, and we shouldn’t expect a single dominant cause of variation across all regions.
The data in the article shed no light on the “forward causal question”: What would happen to deaths if gun-ownership rates changed? For policy purposes, that’s what we really want to get at. (If you want to dive deeper on reverse and forward causal questions, check out this great post by psychologist Drew Bailey.)
Rather than using cross-sectional data — i.e. comparing data from different states at one point in time — a better way to get at the question is to use a so-called “fixed effects” approach. That is: track gun ownership and gun deaths over time within a given state. All those other factors the FBI highlighted (and more) won’t vary as quickly across time in a given location as they will between different locations at a single time. It’s not perfect, but it’s a lot better.
So, I went looking for studies like that.
Gun Ownership and Gun Deaths Over Time
An apt study examined gun ownership and gun deaths in every single state from 1981 to 2010 — a period during which “the prevalence of gun ownership decreased by about 36%.” This group of researchers analyzed variation within states, and between states, over time. They tried to further pin down the impact specifically of gun ownership by making statistical adjustments for a variety of state-level factors, including: proportion of young men; level of unemployment; divorce rate; income inequality; educational attainment; the general rate of violent crime in an area; and a bunch of other factors.
They found that changes in gun ownership across states and within a state over time did change the rate of gun death. According to their model, “for each percentage point increase in gun ownership, the firearm homicide rate increased by 0.9%.” So, if Mississippi had only as many gun-owning households as the average of all states, the model suggests it would have a 17% lower firearm-homicide rate.
The study found a host of other variables that influenced firearm-homicide rates in a state over time, including economic inequality. Actually, the effect of gun ownership on gun-death rates in a state in this study was somewhat smaller than I would have guessed. That’s how we learn! I’ll tweak my beliefs accordingly.
Gun Deaths and Changes in Gun Laws Over Time
Since this is a complicated (and politicized) issue, I wanted to search at least a little for studies that dispute or support the one I discussed above. So, I plugged that study into a really cool startup service I subscribe to: Scite.ai. It’s a database that helps you quickly to get a picture of the network of other studies that reference any particular paper. And, if you pay for the schmancy version (I do), you can get a list of “citation statements” — i.e. the specific context in which those other papers reference the study you’re interested in. Very handy!
When I did that, one of the studies that popped up had analyzed gun deaths in states from 1980 to 2016. This time, instead of looking at how changing gun-ownership rates affected gun deaths, the researchers looked at the influence of changing gun laws in a state over time.
In an unrealistic ideal world, we could set up an experiment where we randomly assign locations to have different gun regulations, and then see what happens. Something tells me voters won’t go for that. The authors of this study capitalized on the fact that changing state gun laws basically act like an experiment.
In a nutshell, they found that when state laws become less restrictive about how guns are stored in homes and cars, or when laws make it easier to carry concealed guns outside the home, the rate of gun deaths rises — modestly.
Changes in gun laws accounted for only a small portion of overall gun deaths, but the researchers concluded that, if the entire country adopted the storage and concealed-carry laws that made a difference in some states, we could save a few thousand lives a year.
I found that particularly interesting, since it suggests that basic regulations for law-abiding gun owners — i.e. people who conform to state laws when they change — can save some lives. This feels to me like traffic laws: some people are just reckless, no matter the laws, and will get people killed; but many others will follow laws (sometimes grudgingly) that force them to be more careful, and some lives will be saved because they follow those laws.
Together, I think the studies above imply a productive suggestion: gun-ownership rates make some difference, but there are many other factors at play; simple state regulations can be one factor that saves lives without infringing on the right to own a gun. So rather than diving once more unto the usual breach (i.e. debating the Second Amendment), maybe this illuminates some politically feasible actions that can save lives.
Low-Hanging Fruit
Even with the “natural experiment” of changing gun laws, cause-and-effect is hard to pin down. As Texas A&M economist Jennifer Doleac, director of the Justice Tech Lab, has pointed out, gun laws don’t just slip silently through state legislatures. They’re high profile, and so when they change it might reflect a larger change in attitudes about guns in a state. Perhaps that attitude change, and not the laws, are changing outcomes.
Doleac (fantastic Twitter follow if you’re interested in this stuff) has written urging lawmakers to focus on programs that aren’t politically polarized (or, less polarized?) and that have been shown to decrease firearm homicides and suicides. I found some of her evidence-backed suggestions intuitive, like summer jobs programs for teens, and Medicaid access in early childhood. Others, I never would have thought of, but now I’m curious. For example, from an article she wrote:
"In addition, repealing duty-to-warn laws for mental health providers—which require that they report a patient’s violent threats, perhaps causing patients to be less honest—could reduce teen suicides by 8 percent and decrease homicides by 5 percent. Repealing juvenile curfews could lower urban gunfire by two-thirds."
Maybe I’ll invite Jennifer Doleac for a Range Widely Q&A. If you have questions she might address, please feel free to share them in the comments.
Why I Wrote About Gun Research, Again
Because I think this topic is of interest to just about everyone right now, but most of the conversations I see are conducted purely with intuition or personal experience.
I have my own intuitions and experiences and emotions tied up in this. I was, for instance, present at a shooting in my high school — not a mass shooting, just a single student shot. And back when I worked at the New York Daily News, I was reporting on homicides every single night.
Like any human, I formed opinions to which I have emotional attachment. And I’m not deeply read in this area of research — not even close. But my hope is that bringing something to this conversation other than just another personal hot take can make it both more interesting, and more productive. I hope you agree.
Thanks for reading.
If you appreciated this post, you can use the button below to share it.
If you share on social media, tag me on Instagram, Twitter, Facebook, or LinkedIn so I can thank you.
If a friend sent this to you, you can subscribe.
Until next time…
David