Guns for Vaccines: Estimating the Tradeoff
Fermi estimation can make you a smarter news consumer
Welcome to Range Widely, the newsletter for generalists — where I'll help you step outside your lane for a few minutes each week. You can subscribe here:
If you left a comment on any post before today, I've responded. Please feel welcome to join the conversation!
*********************************************************************
Vaccine lotteries have been in the news periodically this fall, in part because it isn't entirely clear whether or not they work. (Update about the current state of evidence later in this newsletter...)
Ohio's "Vax-a-Million" program, which entered Covid-19 vaccine recipients in a lottery for a million dollars, kicked off the hopeful headlines this summer. The West Virginia lottery, meanwhile — in which getting a shot might win you a shotgun — drew some predictable online mockery.
Though my gut reflex is that we shouldn’t have to give out guns to get people vaccinated, one of the habits I’ve tried to cultivate — after reading a lot of research about judgment and decision-making — is to put friction between my gut reflexes and my conclusions. That can be as simple as trying to view a problem from the perspective of an outsider with no vested interest, or just asking a question that challenges my intuition. So when I first read about the West Virginia lottery, I stopped to ask: “But is the public health tradeoff worth it?” In other words: If the lottery actually works, then strictly in terms of deaths, will the public health impact of more guns be worth more vaccinated citizens?
I’m a huge fan of Fermi estimation — breaking a problem down into many pieces and making quick and extremely rough estimates. So I started there.
From reading the news, I know that there are about 40,000 gun deaths a year. From my time years ago as a crime reporter, I recalled some stat that there were as many civilian-owned guns in the U.S. as people. (I made my initial estimates without Googling, to take advantage of the "generation" and "hypercorrection" effects. That is, if you force yourself to come up with an answer, even if it's wrong, you'll subsequently better retain the correct information when you learn it.) I also know that the U.S. death total from Covid-19 is roughly similar to the Civil War, which was something like 700,000. So, right away, I’ve gained some insight: there have been far fewer Covid-19 infections in the U.S. than there are guns, but Covid led to something like 10 times as many deaths over a year. Maybe that lottery is a pretty clever idea.
But this seemed like a thought experiment worth a second opinion. So I mentioned it to Daniel Green, my brother-in-law, and a young professor at the Harvard Business School (who has done interesting work on public-sector employment during Covid). I want to stress that I was in no way suggesting that we do a rigorous, formal analysis of the West Virginia lottery, but rather just a little West Virginia-inspired estimation of the variety that an astute consumer of the news can do quickly. Daniel replied with exactly that. (**Note: Daniel and I were discussing this over the summer, so some of the numbers below are old! But the point was simply to consider the tradeoffs in general; it's a thought exercise.)
He availed himself of Google’s services and noted that there are 390 million civilian guns in America (I had underestimated, that’s 1.2 for every person!) and 38,000 gun deaths per year. Thus, there are about 10,000 guns per gun-death per year (~390 million/38,000).
On the vaccine side, Daniel used data from the Johns Hopkins Coronavirus Resource Center and went with a U.S. case-fatality rate (different from infection-fatality rate) of about 2 percent, meaning that to save one life we’d have to prevent 50 cases. (Of course, this varies highly based on age and other factors, but we’re not doing epidemiology here, just very rough estimates for a thought experiment that might help us think critically about headlines, and our own intuition.)
In early June — the time of the first West Virginia lottery headlines — there were circa 150 million unvaccinated Americans, and about 16,000 new cases per day, or 5.8 million annually at that rate. Using a 2 percent case-fatality rate, 116,000 of those 5.8 million infected people would die in a year, assuming none of them are vaccinated along the way. Vaccinating 150 million people would, then, prevent around 116,000 deaths over the year. In other words, to prevent one death, we'd need to get 1,300 people vaccinated (150 million / 116,000).
Doctors sometimes refer to that as the “number needed to treat,” or NNT — how many people need to get the particular medicine in order to help one person avoid a particular bad outcome (in this case, death); 1,300 sounds like a lot, and it is, but that’s because most people won’t get Covid in the given year in the first place, and most who do won’t die.
On the other hand, applied to a large population, the payoff is ginormous. Recall: there are about 10,000 guns for every gun-death per year, whereas according to the above estimate, it only takes around 1,300 vaccinations to save a life. As far as the impact on lives saved, vaccinating 1,300 people is equivalent to getting rid of 10,000 guns. Looking at it another way, in terms of balancing deaths caused by guns and prevented by vaccines, you’d have to give away 7.5 guns (10,000/1,300) to offset the public health impact of a single vaccinated person. West Virginia was giving away 10 guns total, so as long as the lottery led at least two people to get vaccinated, it looks pretty good from the death-ledger perspective.
SO MANY ASSUMPTIONS
As you may be aware, I made a backhoe-load full of assumptions in order to simplify the question above. There are all sorts of details that I left out, some of which weight the comparison in one direction, and some in the other. For example: West Virginia was giving away hunting rifles and shotguns, which are not the types of guns typically involved in gun deaths; older people who get Covid are much, much more likely to die than younger or middle-aged people, and older people are also more likely to have been vaccinated already; infection rates do not stay steady, but can decline or rise as more people get vaccinated or new variants emerge; guns last more than one year, and we aren’t yet sure how long a vaccine's protection against death lasts; etc. etc.
But the point of Fermi estimation is to get a foothold on a problem that’s better than your first reflex, and then from there, you can start tallying assumptions that warrant deeper exploration. For me, one benefit of this approach is that in thinking about the assumptions I'm making, I become more attuned to the complexity of the problem.
It reminds me of a particular interview I did while writing Range; it was with the late James Flynn, an eminent professor who studied cognitive abilities. Flynn told me that it was no longer enough for people to specialize in a single discipline, or even two or three. Rather, he said, in order to be a responsible thinker in an age of information deluge, everyone needs habits of mind that allow them to dance across disciplines. I think Fermi estimation is one of those; a little practice has improved my number sense, no matter the specific topic, and shown me (annoyingly) that my first reflex is often wrong.
If you're eager for a little practice, you can try the famous “piano tuners” estimation question, and be sure to take a crack at it before looking at the answer! It doesn't matter at all if you're way wrong. The exercise itself is valuable, and you'll get better quickly with practice.
David’s Digressions (and an update on vaccine-lottery research)
-If lotteries intrigue you, read about “regret lotteries.” In regret lotteries, everyone is entered automatically, but when names are drawn, if the winner didn’t do whatever it was they were supposed to do in order to qualify (like get vaccinated), they don’t actually get the prize. Gut punch. “Research suggests that can be highly motivating,” according to behavioral scientist Katy Milkman.
-Milkman helped design a vaccine lottery (with regret feature) for Philadelphia. She also co-authored a brand new study which found that the Philly lottery didn't really work. The Ohio vaccine lottery also looks like a disappointment in a new analysis. This is a bummer, but it's also an important lesson; kudos to the researchers for following up.
-If instead of a one-time-thing, the goal is sustained behavioral change (say, getting people to exercise), lotteries with smaller but easier-to-win prizes might be effective.
-If you’re up for a suspenseful narrative, check out “The Man Who Cracked the Lottery”, or just get intrigued by this short news clip about a group of young people who appear to be using a data-informed, blunt force approach to winning lotteries.
-If you want to think more about risk and inoculation, read about how George Washington inoculated the Continental Army against smallpox, using a treatment that had a 5 to 10 percent fatality rate!
-Finally, for a deeper, more academic dive on how people weigh potential rewards, check out “Prospect Theory: An Analysis of Decision Under Risk,” the most cited of all papers by Nobel laureate Daniel Kahneman, whose work famously illuminated our cognitive biases.
Thanks for reading. Until next week...
David
p.s. If you liked today’s post, please share it! You can find it (and other posts) here.