Journal of Quantitative Criminology, Vol. 14, No. 2, 1998
Defensive Gun Uses: New Evidence from a
National Survey
Philip J. Cook1,3 and Jens Ludwig2
The number of civilian defensive gun uses (DGUs) against criminal attackers is
regularly invoked in public policy debates as a benefit of widespread private
ownership of firearms. Yet there is considerable uncertainty for the prevalence
of civilian DGUs, with estimates ranging from 108,000 (using the National Crime
Victimization Survey) to 2.5 million (using smaller telephone surveys) per year.
In this paper we analyze the results of a new national random-digit-dial telephone
survey to estimate the prevalence of DGU and then discuss the plausibility of
the results in light of other well-known facts and possible sources of bias in
survey data for sensitive behaviors. Because DGU is a relatively rare event by
any measure, a small proportion of respondents who falsely report a gun use can
produce substantial overestimates of the prevalence of DGU, even if every true
defensive gun user conceals his or her use. We find that estimates from this new
survey are apparently subject to a large positive bias, which calls into question
the accuracy of DGU estimates based on data from general-population surveys.
Our analysis also suggests that available survey data are not able to determine
whether reported DGU incidents, even if true, add to or detract from public
health and safety.
KEY WORDS: defensive gun use; firearms; survey bias.
1. INTRODUCTION
The widespread private ownership of firearms in the United States may
have both positive and negative effects on the level (and lethality) of violence
that we experience. Firearms in private hands may reduce injuries by
1
Sanford Institute of Public Policy, Duke University, Durham, North Carolina, and National
Consortium for Violence Research and National Bureau of Economic Research.
2
Georgetown Public Policy Institute, Georgetown University, Washington, DC, and Northwestern University/University of Chicago Poverty Center, Evanston, Illinois.
3
To whom correspondence should be addressed at Sanford Institute for Public Policy Studies,
Box 90245, Duke University, Durham, North Carolina 27708. e-mail: [email protected].
l11
112
Cook and Ludwig
providing citizens with the means of protecting themselves against criminal
attackers (Kleck, 1988, 1991; Kleck and Gertz, 1995). On the other hand,
firearms may increase the lethality of suicide attempts and criminal attacks,
as well as causing accidental injury (Zimring, 1968, 1972; Cook, 1991; Kellermann et al., 1992, 1993). Understanding the relative magnitudes of these
benefits and costs is crucial for public policy.
Yet there is currently considerable disagreement concerning the number
of defensive gun uses (DGUs) each year. One data source for estimating
the incidence of civilian DGU is the National Crime Victimization Survey
(NCVS), a nationally representative survey of 59,000 households conducted
by the Census Bureau for the U.S. Bureau of Justice Statistics (McDowall
and Wiersema, 1994). The NCVS attempts to interview each individual age
12 or older in eligible households; housing units are retained in the sample
for 3 years, and the residents are interviewed each 6 months. The most recent
published estimates from NCVS data suggest 108,000 defensive gun uses
annually (Cook et al., 1997).
In sharp contrast are the results of a number of relatively small telephone surveys conducted in the last 20 years by private polling organizations.
After adjusting for differences in methods and coverage, the estimates from
these surveys range between 770,000 and 3.6 million people using guns
defensively against humans each year (Kleck and Gertz, 1995). The most
recent survey is also the first to be designed expressly for examining DGU
and has been used to estimate 2.5 million defensive gun users per year (Kleck
and Gertz, 1995). The Kleck and Gertz estimate of 2.5 million DGUs each
year has received considerable attention in public policy debates and is
regularly invoked within the research community as well as by media, politicians, and even the Congressional Research Service (Bea, 1994).
In 1994 the Police Foundation, under a grant from the National Institute of Justice, conducted a survey on gun ownership and uses in the United
States. The survey includes a sequence of questions on DGU very similar
to the one used by Kleck and Gertz and, thus, provides an opportunity to
study more closely the nature, reliability, and implications of survey-based
estimates for such uses. Our analysis of these data call into question the
accuracy of DGU estimates based on data from general-population surveys.
Our analysis also suggests that available survey data are not able to determine whether reported DGU incidents, even if true, add to or detract from
public health and safety.
In Section 2 we discuss the possible sources of bias in estimating the
prevalence of DGU from survey data. Section 3 contains details about the
survey data that we analyze. In Section 4 we present the results of our
analysis, followed by a discussion of the implications of our results for future
research and public policy.
Defensive Gun Uses
113
2. ESTIMATING THE PREVALENCE OF DEFENSIVE GUN
USE (DGU)
Criminologists frequently make use of survey data to overcome the
limitations of administrative records in estimating parameters of interest. In
the case of civilian DGU, the concern with using official law enforcement
records is that many such uses go unreported. Some defensive gun users
may fail to report their gun uses because they own or carry their firearms
in violation of local, state, or Federal law or are uncertain about the legality
of their actions (Kleck and Gertz, 1995). The hope is that such gun users
will be more willing to report their gun uses to survey interviewers than to
law enforcement authorities and, in turn, that surveys will produce more
accurate estimates than those derived from police records.
Yet survey estimates may themselves be subject to bias, because some
respondents who have not used a gun defensively may report that they have
(a "false positive"), and others who have used a gun defensively may conceal
their use (a "false negative"). Following Hemenway (1997), we describe the
bias in a parameter estimate as the ratio of the estimated prevalence divided
by the true prevalence:4
Bias = [estimated prevalence]/[true prevalence]
In the formula described in Eq. (1), W represents the number of observations
in the sample, A represents the number of people who experience some event
and report accurately (the number of "true positives"), (N-A) represents
the number of people who do not experience the event, and f+ and f- represent the rates of false positives and false negatives, respectively.
Misreporting may be unintentional or intentional. Unintentional misreporting may occur when respondents forget about the behaviors or experiences in question. The degree to which this problem may plague DGU
estimates is unclear, though previous research suggests that forgetting is less
likely with more consequential events (Skogan, 1990). Alternatively, it is
possible that some respondents are confused during the survey. A nationally
representative survey is likely to include many respondents who will be
cognitively impaired as a result of substance use, mental illness, or dementia
and, hence, unreliable reporters.5
''This is a slight modification of Hemenway's (1997) formula.
'Recent estimates from the National Institute for Mental Health suggest that 51.3 million
American adults have "one or more mental or addictive disorders" (Bourndon el al., 1994).
114
Cook and Ludwig
Respondents may also be unclear about how long ago the events transpired (which may lead to the well-known phenomenon of "telescoping").
In principle, telescoping can cause estimates to be too high or too low.
Respondents either may mistakenly report on events that occurred outside
of the recall period or may fail to report events that happened during the
recall period but are mistakenly remembered to have occurred outside of
the time frame in question. In practice, the former seems to dominate. As
Skogan (1990, p. 262) notes, in criminal victimization surveys telescoping
can increase estimates "by between 40 and 50% depending on the time of
crime; the inflation rate is greatest for violent crimes and those (often more
serious) that were reported to the police." Surveys such as the NCVS attempt
to address this problem by exploiting the fact that sampled households are
interviewed every 6 months; the results of the first interview are used to
"bound" the reference period that is asked about on the follow-up survey.
Telephone surveys that interview respondents only once are obviously not
able to make this correction.
The incentives to intentionally misreport will depend on the behavior
or experience in question. Some questions may ask respondents to report
illegal behaviors that they or others have engaged in, in which case respondents may be further dissuaded from accurately reporting for fear of possible
legal action if the survey data is shared with law enforcement. For example,
previous research suggests that assaults by family or friends are underreported in surveys (Skogan, 1981). In the case of defensive gun use, some
respondents who actually experience a DGU may have incentives to misreport intentionally (report a false negative) because of concerns about the
legality or legitimacy of their gun uses. They may have been carrying the
gun illegally at the time or involved in drug dealing or other illicit activity.
Or they may simply be unsure of whether they had been within their rights
when they threatened another person with a gun.
On the other hand, previous research suggests that survey respondents
wish to make themselves look good in the eyes of the interviewer, which leads
to "social desirability bias" (Sudman and Bradburn, 1974). For example,
respondents are likely to overreport socially desirable behaviors such as
voting and underreport socially undesirable behaviors such as bankruptcy
(Bradburn et al, 1979). The false-positive rate may be higher with survey
questions about DGU than with other potentially illegal behaviors or forms
of criminal victimization because DGU reports may be subject to positive
social desirability bias. Unlike victimization or engaging in some illegal activity, successfully using a gun against a criminal attacker may be viewed as a
heroic act. For example, since 1932 the NRA's publication American
Rifleman has published abbreviated newspaper accounts of DGU. These
accounts reflect the accolades that defensive gun users receive when they
Defensive Gun Uses
115
prevent a crime, including the Kentucky State Police's highest civilian honor
and an award from the Ross County (Ohio) Law Enforcement Officers'
Association.6 As one newspaper account reported,
An unidentified NRA member became famous throughout Texas as "The
Hunter" when he and his son heard a distress call on their CB radio. Two college
coeds saw a Waco man shoot Sammy Long, a Texas Department of Public Safety
officer, and called for help. The hunter arrived on the scene too late to save
Long's life, but killed the thug with a 6 mm rifle. Upton County District Attorney
Aubrey Edwards said the coeds and the hunter requested their names not be
made public and said the hunter "deserved a medal" for his action.7
Yet as suggested by Eq. (1), the bias of a parameter estimate depends
on both the false-positive and the false-negative rates and the true prevalence
of the behavior in the population (that is, the relative number of respondents
who have an opportunity to provide false positives versus false negatives).
Hemenway (1997) notes that if an event is rare—that is, A is small relative
to (N-A)—even a low false-positive rate may lead to overestimates of the
population prevalence, regardless of how high the false-negative rate is.
For example, suppose that we have a sample of a size similar to that
used in the Kleck and Gertz (1995) survey, with N= 5000. For the sake of
our example, assume that the true prevalence of DGU is 0.001, twice the
prevalence rate that has been estimated from recent years of the NCVS.8 In
this case, we expect 5 respondents to have experienced a DGU and have the
chance to report a false negative, while the remaining 4995 respondents have
the opportunity to report a false positive. Suppose that the false-positive
rate is 0.01.9 The resulting estimate from this survey will overstate the prevalence of DGU by a factor of 10 even if the false-negative rate is as high as
100% (that is, none of the defensive gun users report their use). What Kleck
and Gertz (1997) refer to as "one-sided speculation" about the importance
"Taken from the National Rifle Association home page on the World Wide Web, 11/29/97.
Web address: http://www.nra.org/research/armdctzn.html. The two accounts reported above
originally came from the Corbin, Kentucky, Times-Tribune (October 1980) and the Chillicothe,
Ohio, Gazette (March 1967).
7
Taken from the National Rifle Association home page on the World Wide Web, 11/29/97.
Web address: http://www.nra.org/research/armdctzn.html. Original article taken from The
Times, San Angelo, Texas, February 1977.
8
The most recent NCVS-based estimate for the prevalence of defensive gun use is 0.0005 (Cook
el al., 1997). This figure is probably somewhat too low, in part because the NCVS does not
ask respondents a direct question about DGU (Smith, 1997). Also, the NCVS only asks
respondents who report a criminal victimization to report the defensive measures that were
taken, and some DGUs may occur in response to crimes that are not asked about in the
victimization screens.
9
By way of comparison, the false-positive rate for self-reported drug use ranged from 0.009
for amphetamines to 0.092 for marijuana compared with urinalysis results (Harrison, 199S,
Fig. 1).
116
Cook and Ludwig
of the false-positive rate is in fact warranted given the algebra associated
with estimating rare events.
Our example also highlights one explanation for at least part of the
discrepancy between the NCVS and the estimates of Kleck and Gertz (1995).
Even if the false-positive and false-negative rates are equal across all of these
surveys, the scope for upward bias is dramatically reduced in the NCVS
since only those respondents who report a criminal victimization are given
the opportunity to answer questions about DGU.
In sum, there is no direct evidence on the false-positive rate for DGU
but good reason to believe that it is not trivial. Given even a small rate of
false positives, the infrequency of DGU (even by Kleck and Gertz's measure)
suggests that surveys are likely to overestimate the prevalence of this event
regardless of the false-negative rate. The next section describes the survey
data we use to test this hypothesis.
3. DATA
In this section, we review the survey design and questions about defensive gun use (DGU). The survey questions about DGU are very similar to
those used by Kleck and Gertz. While the response rate of the survey analyzed in this article is somewhat low, it is no lower than that produced in
the Kleck and Gertz survey.
3.1. Survey Design
The National Study of Private Ownership of Firearms (NSPOF) was
a nationally representative, random-digit-dial telephone survey. Interviews
were conducted during November and December of 1994 by Chilton
Research Services.
The instrument itself was designed by the Police Foundation. Chilton
interviewers and supervisors attended a training session that provided background on the purposes of the study and a review of each question in the
survey instrument; interviewers also participated in practice interviews under
the supervision of senior Chilton staff. The results of 10 pretest interviews
led to a few minor changes to the survey instruments. During the study,
approximately 10% of all interviews were randomly selected to be monitored
by senior Chilton staff.
Households were selected by first choosing a central telephone office,
characterized by the first six digits of the telephone number, and then randomly selecting a household from each central office. Sampling quotas were
defined on the basis of race (Hispanic, African-American, other) and gun
Defensive Gun Uses
117
Table I. Sample Disposition and Response Rate"
Total telephone numbers
29,917
Total ineligible numbers
Nonworking, nonresidential, language barrier, other
Sample quota filled
15,948
9,615
6,333
Total numbers for which eligibility unknown
No answer/busy
Answering machine
Breakoffs
10,701
3,039
1,685
5,977
Total eligible numbers
Completed surveys
Refusals
Response rate
p=(No. eligibles)/(No. eligibility determined)
R = (No. completed surveys)/[(No. eligibles) + (p) . (No. eligibility unknowns)]
3,268
2,568
700
20.5%
47.0%
a
Response rate calculation comes from the definition developed by the Council of the American
Survey Research Organization (White, 1983). Breakoffs are cases in which Chilton contacted
a person but was unable to ask enough questions to determine the household's eligibility
status.
ownership. The 6% of households without telephones are not in the sample
frame for the NSPOF.
Each selected telephone number was scheduled to receive as many calls
as needed to complete the interview, up to a maximum of six. Using the
last-birthday method, an adult (age 18 or older) was selected to be interviewed from each contacted household. Households who initially refused
were recontacted by Chilton interviewers who had unusual skill in converting
refusers into participants. Respondents who were not fluent in English, but
were fluent in Spanish, were recontacted by a Spanish-speaking Chilton
interviewer.
The sample disposition is shown in Table I. Of the 29,917 telephone
numbers that were randomly selected, 9615 were ineligible because the numbers were not working or not residential, and in 6333 cases Chilton determined that the sample cell quota had been filled on the basis of the initial
survey questions and terminated the interview. (Data from partially completed surveys were not used in this analysis.) In 4724 cases Chilton was
unable to reach a person, and in 5977 cases Chilton contacted someone but
was unable to obtain information on the screening questions to determine
eligibility. We used the Council of American Survey Research Organization's
definition for the response rate to calculate a figure of 47% (White, 1983).
While this response rate is somewhat low, there is no reason to believe
that the NSPOF is a less representative sample than other telephone surveys
118
Cook and Ludwig
that have been used to study DGU. For example, Kleck and Gertz (1995)
report a response rate of 61% for the survey used in their analysis, where
the response rate is defined as the number of households willing to participate
divided by the number of completed calls to eligible households. If we followed this procedure, Chilton's response rate would compare favorably with
theirs: of the 3268 numbers in which Chilton contacted a person and determined that the household was eligible, surveys were completed in 79% of
the cases.
In order to produce nationally representative estimates, Chilton provided population projection weights which adjust for the sampling method.
The weights were calculated using control variables for age, race, sex, education, household income, and the number of adults in each household.
Because households with multiple working telephone lines had a higher
probability of selection, we divided the Chilton weights by the number of
phone lines in the home. Since the average number of phone lines per
sampled household is greater than 1, the Chilton population weights divided
by the number of phone lines per household no longer sum to the entire
U.S. population. In order to correct for this, we multiplied all of the phone
line-adjusted weights by 1.0946 so that the sum of the weights equals the
U.S. adult population.
All of our results were calculated using the sampling weights. The
weights differ widely across the NSPOF sample, which serves to increase the
standard errors of our point estimates compared to a uniformly weighted
(simple) random sample of the same size. Our calculations suggested that
the appropriate standard errors in our analysis are 1.435 times the standard
errors that would be derived if the NSPOF were a simple random sample
(Mendenhall et al., 1990).
Table II presents descriptive statistics of the NSPOF sample and, for
comparison, estimates for the U.S. adult population (ages 18 and over) from
the U.S. Census (U.S. Bureau of the Census, 1995).
3.2. DGU Questions
The NSPOF survey instrument begins with 53 questions on topics such
as perceptions about local crime trends, attitudes toward a handgun registration law or a handgun ban, household and personal gun ownership, gun
carrying, and recent victimization experiences (for the crimes of robbery and
burglary).
Each respondent in the NSPOF was then asked the question, "Within
the past 12 months, have you yourself used a gun, even if it was not fired,
to protect yourself or someone else, or for the protection of property at
home, work, or elsewhere?" Respondents who said yes were then asked,
119
Defensive Gun Uses
Table II. Characteristics of the NSPOF Samplea
1994
NSPOF
(N=2568)
Percentage of respondents who own gun
Male
Race
White
Black
Hispanic
Other
Income
$0 to <10K
$10 to <15,000
$15 to <50,000
$50,000 plus
Educational attainment
Less than high school
High school or some college
College or more
R has been arrested for non-traffic offense
Marital status
Never married
Married
Widowed
Divorced/separated
HHs with children <18 living in HH
Census region
Northeast
Midwest
South
West
Census
estimate
24.6
47.5
N/A
79.6
11.5
76.5
11.5
4.2
4.7
8.0
4.0
11.4
8.7
6.9
N/A
6.7
48.9
33.0
48.8
37.1
21.3
55.7
23.0
19.1
58.8
22.2
6.1
N/A
19.8
64.8
23.3
60.6
5.8
9.6
7.0
9.2
41.7
35.0
20.2
24.2
34.8
20.8
19.7
23.6
34.8
21.8
a
Descriptive statistics calculated using NSPOF sampling weights divided by
number of telephone lines in household (to control for higher probabilities
of selection for multiple-telephone line households) ; because the average
number of telephone lines per household in the sample was greater than one,
we multiplied each of the weights by the appropriate number (1.0946) to
ensure projection to the U.S. adult population. Census estimates from the
Statistical Abstracts (U.S. Bureau of the Census, 1995). Also note that the
income categories provided in the NSPOF and the Statistical Abstracts
between $15,000 and $50,000 do not correspond, hence no comparisons are
possible.
"How many different times did you use a gun, even if it was not fired, to
protect yourself or your property in the past 12 months?" (emphasis in
original). Respondents who did not report a DGU during the past year
were then asked, "Have you ever used a gun to defend yourself or someone
120
Cook and Ludwig
else?" Respondents who had reported a DGU either in the past year or ever
were asked, "Let's talk about the most recent incident. About how long ago
was that?" The responses to this question allowed us to identify those gun
uses that reportedly occurred within the past 5 years.
Respondents who reported a DGU experience are then asked 30 additional questions concerning the most recent such experience. Topics covered
include whether the use was against an animal or a human, the relationship
between the respondent and the perpetrator, the location of the incident,
the crime involved, whether the perpetrator was armed, and what the
respondent did with the firearm in the incident. Chilton interviewers were
asked to provide their own assessment of whether the respondent was inventing the most recent gun use incident.
By way of comparison, in the Kleck and Gertz (1995) survey, respondents were asked (p. 161): "Within the past five years, have you yourself or
another member of your household used a gun, even if it was not fired, for
self-protection or for the protection of property at home, work, or elsewhere?
Please do not include military service, police work, or work as a security
guard" (emphasis in original).
In sum, the NSPOF is a state-of-the-art RDD telephone survey. Compared with the survey employed by Kleck and Gertz (1995), the NSPOF
DGU questions are very similar and the response rates are comparable.
Because the survey conducted by Kleck and Gertz was conducted for the
express purpose of examining DGU, only those respondents who reported
a DGU plus a randomly selected third of other respondents were asked to
complete the full survey questionnaire. The result is that 4977 respondents
were asked whether they had experienced a DGU, of whom 1832 completed
the full survey. Because the NSPOF was intended to be a comprehensive
survey of gun ownership and use, all eligible respondents were asked to
complete the survey questionnaire. The tradeoff is that a smaller number of
people reported on DGUs in the NSPOF relative to the Kleck and Gertz
survey (2568 versus 4977), though a larger number of respondents in the
NSPOF answered broader questions about gun ownership, acquisition, and
use.
4. RESULTS
While the survey responses to the NSPOF imply defensive gun use
(DGU) estimates closer to those of Kleck and Gertz than to those produced
using the NCVS, we find evidence to support our earlier hypothesis that
population-based surveys of DGU are subject to upward bias.
Defensive Gun Uses
121
Table III. DGU Estimates for 1- and 5-Year Recall Period: Comparison of NSPOF with KGa
1-year recall
No. respondents who report a DGU
Selected
Kleck & Gertz
NSPOF:
NSPOF DGUs
(1995)
all DGUs against humans
"Type A"
against
after applying
DGU
humans exclusion criteria
estimate
45
19
66
Estimated No. defenders (millions)
(95% confidence interval)
3.12
(1.77-4.54)
1.46
(0.52-2.44)
2.55
(2.09-3.14)
No. defenders as percentage of population
(95% confidence interval)
1.64
(0.92-2.36)
0.77
(0.27-1.27)
1.33
(1.01-1.65)
Estimated No. DGUs (millions)
(95% confidence interval)
23.0
(12.9-33.1)
4.7
(1.2-7.9)
2.55
(2.09-3.14)
a
The second and third columns both exclude the five defensive gun uses that reportedly occurred
within the past year but were suspected to be false by the Chilton interviewers. Estimates
based on exclusion criteria include only those DGUs that are against humans, do not occur
as part of protective service work, and in which the respondent saw a perpetrator, included
a specific crime as part of the description, and reported using the gun in some way. Population
projections are calculated by multilying prevalence estimates from the NSPOF by the adult
(18 and over) population in the United States in 1994.
4.1. DGU Prevalence
A total of 54 respondents in the NSPOF reported a DGU during the
past 12 months, of which 50 were against a human. After excluding the 5
cases in which Chilton interviewers thought the respondent was inventing
the incident, the 45 remaining respondents who report a gun use against a
person in the past year implied 3.1 million defensive gun users per year, with
a 95% confidence interval ranging from 1.8 to 4.5 million (Table III).10
For comparability with the most recent published DGU survey of Kleck
and Gertz (1995), we made a number of additional reasonable exclusions
and are left with just 19 respondents. The exclusions included respondents
whose most recent DGU was part of military or protective service work,
respondents who did not include a report of a specific crime, respondents
'"Unfortunately, Chilton was unable to provide documentation for the specific instructions
that were given to interviewers for identifying untruthful DGU reports. As far as we know,
there is no research on whether social science interviewers are able reliably to identify
intentional misreporting on social science surveys. We exclude the five cases in which a respondent reported a DGU during the past year and the interviewer reported possible misreporting
in order to maintain comparability with Kleck and Gertz (1995), who also take the approach
of eliminating DGU cases from their survey with "any indication that the incident might not
be genuine" (p. 163). As shown in the Appendix, two of the five cases in which the NSPOF
interviewer suspected misreporting meet at least one of our other criteria for exclusion.
122
Cook and Ludwig
who did not use the firearm (or even mention it to the perpetrator), and
respondents who did not actually see a perpetrator. (The numbers of cases
excluded by each restriction are shown in the Appendix.) As shown in Table
III, these 19 reports implied that 0.8% of American adults (1.5 million) were
involved in a DGU in 1994; the 95% confidence interval ranges from 0.3 to
1.5%, implying 0.6 to 2.9 million DGUs. The operational definition on
which this estimate is based is similar to that used by Kleck and Gertz
(1995), and in fact the 95% confidence intervals overlap (Table III).
Data from the 45 respondents who report having used a gun defensively
against a human during the year preceding the survey imply 23.0 million
DGUs. We modeled the random variable indicating the number of DGUs
reported by a respondent as the product of two random variables: an indicator of whether or not the respondent reports any DGUs (governed by the
binomial distribution) and the number of uses reported by DGU reporters
(governed by the Poisson distribution). Applying the formula for the variance of the product of two random variables (Becker, 1971, p. 180) suggests
a 95% confidence interval that ranges from 12.9 to 33.1 million. Of the 19
defensive gun users who met the Kleck and Gertz-type exclusion criteria, 7
reported more than 1 use per year, with an average of 6.7 uses per year; 1
respondent reported 20 uses. Taken together, the responses of the 19 users
implied 4.7 million uses (with a confidence interval of 1.2 to 7.9 million). In
comparison, Kleck and Gertz (1995) find that respondents in their survey
who report a DGU within the past 5 years report an average of about 1.5
uses. (They do not ask about frequency of use during the 1-year recall
period.) If we exclude the NSPOF respondent who reported 20 uses, the
average number of uses during the past year by defensive gun users in the
NSPOF would be about 2.
We also calculated results for the prevalence of DGU involvement during the 5 years preceding the survey by focusing on those DGUs that were
reported to occur 5 or fewer years ago. (Respondents were not asked about
the frequency of such involvement.) Our calculations suggested that 4.11%
of the U.S. adult population has experienced a DGU during the 5 years
preceding the survey (with a confidence interval of 2.97 to 5.25%). After
applying our exclusion restrictions, we estimated that 1.71% of American
adults have used a gun defensively (with a 95% confidence interval ranging
from 0.97 to 2.45%).
4.2. Comparisons with Other Measures
The NSPOF produces estimates for the prevalence of DGU that are
consistent with those produced by Kleck and Gertz (1995), though whether
these results are unbiased is an open question in light of our earlier discussion
Defensive Gun Uses
123
of the possibility and influence of false-positive reports. One way to explore
the credibility of the estimates of DGU from our survey is to compare them
with related estimates from various other sources.
First, based on the NSPOF our best estimate of the number of DGUs
in 1994 is 4.7 million, which is 35 times larger than the corresponding estimate based on the U.S. Bureau of Justice Statistics' National Crime Victimization Survey (NCVS). As noted earlier, one difference between the two
surveys is that the NCVS asked only questions concerning self-defense of
respondents who first report that they have been victimized. In contrast, the
NSPOF asked questions about gun self-defense of all respondents, including
those who said no to the two victimization questions: "During the past 12
months—that is, since last November—has anyone robbed, tried to attack,
or attacked you?" and "During the past 12 months, did anyone break into
or somehow illegally get into your home?" Of the 19 respondents who we
counted as defensive gun users, 7 said yes to either of these questions. Of
the 12 respondents who said no to both of these victimization questions but
later reported a self-defense gun use, all but 4 indicated that the crime against
which they were defending was robbery, rape, assault, or a break-in.
The number of crimes that are reportedly defended with a gun is difficult
to reconcile with the number of crimes estimated from victimization surveys.
Respondents were asked to describe the crime or crimes involved in their
most recent defensive gun use. The DGU reports in the NSPOF implied
point estimates for the number of crimes defended with a gun (and 95%
confidence intervals) of 322,000 rapes (12,000-632,000), 462,000 aggravated
assaults (101,000-562,000), and 527,000 robberies (130,000-924,000). By
way of comparison, NCVS data estimated that the total number of rapes
(including attempts) was 316,000, that of aggravated assaults was 2.48
million, and that of robberies was 1.30 million in 1994 (Bureau of Justice
Statistics, 1996). The NSPOF estimate of the number of defensive gun users
thus implies that every rape or rape attempt was thwarted by a DGU, as
was one of five aggravated assaults and two of five robberies. Even if the
NCVS-based estimates of criminal victimization rates are off by an order of
magnitude, the NSPOF-based estimates for DGU are implausible.
We can also make this comparison using victimization estimates based
on the victimization questions (cited above) in the NSPOF. These turn out
to be quite close to the 1994 NCVS estimates: 10.4 million robberies,
assaults, and attempted assaults (10.8 million if we also include those crimes
reported in the DGU sequence but not captured by the victimization questions) versus 10.9 million in the NCVS, and 5.0 million burglaries versus
5.5 million in the NCVS. In comparison, the number of self-defense gun
uses reported in the NSPOF against an assault (attack or fight), rape, or
robbery in 1994 was 938,000.
124
Cook and Ludwig
Each defensive gun user is asked to assess the likelihood that someone
would have been killed had a gun not been used defensively. The results
implied that 629,000 lives were "likely," "somewhat likely," or "very likely"
saved through the DGU (95% confidence of 196,000 to 1,062,000). In comparison, data gathered by the FBI indicated 21,100 actual homicides in 1994
(Federal Bureau of Investigation, 1995). Even if we assume that only 1 of
every 10 respondents was correct in believing that the gun use saved a
life, the NSPOF estimates would imply that around three-quarters of all
potentially fatal attacks are prevented by a gun-wielding citizen. More likely,
either respondents are inventing the DGUs that they report or people who
use guns against people are very poor at judging the danger that they or
others face. The latter possibility calls into question the social benefits of
these gun uses, a point to which we return below.
Finally, Kleck and Gertz have argued that comparisons of survey-based
DGU estimates with the number of crimes implied by victimization surveys
are invalid because "a large share [of DGUs] are probably outside the scope
of incidents that realistically are likely to be reported to either the NCVS
or police" (Kleck and Gertz, 1995, p. 167).11 In the NSPOF, we find that
49% of respondents who report a DGU also indicate that the police found
out about the incident. The NSPOF estimates thus implied that law enforcement authorities find out about 750,000 DGUs each year, including 526,000
uses that were in the context of a serious violent crime.
In particular, based on the NSPOF we estimated that there are 265,000
attempted rapes each year in which the victim used a gun and the police
were notified; we estimated that an additional 141,000 attempted robberies
with gun self-defense were reported to the police. But according to the FBI,
the total number of robberies known to the police in 1994 was 619,000,
which, together with the NSPOF results, seems to imply that 23% of all
robberies known to the police involved the victim defending himself with a
gun; and the total number of rapes (including attempts) known to the police
in 1994 was 102,000, implying that the number of gun uses against rapists
known to the police is 2.6 times the number of rapes or attempted rapes
known to the police (Federal Bureau of Investigation, 1995). Another source
for estimating the number of robberies and rapes known to the police is the
NCVS. According to NCVS estimates from 1994, a total of 719,000 robberies
and 137,000 rapes, attempted rapes, or sexual assaults were reported to the
police (Bureau of Justice Statistics, 1996). In any case, the comparisons for
robberies are implausible, while the comparisons for rapes are impossible.
4.3. Inconsistencies in Gun Use Reports
Another way to examine the reliability of DGU responses is to examine
the reports for internal consistency. Kleck and Gertz (1997, p. 1449) suggest
"See also Kleck and Gertz (1997, p. 1452) and Kleck (1997, pp. 158-159).
125
Defensive Gun Uses
Table IV. Inconsistencies in DGU Reports
Number of
cases
Total reports that contain at least one inconsistency
Report contains Inconsistency 1 : Respondent reports that serious crime (rape
or robbery) was involved, and also reports that "no crime was involved"
Report contains Inconsistency 2: Respondent reports that serious crime (rape,
robbery or attack) was involved and also reports that perpetrator neither
attacked nor issued threat
Report contains both Inconsistency 1 and Inconsistency 2
7
1
4
2
DGU report contains neither inconsistency
12
Total number of DGU reports
19
that false positives are unlikely with their own survey of DGU because of the
difficulties respondents face in "provid [ing] as many as nineteen internally
consistent responses covering the details of the alleged [DGU] incident."
While Kleck and Gertz (1995) do not report on the internal consistency of
the DGU reports in their survey, we find that a large share of the DGU
reports with the NSPOF contains what appears to be some form of internal
contradiction (see Table IV).
Each respondent was asked "Which of the following best describes what
was happening when you used the gun defensively?" The respondent was
then read the following list, one at a time, to which the respondent provides
a yes or no answer: rape, attempted rape, other sexual assault; some other
kind of attack or threat of violence; burglary or theft at home without
threat; robbery—someone took or tried to take something with the threat
or use of force; some other kind of theft or attempted theft; a fight; trespassing; and no crime was involved. Respondents were also asked, "Did the
perpetrator threaten, attack, or injure you?"
In 8 of the 19 DGUs that met our exclusion criteria, the respondent
reported that a serious crime (rape or robbery) was involved. Yet in three
of the eight incidents the respondent also included "no crime involved" as
part of the incident description. Further, in 6 of the 14 cases in which the
respondent indicated that the DGU involved a rape, robbery or attack, the
respondent also indicated that the perpetrator did not attack or even issue
a threat (Table V). Taken together, 7 of the 19 reported gun uses contained
at least one of these apparent inconsistencies.
It seems unlikely that these contradictions can be explained by coding
errors because senior Chilton staff monitored interviews and response coding
for a random 10% of calls throughout the study and these apparent inconsistencies arise in such a large proportion of DGU reports (over a third). It is
possible that some of these respondents are referring to serious crimes that
they believe to have been preempted by the presence of a gun (that is, a gun
Cook and Ludwig
126
Table V. Reported Type of Crime and Use of Force by
Perpetrator in 19 DGUs"
Crime?
(most serious) "
Attack?
Threat
Attack
Neither
1
1
3
0
1
1
0
1
2
0
0
0
2
3
1
1
0
2
Rape
Robbery
Attack
Burglary
Theft
Trespass
"The columns classify DGUs on the basis of their
answers to question 75 "Did the perpetrator threaten,
attack, or injure you?" The rows classify DGUs on the
basis of their answers to question 72, "Which of the
following best describes what was happening when you
used the gun defensively?" Q72 permitted multiple
answers; each DGU is categorized by the most serious
crime mentioned, with the hierarchy of "seriousness"
defined by the order in which the crime types are listed.
prevented the attack or threat from occurring). This possibility raises the
question of whether brandishing a gun against someone who has not even
issued a threat should be counted as a social benefit, a subject to which we
return below. At the very least, these apparent contradictions are consistent
with the idea that a sizable share of DGU reporters are falsifying part of
their accounts.
5. DISCUSSION
This paper used survey methods similar to those employed in a recent
widely cited study by Kleck and Gertz (1995) and produced comparable
results; yet our comparison of estimates based on NSPOF with other
sources, together with puzzling inconsistencies in over a third of the defensive
gun use (DGU) reports, lead us to conclude that the estimates are far too
high. We conclude with a discussion of what information these kinds of
DGU estimates (even if they were unbiased) can provide about the social
benefits of widespread gun ownership. We then offer some suggestions
on how future surveys of DGU might produce more reliable and useful
estimates, as well as some concluding thoughts.
5.1. Interpreting Gun Use Reports
Estimates for the prevalence of DGU have been used in arguments
against restrictions on gun ownership and carrying. While we will not evaluate such policies here, it is important to note that the available survey data
Defensive Gun Uses
127
provide little information about whether DGUs provide a net benefit to
society or whether they are legally or morally justifiable. Even if the respondent is providing an accurate report, it is typically quite difficult to determine
whether he or she acted appropriately.
As shown in Table IV, in 5 of the 19 gun use reports that met our
exclusion criteria, the most serious crime involved was a fight or attack. The
DGU report in these cases is the testimony of one party to an encounter.
We are given little context with which to judge these reports and, of course,
have no way to learn the other party's view of what happened.
Further, alternative courses of action available to the respondent cannot
be determined from these surveys. Claims about the benefits of DGU assume
that the firearm produces an outcome that is preferable to some (unobserved) alternative sequence of events. Citizens with access to firearms may
use their guns in situations where other effective responses may be available.
For example, in 6 of the 19 gun uses that met our exclusion criteria,
the respondent reported that the encounter occurred "near the respondent's
home" and also indicated that, when he first wanted to use his gun for
protection, the gun was stored somewhere in the home. In other words, in
about one-third of these events the respondent apparently had the option
of staying inside and calling the police.
Finally, DGU estimates cannot provide information about other important consequences of widespread gun ownership and carrying. Higher rates
of gun ownership may increase the likelihood that criminals arm themselves.
On the other hand, widespread gun ownership and carrying may deter criminal activity. Neither of these effects is reflected in DGU estimates.
5.2. Suggestions for Future Research
Below we offer three suggestions for future survey work on DGU that
may improve our understanding of the prevalence and nature of this event.
First, greater attention should be devoted to screening out false positives
when attempting to estimate the prevalence of rare events. As suggested by
Section 2 in this paper, the false-positive problem is not limited to studies
of DGU; for example, analysis of the NSPOF data suggested 315,000
unintentional gunshot woundings per year, much higher than the 17,000
unintentional gunshot wounds treated in hospital emergency rooms each
year (Sinauer et al., 1996). One possibility for screening out false positives
in the DGU application might be to build more redundancies into the
sequence of questions about the details of the encounter. Another possibility
would be to attempt to match survey reports with official police records in
those cases in which the respondent reports that law enforcement were notified about the gun use. Unfortunately this approach would require survey
128
Cook and Ludwig
participants to identify themselves and could only be performed for a subset
of reported DGUs. Yet another possibility would be to include questions
about DGU that are asked of every respondent in a longitudinal survey, to
gauge the effects of telescoping with one-time surveys (Smith, 1997).
It is worth noting that the NCVS asks DGU questions only of those
respondents who report a victimization. This method dramatically reduces
the scope of the false-positive problem, though at the cost of excluding those
gun uses associated with crimes omitted from the NCVS crime list (such as
trespassing). Of the 19 DGU reporters in the NSPOF, 7 answered yes to
questions about criminal victimization that came earlier in the survey. Thus
something like two-thirds of the difference between the NSPOF- and the
NCVS-based DGU estimates may be accounted for by the fact that the
NCVS asks only self-defense questions of victims. But even with that adjustment, the NSPOF estimate is an order of magnitude greater than the NCVS
estimate. It would be helpful to know whether more vigorous attempts to
identify false positives in both surveys would reduce the discrepancy yet
further.
Second, surveys should include questions about the sequence of events
leading up to the gun use. This information may help researchers understand
both the respondent's role (if any) in precipitating the incident and alternative courses of action available to the respondent prior to using a gun in the
encounter.
Finally, additional information is required about the exact nature of
these events and how the survey responses differ from what a neutral
observer would report. It may be useful to conduct a study that starts with
a sample of gun encounters that are reported to the police and then conduct
follow-back surveys of every participant in the encounter. The gun uses that
are reported to the police will not be representative of all gun uses, but
a study along these lines would, nevertheless, substantially improve our
understanding of the social benefits or costs implied by DGU reports in
household surveys.
5.3. Conclusion
Survey estimates for the annual number of DGUs have been offered as
a measure of the protective value of private gun ownership and carrying.
Because several recent estimates from household survey data suggest that
there are millions of DGUs each year, some have argued that widespread
gun ownership and carrying are effective in reducing injury from criminal
victimization. It is therefore important for public officials to be aware that
estimates of the prevalence of DGU based on data of the sort analyzed here
appear to suffer from a large positive bias and greatly overstate the prevalence of DGU. Further, these data do not provide sufficient information to
Defensive Gun Uses
129
distinguish between virtuous and objectionable uses. Hence these estimates
contribute little to evaluating the benefits of widespread gun ownership and
carrying.
APPENDIX
Table AI. Restrictions for DGUs Using 1-Year Recall Period"
No. cases
Total DGUs against humans in past year reported by NSPOF respondents
Reason for exclusion
Chilton interviewer thought R inventing incident ("invented")
Invented AND R did not report any crime as part of incident ("no crime")
Invented AND R did not report using gun in any way as part of incident ("gun
not used") AND no crime AND R used gun as part of law enforcement/
protective service work ("police use")
R reports he/she did not see perpetrator ("not see perp")
Gun not used
R not see perp AND no crime
Gun not used AND no crime
Gun not used AND not see perp
Gun not used AND no crime AND police use
Gun not used AND no crime AND invented
Total cases excluded
Total gun uses against humans after "restrictive exclusions"
50
3
1
1
6
7
1
1
5
5
1
31
19
"The exclusions reported above (with the exception of "invented") follow the method reported
by Kleck and Gertz (199S). See text for additional details.
ACKNOWLEDGMENTS
This research was sponsored in part by a grant from the National
Institute of Justice to the Police Foundation, though the views reflected
herein do not represent those of either funding agency. The data upon which
this study was based have been archived with the Interuniversity Consortium
for Political and Social Research (ICPSR) at the University of Michigan
and are available to the public according to the usual ICPSR regulations.
Thanks go to Brian Komar for valuable research assistance and to Earl
Hamilton, David Kennedy, Dave Lambert, Michael Maltz, James Mercy,
Lois Mock, Wesley Skogan, Eugene Volokh, and three anonymous reviewers
for helpful comments. Any remaining errors are our own.
REFERENCES
Bea, K. (1994). Issue Brief: Gun Control, Congressional Research Service, Washington, DC
(Order Code IB94007; revised Sept. 19).
130
Cook and Ludwig
Becker, G. (1971). Economic Theory, Alfred Knopf, New York.
Bourndon, K., Rae, D., Narrow, W., Manderscheid, R., and Regier, D. (1994). National
prevalence and treatment of mental and addictive disorders. In Mental Health, United
States, 1994, U.S. Department of Health and Human Services, Center for Mental Health
Services, Washington, DC, pp. 22-35.
Bradburn, N., and Sudman, S. (1979). Improving interview Method and Questionnaire Design,
Jossey-Bass, San Francisco.
Bureau of Justice Statistics (1996). National Crime Victimization Survey: Criminal Victimization
1994, U.S. Department of Justice, Washington DC.
Cook, P. J. (1985). The case of the missing victims: Gunshot woundings in the National Crime
Survey. J. Quant. Criminal. 1: 91-102.
Cook, P. J. (1991). The technology of personal violence. In Tonry, M. (ed.), Crime and Justice:
An Annual Review of Research, Vol. 14, University of Chicago Press, Chicago, pp. 1-71.
Cook, P. J., Ludwig, J., and Hemenway, D. (1997). The gun debate's new mythical number:
How many defensive uses per year? /. Policy Anal. Manage. 16(3): 463-469.
Federal Bureau of Investigation (1995). Uniform Crime Reports for the United States, 1994,
U.S. Government Printing Office, Washington, DC.
Harrison, L. D. (1995). The validity of self-reported data on drug use. J. Drug Issues 25(1):
91-111.
Hemenway, D. (1997). Survey research and self-defense gun use: An explanation of extreme
overestimates. J. Crim. Law Criminal. 87(4): 1430-1445.
Kellermann, A. L., Rivara, F. P., Somes, G., Reay, D. T., Francisco, J., Banton, J. G., Prodzinski, J., Rligner, C, and Hackman, B. B. (1992). Suicide in the home in relation to gun
ownership. N. Engl. J. Med. 327: 467-472.
Kellermann, A. L., Rivara, F. P., Rushforth, N. B., Banton, J. G., Reay, D. T., Francisco, J.
T., Locci, A. B., Prodzinski, J., Hackman, B. B., and Somes, G. (1993). Gun ownership
as a risk factor for homicide in the home. N. Engl. J. Med. 329: 1084-1091.
Kleck, G. (1988). Crime control through the private use of armed force. Sac. Problem 35(1):
1-21.
Kleck, G. (1991). Point Blank: Guns and Violence in America, Aldine De Gruyter, New York.
Kleck, G. (1997). Targeting Guns: Firearms and Their Control, Aldine de Gruyter, New York.
Kleck, G., and Gertz, M. (1995). Armed resistance to crime: The prevalence and nature of
self-defense with a gun. J. Crim. Law Criminal. 86(1): 150-187.
Kleck, G., and Gertz, M. (1997). The illegitimacy of one-sided speculation: Getting the defensive gun use estimate down. J. Crim. Law Criminal. 87(4): 1446-1461.
McDowall, D., and Wiersema, B. (1994). The incidence of defensive firearm use by U.S. crime
victims, 1987 through 1990. Am. J. Public Health 1982-1984.
Mendenhall, W., Wackerly, D., and Scheaffer, R. (1990). Mathematical Statistics with Applications, 4th ed., PWS-Kent, Boston.
Sinauer, N., Annest, J. L., and Mercy, J. A. (1996). Unintentional, nonfatal firearm-related
injuries: A preventable public health burden. JAMA 275(22): 1740-1743.
Skogan, W. (1981). Issues in the Measurement of Victimization, U.S. Department of Justice,
Bureau of Justice Statistics, Washington, DC.
Skogan, W. (1990). The National Crime Survey Redesign. Public Opin. Q. 54: 256-272.
Smith, T. W. (1997). A call for a truce in the DGU war. J. Crim. Law Criminal. 87(4): 14621469.
Sparks, R. F. (19xx). Surveys of victimization—An optimistic assessment. In Tonry, M., and
Morris, N. (eds.), Crime and Justice: An Annual Review of Research, Vol. 3, University of
Chicago Press, Chicago, pp. 1 -60.
Sudman, S., and Bradburn, N. M. (1974). Response effects. In Surveys: A Review and Synthesis,
Aldine, Chicago.
Defensive Gun Uses
131
U.S. Bureau of the Census (1995). Statistical Abstract of the United States: 1995, 115th ed.,
U.S. Government Printing Office, Washington, DC.
White, A. (1983). Response rate calculation in ROD telephone health surveys: Current practices. In Proceedings of the American Statistical Association, Section on Survey Research
Methods, American Statistical Association, Washington, DC, pp. 277-282.
Zimring, F. E. (1968). Is gun control likely to reduce violent killings? Univ. Chicago Law Rev.
35:721-737.
Zimring, F. E. (1972). The medium is the message: Firearm calibre as a determinant of death
from assault. J. Legal Stud. 1: 97-124.