American Journal of Epidemiology
Copyright C 1997 by The Johns Hopkins University School of Hygiene and Public Health
All rights reserved
Vol. 148, No. 3
Printed In USA.
Evaluating Sources of Traumatic Spinal Cord Injury Surveillance Data in
Colorado
Renee L Johnson, 1 ' 2 Barbara A. Gabella,2 Kenneth A. Geitiart,1 Jeannie McCray,2 Jean C. Menconi,2 and
Gale G. Whiteneck1
The purpose of this study is to evaluate the sources reporting hospitalized spinal cord injury cases to the
statewide, population-based surveillance system in Colorado for the year 1994. Three reporting sources were
evaluated: clinical contact persons, medical records departments, and a centralized statewide hospital
discharge database. Two evaluation strategies were utilized; these include both measures of accuracy and
estimates of missed cases. For the latter, capture-recapture techniques were used to estimate the number of
hospitalized spinal cord injury cases missed by all three reporting sources. The clinical contact persons
reported 84 confirmed cases, missed 80 confirmed cases, and reported 10 cases that were later determined
not to have spinal cord injuries, resulting in a sensitivity of 0.51. Medical records departments and the
discharge database reported 143 and 147 cases, respectively, missed 21 and 17 confirmed cases, and
reported 118 and 69 cases that were later determined not to be cases of hospitalized injuries of the spinal cord,
resulting in sensitivities of 0.87 and 0.90. Capture-recapture results indicate all three sources combined
missed an estimated 1-5 cases, yielding a total annual incidence rate for hospitalized spinal cord injury ranging
from 45.1 to 46.3 per million population. Am J Epidemiol 1997; 146:266-72.
evaluation studies; population surveillance; spinal cord injuries
Accurate surveillance is required to quantify the
morbidity and mortality associated with injury (1). As
with disease prevention programs, the ability to conduct accurate surveillance is also key to developing
and sustaining effective injury control programs (1-3).
Spinal cord injury has been a model or prototype for
injury surveillance (3), because it is a relatively rare
condition that is clinically identifiable and generally
unequivocally diagnosed. Indeed, it is often severely
disabling, affects primarily young people, and is extremely costly in personal, financial, and societal
terms. Previous reported annual hospitalized incidence
rates of spinal cord injury range from 28.5 to 85.0 per
million population (4-10).
In 1987 the Council of State and Territorial Epidemiologists recommended that spinal cord injury be the
first injury condition to be reported to the National
Notifiable Disease System, which, under the auspices
of the Centers for Disease Control and Prevention,
assembles and analyzes data collected by the individual states (3). Colorado is one of the 34 states currently
involved in some form of spinal cord injury surveillance. Along with the other 33 states, Colorado faces
several surveillance challenges. With limited resources available, surveillance must strive for efficiency by maximizing both the sensitivity and the
positive predictive value of reporting sources (1, 2,
11-14). Each case that surveillance fails to identify not
only reduces the accuracy of the estimates, but also
diminishes the opportunity for individuals to receive
needed services. When specific types of cases or individuals are systematically missed, interventions may
be mistargeted, research may be misguided, and efficacy evaluations may be inaccurate. On the other
hand, a case reported to the surveillance program that
does not meet the system's carefully defined case
definition (a "false positive" report) takes staff time to
review, verify, and exclude, time that could be devoted
elsewhere.
The purpose of this study is to describe the Colorado
spinal cord injury surveillance system and to evaluate
the effectiveness of three distinct, yet overlapping
sources of hospitalized spinal cord injury data in Colorado. These sources are 1) key clinicians in designated hospitals, 2) medical records departments, and
3) a centralized statewide hospital discharge database.
Received for publication November 8, 1996, and accepted for
publication April 7, 1997.
Abbreviation: ICD-9-CM, International Classification of Diseases,
Ninth Revision, Clinical Modification.
"* Craig Hospital, Research Department, Englewood, CO.
2
Colorado Department of Public Health and Environment, Injury
Epidemiology Program, Denver, CO.
Reprint requests to Renee Johnson, Craig Hospital, 3425 S.
Clarkson Street, Englewood, CO 80110.
266
Evaluating Sources of Traumatic Spinal Cord Injury Surveillance Data
Effectiveness was evaluated using two approaches.
First, the traditional approach of calculating both sensitivity and positive predictive value was used to evaluate the accuracy of each source (1, 14, 15). Second,
the number of cases missed by each reporting source
was estimated using capture-recapture analysis (1621). In addition, the annual incidence of hospitalized
spinal cord injury in Colorado was estimated.
MATERIALS AND METHODS
The Colorado Spinal Cord Injury Early Notification
System is a statewide, population-based, traumatic
spinal cord injury surveillance system. New cases of
hospitalized spinal cord injury are included in the
surveillance system if they meet all the criteria in the
following case definition. 1) The injury results in some
degree of spinal cord-related motor, sensory, and
bowel and/or bladder impairment, which can be temporary or permanent. 2) The injury is sustained by a
Colorado resident, whether or not the injury event
occurred within the state. 3) The injury is caused by
trauma.
Three sources were utilized to identify potential
cases of spinal cord injury diagnosed during 1994.
First, designated clinicians, in key hospitals equipped
to treat neuro-trauma survivors, reported new injuries
upon admission. Second, a medical records reporting
system identified cases postdischarge, using the International Classification of Diseases, Ninth Revision,
Clinical Modification (ICD-9-CM), codes 806 (fracture of vertebral column with spinal cord injury) and
952 (spinal cord injury without evidence of spinal
bone injury) (11, 22); a state Board of Health regulation mandates such reporting by all acute care and
rehabilitation facilities. Finally, the third source of
spinal cord injury cases was the Colorado Hospital
Association discharge data set. The Colorado Hospital
Association is a voluntary membership organization to
which all nonfederal acute care facilities in Colorado
report hospital discharge data. These acute care facilities also report their inpatient rehabilitation discharge
data to the Colorado Hospital Association. Additionally, four of Colorado's seven freestanding inpatient
rehabilitation facilities also report to the hospital association. The 15 discharge diagnosis fields in the
Colorado Hospital Association data set were queried
again using the ICD-9-CM codes 806 and 952 (11,
22). Both the medical records and hospital association
reports were additionally queried for the first half of
1995 to ensure inclusion of persons who had sustained
their injuries in 1994 but were not discharged until
1995.
All spinal cord injury reports, regardless of reporting source, were subjected to a medical records review
Am J Epidemiol
Vol. 146, No. 3, 1997
267
for validation of spinal cord injury. This validation of
the spinal cord injury diagnosis was conducted by
reviewing for the following components of the International Standards for Neurological and Functional
Classification of Spinal Cord Injury, Revised 1992
(23): absence or impairment of motor and/or sensory
functioning, establishment of a neurologic level, and
the presence of a clinically documented spinal cord
syndrome. These key clinical features had to be noted
in the medical records. Motor and sensory testing,
neurologic examinations of both spinal and peripheral
nerves, and diagnostic imaging reports were reviewed.
This records review was conducted by staff specifically trained in the diagnosis and/or care of spinal cord
injury and included both a physician and a physical
therapist. When a report did not meet the eligibility
criteria, reasons for this "false positive" report were
recorded.
The accuracy of each reporting source was evaluated in terms of the number of the total true positive
cases as well as the number of false positive cases that
were reported by each source. This information was
used to calculate sensitivity, that is, the proportion of
cases of a disease or health condition that are detected
by each source, and to determine positive predictive
value, the proportion of reports by each source that
actually have die condition being monitored (14, 15).
In calculating sensitivity, we used the "gold standard"
of the total number of confirmed cases from all
sources as the denominator (the total number of "true
positive" cases).
Capture-recapture statistical techniques (16-21)
were used to estimate the number of cases not ascertained by any of the reporting sources and to estimate
overall incidence, using both the number of cases
common to sources and the number of cases unique to
each source. As described by Hook and Regal (17, 20),
the nearly unbiased estimator based on the maximum
likelihood estimates from the log linear model, which
adjusts for small sample bias, was used in this analysis. Since all three sources provided admission-based
data, one might expect these sources to be positively
dependent. An extreme example of positive dependence would be when the list of cases from one source
for a particular hospital is directly obtained from other
sources for the same hospital (18). Positive dependence results in an underestimate of the total injuries
in the population (17-20). To estimate the number of
missed cases and the total incidence while adjusting
for all possible dependencies among sources, we used
all three sources simultaneously by pooling two of the
three sources and comparing them as a unit with the
third source (referred to as "source X" in Results) (17,
19, 20). This method was repeated three times, by
268
Johnson et al.
changing which two reporting sources were pooled
and which source was not. Next, the three sources
were used separately, and the potential dependencies
were modeled explicitly by testing all two-way interaction terms (19, 21).
Given the nature of spinal cord injury and the potential for each individual case to have more than one
hospital admission, some cases had more than one
opportunity to be reported by each source, particularly
those patients who required inpatient rehabilitation in
addition to their acute care hospitalization. To examine this difference in reporting opportunity, the reports
were stratified by whether or not inpatient rehabilitation was utilized (17).
RESULTS
The combination of all three sources (clinical contacts, medical records, and the Colorado Hospital Association) identified a total of 311 reports of suspected
spinal cord injury diagnosed in 1994. Of these, 164
reports were verified as meeting the surveillance case
definition (true positive cases), and the remaining 147
reports did not meet the case definition (false positive
cases).
Persons who had some form of paralysis but who
did not meet the case definition or persons who had
sustained their spinal cord injury in a previous year
accounted for 32.0 percent of the total 147 false positive reports. Persons who had a bone fracture of the
vertebral column with no associated spinal cord
injury-related paralysis generated 28.6 percent of the
false reports. Nineteen percent of the persons excluded
had unilateral symptoms at one neurologic level that
TABLE 1.
1994
were most likely due to a radicular or peripheral nerve
deficit. Finally, 17.7 percent of the false reports appeared to be inexplicable ICD-9-CM coding errors.
These persons with miscoded records had a wide variety of diagnoses, including neck and back pain, outpatient radiologic and laboratory procedures, an upper
extremity amputation, a brachial plexus injury, and a
contusion of the coccyx.
Table 1 displays the numbers of confirmed hospitalized spinal cord injury cases (true positive cases)
identified by each source. Among the true positive
cases, 21 (13 percent) were identified by only one
source: five were reported by a clinical contact only,
nine were reported solely by a medical records department, and seven were reported only by the hospital
association. A total of 76 cases (46 percent) were
identified by only two sources, and the remaining 67
cases (41 percent) were reported by all three sources.
Similarly, table 1 also displays the numbers of false
positive cases identified by each source.
It can be seen in table 2 that clinical contact persons
reported 84 (51 percent) of the confirmed cases, medical records reported 143 (87 percent), and the Colorado Hospital Association reported 147 (90 percent) of
the confirmed cases. The sensitivity and positive predictive value of each of the reporting sources are
presented in table 2. Clinical contact persons were the
least sensitive. Conversely, the hospital association
data set was the most sensitive. Among the three
sources, the clinical contact persons had the highest
positive predictive value. Also displayed in table 2 is
the fact that, when any two of the reporting sources are
combined, the resulting sensitivity was 0.95 or higher.
Total number of true and false positive hospitalized spinal cord injury (SCI) cases by reporting sources, Colorado,
Reported
only by
dWcal
contacts
(CC)
Reported
only by
medfcaJ
records
(MR)
Reported
only by
Colorado
Hospital
Association
(CHA)
Reported
by both
CCandMR
Reported
by both
MR and CHA
Reported
by both
CC and CHA
Reported
byCC,
MR, and CHA
All reports stratified by case inclusion status
Confirmed cases of hospitalized
SCI cases ( n o 164)
5
9
7
3
64
67
False positive reports of
hospitalized SCI ( n - 1 4 7 )
9
68
20
1
49
0
Case inclusions stiatified by utilization ofinpatient rehabilitation services
Hospitalized cases that reported
inpatient rehabilitation
(n-102)
3
1
3
3
22
Hospitalized cases that did not
require inpatient rehabilitation
(n-62)
2
8
4
0
42
Am J Epidemiol
63
Vol. 146, No. 3, 1997
Evaluating Sources of Traumatic Spinal Cord Injury Surveillance Data
269
TABLE 2. Senaitlvfty and positive predictive value of hospitalized spinal cord injury (SCI)* by reporting
sources, Colorado, 1994
(lateen))
Raise
mgativas
(lalseH)
SenslMtyt
(tnie(+V
[true{+) +
false(-)D
Positive
predictive vaKiet
(true<+V
[tme(+) +
tatoe(+)D
84
143
10
118
80
21
0.51
0.87
0.89
0.55
147
69
17
0.90
0.68
157
155
159
127
79
138
7
9
5
0.96
0.95
0.97
0.55
0.66
0.54
True
positives
Cnje(+))
Clinical contacts (CC)
Medical records (MR)
Colorado Hospital
Association (CHA)
CC and MR combined
CC and CHA combined
MR and CHA combined
Reporting
sources
False
posllves
* Total confirmed cases (true(+) + false(-)): n • 164 for all three sources (table 1).
t Klaucke DN. Evaluating public health surveillance. In: Teutsch SM, Churchhill RE, eds. Principles and practice
of public health surveillance. Oxford: Oxford University Press, 1994:158-74.
Based on the capture-recapture method, table 3
demonstrates that the estimates of 1994 spinal cord
injury incidence in Colorado range from 164.8 to
169.0. Regardless of which single source ("source X")
was evaluated against the combination of the other two
sources ("pooled source"), the resulting annual spinal
cord injury rates varied little, from 45.1 to 46.3 per
million population. Using the three sources separately
with an interaction term for medical records and the
hospital association, the model resulted in an estimate
of 4.8 missing cases or an annual total incidence of
168.8 cases. Medical records and the Colorado Hospital Association were positively dependent (p <
0.01, G2 = 3.45, df = 2), while the evaluation of the
other two-way interactions indicated that they were
not dependent (19, 21).
Cases who received inpatient rehabilitation had a
minimum of two separate hospital admissions and,
therefore, a greater probability of being reported than
cases who did not. Table 1 examines these differences
in reporting among the two types of cases. Clinical
contact persons were able to identify 75 percent of the
cases that went to inpatient rehabilitation, while they
identified only 13 percent of the cases that did not. On
the other hand, medical records and the Colorado
Hospital Association were equally good at identifying
both rehabilitation and nonrehabilitation cases. When
the capture-recapture estimates of missed cases from
each stratum are summed, the total estimate of missed
cases is 19, based on the three-source model with a
hospital association-medical record interaction.
DISCUSSION
This study examined three sources of spinal cord
injury surveillance data: two that have been used by
Colorado regularly since 1989 as a part of its comprehensive surveillance and a third that was identified and
tapped particularly for the research reported here. The
first two sources, clinical contact persons based in key
acute care hospitals and trauma centers and retrospective ICD-9-CM code-based reporting by all of Colorado's hospital medical records departments, reported
157 confirmed cases. They also accounted for another
127 falsely identified cases, giving Colorado's regular
surveillance system a sensitivity of 0.96 and a positive
TABLE 3. Estimates of number of cases missed and total incidence of hospitalized spinal cord injury
(SCI) based on the capture-recapture method, Colorado, 1994
Source
X
Clinical contacts
Colorado Hospital
Association
Medical records
In source
Xandlnlhe
pooled
source*
(a)
Unique
to
source
X
Not In
source X
but hi the
pooled
Total In
source X
Estimate
of
missed
Estimated
total
Incidence
[(a+ 6 + 1 )
(a+1))
source
79
5
80
84
5.0
(a + 1)]-1
169.0
140
134
7
9
17
21
147
143
0.8
1.4
164.8
165.4
(c)
Estimated
annual
Incidence
rate/
rrtDtont
46.3
45.1
45.3
* Total confirmed cases (n « 164 for all three sources).
t The Colorado population figure of 3,653,600 for 1994 is a 1993-based projection from the Demographic
Section, Colorado Department of Local Affairs.
Am J Epidemiol
Vol. 146, No. 3, 1997
270
Johnson et a).
predictive value of 0.55. These findings compare favorably with those reported by spinal cord injury surveillance programs in both Utah and Oklahoma. Utah
has reported a sensitivity of 0.89 and a positive predictive value of 0.61 for a hospital discharge databased system (7). Oklahoma reports an overall spinal
cord injury surveillance system sensitivity of 0.77 but
reports no information on cases that are reported and
then later excluded (8).
The third data source, the Colorado Hospital Association's hospital discharge data set, is one that has
tended to become available much later after discharge
than the other two and, thus, up until the time of this
research, it has not been a useful means of timely
spinal cord injury surveillance in Colorado. Nonetheless, in yielding a sensitivity of 0.90 and a positive
predictive value of 0.68, its data were slightly better
than clinical and medical records reporting in Colorado and at least comparable to the values reported by
Oklahoma and Utah.
In terms of any one single source of case identification, the total number identified by either the hospital association (147 cases) or medical records (143
cases) provided 87.0 percent or 84.6 percent, respectively, of the highest capture-recapture estimate. We
feel that either of these two sources alone, followed by
thorough medical records review, may provide adequate case identification for the purpose of basic surveillance.
As expected, medical records departments and the
Colorado Hospital Association were found to be positively dependent, meaning that being in one source
influences the probability of being captured by the
other source. This positive dependence explains why
the incidence estimates generated when using only two
sources were so similar to estimates generated by
using all three sources (with an interaction term for
medical records and the hospital association). Both
medical records and the hospital association reported a
similar number of true positive cases. In fact, 131 of
the 164 confirmed cases (80 percent) were common to
both data sources. These two sources also had similar
values for sensitivity and a similar (though low) positive predictive value. The absolute number of false
positive reports from both sources was large, accounting for 138 (94 percent) of all 147 false positive
reports.
The high number of false positive cases reported by
both the hospital association and medical records
causes one to consider excluding the subset of 806 and
952 coded cases that have either a fourth or fifth digit
that indicates an "unspecified spinal cord injury." To
address this issue, we have inspected die hospital
association data set. All reports of admissions were
considered; the 216 cases identified by the hospital
association resulted in 427 admissions reported. When
the hospital association reports with unspecified codes
are grouped and compared with the reports from the
hospital association without unspecified codes, we
find that 28 percent of those reports that were later
excluded had an unspecified code while 33 percent of
those reports later included had unspecified codes.
This difference is nonsignificant (chi-square = 1.151,
df = 1, p = 0.28); thus, excluding the "unspecified"
coded records would not be a useful refinement to the
surveillance system.
Perhaps most importantly, the large number of false
positive reports from medical records departments or
the hospital association was costly, in terms of the
staff time needed to review medical records and to
verify case inclusion. Yet these reviews could not be
eliminated given the low positive predictive value of
either source. Overall, both medical records departments and the hospital association contributed a similar number and type of cases to surveillance. The only
major difference in reporting between medical records
and the hospital association that impacts surveillance
activities was the current lack of specific personal
identifiers available from the hospital association data.
This lack of an identifier unique to the person, such as
a name, makes identifying the same case from multiple sources difficult and time consuming.
Clinical contact persons reported only 84 (51 percent) of the confirmed spinal cord injury cases in
1994, similar to the number in previous years (4).
Clinical contact persons, however, performed better
with respect to persons who went to inpatient rehabilitation. Also, clinical contact persons reported few
false positive cases, thereby reducing the resources
wasted by reviewing the medical records of false positive reports. In fact, 84 (89 percent) of the 94 reports
by clinical contact persons truly had a spinal cord
injury. Neither medical records nor the hospital association was as accurate (68 percent and 55 percent,
respectively).
The accuracy of each individual reporting source is
impacted by the accuracy of the chosen "gold standard." The fact that the total number of confirmed
spinal cord injury cases (164 cases) included in the
"gold standard" was so close to the lowest estimate
generated by capture-recapture (164.8 cases) indicates
the validity of the calculations presented in table 2. In
fact, when the highest estimate generated by capturerecapture (169.0 cases) was used to calculate the measures of accuracy, the sensitivity for all sources drops
only slightly: clinical contacts from 0.51 to 0.50, medical records from 0.87 to 0.85, and the Colorado Hospital Association from 0.90 to 0.87. The positive preAm J Epidemiol
Vol. 146, No. 3, 1997
Evaluating Sources of Traumatic Spinal Cord Injury Surveillance Data
dictive value is not affected by the change in the
number of false negative (unidentified) cases.
With regard to persons who did not receive inpatient
rehabilitation, medical records departments and the
hospital association were better sources for reporting
than were the clinical contact persons. This trend is
evidenced by Colorado's own spinal cord injury surveillance history. The spinal cord injury surveillance
system began in 1986 with clinical contact persons as
the only reporting source. Based on this source, the
number of cases ranged from 86 in 1986 to 80 in 1988.
Correspondingly, annual spinal cord injury rates
ranged from 26 to 24 per million. In 1989, hospitals
became subject to a Board of Health regulation requiring them to report hospitalized spinal cord injury to the
state health department. With the addition of this second source, incidence increased to 121 in 1989, with
an annual rate of 36.8 per million population (4). As
the incidence rate rose, it was clearly due to the
inclusion of persons with less severely injured spinal
cords.
The three reporting sources varied greatly with regard to timeliness. Clinical contacts were the most
timely, generally reporting cases within the first few
days of injury. This not only allows for timely surveillance but also for provision of services to the newly
injured person. Bom medical records and the hospital
association had a lengthy delay in reporting, and they
both reported after discharge, making early contact
with survivors and provision of services impossible.
Medical records departments respond to regular mailings and requests for cases identified out of their
database. In Colorado these requests are made either
every other month or every 6 months. Hospitals participating in the clinical contact system are queried
every 6 months. Both request patterns allow for data
collection and medical records review to be an ongoing process throughout the calendar year. A full year
of data on discharges in the prior calendar year was
available from the Colorado Hospital Association 6
months after the close of the prior calendar year.
Since the current method of surveillance uses two
data sources and a third source became available, we
were interested in seeing how the addition of or a
change to the third source would impact the overall
surveillance results. A pooled approach was used to
adjust for possible dependencies between the data
sources. Based on goodness-of-fit statistics (G2, the
Akaike Information Criterion, and the Bayesian Information Criterion), more than one model provided a
good fit for these data. Statisticians do not agree on
which goodness-of-fit statistic to use (19). We chose
the model with only one interaction (hospital association and medical records) based on both the G2 staAm J Epidemiol
Vol. 146, No. 3, 1997
271
tistic and on the fact that this interaction was the only
one that was statistically significant. This model is
equivalent to pooling hospital association and medical
records sources, except that it provides the usual maximum-likelihood estimates and, therefore, does not
adjust for small samples. Additionally, the positive
dependency seen between the hospital association and
medical records is plausible given what we know
about the data sources. Both the medical records and
hospital association sources are based on the medical
record. In contrast, the clinical contacts are a variety of
health care professionals located throughout the hospital who do not necessarily impact the diagnostic
coding that appears on the medical record.
We think that the chosen model gives a reasonable
estimate of overall incidence. The estimate of 19
missed cases, based on stratifying by use of inpatient
rehabilitation, provides an outside range. This stratification, however, introduces error related to the hospital association's inability to identify all rehabilitation
admissions and the clinical contacts' decreased likelihood of needing to provide care and services to the
less severely injured.
One limitation of capture-recapture analysis is that it
can only estimate the cases that are systematically
missed by all three sources if the case was in the pool
of cases to be captured. Therefore, the number of
Colorado residents injured out of state and who do not
return to Colorado for inpatient services cannot be
estimated by the capture-recapture method. In addition
to this group of potentially missed cases, there is also
a very small group of patients that had a reduced
chance of being reported. The Colorado Hospital Association, a voluntary membership organization, does
not capture patients who were seen at Colorado's
federal facilities, although they are covered by the
medical records reporting requirement.
In summary, this study measured sensitivity and
positive predictive value and described the timeliness
of three reporting sources of spinal cord injury in
Colorado. Evaluation of these commonly assessed attributes of surveillance systems (14, 15) has resulted in
the following conclusions/recommendations:
1. Careful review of all suspected cases of spinal
cord injury is necessary because of the substantial number of false positive cases identified.
2. Both medical records reports and the hospital
discharge data set were very complete and may
be best for case identification of all types and
severities of injury.
3. Clinical contacts provided the most timely reporting of cases that were transferred to inpatient
rehabilitation; however, they were less likely to
identify less severely injured cases.
272
Johnson et al.
4. The purpose for surveillance data is key to
choosing a case identification method. The type
and timeliness of the surveillance information
required may influence the method chosen.
The results of this study indicate the potential benefits and limitations of similar reporting sources in
other states. This study reinforces the need to evaluate
surveillance in other states as surveillance priorities
are both established and changed.
ACKNOWLEDGMENTS
Supported in part by Disability Prevention Cooperative
Agreement U59/CCU803364 from the Centers for Disease
Control and by grant H133N5OOO1 from the National Institute on Disability and Rehabilitation Research.
The authors especially thank Russell Rickard for his
statistical expertise.
REFERENCES
1. Sniezek JE, Finklea JF, Graitcer PL. Injury coding and hospital discharge data. JAMA 1989;262:2270-2.
2. Graitcer PL. The development of state and local injury surveillance systems. J Safety Res 1987;18:191—8.
3. Harrison CL, Dijkers M. Spinal cord injury surveillance in the
United States: an overview. Paraplegia 1991;29:233-46.
4. Colorado Department of Public Health and Environment.
1994 annual report of the spinal cord injury early notification
system. Denver, CO: Colorado Department of Transportation
Printing Office, 1995.
5. Fine PR, Kuhlemier KV, Devivo MJ, et al. Spinal cord injury:
an epidemiologic perspective. Paraplegia 1979;17:237-50.
6. Kraus JF, Franti CE, Riggins RS, et al. Incidence of traumatic
spinal cord lesions. J Chronic Dis 1975 ;28:471-92.
7. Thurman DJ, Burnett CL, Jeppson L, et al. Surveillance of
spinal cord injuries in Utah, USA. Paraplegia 1994;32:665-9.
8. Price C, Makintubee S, Herndon W, et al. Epidemiology of
traumatic spinal cord injury and acute hospitalization and
rehabilitation charges for spinal cord injuries in Oklahoma,
1988-1990. Am J Epidemiol 1994; 139:37-47.
9. Acton PA, Farley T, Freni LW, et al. Traumatic spinal cord
injury in Arkansas, 1980 to 1989. Arch Phys Med Rehabil
1993;74:1035-40.
10. Warren S, Moore M, Johnson MS. Traumatic head and spinal
cord injuries in Alaska (1991-1993). Alaska Med 1995;37:
11-19.
11. Thurman DJ, Sniezek JE, Johnson D, et al. Guidelines for
surveillance of central nervous system injury. Atlanta: Centers
for Disease Control and Prevention, 1995.
12. Goldberg J, Gelfan HM, Levy PS. Registry evaluation
methods: a review and case study. Epidemiol Rev 1980;2:
210-20.
13. Hanrahan LP, Moll MB. Injury surveillance. Am J Pubic
Health 1989;79(suppl):38-45.
14. Klaucke DN. Evaluating public health surveillance. In:
Teutsch SM, Churchhill RE, eds. Principles and practice of
public health surveillance. New York: Oxford University
Press, 1994:158-74.
15. Klaucke DN, Buehler JW, Thacker SB, et al. Guidelines for
evaluating surveillance systems. MMWR Morb Mortal Wkly
Rep 1988;37(suppl5):l-18.
16. Fienberg SE. The multiple recapture census for closed populations and incomplete 2k contingency tables. Biometrika
1972;59:591-603.
17. Hook EB, Regal RR. Effect of variation in probability of
ascertainment by sources ("variable catchability") upon
"capture-recapture" estimates of prevalence. Am J Epidemiol
1993;137:1148-66.
18. Wittes JT, Colton T, Sidel VW. Capture-recapture methods
for assessing the completeness of case ascertainment when
using multiple information sources. J Chronic Dis 1974;27:
25-36.
19. Hook EB, Regal RR. Capture-recapture methods in
epidemiology: methods and limitations. Epidemiol Rev 1995;
17:243-64.
20. Hook EB, Regal RR. The value of capture-recapture methods
even for apparent exhaustive surveys. Am J Epidemiol 1992;
135:1060-7.
21. SAS Institute, Inc. The CATMOD procedure. In: SAS/STAT
user's guide, release 6.03 ed. Cary, NC: SAS Institute, Inc.,
1988:189-282.
22. US Health Care Financing Administration. International classification of diseases. Ninth Revision, clinical modification.
Vol 1. Washington, DC: US GPO, 1989. (DHHS publication
no. (PHS) 89-1260).
23. American Spinal Injury Association and International Medical
Society of Paraplegia. Internationa] standards for neurological
and functional classification of spinal cord injury, revised
1992. Chicago: American Spinal Injury Association, 1994.
Am J Epidemiol Vol. 146, No. 3, 1997