A POPULATION-BASED COMPARISON OF CIREN AND
NASS CASES USING SIMILARITY SCORING
Joel D. Stitzel1,2, Patrick Kilgo3, Brian Schmotzer3, H. Clay Gabler2,
J. Wayne Meredith1
1
Wake Forest University School of Medicine, Medical Center Blvd,
Winston-Salem, NC 27157
2
Virginia Tech – Wake Forest University Center for Injury
Biomechanics, Medical Center Blvd, Winston-Salem, NC 27157
3
Emory University, Atlanta, GA 30322
ABSTRACT
The Crash Injury Research and Engineering Network
(CIREN) provides significant details on injuries, and data on patient
outcomes that is unavailable in the National Automotive Sampling
System (NASS). However, CIREN cases are selected from specific
Level I trauma centers with different inclusion criteria than those
used for NASS, and the assertion that a given case is similar to the
population of NASS cases is often made qualitatively. A robust,
quantitative method is needed to compare CIREN to weighted NASS
populations. This would greatly improve the usefulness and
applicability of research conducted with data from the CIREN
database. Our objective is to outline and demonstrate the utility of
such a system to compare CIREN and NASS cases.
This study applies the Mahalanobis distance metric
methodology to determine similarity between CIREN and
NASS/CDS cases.
The Mahalanobis distance method is a
multivariate technique for population comparison. Independent
variables considered were total delta V, age, weight, height,
maximum AIS, ISS, model year, gender, maximum intrusion,
number of lower and upper extremity injuries, and number of head
and chest injuries. The technique provides a unit-independent
quantitative score which can be used to identify similarity of CIREN
and NASS cases.
Weighted NASS data and CIREN data were obtained for the
years 2001-2005. NASS cases with Maximum AIS 3 resulted in a
subset of 1,869 NASS cases, and 2,819 CIREN cases.
Results of the analysis demonstrate the utility of the distance
technique to identify similarity of CIREN cases with the average
NASS case. All NASS means were within 10% of CIREN and higher
except Total Delta V was 9% higher for CIREN, CIREN cases were
50/50 male:female, and mortality of CIREN cases was 38% lower
than for NASS/CDS. Results demonstrate that on average the
CIREN cases analyzed had a greater proportion clustered about the
mean distance for NASS cases than did the NASS cases, with a very
similar average distance (similarity score) of 3.75. Maximum
distance (worst similarity score) was 14.75. The mean peak in
probability density for CIREN (0.37) is slightly higher than for
NASS (0.34). The distribution in the main body for both datasets is
unimodal and nearly symmetric, and the overall distribution is
slightly skewed right. For body region specific injuries, similiarity
increases then decreases gradually with increasing number of injuries
for distance scores between 1 and 2. Maximum AIS similarity
increases and then begins to decrease with a minimum distance of
about 1.5. Distance is very high (9) for very low age (<1 year), and
similarity improves to 3-4 only around 12-13 years of age,
representing the dissimilarity between children and adults in the
CIREN and NASS populations. Model year distribution indicates hat
older and more recent model years are not more associated with a
lower similarity score in and of themselves.
This study has important implications for CIREN research
studies, for which a similarity score could be assigned to each
CIREN case based on overall, crash, anthropometric, or injury
severity to the NASS population or another population of interest.
The result is a tool that can be used with CIREN data to make
stronger conclusions about the biomechanics and outcome of
injuries, by quantitatively demonstrating real-world relevance.
INTRODUCTION
Since its inception in 1998, the Crash Injury Research and
Engineering Network (CIREN) has been involved in research into
the causation of injury in automobile crashes (Runge 1996; Scally,
McCullough et al. 1999; NHTSA 2001; Wang 2001). A unique
feature of CIREN cases is that all case occupants receive care at a
Level I trauma center and receive a full case review with medical
personnel. Case occupants in the CIREN database also receive
followup interviews, providing valuable data on outcomes of persons
involved in car crashes.
However, CIREN case inclusion criteria are very different
from National Automotive Sampling System (NASS) inclusion
criteria, in which cases are sampled on a regional geographic basis,
using two zone centers and 27 primary sampling units (PSUs) spread
throughout the United States. NASS cases use a statistical weighting
technique designed to make the sample population based, i.e. the
NASS database is intended to be a national sample which represents
accurately the spectrum of types of crashes in the United States.
CIREN is a powerful tool for collecting detailed injury data
which relates to clinical outcomes. CIREN also collects the full set of
National Automotive Sampling System (NASS) data (crash data), but
also collects other medical data. These data include the International
Classification of Diseases, 9th edition (ICD-9) codes, Orthopedic
Trauma Association (OTA) codes, and other more common clinical
injury data, as well as long-term 6 and 12 month outcomes through
the 36-item Short Form Health Survey (SF-36) and the Pediatric
Quality of Life Inventory (PedsQL) reports, radiology reports and
operative notes, a detailed hospital course for case occupants, and the
case occupant’s radiology studies.
In the past CIREN has been used to analyze crash and vehicle
characteristics and their relationship to injury patterns. Early CIREN
work focused on factors influencing injury pattern and outcomes in
car vs. car impacts versus other vehicle categories (Siegel, Loo et al.
2001). Leg and lower extremity injuries quickly surfaced as
important, as those involved in CIREN research realized the
outcomes from these injuries were of great importance, and some of
the first CIREN biomechanics studies were undertaken (Assal, Huber
et al. 2002; Tencer, Kaufman et al. 2002). One of the earliest CIREN
studies focused also on aortic injury and the relationship of outcome
to the associated injuries as well as the crash characteristics involved
(Siegel, Smith et al. 2002).
Subsequently, several researchers have attempted to combine
CIREN and NASS analyses into one study, drawing on CIREN for
information about biomechanical causation of injury while also
performing a side-by-side analysis of the same injury using NASS
data. These studies have focused on biomechanics of neck injuries,
specifically C2 Dens and Odontoid fractures (Yoganandan, Pintar et
al. 2004; Yoganandan and Pintar 2005) the proper use of automatic
crash notification systems to identify patients with serious injuries
(Augenstein, Perdeck et al. 2003), and pelvic and thoracic injuries
(Tencer, Kaufman et al. 2005), the last of which is notable for
including an analysis of NCAP data as well.
The detailed injury data available in CIREN also lends itself
to prediction of injury using computational models. These types of
studies have been undertaken for the pelvis (Tencer, Kaufman et al.
2007), thoracic aorta (Siegel, Yang et al. 2006) and head (Moran,
Key et al. 2004).
A number of general studies outline characteristics of cases in
CIREN and may be organized by occupant type, crash type, or body
region. Pediatric-specific CIREN studies have been undertaken
(Brown, Jing et al. 2006), as well as studies focusing on the
protective role of subcutaneous fat against injuries in the abdominal
region (Wang, Bednarski et al. 2003). One crash type study focused
on near-side impact and the effect of door characteristics (Tencer,
Kaufman et al. 2005), and another on vehicle mismatch (specifically
light truck and passenger vehicle) (Acierno, Kaufman et al. 2004).
Body-region specific studies form a greater proportion of the studies
undertaken utilizing CIREN data. Maxillofacial and orbital injury
studies focus on the role of airbags in orbital blowouts (Francis,
Kaufman et al. 2006) and in maxillofacial injuries (Brookes 2004)
and also highlight the role of the a-pillar in maxillofacial injuries
(Brookes, Wang et al. 2003). Pelvic injuries are of frequent interest
in CIREN studies and tend to focus on biomechanics and long term
consequences (Stein, O'Connor et al. 2006). Upper extremity studies
(Conroy, Schwartz et al. 2006), and lower extremity studies also
exist, a few of the lower extremity studies focusing on calcaneal
fractures (Benson, Conroy et al. 2007), lower extremity injuries and
poor long term outcome (Read, Kufera et al. 2004), and femur
fractures in low speed crashes and the effect of muscle forces
(Tencer, Kaufman et al. 2002). Spinal cord (Smith, Siegel et al.
2005) and thoracic aortic injury studies exist (Siegel, Smith et al.
2004).
Finally, mild traumatic brain injury (MTBI) studies
(Dischinger, Read et al. 2003) which focus on long term outcomes,
and head injury (Nirula, Mock et al. 2003) studies which focus on
vehicle contact points are of course included.
A unifying theme in all of these studies has been their use of
detailed injury data: injury data much more detailed than can be
obtained from the NASS database, and their increased focus on longterm outcomes of patients. Many of these studies are outlined as
investigations into mechanism of injury and injury outcomes, and
many are presented as case studies with guarded assertions about
their relationship to injury prevalence in the total population of motor
vehicle crashes. In many of the studies very detailed information is
available about the injury, yet the full vehicle investigation (NASS
data) is still included, facilitating the understanding of vehicle crash
performance and safety system performance to the injury sustained.
A potential limitation to extending the conclusions of many
of these studies to the full population of cases in the United States
has been the use of CIREN data itself. Because the CIREN dataset is
a small dataset that does not represent a population-based sample, the
conclusions about individual cases or a group of cases within CIREN
carry some weight but are more difficult to generalize to the general
population of crashes in the field. It is difficult to determine whether
a given CIREN case represents a ‘one in a million’ crash or injury
scenario that is not expected to be representative of a typical realworld crash.
There is a key similarity between NASS and CIREN, which
enables one to make an important comparison between CIREN and
NASS. Because the CIREN data system uses the NASS crash data
structure as a backbone for all of its crash data, all of the entries are
the same and this part of the database is virtually identical between
NASS and CIREN. That means that one can use the fields entered in
NASS for a CIREN case to make a comparison between case
similarity for the two databases.
Fortunately, analytical and statistical techniques allow easy
comparison between datasets and there are statistical techniques to
determine if a given case or set of cases is similar to another
population of cases. Thus, the objective of this study is to present a
method to quantify the similarity between a given CIREN case, or a
subset of CIREN cases of interest, or the entire CIREN database, and
the population of NASS cases. In so doing, it would be possible to
say more definitively that measures from CIREN are highly
applicable to real world crashes, based on their similarity to NASS
crashes. The end goal is a ‘similarity score’ that can be assigned to a
CIREN case to compare the datasets. Hypothetical questions that
might be answered using this information include: Given a subset of
NASS cases of interest,
1) How do we identify CIREN cases that ‘match’ it, or are very
similar?
2) What is the best way to make a comparison?
3) What variables in NASS or CIREN are important in describing
the differences? All these questions are not answered but are
proposed as a framework for future research.
Figure 1 is a pictorial representation of the question. The
population of NASS cases can be described as a large circle, and a
population of CIREN cases as a smaller circle. A ‘similarity score’
can be used to determine if CIREN cases are a subset of NASS/CDS,
what the nature and size of this overlap is, and whether CIREN
contains crashes that are very dissimilar, or too dissimilar to be
considered to be within the universe of NASS crashes.
NASS/CDS
NASS/CDS
NASS/CDS
CIREN
CIREN
Is CIREN a subset
of NASS/CDS?
What is the nature
and size of the
overlap?
CIREN
Does CIREN contain
crash scenarios not
in NASS/CDS or
vice - versa?
Figure 1. Schematic of ways to think about NASS/CDS and CIREN databases.
Depending on the nature of a given study, conclusions from
the CIREN database may be criticized because of the sampling
technique. It is easy to argue that a given CIREN crash may or may
not be representative of the population in a qualitative way.
However, few of these arguments are made quantitatively.
The objective of this study is to present a quantitative method
by which a given CIREN case or a population of CIREN cases may
be compared quantitatively to the population of NASS cases, so that
more substantive conclusions can be made using CIREN data.
METHODS
The general approach is to seek to compute a ‘similarity score’
between CIREN cases and the average NASS case. This similarity
can be computed using any subset of variables contained in both
databases, resulting in a k-dimensional distance, where k is the
number of variables common between the data sources that are being
compared.
The Mahalanobis distance computation can be used to make
this comparison, and the resulting Mahalanobis Distance (DIST) can
be thought of as a similarity score to compare CIREN cases to the
‘average’ NASS case.(Mahalanobis 1936) Mahalanobis distance is a
multivariate measure of distance. All variables are compared on the
same scale - DIST standardizes them, so individual units for each
measurement don’t matter. The method takes into account the
correlation between the k variables as well. So, for instance, if high
Delta-V crashes would normally result in large intrusions, it is low
delta V crashes with high intrusions or high delta V crashes with low
intrusions that would stand out, not necessarily a very high delta V
crash. These correlations statistically are made utilizing every entry
considered in the analysis.
The distance score is based on correlations between variables.
Mahalanobis distance differs from Euclidean distance as it takes into
account the correlations of the data set and is invariant to scale– i.e. it
is not dependent on the scale of measurements, or the units of the
individual variables.
The Mahalanobis distance is computed using a group of k
variables with mean μ:
μ = (μ1 , μ 2 , μ3 ,..., μ k ) T
And a covariance matrix ∑ for the vector x
T
x = ( x1 , x2 , x3 ,..., xk )
Mahalanobis Distance (DIST) is defined as:
DIST ( x ) =
(x − μ )T ∑ −1 (x − μ )
A higher DIST score means that a particular sample is less
similar to a population, and a lower DIST score means that the
sample is more similar to the population. A DIST score of 0 would
indicate a case with the means exactly the same as the population (xμ=0, a scenario which becomes less probable with smaller datasets
and greater numbers of variables being compared.
An example of this is to take the height and weight of two
men (Figure 2). In general, weight is higher for people who are taller,
and height is lower for people who weigh less. Male #1 is 6 inches
above average height and 100 lb above average weight. Male #2 is 6
inches above average height, and 100 lb below average weight.
Using Euclidean distance, these two cases are equidistant from the
average (centroid). Each person is equidistant in the x-y direction
from the average. However, by Mahalanobis distance, Male #2 is
much further from the average. The DIST technique identifies the
Male who is outside the average for the population (Male #2) and
assigns a larger distance to him than for Male #1.
Male #1
Weight
1σ
Average
Male
Male #2
Height
Figure 2. Simple example illustrating Mahalanobis distance. Male 1 and 2 are
equidistant from the average in Cartesian coordinate space, but Male 2 is much
further from the average and would have a higher DIST or less similarity.
Mahalanobis distance is therefore an effective multivariate
measure for how far points are apart in k-space in the context of the
correlations between them. DIST can be used as a similarity score,
and this is often how it is used.
Since DIST is a multivariate measure and the score is
computed in k-space, k can be any number of variables, and could
include every point in the dataset. For the current study, the
variables in Table 1 were included. They are separated into Crash,
Injury, and Anthropometric Variables.
Table 1. Variables included in analysis of NASS and CIREN databases.
Crash Variables
Total Delta V
Vehicle Model Year
Maximum Intrusion
Anthropometric
Variables
Occupant Age
Occupant Gender
Injury Variables
Maximum Abbreviated
Injury Score (MaxAIS)
Injury Severity Score (ISS)
Number of Lower
Extremity Injuries
Number of Upper
Extremity Injuries
Occupant Height
Occupant Weight
Number of Head Injuries
Number of Chest Injuries
Included in the analysis were, for NASS, all cases with
Maximum Abbreviated Injury Score (MAIS) ≥ 3, from years 2001 –
2005. This resulted in a subset of 1869 NASS cases. For CIREN, all
CIREN cases from 2001 to August 2006 were chosen, resulting in a
subset of 2,819 cases.
Because the DIST score requires a mean for each variable
included, and there are data missing from both the CIREN and NASS
databases, missingness must be addressed.
Three common
possibilities are: 1. impute with averages, 2. estimate the covariance
structure with multiple imputation methods, or 3. delete the
observations. The selection of an imputation method taking into
account the covariance structure would be subject to some debate and
is probably deserving of a study in itself. Deleting the observations
would result in a much reduced and incomplete dataset, and therefore
was not performed. For this study, missing variables were imputed
with column-wise averages.
The weighting system in NASS is part of what makes NASS
the population-base sample that it is. In this analysis, NASS sampling
weight coefficients were used to weight the covariance matrix. This
approach is basically as if each row of data (each case occupant) was
repeated k times, where k is the NASS weight for that row. This has
the effect of expanding the population of NASS cases greatly, but
that is what happens in all NASS analyses.
RESULTS
PART I: GENERAL COMPARISON OF NASS AND
CIREN DATABASES: Comparing the NASS and CIREN databases
for one year (2005) highlights an important difference between the
databases. Because the primary NASS/CDS selection criteria is that
the case involve a ‘tow-away’ crash, NASS contains more than half
(57%) MAIS 0 crashes, and 35% MAIS 1 crashes, and 6% MAIS 2
crashes. These are largely low injury crashes with very low threat to
life. CIREN contains stipulations for including low MAIS (usually
MAIS 2) but contains mostly MAIS 3, 4, and 5 cases (Figure 3).
60%
NASS/CDS (weighted)
CIREN
50%
0
4%
0.1%
13%
0.2%
20%
47%
0.5%
0%
1.6%
0%
4%
10%
12%
35%
20%
6%
30%
57%
% Cases
40%
1
2
3
4
5
Maximum Abbreviated Injury Score (Max AIS)
6
Figure 3. Distribution of cases by Maximum AIS – NASS/CDS 2005 versus
CIREN. NASS is composed of lower severity cases.
Selecting NASS/CDS and CIREN for AIS 3+ cases only in
the year 2005, weighted and unweighted NASS/CDS and CIREN are
compared (Figure 4). NASS/CDS and CIREN are very similar in
terms of distribution of Maximum AIS, with lower Max AIS slightly
underrepresented and higher Max AIS slightly overrepresented in
CIREN. This effect seems to be enhanced by the weights in NASS,
as unweighted numbers are closer to CIREN numbers.
70%
NASS/CDS (weighted)
NASS/CDS (unweighted)
CIREN
60%
56%
30%
56%
40%
67%
% Cases
50%
5%
6%
0%
3%
16%
16%
9%
23%
10%
22%
21%
20%
3
4
5
6
Maximum Abbreviated Injury Score (Max AIS)
Figure 4. Distribution of MAIS3+ cases – NASS/CDS 2005 versus CIREN.
NASS/CDS and CIREN are roughly comparable.
Weighted NASS and CIREN means for some of the variables
included in the analysis are shown in Table 2.
Table 2. Weighted NASS means and CIREN means for study dataset.
Variable
Total Delta V (kph)
Age (years)
Weight (kg)
Weighted
NASS Mean
60.4
38.0
75.7
CIREN
Mean
65.8
36.7
72.8
Height (m)
Maximum AIS
ISS
Mortality
Vehicle Model Year
Male:Female
1.69
3.70
23.6
0.24
1994
56:44
1.64
3.39
21.6
0.15
1997
50:50
By normalizing CIREN means to NASS means (NASS
Mean/NASS Mean=1, CIREN Mean/NASS Mean=X) an easier
comparison can be made. All CIREN variables compared are within
10% of their NASS means (vary from 0.9 to 1.1) except for
mortality, which is about 63% of the NASS mean for mortality, and
gender, which is 89% of the NASS mean. In the CIREN sample the
ratio of males to females is more equal (50:50), and the mortality of
CIREN patients is on average much lower than that of NASS
patients. All CIREN patients go to a Level I trauma center, a
potential partial explanation for differential mortality which was not
tested in this study. A more likely explanation is that fatalities are
not avoided in NASS, but are generally avoided in CIREN unless
they occur after the person has been admitted to the trauma center.
This finding, in fact, underscores the need for a metric to compare
NASS to CIREN, as mortality variation would tend to change the
average DIST for a NASS case versus a CIREN case. This is likely a
variation clearly influenced by inclusion criteria, which must be dealt
with in analysis by selecting cases for comparison based on the
criteria, or coming up with a comparison metric like DIST.
Normalized NASS means
1.1
Normalized CIREN means
Normalized Mean
1
0.9
0.8
0.7
Male
Year
Mode
l
Mort
ality
ISS
S
Max
AI
Heig
ht (m
)
Age
ht (kg
)
Weig
Total
Delta
V
0.6
Figure 5. Normalized CIREN to NASS Means
Using the DIST score one of the most informative
comparisons to make is to compare the NASS and CIREN
populations directly by looking at probability density of the
similarity scores. In the calculations, the similarity score for a
CIREN case is calculated using the covariance matrix for the NASS
population, and the NASS population similarity scores are also
calculated and plotted for comparison.
The distance distributions are roughly the same. In fact there
are a greater proportion of cases clustered about the mean distance
for CIREN data than there are for NASS data. The mean peak for
CIREN is slightly higher (0.37) than for NASS (0.34). The
distribution in the main body for both datasets is unimodal and nearly
symmetric, and the overall distribution is skewed toward lower
similarity cases. Near the lowest distances there is a small collection
of CIREN data that is very near the average which may represent
imputed data. One might infer from this graphic, particularly from
the greater proportion of CIREN data clustered about the mean
distance of 2.5 to 4.0, that CIREN is more like the average NASS
case than NASS, but NASS is the standard to which CIREN is being
compared and though the distributions are similar they are not
identical.
0.4
CIREN
NASS
CIREN
Probability Density
0.3
0.2
0.1
14
12
10
8
6
4
2
0
0
Mahalanobis Distance
Figure 6. Probability density versus Mahalanobis distance.
The NASS and CIREN means are similarly distributed, with a greater proportion of
CIREN cases clustered about the mean than for NASS cases.
Unfortunately the DIST score, being a calculation in k-space
and a multivariate measure, does not lend itself to simple
visualization of the populations being compared.
However,
simplifications allow more direct comparison between the databases.
One potential method is to perform a principal components (PC)
analysis, whereby linear combinations of the variables best
describing the variability within the datasets are calculated. Taking
the first 2 principal components allows one to graphically depict the
maximum amount of variation in the datasets while also allowing
visualization in a two-dimensional (x and y axis) plot. This allows
one to look at the data in a figure something like Figure 2, and
compare population overlap. By plotting the first two principal
components for each entry in the database and separating CIREN
from NASS one can visualize the overlap between the two datasets.
The result is Figure 7.
There is a remarkable amount of overlap between the two
datasets. The two principal components here together explain 39.0%
of the point variability and so may be interpreted to give a pretty
good picture of the overall variability in the dataset. They are not as
comprehensive as the DIST score, but explain much of the
variability. Some differences are apparent. Looking at the variable
coefficients from the principal components, some of the differences
(the CIREN data skewed to the right on the PC1 axis) are largely due
to some low MAIS included in the analyses of CIREN that were not
included in NASS.
One could also draw a smaller concentric ellipse centered on
the CIREN data mean (PC1=PC2=0) represented by the thick black
line but stopping at the maximum extent of the NASS population. In
so doing one would minimize the DIST score as well. This has
important implications, as in a given analysis the NASS or CIREN
data of interest can be chosen at will, and by minimizing the DIST
score or controlling the PC, one can choose the data from the other
source with greater similarity to the mean. This effect is
demonstrated visually using the PC plots, but in practice one could
use one score – DIST – to accomplish this task.
- NASS
- CIREN
6
Principal Component 2
4
2
0
-2
-4
-10
-5
0
Principal Component 1
5
Figure 7. Principal Component plots for NASS and CIREN data showing dataset
overlap. The first 2 principal components explain 39.0% of the variability in the
datasets. A subset (dark black line) of CIREN data has been chosen which lies
within the area containing NASS data.
The next few figures illustrate DIST versus some of the
region-specific injury variables included in the analysis. For number
of head, chest, upper extremity, and lower extremity injuries the
minimum DIST increases gradually with increasing number of
16
16
14
14
Mahalanobis Distance
Mahalanobis Distance
injuries. With increasing number of injuries the minimum DIST
increases very quickly and most quickly for lower extremity injuries.
12
10
8
6
4
2
0
10
8
6
4
2
0
0
5
10
# Head Injuries
15
0
16
16
14
14
Mahalanobis Distance
Mahalanobis Distance
12
12
10
8
6
4
2
0
5
10
# Chest Injuries
15
12
10
8
6
4
2
0
0
5
10
# Upper Extremity Injuries
15
0
5
10
15
20
# Lower Extremity Injuries
25
Figure 8. Mahalanobis Distance versus Number of Injuries to the Head, Chest,
Upper, and Lower Extremities
16
16
14
14
Mahalanobis Distance
Mahalanobis Distance
Figure 9 shows DIST versus crash and vehicle characteristics
for Total Delta V, Maximum Intrusion, and Model Year. More of
the data is clustered below the mean (Minimum DIST) than for
injury variables. Model year alone has a relatively weak relationship
to DIST.
12
10
8
6
4
2
0
12
10
8
6
4
2
0
0
20
40
60 80 100 120 140 160
Total Delta V (kph)
0
30
60
90
120
Maximum Intrusion (cm)
150
Mahalanobis Distance
16
14
12
10
8
6
4
2
0
1980
1985
1990 1995 2000
Model Year
2005
2010
Figure 9. Mahalanobis distance versus Total Delta V, Maximum Intrusion, and
Model Year
For injuries, Maximum AIS drops between 3 and 4 for
minimum distance (about 1.5), and for very low or very high Max
16
16
14
14
Mahalanobis Distance
Mahalanobis Distance
AIS, DIST increases quickly to 3 and higher, even for AIS 2 or 5.
The ISS distribution is distinctly different from many of the graphics,
because ISS has a collection of possible values which are not equally
spaced, there is a less discernable V shape to the minimum DIST in
the distribution and DIST does not seem to be very strongly
influenced by ISS, but has a lower minimum at ISS of around 21.
12
10
8
6
4
2
0
12
10
8
6
4
2
0
0
2
4
Maximum AIS
6
8
0
10
20
30
40
50
60
Injury Severity Score
70
80
Figure 10. Mahalanobis distance versus Maximum AIS and ISS
16
16
14
14
Mahalanobis Distance
Mahalanobis Distance
Figure 11 shows anthropometric variables. For age, DIST is
very high for very low age representing the dissimilarity between
children and adults in the CIREN population. Children are not as
well represented in the CIREN or NASS databases as adults and so
their anthropometric characteristics tend to create high DIST scores.
12
10
8
6
4
2
0
12
10
8
6
4
2
0
0
10 20 30 40 50 60 70 80 90 100
Age
0
50
100
Weight, kg
150
200
Mahalanobis Distance
16
14
12
10
8
6
4
2
0
40
60
80 100 120 140 160 180 200 220
Height, cm
Figure 11. Mahalanobis Distance versus Age, Weight, and Height
DISCUSSION
Examining the DIST versus independent variable
distributions does not give a complete picture of the relationship
between that variable and DIST, as DIST is a multivariate measure,
but does infer the effect of altering inclusion criteria in the CIREN
database. It can hint at how to alter the independent variable
distributions chosen for inclusion in a given analysis, to minimize
overall distance.
For instance, one could simply decide to take cases for an
analysis for which the overall similarity score is no higher than 5.
Using the current approach, that dataset would include 62.5% of the
CIREN data. One could account for much of the 37.5% of the data
excluded from such an analysis by using the thresholds in Table 3.
These values are obtained by examining the minimum distance at
each of the levels of the independent variable, and establishing that
value as a threshold above which any value of the independent
variable is likely to result in a DIST score of 5 or more.
The values in Table 3 represent a hypothetical scenario
created to suggest how the inclusion criteria in a given analysis might
be changed in order to influence the similarity score, without
restricting the similarity score directly. A possible use of this
information is to utilize similarity scoring to restrict future inclusion
criteria in CIREN database in order to prospectively change inclusion
criteria for the database to contain cases more similar to NASS cases.
CIREN inclusion criteria are subject to variation over time but these
criteria are usually changed based on expert discussion and
qualitative reasoning. The similarity score gives a way potentially to
quantify this reasoning.
However, there are limitations to this approach. Holding the
independent variable within these ranges will eliminate high DIST
scores certain to occur outside the range of interest. However
limiting DIST by limiting a single independent variable is not
sufficient to eliminate all low similarity cases within the ranges
chosen. For that, a similarity score threshold would be needed which
would have the effect of limiting all independent variables.
Table 3. Hypothetical CIREN minimums and maximums for inclusion criteria
to help attain a similarity score (DIST) of 5 or less
Variable
Total Delta V (kph)
Age (years)
Weight (kg)
Height (cm)
Maximum AIS
ISS
# Head Injuries
# Chest Injuries
# Upper Ex Injuries
# Lower Ex Injuries
Max Intrusion (cm)
Vehicle Model Year
CIREN
Minimum
0
0
7
97
1
0
0
0
0
0
0
1994
CIREN
Mean
65.8
36.7
72.8
164
3.39
21.6
1.131
1.236
1.542
2.72
30.8
1997
CIREN
Maximum
94
94
159
N/A
6
75
9
8
9
11
101
N/A
Another possible use for the similarity score is for quality
control of the data. CIREN and NASS, like all databases, are subject
to misentry of data or incorrect data. The Mahalanobis distance
technique identified a number of cases in CIREN and NASS where
there appeared to be errors in entry. An additional benefit of
utilizing a similarity score would be that obvious errors in entry
(entries off by an order of magnitude because of an extra digit typed,
etc) would artificially increase the distance score and would draw
further scrutiny during the QC process. For example, a 5 kph delta V
crash with a maximum intrusion of 60 cm is highly unlikely and
might indicate that 5 kph was entered when 50 kph was intended, or
60 cm was entered when 6 cm was intended. However it is notable
that there are only 4 or 5 of these cases in CIREN, and similarly
small percentage in NASS, which indicates a high level of quality
control in these databases. The 6-10 highest DIST scores in CIREN
are all children, with large delta v’s and low injuries, or high ISS
with survival. This points out the uniqueness of the pediatric
population. Nevertheless, the DIST score could be used as one form
of quality control for the database.
One important point relates to missingness in the CIREN and
NASS datasets. Since the dataset chosen in this study had small
amounts of missing data column-wise averages were imputed rather
than taking a more sophisticated approach like multiple imputation
(Schafer 1999). For datasets with moderate to large amounts of
missing data the use of these methods could have some utility and
might confer some estimation advantages above and beyond simply
imputing with sample means. This issue should be investigated
further within the context of database comparisons using
Mahalanobis Distance.
A final point regarding this method is as it relates to the
pediatric population. Examining Figure 11 it is difficult to avoid the
inference that age-related variables particularly in younger age
ranges seem to be strongly influencing the DIST score. In terms of
the implications for analysis of pediatric cases, the analysis suggests
that aspects of pediatric anthropometry tend to change the DIST
score substantially, but this is based on an analysis using relatively
few variables. In practice, it may be prudent to select crash,
demographic, or injury criteria of interest and then use the DIST
metric to select a population of cases from NASS and CIREN that
are similar. It would be up to the person(s) doing the analysis to
decide which variables should be limited to select cases for
comparison, and which should then be included in that comparison,
or whether all variables should be included in that comparison. This
analysis suggests that when two to three of 13 variables are highly
variable for the pediatric population (height, weight, and age), those
variables tend to create large variation in DIST scores. This may be
related to the fact that while other variables of interest may be
expected to vary normally, height and weight tend to change
exponentially with age. Future analysis using this method may need
to account for variables which have an exponential dependence on
other variables included in the analysis.
RECOMMENDATIONS FOR IMPLEMENTATION: One of the
most useful methods to implement this approach may be to assign a
similarity score to each CIREN case retrospectively and
prospectively. One could create 4 additional derived variables in the
CIREN database: overall similarity, anthropometric similarity, crash
similarity, and injury similarity.
The overall similarity score could include all of the variables
in the NASS and CIREN databases common between both databases,
or a comprehensive subset determined by a panel of experts. The
anthropometric similarity score would consist of height, weight,
gender, etc. The crash similarity score would consist of PDOF, Delta
V, Crush, Collision Deformation Classification, etc. The injury
similarity would consist of MAIS, ISS, body region specific
variables, outcome, etc. Each of these criteria should probably be
determined by a group of experts familiar with CIREN and NASS
and should be implemented into the CIREN database as a standard.
Doing so would allow for the selection of CIREN cases for a given
study.
The similarity scoring approach for selecting CIREN cases
would not always need to be NASS mean-specific either. For instance
if one were interested in doing a study of injuries in the elderly in
CIREN, one could use an age criteria to select cases in NASS, and
then calculate a similarity score for CIREN cases to identify that
population in CIREN. Doing so would allow for greater control over
the population selected, a quantitative measure of that populations’ or
case’s similarity, and the conclusions drawn from such an analysis
could be made in a quantitatively stronger way.
The similarity scoring approach for selecting CIREN cases
would also not always need to be NASS specific. For instance if one
were interested in doing a more exhaustive study of outcomes, the
National Trauma Databank (NTDB), a collection of over 2 million
cases with detailed injury and outcome data (containing the fact that
an individual was involved in a car crash but not detailed data about
the crash) could be used. One could include some of the
anthropometric and all of the injury variables in CIREN to determine
an injury similarity to the NTDB, and utilize the two databases to
draw conclusions about injury and outcome in motor vehicle crashes.
Finally, the similarity score could be used as a sort of merge
variable, to allow the strengths of NASS (population based) and
CIREN (detailed injury data) to be utilized concurrently. One could
perform NASS analyses and infer injury and outcome information
from similar cases in CIREN. Conversely, one could put a set of
CIREN injuries or outcomes into context by presenting a set of
similar cases from NASS and inferring real world incidence using
population-based data.
CONCLUSION
For the variables chosen, CIREN crashes are similar to NASS
crashes for serious to fatal injuries (MAIS 3+),and taking into
account all of the assumptions made in this study. This is a valuable
approach to improve understanding and use of CIREN data. The
method could be refined to be able to assign an overall NASS
similarity score to each CIREN crash.
More importantly, one might envision a methodology to
describe crash similarity, anthropometric similarity, or injury
similarity alone. Any of these could be used to create a comparison.
However, one could extend this to include all variables common
between the datasets, and eliminate some of the criticism of the
analysis method pertaining to the use of a non-population based
sample. Lastly, a corollary to the study, is that Mahalanobis distance
represents a good method to quality control data in CIREN and
NASS by pointing out entries that are far from the ordinary.
This is the first study to quantitatively show the similarity
between CIREN and NASS/CDS. The similarity score distributions
are alike, showing CIREN cases are similar to NASS cases with
serious injuries. A NASS similarity score assigned to each CIREN
case could be used to select similar cases based on crash,
anthropometric or injury characteristics. This has important
implications for the usefulness of CIREN in rulemaking, for which
NASS is currently the gold standard. An approach to incorporate the
technique has been outlined. Given the detailed data on biomechanics
of injuries and outcomes that is available from CIREN, this
technique will allow CIREN data to have an increased level of
relevance to organizations making decisions about how to improve
crash safety.
ACKNOWLEDGEMENTS
Work was performed for the Crash Injury Research and
Engineering Network (CIREN) Project at Wake Forest University
School of Medicine in cooperation with the United States
Department of Transportation/National Highway Traffic Safety
Administration (USDOT/NHTSA). Funding has been provided by
Toyota Motor North America Inc. under Cooperative Agreement
Number DTNH22-05-H-61001. Views expressed are those of the
authors and do not represent the views of any of the sponsors or
NHTSA.
REFERENCES
Acierno, S., R. Kaufman, et al. (2004). "Vehicle mismatch: injury
patterns and severity." Accid Anal Prev 36(5): 761-72.
Assal, M., P. Huber, et al. (2002). "Are drivers more likely to injure
their right or left foot in a frontal car crash: a crash and
biomechanical investigation." Annu Proc Assoc Adv
Automot Med 46: 273-88.
Augenstein, J., E. Perdeck, et al. (2003). "Characteristics of crashes
that increase the risk of serious injuries." Annu Proc Assoc
Adv Automot Med 47: 561-76.
Benson, E., C. Conroy, et al. (2007). "Calcaneal fractures in
occupants involved in severe frontal motor vehicle crashes."
Accid Anal Prev.
Brookes, C., S. Wang, et al. (2003). "Maxillofacial injuries in North
American vehicle crashes." Eur J Emerg Med 10(1): 30-4.
Brookes, C. N. (2004). "Maxillofacial and ocular injuries in motor
vehicle crashes." Ann R Coll Surg Engl 86(3): 149-55.
Brown, J. K., Y. Jing, et al. (2006). "Patterns of severe injury in
pediatric car crash victims: Crash Injury Research
Engineering Network database." J Pediatr Surg 41(2): 362-7.
Conroy, C., A. Schwartz, et al. (2006). "Upper extremity fracture
patterns following motor vehicle crashes differ for drivers and
passengers." Injury.
Dischinger, P., K. Read, et al. (2003). "Causes and outcomes of mild
traumatic brain injury: an analysis of CIREN data." Annu
Proc Assoc Adv Automot Med 47: 577-89.
Francis, D. O., R. Kaufman, et al. (2006). "Air bag-induced orbital
blow-out fractures." Laryngoscope 116(11): 1966-72.
Mahalanobis, P. C. (1936). "On generalised distance in statistics."
Proceedings of the National Institute of Sciences of India 12:
49-55.
Moran, S. G., J. S. Key, et al. (2004). "The applicability of a
computer model for predicting head injury incurred during
actual motor vehicle collisions." J Trauma 57(1): 99-103.
NHTSA (2001). "Crash Injury Research and Engineering Network
(CIREN): an update." Ann Emerg Med 38(2): 181-2.
Nirula, R., C. Mock, et al. (2003). "Correlation of head injury to
vehicle contact points using crash injury research and
engineering network data." Accid Anal Prev 35(2): 201-10.
Read, K. M., J. A. Kufera, et al. (2004). "Life-altering outcomes after
lower extremity injury sustained in motor vehicle crashes." J
Trauma 57(4): 815-23.
Runge, J. W. (1996). "NHTSA Crash-Injury Research and
Engineering Network." Ann Emerg Med 28(4): 450-1;
discussion 451-2.
Scally, J. T., C. A. McCullough, et al. (1999). "Development of the
Crash Injury Research and Engineering Network." Int J
Trauma Nurs 5(4): 136-8.
Schafer, J. L. (1999). "Multiple imputation: a primer." Stat Methods
Med Res 8(1): 3-15.
Siegel, J. H., G. Loo, et al. (2001). "Factors influencing the patterns
of injuries and outcomes in car versus car crashes compared
to sport utility, van, or pick-up truck versus car crashes: Crash
Injury Research Engineering Network Study." J Trauma
51(5): 975-90.
Siegel, J. H., J. A. Smith, et al. (2004). "Change in velocity and
energy dissipation on impact in motor vehicle crashes as a
function of the direction of crash: key factors in the
production of thoracic aortic injuries, their pattern of
associated injuries and patient survival. A Crash Injury
Research Engineering Network (CIREN) study." J Trauma
57(4): 760-77; discussion 777-8.
Siegel, J. H., J. A. Smith, et al. (2002). "Deceleration energy and
change in velocity on impact: key factors in fatal versus
potentially survivable motor vehicle crash (mvc) aortic
injuries (AI): the role of associated injuries as determinants of
outcome." Annu Proc Assoc Adv Automot Med 46: 315-38.
Siegel, J. H., K. H. Yang, et al. (2006). "Computer simulation and
validation of the Archimedes Lever hypothesis as a
mechanism for aortic isthmus disruption in a case of lateral
impact motor vehicle crash: a Crash Injury Research
Engineering Network (CIREN) study." J Trauma 60(5):
1072-82.
Smith, J. A., J. H. Siegel, et al. (2005). "Spine and spinal cord injury
in motor vehicle crashes: a function of change in velocity and
energy dissipation on impact with respect to the direction of
crash." J Trauma 59(1): 117-31.
Stein, D. M., J. V. O'Connor, et al. (2006). "Risk factors associated
with pelvic fractures sustained in motor vehicle collisions
involving newer vehicles." J Trauma 61(1): 21-30; discussion
30-1.
Tencer, A. F., R. Kaufman, et al. (2005). "The role of door
orientation on occupant injury in a nearside impact: a CIREN,
MADYMO modeling and experimental study." Traffic Inj
Prev 6(4): 372-8.
Tencer, A. F., R. Kaufman, et al. (2007). "Reducing primary and
secondary impact loads on the pelvis during side impact."
Traffic Inj Prev 8(1): 101-6.
Tencer, A. F., R. Kaufman, et al. (2005). "Factors affecting pelvic
and thoracic forces in near-side impact crashes: a study of
US-NCAP, NASS, and CIREN data." Accid Anal Prev 37(2):
287-93.
Tencer, A. F., R. Kaufman, et al. (2002). "Femur fractures in
relatively low speed frontal crashes: the possible role of
muscle forces." Accid Anal Prev 34(1): 1-11.
Wang, S. C. (2001). "Today's CIREN: news you can use." Ann
Emerg Med 38(2): 182-3.
Wang, S. C., B. Bednarski, et al. (2003). "Increased depth of
subcutaneous fat is protective against abdominal injuries in
motor vehicle collisions." Annu Proc Assoc Adv Automot
Med 47: 545-59.
Yoganandan, N., F. Pintar, et al. (2004). "Injury biomechanics of C2
dens fractures." Annu Proc Assoc Adv Automot Med 48:
323-37.
Yoganandan, N. and F. A. Pintar (2005). "Odontoid fracture in motor
vehicle environments." Accid Anal Prev 37(3): 505-14.