Project APOLLO/W2/AISE
Final Report on Injury Severity Assessment
Module
Validation of the new ICD9-AIS extended method and calculation of
patients injury severity in Europe.
Alessio Pitidis, Giuseppe Balducci Italian National Institute of Health - Department of Environment
and Primary Prevention;
Osvaldo Chiara, Stefania Cimbanassi General Hospital “Niguarda Ca’ Granda” Milan - Trauma
Team;
Stefano Calderale, Sergio Ribaldi General Hospital ”Umberto I” Rome - Emergency Department;
Maria Segui-Gomez, Rafael Heredero University of Navarra – Department of Preventive Medicine
and Public Health;
Marco Dalmasso, Carlo Mamo Piedmont Region – Inter-Territorial Epidemiologic Observatory;
Gabriella Poli, Carlo Olivari Liguria Region - Health Information System;
Adriano Murgano Abruzzo Region – Epidemiologic Observatory;
Rome November 25th 2008
Final Report
AIM
The final phase of the injury severity assessment module have been dedicated to the
preparation of:
1) Review Report on the state of the art in injury severity assessment
methods;
2) Selection, among different methods based on anatomical or physiological
(or their combination) measures of the patient health status, of those
indicators more efficient in the prediction of an outcome measure (i.e.
lethality rate) and applicable with less information cost to the existing
clinical documents or hospital information systems.
3) Validation of the new ICD9-AIS extended method and calculation of
patients injury severity in Europe by mean of developing an automatic
software application for severity assessment on current data extracted
from Hospital Discharge Register databases or from other ICD-9/ICD-9CM databases.
Our analysis starts from the consideration that a precise injury classification by type and
severity is fundamental to the study of trauma relevance, diffusion and determinants, from
both statistical/epidemiological and clinical point of view. Furthermore, classification of
injuries and individual assessment of trauma patients by severity is central to the research
of injured patients outcome evaluation.
1
THE STATE OF THE ART
The first phase of the injury severity assessment module have been dedicated to the
drafting of a review report on the state of the art of trauma severity assessment methods.
Our analysis starts from the consideration that a precise description of the injury and its
damages is necessary for clinical and epidemiological aims, more in general we can name
them evaluation purposes. The need for a common language in trauma description has led
through the years to the development of trauma dictionaries based on two fundamental
language approaches: specific for trauma and non-specific.
In the first category we can include the Abbreviated Injury Scale (AIS) method and the
assessment methodologies deriving from it, in the second group we can include the
methods based on the International Classification of Diseases (ICD) system.
The construction of a common dictionary for trauma description fulfils two fundamental
needs, on one hand it consents to describe accurately and unequivocally a determined
trauma typology, on the other hand it defines an information base for the development of
injury severity indicators. This last item in particular interests us in our analysis.
In the AIS system the severity score attribution is based on the agreement among groups
of experts. This severity scale is based essentially on the opinion of the clinicians and
other experts that developed it. It can be defined, then, a subjective iso-severity patient
classification system.
Nevertheless, theoretically such an indicator should be an objective measure of a well
determined outcome such as the survival probability or the disability level. It should be an
indicator predictive of outcome expressed as individual or population health status
measure (i.e. mortality or morbidity probability ratio) or health care productivity measure
(i.e. inpatient length of stay or average cost).
Concerning clinical outcome an injury can be described on two basic dimensions:
anatomical damage and physiological consequences.
Using a simplified scheme, the main variables determining the outcome can be included,
as Osler indicated, in the following model:
OUTCOME = f(A, F, H)
(1)
2
where
A
= ANATOMICAL DAMAGE
F
= PHYSIOLOGICAL DAMAGE
H
= GENERAL HEALTH STATUS
The concrete functional form of the model depends obviously on the measures effectively
used.
The outcome more immediate measure is the survival (death) probability.
Physiological consequences of trauma can be measured by dedicated scales such as
Revised Trauma Score (RTS).
The anatomical damage can be measured i.e. by the AIS method and its extension to the
whole patient: the Injury Severity Score (ISS).
With regard to the pre-existing health status of the patient a proxy variable generally used
is the age of the patient, in the hypothesis that that it is a fundamental component of his
general health status.
Until now we have discussed about objective or subjective methods expressed by mean
of
indexes of scales considering the validity of such indicator in terms of prediction
capacity of the outcome. We investigated the relationship between severity indicator (i.e.
ISS score) and outcome measure (i.e. mortality rate), but the trauma severity level can be
directly expressed by mean of an outcome measure. As examples can be indicated at
least two methods measuring the severity of trauma directly by mean of a survival
probability.
The first one is the TRISS method (TRauma score, Injury Severity Score, age combination
index) that combines all the outcome determinants considered in the above indicated
model: ISS as anatomic measure, RTS for physiological damage, age as proxy of the
general health status of the patient (physiological reserve). The basic idea of this indicator
is using simultaneously health status measures describing different aspects of the damage
for having a greater accuracy in outcome prediction than the one obtainable by measuring
each single dimension alone.
In the TRISS method you start from anatomical and physiological measures of trauma
severity for estimating their relationship with the survival probability in a determined
population and then using that probability as measure of global injury severity.
Although survival probability can be used in itself for directly grading the different levels of
severity of a determined pathology without necessarily calculating its relationship with
3
anatomical and physiological damages. That is the Rutledge idea for ICISS (International
Classification of diseases-9 Injury Severity Score) indicator that assesses the trauma
patient severity as product of the survival probabilities calculated for each ICD-9-CM
diagnosis as survival ratio observed in a wide study population.
REVIEW OF THE STATE OF ART IN INJURY
SEVERITY ASSESSMENT METHODS
INTRODUCTION
In the period from the early ‘70s up to the late ‘90s an increasing need for a common
language in trauma description has internationally led the researchers toward the
development of trauma dictionaries and methodologies useful for an accurate assessment
of trauma severity based on two fundamental language approaches: specific for trauma
and non-specific.
In the first category we can include the Abbreviated Injury Scale (AIS)1 method and the
assessment methodologies (e.g., ISS, NISS, etc.) deriving from it as well as the
assessment methods related to physiological descriptors (e.g., RTS). The second category
is represented by methods such as ICISS2, based on universal languages used for the
description of every kind of pathology e.g. the International Classification of Diseases3
system.
INJURY SEVERITY
Growth over the past two decades in the number and variety of injury severity scales
reflects recognition that severity classification is critical for surveillance, epidemiological
4
investigations and evaluations of programs and policies aimed at mitigating the impact of
injury at both the individual and social levels4,5.
Injury severity generally describes the impact of an injury in terms of the extent of tissue
damage (that is, the pathologic evidence of trauma) and/or the physiologic response of the
body to that damage. Host factors are known to mediate the effect of injury severity on
outcome, such as: age, gender, and the presence of certain pre-existing conditions,
including intoxicants.
Since many of the early indices were initially developed for either patient triage or for use
as covariates in evaluating the impact of services or systems on patient survival, these
indices defined severity in terms of a "threat to life" criterion. Given the increased survival
rates of trauma patients due to advances in injury control and emergency medical
services, the scope of evaluation research has been broadened to include studies in which
functional outcomes and disability are of increasing interest. As discussed below, current
initiatives in severity measurement are focused on developing indices that measure these
non-fatal outcomes. Interest has also been directed towards evaluating currently available
indices (or developing new ones) for predicting consumption of health care resources and
the associated costs.
Here in after a detailed analysis of the main adopted trauma severity assessment systems
specific for trauma or non specific.
A. TRAUMA-SPECIFIC APPROACHES
1. Severity Scoring Systems Based On Anatomic Descriptors
a) Abbreviated Injury Scale (AIS)
The Abbreviated Injury Scale (AIS) was introduced in 19711 and is in its 5th revision6. It is
a specialized trauma classification of injuries based mainly on anatomical descriptors of
the tissue damage caused by the injury. It was originally developed for use by
multidisciplinary vehicular crash investigators in the 1970s as a standardized injury
severity assessment tool. Subsequently, it gained acceptance with the trauma community
for clinical research application. Currently, it is generally applied directly to records in
5
specialized trauma registries. To this day, a large AIS-user population consists of nonclinical specialists in government (other than health agencies), academia and the
automotive industry.
The AIS classification system was designed to distinguish between types of trauma of
clinical importance as well as types of trauma of interest to vehicle designers and research
engineers. It has been shown to provide a good basis for valid measurement of probability
of death. The revisions to the classification of the injuries were also designed to provide a
basis to support severity measures framed in terms other than probability of death, for
example, probability of disability or functional impairment. However, to date, there has
been little empirical data to support the notion that AIS does well in terms of probability of
disability. Predicting functional impairment or disability was an important consideration in
the 2005 Revision of the AIS classification system6.
The AIS has two components: (1) the injury descriptor (often referred to as the ‘pre-dot”
code) which is a unique numerical identifier for each injury description; and (2) the severity
score (can be referred to as the ‘post-dot’ code).
The severity score ranges from 1 (relatively minor) to 6 (currently untreatable), and is
assigned to each injury descriptor. The severity scores are consensus assessments
assigned by a group of experts and implicitly based on four criteria: threat to life,
permanent impairment, treatment period, and energy dissipation. AIS values are
considered to be well-ordered within body region, while there are documented
discrepancies of the symmetry in scores between body regions (e.g. an AIS of 4 to the
head may not be equal to an AIS of 4 to the extremities)7.
Injuries should be coded to the AIS by trained staff of trauma services or by specialists in
injury data collection. Assigning AIS scores based on medical records is time consuming;
it is not routinely done outside of trauma centers.
The AIS is a severity measure for a single injury. Many derivative severity scales have
been developed to combine multiple injuries to create a single composite score for each
patient including: Injury Severity Score (ISS)8, The New Injury Severity Score (NISS)9, the
Anatomic Profile (AP)10 and the Anatomic Profile Scale (APS)11.
AIS is the most widely used severity score based on anatomic descriptors of injury; it is the
official injury data collection tool of the National Highway Traffic Safety Administration
(NHTSA) crash investigation teams. Although the reliability and validity of the AIS (and its
derivatives for assessing the combined effect of multiple injuries) have been
6
demonstrated, their utility for population-based research is limited due to the time it takes
to assign adequate scores12.
b) Injury Severity Score (ISS)
Until recently, the Injury Severity Score (ISS)13 was the accepted method of classifying
anatomic injury severity in cases of multiple injury. The ISS is derived from the
Abbreviated Injury Scale (AIS)1, which assigns a severity score to an injury on a scale
from 1 (minor) to 6 (major). An AIS 1 injury will, generally, not require inpatient hospital
treatment, whereas an AIS 6 injury is almost certainly fatal.
The ISS score takes values from 0 to 75. If an injury is assigned an AIS of 6 (unsurvivable injury),
the ISS score is automatically assigned to 75. The ISS score is virtually the only anatomical
scoring system in use and seemed to correlate linearly with mortality and morbidity rates, hospital
stay and other measures of severity8.
It's weaknesses are that any error in AIS scoring increases the ISS error, many different injury
patterns can yield the same ISS score and injuries to different body regions are not weighted. Also,
as a full description of patient injuries is not known prior to full investigation and operation, the ISS
(along with other anatomical scoring systems) is not useful as a triage tool.
For that and other reasons, in more recent years, a number of other AIS-based measures have
been suggested as alternatives to the ISS. These include the New Injury Severity Score (NISS)9,
introduced in 1997; the modified Anatomic Profile (mAP)10 and the Anatomic Profile Score (APS)11.
It has been suggested that these measures better predict the probability of a patient surviving than
does the ISS.
The AIS classification, which forms the basis of all of these approaches, requires a manual review
of the record of each patient, which is estimated to take 10 to 20 minutes12. This makes it
prohibitively expensive for the coding of injury severity in large datasets.
MacKenzie et al. made possible the application of AIS coding to large datasets by developing the
software package ICDMAP-8512, released in 1986. This provided a map for conversion of
International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM)14
diagnosis codes to AIS codes. Therefore, ISS could be calculated simply by knowing the ICD
diagnosis codes of a patient’s injuries. A revised version, ICDMAP-9015, was released in 1997 after
changes to the AIS16. To differentiate between injury severity measures based on AIS codes
derived using ICDMAP and those coded by hand, the ICDMAP-based measures have been given
the prefix “ICD/” (e.g., ICD/ISS when ISS is based on AIS codes from ICDMAP).
7
c. New Injury Severity Score (NISS)
Recently, researchers have proposed a new injury severity score (NISS)9 which, unlike the ISS,
considers the three most severe injuries, regardless of body region. The NISS is computed as the
simple sum of squares of the three most severe AIS (1990 revision)16 injuries. To date, two studies
have reported that the NISS is more predictive of survival and performs better, statistically, than
the ISS9,17.
d. modified Anatomic Profile (mAP)
The mAP has four separate characterizations of the patient’s injury severity: the maximum AIS
value and the square root of the summed squares of the AIS values for all serious injuries (AIS ≥
3) in each of three body regions (head/brain and spinal cord, thorax and front of neck, all other)11.
e. Anatomic Profile Score (APS)
The APS is a weighted sum of the four mAP components, defined as APS _ 0.3199 (A) _ 0.4381
(B) _0.1406 (C) _ 0.7961 (maximum AIS). These four weights come from data from level 1 trauma
centers participating in the Major Trauma Outcome Study19.
2. Severity Scoring Systems Based On Physiological Descriptors
a. Revised Trauma Score (RTS)
The Revised Trauma Score (RTS) is a physiological scoring system, with high inter-rater reliability
and demonstrated accurracy in predicting death. It is scored from the first set of data obtained on
the patient, and consists of Glasgow Coma Scale, Systolic Blood Pressure and Respiratory Rate.
Glasgow Coma Scale Systolic Blood Pressure Respiratory Rate Coded Value
RTS
(GCS)
(SBP)
(RR)
13-15
>89
10-29
4
9-12
76-89
>29
3
6-8
50-75
6-9
2
4-5
1-49
1-5
1
3
0
0
0
0.9368 GCS + 0.7326 SBP + 0.2908 RR
8
=
Values for the RTS are in the range 0 to 7.8408. The RTS is heavily weighted towards the
Glasgow Coma Scale to compensate for major head injury without multisystem injury or major
physiological changes. A threshold of RTS < 4 has been proposed to identify those patients who
should be treated in a trauma centre, although this value may be somewhat low. The RTS
correlates well with the probability of survival19:
9
B. NON SPECIFIC APPROACHES
1. International Classification of Diseases (ICD)
The International Classification of Diseases (ICD) is a general purpose classification of diagnoses
and related matters for all health conditions and includes diagnostic codes for both nature of injury
and external causes of injury. The ICD does not incorporate an explicit severity dimension. The
ICD is widely used to classify health conditions in the clinical, administrative, public health
promotion, and research settings in the U.S. and abroad. The Barell Matrix is a framework for ICD9 CM injury codes by anatomic region and type, but does not consider relative severity20. In the
United States of America (USA), mortality data are currently coded using the 10th revision of the
ICD, ICD-1021 and morbidity data are currently coded using the clinical modification of the 9th
revision of the ICD, ICD-9-CM14. In many European countries (i.e. ltaly and other southern
European countries) mortality data are yet coded using the 9th revision of the ICD, while morbidity
data are coded using the Clinical Modification of 9th revision of the ICD (ICD-9 CM). Due to its
widespread use, there has long been interest in basing the assessment of injury severity on the
ICD taxonomy.
2. ICDMAP
One approach for using the ICD for severity assessment has been the development of software
called ICDMAP that translates ICD-9 CM coded discharge diagnoses into AIS pre-dot codes, injury
descriptors, and severity scores15. The software then uses the AIS severity scores to compute ISS,
NISS, the components of the AP and the APS for the injured individual. The mapping does result
in some loss of information due to differences in the injury classification systems. Resulting
severity scores referred to as ICD/AIS scores are considered to be conservative measures of injury
severity.
ICDMAP has been validated and shown to be useful in categorizing the severity of injuries when
only ICD rubrics are available7,12,22. However, ICDMAP is becoming out of date; there is no map
available for the updated ICD-9 CM, (or ICD-10) codes and there is no map to the AIS2005.
Both the AIS and ICDMAP are proprietary. This poses a major barrier for many potential users.
There is a strong and growing consensus that both should be non-proprietary.
Furthermore, and often because of the proprietary nature of the AIS and its map to ICD,
researchers have been developing empirically derived measures of severity based on ICD
10
diagnoses and short term outcomes such as hospital mortality coded with ICD-9 CM. Although the
development of such measures was attempted as early as the 1970s23,24, not until recently have
we had sufficiently large enough databases and the statistical tools to develop and validate this
approach to severity classification.
3. Measure Of The Severity Of Trauma Directly By Mean Of A Survival Probability
Until now we have discussed about objective or subjective methods expressed by mean
of indexes of scales considering the validity of such indicator in terms of predictive
capacity of the outcome. We investigated the relationship between severity indicator (i.e.
ISS score) and outcome measure (i.e. mortality rate), but the trauma severity level can be
directly expressed by mean of an outcome measure. As an example can be indicated at
least two methods measuring the severity of trauma directly by mean of a survival
probability.
a) Trauma Score - Injury Severity Score (TRISS)
The first one is the TRISS method (TRauma score, Injury Severity Score, age combination
index) that combines all the outcome determinants considered in the above indicated
model: ISS as anatomic measure, RTS for physiological damage, age as proxy of the
general health status of the patient (physiological reserve). The basic idea of this indicator
is using simultaneously health status measures describing different aspects of the damage
for having a greater accuracy in outcome prediction than each single dimension measure
alone. In the TRISS method you start from anatomical and physiological measures of
trauma severity for estimating their relationship with the survival probability in a determined
population and then using that probability as measure of global injury severity.
TRISS25 determines the probability of survival (Ps) of a patient from the ISS and RTS
using the following formulae:
Where 'b' is calculated from:
The coefficients b0 - b3 are derived from multiple regression analysis of the Major Trauma
Outcome Study (MTOS)15 database. AgeIndex is 0 if the patient is below 54 years of age
or 1 if 55 years and over. b0 to b3 are coefficients which are different for blunt and
11
penetrating trauma. If the patient is less than 15, the blunt coefficients are used regardless
of
mechanism.
Blunt
Penetrating
B0
-0.4499
-2.5355
B1
0.8085
0.9934
B2
-0.0835
-0.0651
B3
-1.7430
-1.1360
The TRISS
determines
probability
from
the
calculator
the
of
survival
ISS,
RTS
and patient's age. ISS and RTS scores can be inputted independently or calculated from
their base parameters.
b) ICD-9 Injury Severity Score (ICISS)
One family of empirically derived measures of injury severity based on the ICD is the ICD-9 Injury
Severity Score, referred to as ICISS. The development of this approach to injury severity
assessment is on-going and shows great promise.
ICISS, first proposed by Osler and colleagues using the North Carolina Hospital Discharge Data, is
based on the calculation of survival risk ratios (SRRs) for each ICD-9 CM code2,26. The SRR’s are
derived by dividing the number of patients that survive a given ICD-9 CM injury diagnosis code by
the number of patients with that diagnosis code. ICISS, then, is the product of the SRRs
corresponding to a patient’s set of injuries. ICISS was shown to outperform several important
competitors, including the Injury Severity Score which is based on the AIS11,27-30.
There are several problems/issues with this early direct estimation approach that have been
addressed: 1) SRRs are database-specific and it cannot be assumed that SRRs from one source
will sustain accuracy and prediction in other sets of patients; 2) SRRs calculated in this manner are
not independent and are contaminated by the presence of outcomes from other injuries in multiple
trauma cases; 3) Interactions of injuries, which are known to exist, are not accounted for. That is, in
some cases, two injuries may combine synergistically to produce a greater effect than the
combination of the two acting independently. [This is equally likely to be a problem for scoring
systems based on AIS and ICD.]
To address some of these concerns, Meredith et al. calculated SRR’s from the National Trauma
Data Bank and highlighted differences between these SRR’s and the original set from NCHDD,
underscoring the need for comparability when using SRR’s from different sources28 Other sets of
SRR’s have since been calculated from various trauma registries. Also, Meredith et al. calculated
and described a set of independent SRRs that were derived from patients with only a single
injury29. These independent SRR’s were shown to be better predictors of survival than their
traditional counterparts. The accurate measurement and incorporation of injury interactions into
trauma severity models has been elusive, though some alternatives have been proposed31.
12
More recently, Kilgo et al. has shown that modelling a patient’s most severe SRR rather than the
full multiplicative ICISS score is more predictive of survival32. This “worst-injury” approach may be
superior because most trauma patients are either so minimally injured as not to pose a direct threat
to life or so severely injured that they cannot be saved. These cases compose approximately 97%
of all trauma cases and the worst-injury approach does very well for these patients.
It could be argued that the SRR is exclusively an empirical observation of hospital survival given
hospital admission, and lacks the independent descriptive value of AIS in terms of body system
involvement. The SRR may therefore be less flexible for modelling outcomes other than hospital
survival.
All measures of prediction (ICISS as well as methods based on AIS) are affected by the large
proportion of deaths that occur before hospital admission as well as the large proportion of deaths
in older patients that occur after hospital discharge. These probabilities will also change over time,
and may be affected by extraneous factors like utilization and service delivery.
One major limitation in applying the ICISS to NCHS data is the dependence of the data on the
system from which it was derived. ICISS scores could be derived from the NCHS National Hospital
Discharge Survey. While the literature has shown that the ICISS can be internally valid, potential
lack of external validity is often mentioned as a caveat. One study of interest to NCHS is the
comparison of national level data from New Zealand and Australia showing both good external and
internal validity33. This study derived SRR’s based on the Australian clinical modification of ICD10
(ICD-10-AM) 33. In the United States, experts involved in its most recent revision of the AIS were
consulted during the development of the injury chapter of ICD-10 CM. This is likely to improve both
the ICDMAP software (should a new version be developed) as well as the ICISS measures of
injury severity.
CONCLUSIONS
In conclusion we have examined two fundamental approaches for injury severity
assessment, the first one based on injury description languages specific for trauma, the
second one referred to universal languages used for the description of every kind of
pathology.
The first approach is very costly for the necessary training of specialized codifying
personnel, the complexity of clinical instrumental information to be registered, the volume
of information to collect and elaborate. The universal languages based systems have
normally a lesser information costs.
13
That difference has determined the development of
two main evolution paths in the
patient injury severity classification systems.
On one side a year 1980 population study by Champion24 calculated the real death
probability for each HICDA-8 (Hospital International Classification of Diseases) codified
diagnosis by mean of the analysis of correlation among lethality and ICD based severity
scores. That study opened the way to systems based on routine clinical information
currently registered, with trauma severity levels attributed directly on the base of observed
outcomes, ICISS index is the most recent of them. Those systems have a good outcome
predictive capacity and a fundamental advantage: can be applied on current existing
clinical data.
That intrinsic advantage has led some researches to follow a second approach in injury
severity assessment, simply consisting in the development of a conversion dictionary
among ICD-9 and trauma specific severity scores such us AIS. After many attempts the
introduction of ICD-9-CM incremented the diagnosis description specificity consenting to
Mackenzie the development of an ICD-9-CM/AIS-90 conversion table (ICDMAP-90)15.
Nevertheless because of the lesser diagnosis description specificity of ICD than AIS there
is a high discordance among manually calculated and automatic (conversion table based)
severity scores for abdominal internal organs and internal head, face and neck injuries
(52% of non appropriately assigned scores).
In our study we will make a selection of indicators to be applied to European data and try
to improve existing methods.
REFERENCES
1. AIS – “Rating the severity of tissue damage: I. The Abbreviated Injury Scale”. JAMA
(1971); 215:277-280.
2. Osler T, Rutledge R et al.: "ICISS: an international classification of disease-9 based
injury severity score". J Trauma (1996); 41(3): 380-6; discussion 386-8.
3. WHO (1977): International Classification of Diseases 1975 (IX Conference Revision).
Geneva, World Health Organization.
4. MacKenzie EJ: "Injury severity scales: overview and directions for future research". Am
J Emerg Med (1984); 2(6): 537-49.
5. Rivara FP (2001). Injury control: a guide to research and program evaluation.
Cambridge; New York, Cambridge University Press.
14
6. Gennarelli T, Wodzin E (2005). The Abbreviated Injury Scale - 2005. Des Plaines, Il,
Association for the Advancement of Automotive Medicine.
7. Clark DE, Winchell RJ: "Risk adjustment for injured patients using administrative data".
J Trauma (2004); 57(1): 130-40; discussion 140.
8. Baker SP, O’Neill B, Haddon W, Long WB: "The injury severity score: a method for
describing patients with multiple injuries and evaluating emergency care". J Trauma
(1974); 14(3): 187-96.
9. Osler T, Baker SP et al.: "A modification of the injury severity score that both improves
accuracy and simplifies scoring". J Trauma (1997); 43(6): 922-5; discussion 925-6.
10. Copes WS, Champion HR et al.: "Progress in characterizing anatomic injury".
J Trauma (1990); 30(10): 1200-7.
11. Sacco WJ, MacKenzie EJ et al.: "Comparison of alternative methods for assessing injury
severity based on anatomic descriptors". J Trauma (1999); 47(3): 441-6; discussion
446-7.
12. MacKenzie EJ, Steinwachs DM, Shankar BS: “Classifying severity of trauma based on
hospital discharge diagnosis: validity of a ICD-9CM to AIS-85 conversion table”. Med
Care (1989); 27:412–422.
13. Baker SP, O’Neill B, Haddon W Jr: "The Injury Severity Score: a method for describing
patients with multiple injuries and evaluating emergency care". J Trauma (1974); 14:187196;
14. United States National Center For Health Statistics. The International Classification of
Diseases, Ninth Revision, Clinical Modification. Vol 1. Ann Arbor, MI: Commission on
Professional and Hospital Activities; 1979.
15. MacKenzie EJ, Sacco W. ICDMAP-90: A User’s Guide. Baltimore, MD. The Johns
Hopkins University School of Public Health and Tri-Analytics, Inc, 1997.
16. Association for the Advancement of Automotive Medicine. The Abbreviated Injury Scale,
1990 Revision. Des Plaines, IL: Association for the Advancement of Automotive
Medicine; 1990.
17. Brenneman FD, Boulanger BR, Mclellan BA et al.: “Measuring injury severity: time for a
change?” J Trauma (1998); 44:580–2
18. Champion HR, Copes WS, Sacco WJ: “Major trauma outcome study: establishing
national norms for trauma care”. J Trauma (1990); 30:1356.
19. Champion HR, Sacco WJ, Copes WS et al.: “A revision of the Trauma Score”. J Trauma
(1989); 29: 623-629.
20. Barell V, Aharonson-Daniel L et al.: "An introduction to the Barell body region by nature
of injury diagnosis matrix". Inj Prev (2002); 8(2): 91-6.
21. World Health Organization. (1992). International statistical classification of diseases and
related health problems (Tenth revision). Geneva, World Health Organization.
22. Mullins RJ, Veum-Stone J et al.: "Outcome of hospitalized injured patients after
institution of a trauma system in an urban area". JAMA (1994); 271(24): 1919-24.
23. Levy PS, Mullner R et al.: "The estimated survival probability index of trauma severity".
Health Serv Res (1978); 13(1): 28-35.
24. Champion HR, Sacco WJ et al.: "An anatomic index of injury severity." J Trauma (1980);
20(3): 197-202.
25. Boyd CR, Tolson MA, Copes WS: "Evaluating Trauma Care: The TRISS Method". J
Trauma (1987); 27:370-378.
26. Rutledge R, Hoyt DB et al.: "Comparison of the Injury Severity Score and ICD-9
diagnosis codes as predictors of outcome in injury: analysis of 44,032 patients". J
Trauma (1997); 42(3): 477-87; discussion 487-9.
27. Hannan EL, Farrell LS et al.: "Predictors of mortality in adult patients with blunt injuries in
New York State: a comparison of the Trauma and Injury Severity Score (TRISS) and
15
28.
29.
30.
31.
32.
33.
the International Classification of Disease, Ninth Revision-based Injury Severity Score
(ICISS)". J Trauma (1999); 47(1): 8-14.
Meredith JW, Kilgo PD et al.: "A fresh set of survival risk ratios derived from incidents in
the National Trauma Data Bank from which the ICISS may be calculated". J Trauma
(2003); 55(5): 924-32.
Meredith JW, Kilgo PD et al.: "Independently derived survival risk ratios yield better
estimates of survival than traditional survival risk ratios when using the ICISS". J
Trauma (2003); 55(5): 933-8.
Hannan EL, Waller CH et al.: "A Comparison Among the Abilities of Various Injury
Severity Measures to Predict Mortality With and Without Accompanying Physiologic
Information". J Trauma (2005); 58(2): 244-251.
West TA., Rivara FP et al.: "Harborview assessment for risk of mortality: an improved
measure of injury severity on the basis of ICD-9-CM". J Trauma (2000); 49(3): 530-40;
discussion 540-1.
Kilgo PD, Osler TM et al.: "The worst injury predicts mortality outcome the best:
rethinking the role of multiple injuries in trauma outcome scoring". J Trauma (2003);
55(4): 599-606; discussion 606-7.
Stephenson S, Henley G et al.: "Diagnosis based injury severity scaling: investigation of
a method using Australian and New Zealand hospitalisations". Inj Prev (2004); 10(6):
379-83.
16
SELECTION OF EFFICIENT INDICATORS
Information cost of severity assessment methods
At the state of the art we can classify then two fundamental approaches for injury severity
assessment, the first one based on injury description languages specific for trauma, the
second one referred to universal languages used for the description of every kind of
pathology.
The first approach lead to the definition of iso-severity groups of patients and is very costly
for the necessary training of specialized codifying personnel, the complexity of clinical
instrumental information to be registered, the volume of information to collect and
elaborate.
Table 1. Workload and costs estimated for the implementation of ISS and ICISS systems
Severity assessment process
ISS
ICISS
Diagnosis ICD-9-CM coding
30 Minutes
30 Minutes
Diagnosis AIS coding
30 Minutes
Unnecessary
20 hours/year
Unnecessary
Multiple trauma (up to 5 injuries)
Admission
Diagnosis and treatment
RTS+Age+ISS coding for TRISS
Comparison expected/observed outcome
Personnel unit training AIS/ISS use
Total time estimated on 2,000 patients/ 2,000 hours/year
hospital/year
Total additional costs ISS vs. ICISS on 10 hours/week
2,000 patients/ hospital/year
$ 6,250,000 hospital/year
Total additional costs ISS implementation $3,125,000/year
in 500 trauma centers/year
Total additional costs ISS implementation $18,750,000/year
in ,.000 ospedali/anno
Source: adapted from Rutledge et al.
17
1,000 hours/year
An estimate of those costs has been provided by Rutledge et al. (1997)1. This example
shows clearly that the information costs of a manual trauma specific classification system
such as the AIS/ISS are largely greater than the ones deriving from an automated system
based on universal classification of pathologies (and trauma).
The universal languages based systems are expected to have a lower information cost.
Severity assessment based on survival probability
The above illustrated difference has determined the development of two main evolution
paths in the patient injury severity classification systems.
On one side in a year 1980 study Champion2 calculated on a sample of 2,165 trauma
patients the real death (survival) probability for each HICDA-8 (Hospital International
Classification of Diseases) codified diagnosis by mean of the analysis of correlation among
lethality and ICD based severity scores. The method was applicable to 1,154 of them and
demonstrated a Pearson correlation index of 0,6 similar to the one calculated for the ISS.
That appreciable result opened the way to the development of iso-severity classification
systems based on the ICD codes or anyway based on routine clinical information currently
registered, with trauma severity levels attributed directly on the basis of observed
outcomes, ICISS index is the most recent of them. It can be seen as an evolution of the
systems based on the analysis of the relation among survival probability and
anatomical/physiological indicators such as the TRISS which is an extension to
penetrating injuries of the Champion’s method investigating the aforesaid relation on a
sample of 2,000 blunt trauma patients3.
While the TRISS is still linked to anatomical and physiological measures in order to predict
the survival probability, the ICISS directly lies on the ground of the observed probability of
survival (death) used as indicator of the severity of trauma.
1
Rutledge R, Hoyt DB, Eastman B, Sise MJ, Velky T, Canty T, Wachtel T, Osler TM (1997): Comparison of the Injury
Severity Score and ICD-9 diagnosis codes as predictor of outcome in Injury analysis of 44.032 patients. The Journal of
Trauma; vol 42, 3: 477-488.
2
Champion HR, Sacco WJ, Lepper RL et al. (1980): An anatomic index of injury severity. The Journal of Trauma; 20:
197.
3
Champion HR, Sacco WJ, Carnazzo AJ et al. (1981) : Trauma Score. Critical Care Medicine; 9:672-676.
18
Rutledge et al.4 used the ICD-9-CM classification for coding the diagnoses and
interventions, grading the severity of injuries by mean of the death (survival) probability per
diagnosis/intervention group observed in a sample of 10,000 inpatients (DRR: Death Risk
Ratio). The observational variable used in this model for the prediction of survival can be
summarized as follows:
Dependent Variable
Independent Variable
Survival (1=alive, 0=dead)
DRR1 – main diagnosis
DRR2 – second diagnosis
DRR3 – third diagnosis
DRR4 – main surgical procedure
DRR5 – second surgical procedure
DRR6 – external cause
Because the outcome variable have been considered dichotomous, logistic regression
methods where appropriate in order to investigate the functional relation among ICD-9
diagnoses and survival, estimating the functional parameters. The outcome has been
defined as ratio between the probability of survival and the one of death: that consented
the regression analysis by mean of a logit transformation for the prediction of association
and predictive power of the independent variables. In formal terms:
⎛ SRR ⎞
k
p
⎟ =b +∑b x
ln(lo ) = ln ⎜⎜
0
d d
⎟
d =1
⎝ DRRp ⎠
(2)
with
k = maximum number of indipendent variables per patient (in this case k = 6)
p = 1, 2,….., n
and
4
Rutledge R, Fakhry S, Baker C, Oller D (1993): Injury severity grading in trauma patients: a simplified technique
based upon ICD-9 coding. The Journal of Trauma; vol 35, 4:497-507.
19
SRRp =
e
b0 +
∑ bdx d
b + b x
1+e 0 ∑ d d
(3)
where
lo
= survival odds
SRRp = alive patients / dead patients = E(Y|x) = survival probability of the p-th typology of
patient
DRRp = 1 - SRRp = 1 - E(Y|x) = death probability of the p-th typology of patient
xd
= severity level of the d-th ICD-9 codes group graded by specific DRRd value of the group
(independent variables)
bd
= coefficients of the independent variables
b0
= intercept
For each typology of patient is defined a pattern of diagnoses and procedures (surgical
and medical) each one associated with its own probability of death identifying the specific
typology of patient and determining his probability of survival. The patterns and their DRRd
if they are observed in sufficiently large sample not affected by selection biases, can be
applied to any other population of trauma patients. The theoretical basis of this approach is
frequentistic and lies on the idea that the mortality ratio for a specific trauma is stable when
the socio-economic contexts observed are similar. Because the mechanism of the
accident and the medical treatment of trauma are similar. In conclusion the mortality ratio
observed in a large population of injured people is sufficiently stable during the time and
across different territorial contexts so that it can be extrapolated to another population. The
literature reports a good prediction capacity for these systems based on the observed
survival probability as we evidenced in the review of the state of the art.
A system based on the above illustrated concepts has been developed by Osler and
Rutledge5 it is named ICISS (International Classification of diseases-9 Injury Severity
Score) and is based on the idea of working on survival probability. The development of this
5
Osler T, Rutledge R, Deis J, Bedrick E (1996): ICISS an international classification of disease-9 based Injury Severity
Score. The Journal of Trauma; vol 41, 3: 380-388.
20
idea started in the precedent studies by Champion et al. and Levy & Goldberg. ICISS
expresses the severity of trauma as multiplicative effect of the survival probabilities
calculated for each diagnosis (coded with the ICD-9-CM system) on the ground of the
survival ratio observed in a large enough sample. In formal terms:
m
ICISS = SRR p = ∏ SRRd
(4)
d =1
with
and
0 ≥ SRRd ≤ 1
p = 1,…….., n
where
d = number of diagnoses per typology of patient
SRRd = alive/totale patients = E(X) = survival probability for the d-th diagnosis
SRRp = survival per typology of patient
p
= typology of patient
It is evident that the severity of trauma in this case is directly expressed in function of the
survival probability per diagnosis. This indicator takes into account the ICD coded
diagnoses only, excluding the medical and surgical procedures and any other clinical
information. The implementation of ICISS is an on-going process, but it results are
showing good promises. As we reported in the review of the state of the art, apart the
results presented by its authors, the comparison studies in literature indicate it outperforms
the existing alternative methods for assessing injury severity whether based on anatomic
descriptors (such as ISS) or including physiologic information (such as TRISS).
Nevertheless there are two main concerns affecting this indicator. The first one regards
the exportability of the SRRs (Survival Risk Ratios) from the reference source to other data
sets. In other words a group of SRRs observed in the reference sample could be specific
of that population and not guarantee accuracy and prediction capacity if applied to other
data sets.
Meredith et al.6 evidenced differences among the SRRs calculated in the National Trauma
Data Bank and the correspondent ones observed in the North Carolina State Hospital
Meredith JW, Kilgo PD et al.: "A fresh set of survival risk ratios derived from incidents in the National
Trauma Data Bank from which the ICISS may be calculated". J Trauma (2003); 55(5): 924-32.
Kilgo PD, Osler TM et al.: "The worst injury predicts mortality outcome the best: rethinking the role of multiple
injuries in trauma outcome scoring". J Trauma (2003); 55(4): 599-606; discussion 606-7.
6
21
Discharge Database (NCHD), that originally was used by Osler and Rutlegde for the
development of ICISS.
The second one concerns the effect of multiple trauma. The proposed multiplicative model
could not be appropriate to measure this effect because it does not take into account that
in multiple trauma the single SRR for ICD diagnosis is not independent from the others
because influenced by the outcomes of the other injuries. The interaction among injuries is
implicitly considered in such multiplicative model, but not separately from the independent
effect of each injury. So that that the effect of multiple trauma could be alternatively
underestimated or overestimated depending from the patterns of ICD diagnoses included
in the calculation of ICISS
Kilgo et al. have shown that modelling a patient’s most severe SRR rather than the full
multiplicative ICISS score is more predictive of survival. This “worst-injury” approach may
be superior because most trauma patients are either so minimally injured as not to pose a
direct threat to life or so severely injured that they cannot be saved. These cases compose
approximately 97% of all trauma cases and the worst-injury approach does very well for
these patients.
Finally one can argue that the consideration of diagnoses only, in the model, could not
capture the effect of factors deriving not from the need of medical assistance (clinical
condition of the patient), but on the contrary depending from the organization of the health
care services and their utilization by the people.
Despite of these concerns Rutledge et al. in a comparison study7 showed an high
correlation among ICISS scores computed on the same patients (test sample) by mean of
SRRs calculated in different reference samples. The test sample was made of 44,032
patients registered during 1985-87 in the SDTR (San Diego Trauma Registry) while the
databases in which the SSRs have been calculated were: SDTR itself, North Carolina
State Hospital Discharge Database (NCHD), North Carolina Trauma Registry Database
(NCTR) and Agency for Health Care Policy Research’s Health Care Utilization Project
Database (HCUP). The sample dimension varies from thousand cases in the trauma
registries (TRs) up to millions of observations in the Hospital Discharge Registers (HDRs),
Results in table 2 indicate a good level of association among the test sample and the other
7
Rutledge R, Hoyt DB et al.: "Comparison of the Injury Severity Score and ICD-9 diagnosiscodes as
predictors of outcome in injury: analysis of 44,032 patients". J Trauma (1997); 42(3): 477-87; discussion
487-9.
22
external ones, in general
good pairwise association is shown between two trauma
registries or a trauma registry and a HDR. Consequently it seems reasonable to
extrapolate SRRs from a trauma registry to another one or from a HDR to a trauma
registry (the bigger the HDR the better the extrapolation).
Table 2. Pearson R correlation coefficient – ICISS SDTR patients (n.=44,032) period
1985-87
Data Base
SDTR SRR
NCHD SRR
NCHD SRR
0,5982
NCTR SRR
0,9092
0,6617
HCUP SRR
0,8888
0,4882
NCTR SRR
p < 0,0001
Yes
Yes
0,8260
Yes
Source Osler, Rutledge et al.
In general one can assume and literature shows internal validity for SRRs at limited
distance of time from reference SRRs observation, while lack of external validity has been
mentioned as a caveat.
Nevertheless more recently a comparison study on national level data in Australia and
New Zeeland has shown both good internal and external validity8.
The concern about the assumption of a multiplicative functional relation among injuries in
multiple trauma and about the independence of survival probability for each injury
maintains its validity. The accurate measurement and incorporation of injury interactions
into trauma severity models is still elusive in the existing studies.
In order to avoid potential problem of external validity we will consider the whole trauma
inpatients in a determined country; for that reason we analyzed the National HDRs of Italy
and Spain extracting from there all the patient having at least one injury coded in the main
or secondary diagnoses. The data from these two countries will be used for calculating
SRRs for ICD-9-CM codes while the data from the Netherlands and Austria will be used for
ICD-10-CM.
We will follow the approach of “worst-injury” calculating the most severe SRR per patient
and comparing it with ICISS. The most severe SRR method appears us promising
8
Stephenson S, Henley G et al.: "Diagnosis based injury severity scaling: investigation of a method using
Australian and New Zealand hospitalisations". Inj Prev (2004); 10(6): 379-83.
23
because in over 1 million trauma inpatients observed in Italy during 2002 the 76 percent
had a single diagnosis coded and 90 percent not more than two diagnoses.
Furthermore in order to overcome the problems implicit in a multiplicative model such as
ICISS we will use a clustering method computing a global SRR per patient directly in
groups of patients having the same pattern of diagnoses (main and secondaries ). This
approach provides a global survival probability for each type of patient directly by
observing it, without assuming any hypothesis about the functional relation among the
single diagnoses forming the pattern: not a single SRR for each diagnosis, but a total SRR
for the whole pattern of diagnoses is directly calculated: SRRp computed not in
multiplicative way but for the pattern as a whole . These patterns can be then them too
aggregated when showing similar survival probabilities. In the Italian 2002 HDR trauma
records we observed 9,621 patterns of diagnoses made of at least 5 persons each one
and representing 89 percent of the whole trauma inpatients.
A serious concern regards the fact that all the trauma severity assessment methods
(based on survival probability and/or anatomical/physiological descriptors) in using
mortality for the validation of injury scaling are affected by biases due to the large
proportion of deaths that occur before hospital admission as well as the large proportion of
deaths in older patients occurring after hospital discharge. We estimate that in Italy in 2002
on the total fatalities caused by trauma at least 50 percent occurred out of hospital.
In order to take into control this issue we decided to include in the analysis mortality data
from the death certificates, being normally ICD codes used for mortality less specific than
ICD clinical modification codes used in HDR. We decided to distribute out of hospital
deaths in the same proportion within each ICD-CM category, formally:
Si
1
SRRa =
⋅
Si + DI i (k + 1)
(5)
where:
k=
∑ DO
i
∑ (S
i
i
+ DI i )
i
24
SRRa = Survival Risk Ratio adjusted by out of hospital mortality
Ì
= i-th diagnosis code within the j-th diagnoses category
S
= number survived people
DI
= number dead people in hospital
DO
= number dead people out of hospital
In conclusion the systems based on survival probability have a good prediction capacity of
outcome and a fundamental advantage: can be (automatically) applied on large data sets
of current existing clinical data.
Severity assessment based on anatomic descriptors
The afore said intrinsic advantage has led some researches to follow a second approach
in injury severity assessment based on anatomic descriptors, simply consisting in the
development of a conversion dictionary among ICD and trauma specific severity scores
such us AIS. After many attempts the introduction of ICD-9-CM incremented the diagnosis
description specificity consenting to Mackenzie9 the development of an ICD-9-CM/AIS-85
conversion table.
In order to validate the ICD based conversion AIS and ISS obtained by mean of the
conversion table were compared with those calculated manually by trained coders
reviewing the complete medical charts for 1,120 trauma inpatients. Almost all the observed
cases were convertible (98.2 percent) and show globally a good agreement among ISS
scores calculated with the two methods; mean values were 10.6 and 13.1 for ICD
conversion and chart review respectively. The ISS scores assigned with the two methods
were within ± 5 points for 70 percent of the cases. Grouping the ISS scores in three ranges
(1-12; 13-19; 20 +), a classification often used in evaluation and research, the two
methods of ISS scoring agreed in 68 percent of the cases. In a previous study10 average
9
MacKenzie EJ, Steinwachs DM, Shankar BS: “Classifying severity of trauma based on hospital
discharge diagnosis: validity of a ICD-9CM to AIS-85 conversion table”. Med Care (1989); 27:412–422.
10
MacKenzie EJ, Shapiro S, Eastham J et al. : Reliability of the AIS-80. Med Care (1985); 23:844.
25
pairwise agreement in ISS among 15 coders reviewing the same medical charts in was 75
percent. These global results show that while the automatic conversion is not perfect, it
provides reasonably good information on severity for large population-based data sets.
Nevertheless, because of the lesser specificity in diagnosis description
for ICD with
respect to AIS, there is a high discordance among manually calculated and automatic
(conversion table based) severity scores for abdominal internal organs and internal head,
face and neck injuries (52 percent of non appropriately assigned scores for head and neck
and 46 percent for abdomen) as you can see in table 3. Even though a significant
proportion of inappropriate severity coding (10 percent on average) depends from errors in
abstracting medical charts and using ICD classification. Furthermore for many head
injuries in the MacKenzie ICD-AIS map were assigned ranges of AIS values for the same
ICD code regarding. These ranges were retained in the final ICD-AIS conversion table to
allow the user to choose the higher or lower values depending on the application.
Table 3. Observed and maximum potential agreement between MAXAIS and ISS scores assigned
by medical charts review versus ICD conversion table.
Inpatients (n=1.120) in two Trauma Centers of Maryland 1989.
Body Region
N. Cases
Observed
agreement Maximum
potential
(%)
agreement* (%)
Head/Neck
398
48
67
Face
173
54
79
Thorax
137
63
85
Abdomen
139
54
82
Spine
94
51
79
External/Burns
420
70
72
Extremities
446
74
87
Grouped ISS
1.099
68
78
*Achievable if errors in abstracting were minimized
Source: Mackenzie et al.
26
DEVELOPMENT OF AN EXTENDED AUTOMATIC SEVERITY SCORES ASSIGNMENT
ALGORITHM.
Among the systems for the classification of the inpatients we choose those based on the
clinical condition of the patient, that as you know can be divided in two groups: a) the
universal ones; b) those specific for trauma. Within the group a) we will use as afore said a
severity assessment approach based directly on the observed survival probability, while
for the
group b) we adopt trauma specific severity indicators based on anatomic
descriptors. We prefer the injury anatomic standardized description approach rather than
physiological measures because the first is more easily observable and stable during the
time. In other words there is more clinical agreement in the definition of anatomic damage
and its changes during the natural course of trauma are more progressive and easily
observable.
We can describe now more precisely the aim of our study. In the model expressed in
equation (1) the outcome of a trauma is a function of three factors: the anatomic injury, the
physiological damage, the general health status of the patient. We use for the anatomic
description the AIS/ISS method; as a proxy of the health status of the patient we can use
his age which on average is a good indicator of his general conditions before the trauma.
We preferred to ignore physiological measures because more difficult to observe and
record, for RTS in example it is controversial the moment in which the physiological
parameters have to be recorded, because they change rapidly during the course of
trauma. Physiological parameters are dynamic measures by definition.
In assigning AIS scores the method of reviewing complete medical charts by trained
coders has a high information cost so that we decided to use the automatic ICD-AIS
conversion method to be applied for the last year available in Italy and Spain to the whole
national hospital discharge records with a trauma indicated as main diagnosis or within the
secondary ones.
In order to solve the problem of specificity for the scoring of head/neck and abdominal
trauma we decided to revise the ICD-AIS conversion map. The main aim of this revision
was to extend to the head/neck body region the bi-univocal correspondence between ICD
codes and AIS scores: in order to improve the automation process to each ICD code has
to correspond not a range, but only one AIS score.
27
AIS scores were assigned to all ICD-9-CM rubrics between 800.00 and 959.90 excluding
foreign body through an orifice codes (939 to 939), traumatic complications (958) and late
effects of injures (905 to 909). In the end 1.250 codes resulted bi-univocal convertible.
A panel of clinicians has been formed within the Departments of Orthopedics and
Emergency Surgery of the General Hospital of the First Sate University of Rome, surgeons
from the Trauma Team of the Niguarda General Hospital of Milan were included in the
panel. This panel has taken as reference and revised the Guide Lines for Severity
Assessment published by the Public Health Agency of Latium Region.
We omit for space reasons the entire conversion table and indicate that – except for
head/neck, gastro-intestinal tract and other particular injuries - the general conversion
criterion has been conservative: were multiple AIS scores were attributable to the same
ICD-9-CM code the lowest score has been used if the difference in the range of scores
was not greater than 1 otherwise the ICD code has been excluded from the conversion
table. For head/neck, gastro-intestinal tract and other particular injuries
a specific
conversion algorithm has been developed. The algorithm for the choice of an AIS score
(within a range) per ICD code lies on the probability to observe a specific score within the
range of possible values, the score with the highest relative frequency will be chosen.
This algorithm and the conversion table process have been automated by software
procedures running in a MS-Windows environment. The computing of SRRs has also been
included in the automatic procedures. Examples of the software user friendly interfaces
are reported in the following images.
28
Fig. 1 The software application main panel
Once validated the conversion process, the national hospital discharge records are being
processed in order to be inputted in the above illustrated severity assessment software.
The complete data process flow is charted in figure 4.
In particular have been converted the diagnoses for inpatients having at least one
diagnosis of trauma (within the range above specified) among the main or secondary
diagnoses. The trauma inpatients of three Italian Regions (Piedmont, Liguria and Abruzzo)
and of the whole Spain have been considered in the year 2004. Up to six diagnoses per
patient have been examined for the Italian admissions and up to three for the Spanish
ones.
29
Fig. 2 The ICD-9 diagnoses selection panel
Fig. 3 The web style injury severity report: Maximum AIS, ISS, SRR
30
Fig. 4
AISE project data processing flow
VALIDATION OF THE CONVERSION TABLE
In order to validate the ICD-AIS revised conversion table a retrospective study has been
conducted. Complete medical charts were examined by expert coders and AIS (ISS)
scores were assigned to 1,054 inpatients in two major hospitals in the metropolitan areas
of Rome and Milan. The retrospective sample was formed by 315 inpatients recorded in
1998 within the Department of Emergency Surgery and the Intensive Care Unit of the First
University General Hospital in Rome, 439 cases observed during 2003 in the Departments
of Orthopedics and Emergency Surgery and the Intensive Care Unit of the same hospital,
300 hospitalized patients recorded during 2005 in the Trauma Centre of the Niguarda
General Hospital in Milan. The expected severity of injuries in the observed hospital units
ranged from low to high severity cases in the two major metropolitan areas of Italy.
The 91% of the cases in the whole sample resulted bi-univocal convertible by our
algorithm, so that a single AIS code was assigned per each ICD-9-CM code.
The validation of the ICD-AIS revised conversion table has being conducted comparing the
AIS-ISS scores manually assigned by medical trained personnel reviewing the complete
31
medical charts for each patient in the sample with the scores automatically assigned by
the conversion software.
In order to assess the goodness of the conversion procedure we calculated the
percentage of concordant maximum AIS scores assigned by body region in the validation
sample, respectively by the manual coders in the chart review versus the automatic
conversion procedure of ICD-9-CM diagnosis codes, based on the discharge abstract
data. In the assessment of agreement (or disagreement) we assumed as reasonable the
adoption of two different criteria of validating the conversion table: the one adopted by
McKenzie et Al.
11
in their ICDMAP procedure and a broader one that we propose in this
study.
In accordance to the Mackenzie et Al. criteria, the identification of significant injuries can
rely on the definition of injuries scored as AIS > 1. Once selected the injuries for which
both methods agree on the presence of an injury, for instance as afore said, comparisons
can be made on the agreement of maximum AIS by body region (MAXAIS) or on ISS
scores per patient. We considered the ISS body regions, not the AIS ones, so that direct
comparisons cannot be made for spine injuries.
We prefer to use an approach that includes in calculations even the disagreement about
the presence of a significant injury between the manual and automatic met. So that we
include in the analysis even the effect of errors due to the different sensibility in detecting
the presence of a significant injury. In other words, assigning the value equal to 0 to a
diagnosis of trauma that we could not convert into an AIS score either by manual or
automatic methods, even those zero values have been included in the analysis of
agreement.
Furthermore, based on the definition of significant injury (AIS score > 1), we introduce a
limit of tolerance in conversion agreement between automatic and manual scores, the limit
has been fixed within +1 and -1 difference in AIS scores assigned for the same diagnosis
by the two methods. It is introduced for testing the sensitivity of the scoring procedure in
varying the conversion method from manual to automatic scoring. In conclusion in the
analysis of agreement between the two methods we suggest to consider explicitly the
possible errors due to the sensibility of the methods, but reducing by construction the level
of sensitivity, with the adoption of an interval of tolerance, in order to avoid the agreement
measure being effected by slight variations along the AIS scale: variations equal to one
11
MacKenzie EJ, Steinwachs DM et Al., cited paper.
32
point on the AIS scale; value at which, according to Mackenzie et Al. an injury is not be
considered significant.
RESULTS
We can anticipate that the obtained results indicate that the extension of the ICD-AIS map
toward a revised conversion algorithm assigning univocal AIS scores even for head/neck
and gastro-intestinal tract maintains the information validity of the conversion process,
despite the extension of bi-univocal code conversion to these body parts.
Applying the above discussed first criterion to the validation sample, in order to see if
significant injuries are detected by both methods, we obtain the following frequency of
cases in which the MAXAIS by body region did not match the definition of significant injury
(AIS > 1) in either manual or automatic methods.
Table 4. Percentage of cases in which there was a disagreement as to the presence of a
significant injury (AIS > 1) to the Body Region. Manual vs. Automatic method (n. = 1.051)
Body Region
% MAXAIS manual > 1 and % MAXAIS manual < 1 and
MAXAIS automatic < 1
Head/Neck
Face
Thorax
Abdomen
Extremities
External/Burns
MAXAIS automatic > 1
35
2
23
4
21
1
29
5
16
2
3
1
In comparing each inpatient by maximum AIS score we observe an agreement of the 49,7
percent of patients between the manual and automatic scoring. If we reduce the sensitivity
of the test by applying a limit of tolerance of + 1 or -1 AIS scores to the difference between
the two methods, the agreement rises up to 82,7 percent of cases. This assumption is
acceptable in our opinion, when you consider that, by the MacKenzie et Al. definition, an
injury must have a severity score greater than one point in order to be considered
significant.
33
The same comparison can be made by body region for MAXAISs equal or greater than
one obtaining the agreement results reported in table 5. These data show that our revised
ICD-AIS map procedure vs. the original one performs slightly better for head/neck and
face, worse for thorax, abdomen and extremities, much better for the external area.
Nevertheless only the difference in proportions of agreements for the extremities (– 10
percentage points p = 0.001) and the external area / burns (+ 28 percentage points p =
0.0000) are statistically significant, given the power of samples. It as also to be considered
that we included the spine injuries in the specific body area, while in the original study they
were considered apart.
Once adopted the above mentioned tolerance limit of 1 point of AIS score the proportions
of agreement in our sample rises up to more than 80 percent in any body region. It
depends on the fact that our manual coders tend to attribute superficial injuries to the
specific body region instead of the external area. It is reflected in the fact that in our
sample the number of cases presenting external injuries is just one third of the cases
observed in the MacKenzie et Al. sample. The tolerance limit implicitly eliminates that
effect, given that the external injuries are in general of minor severity. Even if we include in
the analysis the cases that could not be converted by the automatic procedure, while
maintaining the same tolerance limit, the agreement proportion in each body region is
greater than the one observed in the original sample.
At global level in our sample the ISS scores difference between the two methods ranged
within ± 5 points for 64.6 percent of patients (95% C.I.: 61.6 – 67.7) in comparison to the
70 percent observed by MacKenzie. Grouping the scores in the three classes used in the
above indicated original study the agreement proportion range from 76.0 percent (95%
C.I.:
72.5 – 79.2) in the ISS group 1-12 to 41.9 percent (95% C.I.:
33.8 – 50.3) in the
ISS group 13-19 and 39.5 percent (95% C.I.: 31.9 – 47.5) for ISS greater than 20.
Ranking the patients by the above indicated ISS groups the total observed agreement
between chart review and (revised-extended) conversion table rises up to 72.7 percent
(95% C.I.: 69.8 – 75.5) in comparison to the 68 percent observed by MacKenzie et al. in
their ICD-AIS conversion map. Our percent value of ISS-grouped scores agreement (72.7)
between automatic and manual method is very near to the interrater average agreement
(75 percent) among manual coders reviewing the same complete medical charts, used by
MacKenzie et al. as reference for validating the conversion table. This agreement value
could even be incremented by eliminating errors in abstracting medical charts. In the HDR
34
records we often observed an incomplete reporting of ICD codes in comparison to the
injuries reported in the complete medical charts and scored by manual coders.
Anyway having the ICD-9-CM diagnosis codes less specificity in the description of trauma
than the AIS ones the agreement could never be perfect. In the following figure we
observe in the validation sample that automatically assigned MaxAIS scores based on the
conversion of ICD-9-CM codes from discharge abstracts tend to underestimate the
severity of injury in comparison to the MaxAIS scores manually assigned by coders from
charts review. It can be noticed how the automatic AIS scores frequencies are
systematically higher than the manual ones for low severity scores and the contrary for the
intermediate and very severe ones.
Table 5. Observed agreement between MAXAIS and ISS scores assigned by medical charts review
versus ICD conversion table.
Inpatients (n=1.051) in two Major Hospitals of Italy 1998-2005.
Body Region
MAXAIS >= 1
MAXAIS >= 0 and tolerance MAXAIS >= 1 and
difference +/- 1 AIS score
tolerance difference
+/- 1 AIS score
Head/Neck
N.
Observed
N.
Observed agreement Observed agreement
Cases
agreement (%)
Cases
(%)
(%)
244
51
405
54
84
90
56
170
63
100
Thorax
240
53
337
65
82
Abdomen
122
44
202
55
84
Extremities
505
64
640
80
96
External/Burn
148
98
349
96
100
1.051
73
Face
s
Grouped ISS
35
Fig. 5
In conclusion these results on the validation sample indicate that with the extension to
head of unequivocal ICD-AIS mapping, instead of loosing information on the severity of
injuries, we significantly improved quality of automatic versus manual scoring.
That consideration lead us to the application of the automatic conversion procedure to the
national population-based data from Hospital Discharge Registers. We obtained a biunivocal assignment of a single AIS score per each ICD-9-CM diagnosis on 60 percent of
the Italian inpatients (62% in Piedmont, 57% in Liguria and 61% in Abruzzo) and on 50
percent of the Spanish ones. More than 366.000 admissions have been considered for
Spain and more than 174.000 for the Italian Regions.
The sex age distribution was similar in the two populations as shown in the following
figures.
36
Fig. 6
Fig. 7
Fig. 8
The maximum AIS for each patient has been calculated; the overall MaxAIS and the ones
by body region present very similar frequency distrubutions in the two Countries. For the
overall MaxAIS the value 0 indicate patient with uncorvetible ICD-9-CM codes.
37
Fig. 9
Fig. 10
38
Fig. 11
Fig. 12
39
Fig. 13
Fig. 14
40
Fig. 15
There is a strong agreement in the two Countries among the overall MaxAIS relative
frequency distributions. We assigned the value 0 to the ICD-9-CM diagnoses uncovertible
into bi-univocal severity scores. The agreement is also strong in consideringe the MaxAIS
scores by body region. It is interesting to note how Head/Neck and Thorax show the
greatest proportions of intermediate and severe injuries in the trauma inpatients of both
countries.
In general the Spanish hospital data show a more selective style of admission for trauma
patients than the Italian ones. It is reflected in the fact that for head/neck, face and thorax
body regions the Spanish cases have a significantly greater frequency than the Italians of
intefrmediate severity injuries. It can be beter seen in the ISS distribution which has
different shape in the two countries. In Italy there is most of the cases have low severity
(more than 60% in the ISS class 0-4) whilst in Spain the main group is represented by
intermediate severity inpatients (more than 80% in the ISS classes 5-9 and 10-16).
41
Fig. 16
In order to check the applicability of our revised ICD-AIS map procedure we applied it to
10 samples of trauma inpatients in each Country. The samples consisting in 1.000 cases
each one, randomly extracted from the Hospital Discharge Register (HDR) data of the
examined Country.
The MaxAIS relative frequency distributions in the observed samples either Spanish or
Italian are more concentrated on low severity scores than the one of the validation sample,
as we expected because the validation sample has been extracted respectively from the
Emergency Departments of the biggest University General Hospital of Rome and the
Trauma Center of Milan, the largest metropolitan areas of Italy. These hospitals are highly
specialized and therefore considered second level emergency units because they attract
severe cases from the other smaller hospitals in which the patient firstly arrived.
The observation of the charts indicates us that the automatically converted samples (from
ICD-9-CM to AIS) are internally strictly concordant as to overall (per patient) MaxAIS
distribution within the same Country. In other words by examining different sets of cases,
42
casually selected and of the same dimension of the validation sample, we always obtain
the same severity distribution for trauma inpatient within the examined Country..
Fig. 17
Furthermore the severity distributions are strongly agreeing even when the samples
observed through the different Countries. Apart the difference in frequency between low
(expected higher in Italy) an intermediate severity scores ((expected higher in Spain),
because the style of admission. In other words our revised conversion procedure is
produces results that are consistent internally (within) and externally (through) to the
examined Countries
43
Fig. 18
In conclusion the revised procedure for automatic conversion of discharge abstract ICD-9CM coded diagnoses into AIS scores produced valid figures in comparison with the
severity scores manually assigned by trained coders, on complete medical chart review, to
inpatients of the same validation sample. Furthermore it proved high consistency of the
results when applied in different samples from two different Countries, so that its
requirements of validity and consistency are fully complied.
44