Vol. 40 No. 5 November 2010
Journal of Pain and Symptom Management
639
Original Article
The Advanced Dementia Prognostic Tool:
A Risk Score to Estimate Survival in Nursing
Home Residents with Advanced Dementia
Susan L. Mitchell, MD, MPH, Susan C. Miller, PhD, Joan M. Teno, MD, MSc,
Roger B. Davis, ScD, and Michele L. Shaffer, PhD
Hebrew SeniorLife Institute for Aging Research (S.L.M.), and Department of Medicine (S.L.M.,
R.B.D.), Beth Israel Deaconess Medical Center, Boston, Massachusetts; Center for Gerontology and
Health Care Research (J.M.T., S.C.M.), Department of Community Health, Brown University Medical
School, Providence, Rhode Island; and Penn State College of Medicine (M.L.S.), Hershey,
Pennsylvania, USA
Abstract
Context. Estimating life expectancy is challenging in advanced dementia.
Objectives. To create a risk score to estimate survival in nursing home (NH)
residents with advanced dementia.
Methods. This was a retrospective cohort study performed in the setting of all
licensed U.S. NHs. Residents with advanced dementia living in U.S. NHs in 2002
were identified using Minimum Data Set (MDS) assessments. Mortality data from
Medicare files were used to determine 12-month survival. Independent variables
were selected from the MDS. Cox proportional hazards regression was used to
model survival. The accuracy of the final model was assessed using the area under
the receiver operating characteristic curve (AUROC). To develop a risk score,
points were assigned to variables in the final model based on parameter estimates.
Residents meeting hospice eligibility guidelines for dementia, based on MDS data,
were identified. The AUROC assessed the accuracy of hospice guidelines to
predict six-month survival.
Results. Over 12 months, 40.6% of residents with advanced dementia
(n ¼ 22,405) died. Twelve variables best predicted survival: length of stay, age, male,
dyspnea, pressure ulcers, total functional dependence, bedfast, insufficient intake,
bowel incontinence, body mass index, weight loss, and congestive heart failure. The
AUROC for the final model was 0.68. The risk score ranged from 1 to 32.5 points
(higher scores indicate worse survival). Only 15.9% of residents met hospice
eligibility guidelines for which the AUROC predicting six-month survival
was 0.53.
Conclusion. A mortality risk score derived from MDS data predicted six-month
survival in advanced dementia with moderate accuracy. The predictive ability of
Address correspondence to: Susan L. Mitchell, MD, MPH,
Hebrew SeniorLife Institute for Aging Research,
1200 Center Street, Boston, MA 02131, USA.
E-mail: [email protected]
Ó 2010 U.S. Cancer Pain Relief Committee
Published by Elsevier Inc. All rights reserved.
Accepted for publication: February 17, 2010.
0885-3924/$ - see front matter
doi:10.1016/j.jpainsymman.2010.02.014
640
Mitchell et al.
Vol. 40 No. 5 November 2010
hospice guidelines, simulated with MDS data, was poor. J Pain Symptom Manage
2010;40:639e651. Ó 2010 U.S. Cancer Pain Relief Committee. Published by Elsevier Inc.
All rights reserved.
Key Words
Survival, advanced dementia, mortality, nursing home, hospice, palliative care
Introduction
Dementia is a leading cause of death in the
United States.1 Accurately estimating life expectancy in advanced dementia has been a major
barrier to providing palliative care to persons
dying with this condition.2e7
Prognostication helps guide end-of-life decision-making.8e10 Nursing home (NH) residents
with advanced dementia whose family members
understand that their prognosis is poor have
reduced likelihood of receiving burdensome interventions in the last 90 days of life.10 Prognosis
is also a determinant for hospice eligibility,
which requires an estimated life expectancy
of less than six months.3e7,11 Although hospice
is beneficial for patients dying with
dementia,12e14 many do not receive hospice services.15 Physicians cite difficulty in assessing
prognosis as the greatest barrier to referring dementia patients to hospice.16 Hospice eligibility
guidelines for dementia have been criticized because they were not derived from empirical data
and do not accurately predict six-month
survival.5,7
Prior efforts to develop and validate statistical
models to predict survival in advanced dementia are limited.2,5,7,17e19 In 2004, our group
demonstrated the feasibility of deriving a mortality risk score based on the Minimum Data
Set (MDS) data20,21 that estimated six-month
life expectancy in advanced dementia with moderate accuracy.5 However, this earlier work had
several limitations including data from only
two states, inclusion of only newly admitted
NH residents, survival truncated at six months,
use of hazard ratios rather than the logistic regression coefficients (log hazard ratios) to derive point scores, and validation of the risk
score using only retrospective secondary data.
To build on this earlier effort and address its
limitations, the National Institutes of Health
funded a four-year study with the following
two objectives: 1) to derive and validate a mortality risk score (Advanced Dementia Prognostic Tool [ADEPT] in a nationwide population
of NH residents with advanced dementia
(not limited only to new admissions) using
2002 MDS assessments and 12-month followup survival data from Medicare files, and 2)
to conduct a prospective validation of the
ADEPT risk score at the bedside of 600 NH residents with advanced dementia. Additional
aims for both the secondary analyses and prospective validation included a comparison of
the accuracy of the ADEPT risk score with current hospice eligibility guidelines to predict
six-month mortality. With the prospective validation in progress, this report presents the
results from the first of these two objectives;
the derivation and validation of the ADEPT
score using secondary data.
Methods
Study Sample
The cohort was identified from the 2002
National Repository of the MDS, a comprehensive, standardized resident assessment instrument federally mandated for use in all
licensed U.S. NHs.20,21 Full MDS assessments
are completed on all residents within 14 days
of admission, annually, and whenever there is
a significant change in status. Abbreviated
MDS assessments are completed on a quarterly
basis.
NH residents with an admission or annual
full MDS assessment completed between
January 1 and December 31, 2002 were selected. If residents only had a single assessment during that period, it was considered
the baseline assessment; if they had multiple
full assessments, then the baseline assessment
was arbitrarily chosen as the one completed
closest to April 1, 2002. Eligible subjects were
Vol. 40 No. 5 November 2010
Mortality Risk Score in Advanced Dementia
age $65 years and had advanced dementia as
determined from their baseline assessment.
Advanced dementia was defined as having a diagnosis of dementia (Alzheimer’s disease or
other causes) and a Cognitive Performance
Score (CPS) of 5 or 6.20,22 The Alzheimer’s disease and dementia diagnoses have been used
for epidemiological research5,12,13 and have intraclass coefficients of 0.89 and 0.79, respectively.21 The CPS uses five MDS variables to
group residents into seven cognitive performance categories: 0 ¼ intact, 1 ¼ borderline
intact, 2 ¼ mild impairment, 3 ¼ moderate impairment, 4 ¼ moderately severe impairment,
5 ¼ severe impairment, and 6 ¼ very severe impairment with eating problems. A CPS score of
5 corresponds to a mean Mini-Mental State Examination score of 5.1 (standard deviation
[SD]) 5.3.
Mortality Data
To determine survival after baseline, the MDS
assessments of eligible subjects were matched
with 2002 and 2003 Medicare denominator
files, which include dates of death obtained
from the Social Security Administration records. Residents were considered to have died
within the 12-month follow-up period when
the date of death recorded in the Medicare denominator file was within 12 months of the date
on the baseline MDS assessment. Matching was
achieved using the residents’ Heath Insurance
Claim (HIC) and Social Security numbers.
Exact matches based on HIC and/or Social
Security numbers were further validated using
date of birth and gender. A match rate of 91%
of eligible subjects was achieved with this
approach.
Predictors of Survival
Based on the literature and our prior
work,2,5,7,11,17e19,23e30 variables were selected
a priori from the baseline MDS assessment
that were potential predictors of survival in
advanced dementia. Factors exogenous to
the health status of the resident that could
be ordered at the discretion of providers or
family members (i.e., advance directives and
treatments) were not considered for analysis.
The categories of predictor variables included demographic data, functional status,
641
cognitive status, comorbid diagnoses, nutrition and hydration markers, and other
health conditions.
Demographic information included age,
gender, and race (non-Hispanic white, black,
Asian, American Indian, and Hispanic). Subjects whose baseline MDS assessment was completed for purposes of NH admission (vs.
annual assessment) were identified and are
hereafter referred to as ‘‘recent admissions.’’
Measures of cognition included a CPS score
of 5 vs. 6 and the following specific MDS items:
cognitive deterioration in prior 90 days
(vs. improved or no change), not awake most
of day, hallucinations or delusions, and rarely
able to make oneself understood.
Functional ability was assessed using the Activities of Daily Living (ADL) scale (range
0e28, higher scores indicate greater disability).31 Other functional status variables included bedfast most of the day, and functional
deterioration over the past 90 days (vs. no
change or improved).
Active medical diagnoses included diabetes
mellitus, arteriosclerotic heart disease, cardiac
dysrhythmias, congestive heart failure, hypertension, peripheral vascular disease, stroke,
Alzheimer’s disease (vs. other causes of
dementia), seizure disorder, Parkinson’s disease,
chronic obstructive pulmonary disease (COPD),
anemia, cancer (type of cancer not specified in
the MDS), renal failure, pneumonia or other respiratory tract infection, urinary tract infection
in the previous 30 days, other infections (septicemia, wound infection, methicillin-resistant
Staphylococcus aureus, or Clostridium difficile), hip
fracture in the prior 180 days, and other fracture
in the prior 180 days.
Markers of nutrition and hydration included
weight loss (>5% in the previous 30 days or
>10% in the previous 180 days), insufficient
oral intake (did not consume almost all liquids
in previous three days or 25% or more of food
uneaten at most meals), chewing or swallowing
problems, and body mass index (BMI) (kg/m2)
(categorized as <18.5, 18.5 < 25.0, 25.0 < 30.0,
30.0 < 35.0, and $35.0).
Other health conditions present in the seven
days before the MDS assessment included
peripheral edema, bowel incontinence (rarely
or never vs. occasionally, frequently, or always),
fever, at least one pressure ulcer $Stage 2, and
shortness of breath. The presence of recurrent
642
Mitchell et al.
lung aspirations in the prior 90 days was also
ascertained.
Determining Hospice Eligibility with MDS
Data
Based on National Hospice and Palliative
Care Organization guidelines,11 patients with
a primary diagnosis of dementia must meet
two requirements to be eligible for hospice.
First, patients must be at Stage 7c on the Functional Assessment Staging (FAST) scale.32 Second, patients must have experienced at least
one of the following medical conditions over
the prior 12 months: aspiration pneumonia,
pyelonephritis or other upper urinary tract infection, septicemia, multiple decubitus ulcers
$Stage 3, recurrent fever after antibiotics,
and severe eating problems or tube feeding accompanied by a >10% weight loss over the past
six months (or serum albumin <2.5 g/dL).
To determine whether the subjects in our cohort were eligible for hospice, hospice eligibility guidelines were simulated with MDS
variables (Table 1). The FAST scale assesses
Vol. 40 No. 5 November 2010
functional status in dementia and consists of
seven major stages with a total of 16 substages
(possible range, Stages 1e7f).32 Stage 7 is the
most advanced phase and consists of substages
7aef, defined as follows: 7a ¼ speech is limited
to five words or less, 7b ¼ all intelligible vocabulary is lost, 7c ¼ nonambulatory, 7d ¼ unable
to sit independently, 7e ¼ unable to smile,
and 7f ¼ unable to hold head up. To be considered at Stage 7c, patients must have progressed
through all the previous FAST stages in a sequential fashion. Therefore, subjects who had
all the MDS variables that most closely matched
the description of the FAST Stages 6 through 7c
on their baseline assessments were identified
(Table 1). These variables included limited or
more extensive assistance needed for dressing
and toileting, supervision or more assistance
needed for bathing, urinary incontinence at
least twice a week (and no indwelling catheter),
fecal incontinence at least twice a week, rarely
or never able to make themselves understood,
and inability to ambulate without extensive
assistance.
Table 1
Description of Hospice Eligibility Guidelines for Dementia Simulated with MDS Variables
Functional Assessment Stage
6a ¼ Improperly putting on clothes without assistance or
cueing occasionally or more frequently over the past weeks
6b ¼ Unable to bathe properly occasionally or more
frequently over the past weeks
6c ¼ Inability to handle the mechanics of toileting
occasionally or more frequently over the past weeks
6d ¼ Urinary incontinence occasionally or more frequently
over the past weeks
6e ¼ Bowel incontinence occasionally or more frequently over
the past weeks
7a ¼ Ability to speak limited to six or less different intelligible
words in an average day
7b ¼ Ability to speak limited to one or less intelligible word in
an average day
7c ¼ Unable to ambulate without assistance
Medical Conditions in Prior Year
Aspiration pneumonia
Pyelonephritis or another upper urinary tract infection
Septicemia
Multiple Stage 3 or 4 decubitus ulcers
Recurrent fevers after antibiotic treatment
Insufficient oral intake or tube feeding with impaired
nutritional status (10% weight loss within the prior six
months or serum albumin <2.5 g/dL)
MDS Variablea
Limited or more extensive assistance required to dress on
least several occasions during the last seven days
Supervision or more assistance required to bathe during the
last seven days
Limited or more extensive assistance to use the toilet at least
several times during the last seven days
Urinary incontinence at least twice a week
Bowel incontinence at least twice a week
Rarely or never makes oneself understood
Extensive assistance (or total dependence) required for
locomotion (i.e., move between locations) during the last
seven days
MDS Variables
Recurrent lung aspiration in past 90 days and pneumoniab
Urinary tract infection in prior 30 days
Septicemia
Multiple Stage 3 or 4 decubitus ulcers in past seven days
Fever
Insufficient oral intake (did not consume almost all liquids in
previous three days or $25% of food uneaten at most
meals) or tube feeding with 10% or more weight loss in the
previous six monthsb
a
MDS variables used to determine functional assessment stage were ascertained from the baseline assessment. A medical condition was considered
present if it occurred on any available MDS assessments completed during the year before baseline or on the baseline assessment.
b
Conditions had to be present on the same assessment.
Vol. 40 No. 5 November 2010
Mortality Risk Score in Advanced Dementia
Variables also were identified from the MDS
that most closely matched the description of
the other medical conditions required for hospice eligibility (Table 1). These conditions
were determined from the subjects’ baseline
MDS assessment and all available full and quarterly assessments from the 12 months before
baseline (i.e., from 2001 to 2002). Subjects
having the following conditions on these assessments were identified: recurrent aspiration
in prior 90 days and pneumonia (both conditions had to be present on the same assessment), urinary tract infection in the prior 30
days, septicemia, multiple decubitus ulcers
$Stage 3, fever, insufficient oral intake (did
not consume almost all liquids in previous
three days or 25% or more of food uneaten
at most meals) or tube feeding, and $10%
weight loss in the previous six months (albumin levels are not available in the MDS).
Hospice eligibility was only assessed among
subjects in the NH at least 12 months, as this
was necessary to determine presence of coexisting medical conditions. Subjects were considered eligible for hospice if they met MDS
criteria for FAST Stage 7c at baseline and had
at least one of the aforementioned medical
conditions present within the prior 12 months.
Statistical Analyses
Descriptive analyses were conducted to examine the frequencies (categorical variables)
and means with SDs (continuous variables)
of all potential predictors (independent variables) of survival. Survival (dependent variable) was analyzed for all subjects. For
subjects who died within 12 months of their
baseline MDS assessment, survival was defined
as the duration between the baseline assessment and death dates. Subjects who did not
die within 12 months were treated as censored
observations.
The first step in deriving the survival model
involved bivariable analyses to examine the associations between each individual independent
variable and survival using unadjusted Cox proportional hazards regression with ties handled
using Efron’s approximation method.33 The results of the bivariable analyses guided the selection of independent variables to be considered
in a multivariable model. Continuous variables
were transformed into categorical variables
based on clinically and statistically meaningful
643
subdivisions to facilitate their application in
a practical risk score. For example, age was categorized into five-year intervals, ADL score was
dichotomized to either a value equal to 28 (total
functional dependence) vs. <28, and BMI was
grouped into 5 kg/m2 intervals, with the lower
group cutoff at 18.5 kg/m2, which is the threshold for being underweight.34 The associations
between survival and transformed variables
were examined in both their original and newly
defined forms. We assessed the proportional
hazards assumption by examining log-log survival plots.35 No substantive violations of the assumption were identified.
Given the number of comparisons and the
large sample size, associations that would ordinarily be classified as statistically significant
(e.g., P < 0.05) may be clinically insignificant.
Rather than choosing an arbitrarily small
P value to decide which variables to retain
for the multivariable analysis, the selection of
independent variables for entry into the multivariable model building procedure was guided
by the relative strength of statistical significance, collinearity, the feasibility of operationalizing the variable in a risk score, and clinical
experience. Our goal was to create a parsimonious and practical risk score with no more than
12 variables.
Multivariable analysis using Cox proportional hazards regression with ties handled using Efron’s approximation method was used to
derive a final survival model. The final model
selection was based on all subsets regression
optimizing the score Chi-square statistic.36
Collinearity was assessed by examining variance inflation factors.37
To create the ADEPT risk score, points were
assigned based on the precise weighting of the
regression coefficients (log hazard ratios) of
each risk factor in the final multivariable survival model. In the unadjusted analysis, the regression coefficients for the five-year age
categories were nearly perfectly linear. Therefore, the five-year age categories were incorporated as a linear term in the model, and the
risk score points were based on the regression
coefficient for the age variable coded in this
manner in the multivariable model. With this
approach, the lowest age category (65 < 70
years) was standardized to one point. Each
five-year age increment was assigned an additional point. For all other variables, point
644
Mitchell et al.
values were assigned by multiplying their regression coefficients by the inverse of the regression coefficient for the age categories. Point
scores were rounded to one decimal place for
ease of calculation. To obtain a risk score for
a subject, point values were summed for all
mortality-related factors present for that individual on the baseline MDS assessment.
The final multivariable model was internally
validated using cross-validation, and its accuracy
was assessed with measures of discrimination
and calibration (reliability). Cross-validation
corrects these measures for overfitting (optimism). Discrimination was measured using
a generalized area under the receiver operating
characteristic curve (AUROC), which quantifies the ability of the prediction model to separate those who died from those who remained
alive.38 Calibration was assessed by dividing subjects into deciles of risk according to their
model predictions, and the observed mortality
rate among the subjects in each decile was plotted against the average predicted probability of
death and compared with the 45 line (perfect
calibration).39 Separate calibration curves
were constructed for six-month and 12-month
survival.
In addition to assessing the accuracy of the
final multivariable Cox proportional hazards
model to predict survival, optimism-corrected
estimates of the AUROCs also were used to
measure the discriminatory power of the final
ADEPT risk score for predicting the dichotomized outcomes of six-month and 12-month
survival.
Finally, the discriminatory power of hospice
eligibility guidelines to predict six-month survival was compared with the newly derived
ADEPT risk score. Whether a subject meets
hospice eligibility was a binary outcome,
whereas the new risk score was an ordinal measure. Therefore, to make a fair comparison of
the two measures, the specificity of the hospice
guidelines in estimating six-month survival was
computed, and the cutoff necessary to give the
same specificity for the ADEPT score was determined. After the two diagnostic criteria were
set to give the same specificity, the AUROCs
of the current hospice eligibility guidelines
and the ADEPT risk score were compared. All
analyses were conducted using SAS (SAS Institute Inc., Cary, NC) and S-PLUS (Tibco Software Inc., Palo Alto, CA).
Vol. 40 No. 5 November 2010
Results
Study Sample
Among the 2,333,662 NH residents with full
MDS assessments between January 1 and
December 31, 2002, 245,132 (10.5%) residents
had met eligibility criteria for advanced dementia. Adequate matching between the
MDS assessments and the Medicare denominator file was achieved for 222,532 (91%) of
those residents. An additional 127 residents
were excluded because their reported death
date preceded the baseline MDS assessment
date, leaving 222,405 subjects in the final cohort. Eligible residents who were excluded
(9%) had similar distributions as the matched
cohort with respect to age, gender, race, NH
length of stay, and CPS score (i.e., 5 vs. 6).
A total of 49% of subjects were between 80
and 90 years (mean age: 84.5 7.5 [SD]
years), 23.0% were male, 84.0% were white,
and 36.2% were recent admissions to the NH
(Table 2). Slightly more than half (52.1%) of
subjects had a CPS score of 6 (vs. 5), 35.5%
were totally functionally dependent (ADL
score ¼ 28), and 9.1% were bedfast. The most
common active medical diagnoses included
hypertension (48.8%), anemia (22.5%), stroke
(21.0%), diabetes (19.0%), and congestive
heart failure (17.0).
With respect to nutritional markers, 9.2% of
subjects had had a recent weight loss, 41.9%
had insufficient oral intake, 53.2% had a chewing and swallowing problem, and 14.0% had
a BMI <18.5 kg/m2. Other health conditions experienced by the cohort included bowel incontinence (87.0%), peripheral edema (15.0%), and
at least one pressure ulcer >Stage 2 (14.7%).
Modeling Survival
A total of 90,324 (40.6%) subjects died within
12 months of their baseline assessments. Table 2
presents the bivariable associations of each independent variable and survival. Owing to the
large sample size and our a priori selection of
variables expected to be associated with mortality, the unadjusted associations between almost
all independent variables and survival were statistically significant. After examining the relative strength of these associations, collinearity,
feasibility of operationalizing a given variable
in a risk score, and applying clinical experience,
the following variables were selected as
Vol. 40 No. 5 November 2010
Mortality Risk Score in Advanced Dementia
645
Table 2
Baseline Characteristics of NH Residents with Advanced Dementia and Their Unadjusted Associations with
Survival (n ¼ 222,405)
Characteristics
Demographics
Age (years)
65 < 70
70 < 75
75 < 80
80 < 85
85 < 90
90 < 95
95 < 100
$100
Male
n (%)
Unadjusted Association with Survival
Hazard Ratio (95% Confidence Interval)
6,766
15,685
33,060
51,970
56,979
39,407
15,379
3,159
51,223
(3.0)
(7.1)
(14.9)
(23.4)
(25.6)
(17.7)
(6.9)
(1.4)
(23.0)
Referent
1.18 (1.12e1.24)
1.30 (1.24e1.37)
1.46 (1.39e1.53)
1.67 (1.53e1.75)
1.91 (1.82e2.00)
2.16 (2.05e2.27)
2.51 (2.36e2.68)
1.54 (1.52e1.57)
Racea
Non-Hispanic white
Black
Hispanic
Asian
American Indian
Recent NH admission
186,554
25,655
7,228
1,935
678
80,394
(84.0)
(11.6)
(3.3)
(0.9)
(0.3)
(36.2)
Referent
0.82 (0.80e0.84)
0.85 (0.81e0.88)
0.93 (0.87e1.00)
0.98 (0.87e1.11)
1.90 (1.87e1.92)
Cognitive status
CPS of 6 (vs. 5)
Cognitive deterioration in past 90 days
Lethargic or not awake most of the day
Hallucinations or delusions
Rarely makes oneself understood
115,902
26,773
60,808
12,642
93,884
(52.1)
(12.0)
(27.3)
(5.7)
(42.2)
1.44
1.72
1.50
0.90
1.07
Functional status
ADL score ¼ 28b
Bedfast most of day
Functional deterioration in past 90 days
79,020 (35.5)
20,116 (9.0)
48,686 (21.9)
(1.42e1.46)
(1.69e1.75)
(1.48e1.52)
(0.87e0.92)
(1.06e1.09)
1.44 (1.42e1.45)
2.09 (2.05e2.13)
1.66 (1.63e1.68)
Active diagnoses
Diabetes mellitus
Arteriosclerotic heart disease
Cardiac dysrhythmias
Congestive heart failure
Hypertension
Peripheral vascular disease
Stroke
Alzheimer’s disease (vs. other dementia)
Seizure disorder
Parkinson’s disease
COPD
Anemia
Cancer
Renal failure
Pneumonia or respiratory tract infection
Urinary tract infection in prior 30 days
Other infectionsc
Hip fracture prior 180 days
Other (non-hip) fracture prior 180 days
42,410
29,915
26,450
37,756
108,602
23,285
46,782
103,838
20,914
20,750
23,007
50,106
14,140
8,567
17,795
28,754
10,159
7,196
5,308
Markers of nutrition and hydration
Weight lossd
Insufficient oral intakee
Chewing or swallowing problem
20,302 (9.2)
93,121 (41.9)
118,230 (53.2)
1.80 (1.77e1.84)
1.47 (1.45e1.49)
1.39 (1.37e1.40)
BMI (kg/m2)a
<18.5
18.5 < 25.0
25.0 < 30.0
30.0 < 35.0
$35.0
30,664
116,209
54,549
14,903
3,030
1.46 (1.43e1.48)
Referent
0.76 (0.74e0.77)
0.66 (0.64e0.68)
0.66 (0.62e0.71)
(19.1)
(13.5)
(11.9)
(17.0)
(48.8)
(10.5)
(21.0)
(46.7)
(9.4)
(9.3)
(10.3)
(22.5)
(6.4)
(3.9)
(8.0)
(12.9)
(4.6)
(3.2)
(2.4)
(14.0)
(53.0)
(24.9)
(6.8)
(1.4)
1.16
1.13
1.42
1.44
1.09
1.10
1.20
0.80
0.91
1.07
1.36
1.17
1.51
1.76
2.03
1.51
2.09
1.37
1.22
(1.14e1.18)
(1.11e1.15)
(1.39e1.44)
(1.42e1.47)
(1.07e1.10)
(1.08e1.13)
(1.19e1.22)
(0.79e0.81)
(0.89e0.93)
(1.05e1.10)
(1.33e1.39)
(1.15e1.18)
(1.47e1.55)
(1.71e1.81)
(1.99e2.07)
(1.49e1.54)
(2.03e2.14)
(1.33e1.42)
(1.17e1.27)
(Continued)
646
Mitchell et al.
Vol. 40 No. 5 November 2010
Table 2
Continued
Characteristics
n (%)
Other health conditions
Peripheral edema
Bowel incontinencef
Fever in prior seven days
Pressure ulcers, at least one $Stage 2
Shortness of breath
Recurrent lung aspirations in prior 90 days
33,241
193,418
7,081
32,661
11,705
2,317
Unadjusted Association with Survival
Hazard Ratio (95% Confidence Interval)
(15.0)
(87.0)
(3.2)
(14.7)
(5.3)
(1.0)
1.37
1.44
2.15
2.14
2.29
2.45
(1.35e1.39)
(1.41e1.47)
(2.09e2.22)
(2.10e2.17)
(2.24e2.35)
(2.33e2.58)
a
Missing data: race, n ¼ 355; BMI, n ¼ 3,050.
ADL score (0e28) is the sum of scores in seven domains of function including bed mobility, dressing, toileting, transfer, eating, grooming, and
locomotion. Each is scored on a 5-point scale (0, independent; 1, supervision; 2, limited assistance; 3, extensive assistance; and 4, total dependence). A score of 28 represents complete functional dependence.
c
Other infections include septicemia, wound infection, methicillin-resistant S. aureus, and C. difficile.
d
Weight loss is defined as >5% body weight in prior 30 days or >10% in prior 180 days.
e
Insufficient oral intake: not consuming almost all liquids in previous three days or $25% of food uneaten at most meals.
f
Bowel incontinence occasionally, frequently, or always (vs. rarely or never).
b
candidates for the multivariable model: age,
gender, recent admission, not awake most of
the day, ADL score, bedfast, dysrhythmias, congestive heart failure, COPD, cancer, renal failure, pneumonia or respiratory tract infection,
urinary tract infection, other infections, weight
loss, insufficient oral intake, chewing or
swallowing problems, BMI <18.5 kg/m2, bowel
incontinence, fever, pressure ulcers, shortness
of breath, and lung aspiration.
Using Cox proportional hazards models
guided by the score Chi-squared criterion, the
12 variables that best predicted survival were
selected (Table 3). The variables in the final
Table 3
Final Multivariable Model of Characteristics Associated with Survival Among NH Residents with Advanced
Dementia (n ¼ 218,088)a
Characteristics
Recent NH admission
Age (in years; per five-year increment)
65 < 70
70 < 75
75 < 80
80 < 85
85 < 90
90 < 95
95 < 100
$100
Male
Shortness of breath
At least one pressure ulcers $Stage 2
ADL score ¼ 28d
Bedfast most of day
Insufficient oral intakee
Bowel incontinencef
BMI <18.5 kg/m
Weight lossg
Congestive heart failure
Adjusted Hazard Ratio
(95% Confidence Interval)
Regression Coefficient
(Log Hazard Ratio)b
1.72 (1.69e1.75)
1.18 (1.17e1.18)
d
d
d
d
d
d
d
d
1.71 (1.68e1.74)
1.57 (1.53e1.61)
1.44 (1.41e1.46)
1.42 (1.40e1.44)
1.41 (1.38e1.44)
1.39 (1.37e1.41)
1.37 (1.34e1.40)
1.35 (1.32e1.37)
1.30 (1.27e1.33)
1.28 (1.26e1.30)
0.54207
0.16431
d
d
d
d
d
d
d
d
0.53623
0.44903
0.36216
0.34929
0.34024
0.32837
0.31275
0.29841
0.26149
0.24739
Points in Risk Scorec
3.3
1.0
2.0
3.0
4.0
5.0
6.0
7.0
8.0
3.3
2.7
2.2
2.1
2.1
2.0
1.9
1.8
1.6
1.5
a
n ¼ 218,088 (vs. n ¼ 222,405 for total eligible cohort) because some subjects (n ¼ 4,317) have missing values for at least one characteristic in the
final multivariable model.
b
Precision of regression coefficients to five decimal places is needed to determine the point score accurately.
c
Based on the regression coefficients (log hazard ratios) in the final model, each variable was assigned points in the ADEPT risk score. The lowest
age category (65 < 70 years) was standardized to 1 point. Each five-year age increment was assigned an additional point. The regression coefficients of all other variables were multiplied by the inverse of the regression coefficient for the age categories (1/0.16431 ¼ 6.08606).
d
ADL score (0e28) is the sum of scores in seven domains of function including bed mobility, dressing, toileting, transfer, eating, grooming, and
locomotion. Each is scored on a 5-point scale (0, independent; 1, supervision; 2, limited assistance; 3, extensive assistance; and 4, total dependence). A score of 28 represents complete functional dependence.
e
Insufficient oral intake: not consuming almost all liquids in previous three days or $25% of food uneaten at most meals.
f
Bowel incontinence occasionally, frequently, or always (vs. rarely or never).
g
Recent weight loss is defined as >5% body weight in prior 30 days or >10% in prior 180 days.
Vol. 40 No. 5 November 2010
Mortality Risk Score in Advanced Dementia
model included recent admission, age, male,
shortness of breath, pressure ulcers, ADL score,
bedfast, insufficient oral intake, bowel incontinence, BMI <18.5 kg/m2, weight loss, and congestive heart failure. The optimism-corrected
AUROC for this final model was 0.68.
Risk Score Derivation and Accuracy
Based on the regression coefficients (log hazard ratios) in the final model, each variable was
assigned points in the ADEPT risk score. The
lowest age category (65 < 70 years) was standardized to one point. Each five-year age increment
was assigned an additional point. The regression
coefficients of all other variables were multiplied
by the inverse of the regression coefficient
for the age categories (1/0.16431 ¼ 6.08606)
(n.b., precision to five decimal places is necessary to calculate the correct point scores). For
example, the regression coefficient for male
gender was 0.53623; therefore, after rounding
to the first decimal place, it was assigned a point
score of 3.3 (0.53623 6.08606). With this approach, the points assigned for each variable
647
were as follows: length of stay <90 days, 3.3
points; age, 65 < 70 years, 1 point, with 1 additional point for every five-year increment;
male, 3.3 points; shortness of breath, 2.7 points;
at least one pressure ulcer greater than or equal
to Stage 2, 2.2 points; ADL score ¼ 28, 2.1 points;
bedfast, 2.1 points; insufficient oral intake, 2.0
points; bowel incontinence, 1.9 points; BMI
<18.5 kg/m2, 1.8 points; recent weight loss, 1.6
points; and congestive heart failure, 1.5 points.
The range of possible total point scores was
1e32.5.
To obtain a risk score for a subject, point
values were summed for all mortality-related factors that individual had on his/her baseline
MDS assessment. A value of ‘‘0’’ was assigned
for factors that a subject did not have on that assessment. For example, a 76-year-old (3.0 points)
male (3.3 points) resident who was a recent admission to the NH (3.3 points), totally functionally dependent (2.1 points), and experienced
recent weight loss (1.6 points), would have a total
of 13.3 points (3.0 þ 3.3 þ 3.3 þ 2.1 þ 1.6).
Table 4 shows the number and proportion of
Table 4
Number (%) of Subjects with Each Possible Total Risk Score and the Six- and 12-Month Probabilities of Death
with Each Total Score (n ¼ 218,088)
Observed Probability of Death Within
Total Risk Score
1 (minimum score)
>1e2
>2e3
>3e4
>4e5
>5e6
>6e7
>7e8
>8e9
>9e10
>10e11
>11e12
>12e13
>13e14
>14e15
>15e16
>16e17
>17e18
>18e19
>19e20
>20e21
>21e22
>22e23
>23e24
>24e25
>25e26
>26e27
>27e28
>28e32
Subjects with Each Score, n (%)
84
236
1,232
2,609
5,859
9,784
14,700
18,439
21,634
23,036
22,509
20,938
18,632
15,038
111,691
9,512
6,721
4,955
3,585
2,547
1,777
1,154
648
385
188
99
58
21
17
(0.04)
(0.11)
(0.56)
(1.20)
(2.69)
(4.49)
(6.74)
(8.45)
(9.92)
(10.56)
(10.32)
(9.60)
(8.54)
(6.90)
(5.36)
(4.36)
(3.08)
(2.27)
(1.64)
(1.17)
(0.81)
(0.53)
(0.30)
(0.18)
(0.09)
(0.05)
(0.03)
(0.01)
(<0.01)
Six Months
12 Months
0.01
0.04
0.05
0.06
0.06
0.08
0.10
0.12
0.15
0.17
0.21
0.25
0.29
0.34
0.40
0.46
0.52
0.57
0.64
0.67
0.73
0.77
0.83
0.83
0.88
0.88
0.83
0.95
1.00
0.06
0.08
0.11
0.13
0.15
0.19
0.23
0.26
0.30
0.33
0.37
0.42
0.47
0.52
0.57
0.62
0.67
0.71
0.76
0.79
0.84
0.87
0.90
0.91
0.94
0.96
0.90
1.00
1.00
0.8
0.6
0.4
0.6
0.8
1.0
0.6
0.8
1.0
1.0
0.8
1.0
0.8
0.6
0.4
0.4
Observed Proportion Surviving 12 Months
0.0
0.2
0.2
0.6
0.4
b
1-Specificity
0.6
0.2
0.4
0.6
0.8
Predicted 12-Month Survival
1.0
0.4
Fig. 2. a) Calibration curves for the risk score’s prediction of six- and b) 12-month survival among NH
residents with advanced dementia (n ¼ 218,088).
Circles represent apparent calibration accuracy; Xs
are estimates corrected for optimism. The 45 line
represents perfect calibration.
0.0
0.2
Sensitivity
0.4
Predicted 6-Month Survival
0.2
Sensitivity
0.2
0.0
b
0.2
a
0.8
a
1.0
subjects who had all possible total point scores
and the observed six- and 12-month mortality
rates for each score. Using our example, the patient with a total score of 13.3 had a 34% chance
of dying within six months and a 52% chance of
dying within 12 months.
Fig. 1 presents the ROC curves for the
ADEPT risk score’s prediction of six-month
and 12-month survival. The AUROCs were
0.73 and 0.70 for six-month and 12-month
Vol. 40 No. 5 November 2010
1.0
Mitchell et al.
Observed Proportion Surviving 6 Months
648
0.0
0.2
0.4
0.6
0.8
1.0
1-Specificity
Fig. 1. a) ROC curves for the risk score’s prediction
of six- and b) 12-month survival among NH residents with advanced dementia (n ¼ 218,088). The
45 line represents chance. The AUROC is 0.73
for six-month survival and 0.70 for 12-month
survival.
survival, respectively. Fig. 2 presents the calibration curves for the risk score, where the
45 line represents perfect calibration. These
curves show that the amount of error, that is,
the difference between the predicted values
and the corresponding optimism-corrected
calibrated values, was small.
Hospice Eligibility Guidelines and Accuracy
Among eligible subjects who resided in the
NH at least 12 months (i.e., reason for baseline
Vol. 40 No. 5 November 2010
Mortality Risk Score in Advanced Dementia
MDS assessment was an annual assessment vs.
NH admission) (n ¼ 142,011), 41.7% met the
MDS-simulated definition of FAST Stage 7c.
The proportions of subjects experiencing
each of the pre-existing conditions specified
in hospice guidelines during the prior 12
months were aspiration pneumonia, 0.7%; urinary tract infection, 24.8%; septicemia, 1.1%;
fever, 6.3%, multiple decubitus ulcers $ Stage
3, 2.9%; and insufficient oral intake (or tube
feeding with weight loss), 15.8%. A total of
38.7% (n ¼ 54,933) subjects had at least one
of these conditions. Taken together, 15.9%
(n ¼ 22,542) of subjects were both at FAST
Stage 7c and had at least one pre-existing condition and, therefore, met the full MDSsimulated guidelines for hospice eligibility.
The AUROC for hospice eligibility was 0.53
for six-month survival. A cutoff score of 12.1
on the ADEPT risk score achieved the same
specificity as that of the simulated hospice
guidelines (0.85). The AUROC for the ADEPT
to predict six-month survival using a cutoff of
12.1 was 0.65 for the entire cohort and 0.59
for residents who resided in the NH at least
12 months (n ¼ 142,011).
Discussion
In this study, clinical data from the MDS
were used to derive and retrospectively validate
a mortality risk score in a nationwide sample of
NH residents with advanced dementia. The
ADEPT score predicted survival with moderate
accuracy and had greater discrimination in estimating six-month mortality compared with
hospice eligibility guidelines simulated with
MDS data.
Our study furthers prior efforts2,5,7,19,40
examining factors associated with survival in
advanced dementia by using a nationwide sample of both long- and short-stay NH residents,
extending follow-up to 12 months, and applying rigorous statistical methods to develop
a mortality risk score. Although methodological differences limit comparisons with prior
studies, characteristics most consistently reported to be associated with increased mortality in advanced dementia include worse
functional status,2,5,7 reduced oral intake,5,19,40
and older age.2,5,7 Only four variables differentiated the 12-item risk scores derived in this
study vs. our earlier effort that examined only
649
newly admitted NH residents with advanced
dementia.5 The prior model included oxygen
use, unstable medical conditions, cancer, and
not being awake most of the day,5 whereas
the ADEPT instrument did not. Oxygen use
was not considered as a predictor in the present study as we purposefully excluded treatments. The MDS variable ‘‘unstable medical
condition’’ also was excluded as a predictor because it was vaguely defined and hard to operationalize in practice. The lack of cancer in the
ADEPT score may reflect the possibility that it
is more strongly associated with survival in only
short-stay (vs. long stay) residents, as suggested
by MDS-based mortality scores derived for the
general NH population.30,41 In place of these
four variables, the ADEPT score included recent admission to the NH, BMI, recent weight
loss, and pressure ulcers. The inclusion of admission status, which was strongly associated with
survival, allows the flexibility of using this tool
in both recently admitted and long-stay residents. Prior studies also have shown that recent
weight loss19,28,42 and pressure ulcers40 are associated with survival in end-stage dementia. Despite these differences, the accuracy of the
ADEPT and our prior risk score (AUROC was
0.74 and 0.70 in derivation and validation cohorts, respectively) were comparable.5 However,
the methodological advancements used to develop the ADEPT (national sample, short and
long stay, and 12-month follow-up) have resulted
in an instrument with greater generalizability
and applicability.
The accuracy of hospice guidelines for dementia, as simulated with MDS data, to predict
six-month survival was only slightly better than
chance (AUROC was 0.53), corroborating
prior research demonstrating the poor predictive ability of these guidelines.4,5,7,19 The discriminatory power of the full hospice
guidelines examined in this study was only
marginally higher than the FAST Stage 7c
alone as reported in our prior work (AUROC
0.51).5 We found the prevalence of several of
the medical conditions included in hospice
guidelines to be very low, which may be attributed, in part, to limitations of ascertainment of
these conditions using MDS data. Among the
medical conditions also considered as predictor variables in our model, only poor oral intake and pressure ulcers were included in the
final risk score.
650
Mitchell et al.
This study has several limitations that deserve comment. First, the ADEPT risk score
was developed using MDS data. Thus, it is possible that important factors associated with survival in advanced dementia are not adequately
captured in MDS assessments. Similarly, the
poor predictive ability we observed for hospice
guidelines may be attributed, in part, to inaccuracies in using MDS data to simulate these
guidelines. Finally, the assessment of the accuracy of the ADEPT in this report was limited to
retrospective analyses of secondary data; thus,
its validity and feasibility as a bedside tool remain unknown. The prospective validation of
the ADEPT and hospice eligibility guidelines
using primary data (i.e., not using MDS variables) that is currently in progress will address
several of these limitations.
Despite our rigorous efforts, the ADEPT
score, while better than hospice guidelines,
has only moderate accuracy in predicting
survival in advanced dementia. Although methodological issues may partly explain our limited
success, it is important to consider the possibility that more accurate estimation of life expectancy in advanced dementia, like other
chronic conditions,43 is simply not possible. If
true, this possibility seriously calls into question
the validity of using prognosis to determine
whether persons with advance dementia should
receive hospice or palliative care services. In the
meanwhile, although hospice guidelines continue to require an estimated life expectancy
of less than six months, the ongoing prospective
validation of the ADEPT risk score will further
inform how well this can be practically achieved
at the bedside of NH residents with advanced
dementia.
Vol. 40 No. 5 November 2010
survival for patients with advanced Alzheimer’s disease. J Am Geriatr Soc 1993;4:535e540.
3. Hanrahan P, Luchins DJ. Access to hospice programs in end-stage dementia: a national survey of
hospice programs. J Am Geriatr Soc 1995;43:56e59.
4. Luchins DJ, Hanrahan P, Murphy K. Criteria
for enrolling dementia patients in hospice. J Am
Geriatr Soc 1997;45:1054e1059.
5. Mitchell SL, Kiely DK, Hamel MB, et al. Estimating prognosis for nursing home residents with advanced dementia. JAMA 2004;291:2734e2740.
6. Volicer L. Hospice care for dementia patients.
J Am Geriatr Soc 1997;45:1147e1149.
7. Schonwetter RS, Han B, Small BJ, et al. Predictors
of six-month survival among patients with dementia:
an evaluation of hospice Medicare guidelines. Am J
Hosp Palliat Care 2003;20:105e113.
8. Weeks JC, Cook EF, O’Day SJ, et al. Relationship
between cancer patients’ predictions of prognosis
and their treatment preferences. JAMA 1998;279:
1709e1714.
9. Murphy DJ, Burrows D, Santilli S, et al. The influence of the probability of survival on patients’
preferences regarding cardiopulmonary resuscitation. N Engl J Med 1994;330:545e549.
10. Mitchell SL, Teno JM, Kiely DK, et al. The clinical course of advanced dementia. N Engl J Med
2009;361:1529e1538.
11. National Hospice Organization. Medical guidelines for determining prognosis in selected
non-cancer diseases. Hosp J 1996;11:47e63.
12. Miller SC, Gozalo P, Mor V. Hospice enrollment
and hospitalization of dying nursing home patients.
Am J Med 2001;111:38e44.
13. Miller SC, Mor V, Wu N, Gozalo P, Lapane K.
Does receipt of hospice care in nursing homes improve the management of pain at the end of life?
J Am Geriatr Soc 2002;50:507e515.
Acknowledgments
14. Shega JW, Hougham GW, Stocking CB,
Cox-Hayley D, Sachs GA. Patients dying with dementia: experience at the end of life and impact of hospice care. J Pain Symptom Manage 2008;35:499e507.
This research was supported by NIH-NIA
R01 AG028423. Dr. Mitchell is supported by
NIH-NIA K24AG033640.
15. Sachs GA, Shega JW, Cox-Hayley D. Barriers to
excellent end-of-life care for patients with dementia.
J Gen Intern Med 2004;19:1057e1063.
References
1. National Center for Health Statistics. National vital statistics reports. 2005. Available from http://
www.cdc.gov/nchs/data/nvsr/nvsr53/nvsr53_15.
pdf. Accessed December 16, 2009.
2. Volicer BJ, Hurley A, Fabiszewski KJ,
Montgomery P, Volicer L. Predicting short-term
16. Brickner L, Scannell K, Marquet S, Ackerson L.
Barriers to hospice care and referrals: survey of physicians’ knowledge, attitudes, and perceptions in
a health maintenance organization. J Palliat Med
2004;7:411e418.
17. van der Steen JT, Ooms ME, Ader HJ,
Ribbe MW, van der Wal G. Withholding antibiotic
treatment in pneumonia patients with dementia:
a quantitative observational study. Arch Intern
Med 2002;162:1753e1760.
Vol. 40 No. 5 November 2010
Mortality Risk Score in Advanced Dementia
18. van der Steen JT, Mehr DR, Kruse RL,
Ribbe MW, van der Wal G. Dementia, lower respiratory tract infection, and long-term mortality. J Am
Med Dir Assoc 2007;8:396e403.
19. Marsh GW, Prochoda KP, Pritchett E, Vojir CP.
Predicting hospice appropriateness for patients
with dementia of the Alzheimer’s type. Appl Nurs
Res 2000;13:187e196.
651
31. Morris JN, Fries BE, Morris SA. Scaling ADLs
within the MDS. J Gerontol A Biol Sci Med Sci
1999;54:M546eM553.
32. Reisberg B. Functional assessment staging
(FAST). Psychopharmacol Bull 1988;24:653e659.
33. Efron B. The efficiency of Cox’s likelihood
function for censored data. J Am Stat Assoc 1977;
72:557e565.
20. Hartmaier SL, Sloane PD, Guess HA, et al. Validation of the Minimum Data Set Cognitive Performance Scale: agreement with the Mini-Mental
State Examination. J Gerontol A Biol Sci Med Sci
1995;50:M128eM133.
34. World Health Organization. Body mass index
classifications. 2004. Available from http://apps.who.
int/bmi/index.jsp?introPage¼intro_3.html. Accessed
December 16, 2009.
21. Hawes C, Morris JN, Phillips CD, et al. Reliability estimates for the Minimum Data Set for nursing
home resident assessment and care screening
(MDS). Gerontologist 1995;35:172e178.
35. Kleinbaum DG, Klein M. Survival analysis: A
self-learning text, 2nd ed. New York: Springer,
2005. pp. 136e145.
22. Morris JN, Fries BE, Mehr DR, et al. MDS Cognitive Performance Scale. J Gerontol 1994;49:
M174eM182.
36. Furnival JM, Wilson RW. Regression by leaps
and bounds. Technometrics 1974;16:499e511.
23. Morrison RS, Siu AL. Survival in end-stage dementia following acute illness. JAMA 2000;284:
47e52.
24. Meier DE, Ahronheim JC, Morris J,
Baskin-Lyons S, Morrison RS. High short-term mortality in hospitalized patients with advanced dementia: lack of benefit of tube feeding. Arch Intern Med
2001;161:594e599.
25. Ahronheim JC, Morrison RS, Baskin SA,
Morris J, Meier DE. Treatment of the dying in the
acute care hospital. Advanced dementia and metastatic cancer. Arch Intern Med 1996;156:2094e2100.
26. Christakis NA, Sachs GA. The role of prognosis
in clinical decision making. J Gen Intern Med 1996;
11:422e425.
27. Casarett DJ, Hirschman KB, Henry MR. Does
hospice have a role in nursing home care at the
end of life? J Am Geriatr Soc 2001;49:1493e1498.
28. Porock D, Oliver DP, Zweig S, et al. Predicting
death in the nursing home: development and validation of the 6-month Minimum Data Set mortality
risk index. J Gerontol A Biol Sci Med Sci 2005;60:
491e498.
29. van Dijk PT, Mehr DR, Ooms ME, et al. Comorbidity and 1-year mortality risks in nursing home residents. J Am Geriatr Soc 2005;53:660e665.
30. Flacker JM, Kiely DK. Mortality-related factors
and 1-year survival in nursing home residents. J Am
Geriatr Soc 2003;51:213e221.
37. Harrell FE. Regression modeling strategies with
applications to linear models, logistic regression,
and survival analysis. New York: Springer, 2001.
38. Hanley JA, McNeil BJ. The meaning and use of
the area under a receiver operating characteristic
(ROC) curve. Radiology 1982;143:29e36.
39. Yates JF. Judgment and decision making. Englewood Cliffs, NJ: Prentice Hall, 1990.
40. van der Steen JT, Mehr DR, Kruse RL, et al. Predictors of mortality for lower respiratory infections
in nursing home residents with dementia were validated transnationally. J Clin Epidemiol 2006;59:
970e979.
41. Flacker JM, Kiely DK. A practical approach to
identifying mortality-related factors in established
long-term care residents. J Am Geriatr Soc 1998;
46:1012e1015.
42. Hirdes JP, Frijters DH, Teare GF. The
MDS-CHESS scale: a new measure to predict mortality in institutionalized older people. J Am Geriatr
Soc 2003;51:96e100.
43. Fox E, Landrum-McNiff K, Zhong Z, et al.
Evaluation of prognostic criteria for determining
hospice eligibility in patients with advanced lung,
heart, or liver disease. SUPPORT Investigators.
Study to Understand Prognoses and Preferences
for Outcomes and Risks of Treatments. JAMA
1999;282:1638e1645.