Better informing decisionmaking with
multiple outcomes costeffectiveness analysis:
case studies in palliative care
Dr Nikki McCaffrey
Palliative Care Clinical Studies
Collaborative (PaCCSC)
Flinders Health Economics Group
Flinders University
South Australia
Adelaide
Dr Nikki McCaffrey Apr 2015
Overview
- Health economics
- Multiple outcome comparison
- Net benefit correspondence theorem
- Analysis in cost-disutility space
- Summary measures
- Implications
Dr Nikki McCaffrey Apr 2015
Health economics is all about…
• Cutting costs
• Saving $$$
McCaffrey, N & Currow, D 2015
Health economics is really about…
• Maximising patient outcomes,
improving care for patients
• Within budget-constrained funds
• Making the money go further
Dr Nikki McCaffrey Apr 2015
Health economics is…
‘…concerned with the optimum use of scarce
economic resources for the care of the sick
and the promotion of health, taking into
account competing use of these resources’
(Mushkin 1958)
Dr Nikki McCaffrey Apr 2015
Why consider health economics?
McCaffrey, N & Currow, D 2015
Evaluating ‘optimum’ use
LOS
Carer QOL
Home death
$/QALY
Cost
consequences
analysis (CCA)
Cost
effectiveness
analysis (CEA)
Cost utility
analysis (CUA)
Cost benefit
analysis (CBA)
$/ LYG
$
Incremental Cost Effectiveness Ratio (ICER) = (Ci – Cj) / (Ei – Ej)
Dr Nikki McCaffrey Apr 2015
Quality adjusted life year (QALY)
- Quantity AND quality
- Health-related quality of life
- Measuring the ‘Q’
-
EuroQol 5D (EQ-5D)
Short Form 6D (SF-6D)
Assessment of Quality of Life (AQOL)
Health Utility Index (HUI)
Van den Hout WB et al 2006
Comparison of MAUI domains
Questionnaire
Domains/ dimensions
EQ-5D
Anxiety; pain; mobility; self-care; usual activities
SF-6D
Mental health; pain; physical function; role limitation;
social function; vitality
HUI3
Ambulation; cognition; dexterity; emotion; hearing;
pain; speech; vision
AQoL
Coping; independent living; life satisfaction; mental
health; pain; relationship; self worth; senses
Dr Nikki McCaffrey Apr 2015
Palliative Care
‘…improves the quality of life of patients and their
families facing the problem associated with lifethreatening illness, through the prevention and
relief of suffering by means of early
identification and impeccable assessment and
treatment of pain and other problems, physical,
psychosocial and spiritual.’ (WHO)
http://www.who.int/cancer/palliative/en/palliative care
Dr Nikki McCaffrey Apr 2015
Agar M et al. 2008; Hearn & Higginson 1997; Higginson & Sen-Gupta 2000;
McCaffrey N et al. 2009; Sutton & Coast 2014; McCaffrey N, Skuza P et al. 2014
Dr Nikki McCaffrey Apr 2015
Multiple outcomes
Health
Equity
Carers
Dying process
Quality
Access
Hoefman R, Al-Janabi H, McCaffrey N et al 2014;
Al-Janabi H, McCaffrey N, Ratcliffe J 2013
Cost-consequences analysis
population
sample
1
2
2
3
4
5
5
mean cost
6
7
8
8
Altman & Bland 2005; Munro 1986; Briggs &
O’Brien 2001; Briggs et al 2002;
Rationale
Alternative methods are needed in
economic evaluations of
interventions with multiple outcome
domains to better inform societal
decision making under uncertainty
McCaffrey N et al. 2015. PLoS ONE; 10(3):e01155442015
Net benefit correspondence theorem
(1)
ICERij = (Ci – Cj) / (Ei – Ej)
(2)
INMBij = λ(Ei – Ej) – (Ci – Cj)
(3)
INMBi = (λ x DUi + Ci) – (λ x DU* + C*)
NLi = ((λ1 x DU1i) + (λ2 x DU2i) + Ci) –
(4)
((λ1 x DU1*) + (λ2 x DU2*) + C*) =
INMB = incremental net benefit; i = strategy 1; j = strategy 2
* = optimal strategy; λ = threshold value; DU = disutility; C = costs
Eckermann 2004; Eckermann et.al. 2008
Cost-effectiveness plane
Incremental cost ($) relative to lowest cost strategy
(A)
$500
-3
D
$400
$300
$200
E
$100
B
C
-2
-1
$0 A
0
1
2
3
4
5
6
-$100
-$200
-$300
Incremental pain-free days relative to the lowest cost strategy (A)
-$400
Adapted from Eckermann, Briggs & Willan 2008
Cost-disutility plane
Incremental cost relative to the least expensive
strategy (A)
$500
$450 D
$400
$350
$300
$250
$200
E
$150
$100
B
$50
C
$0
A
0
1
2
3
4
5
6
7
Incremental days WITH pain relative to the most effective strategy (D)
Adapted from Eckermann, Briggs & Willan 2008
8
Palliative Care Extended
Packages At Home (PEACH)
McCaffrey N, Agar M, Harlum J, Karnon J, Currow D,
Eckermann S 2013. Is home-based palliative care costeffective? An economic evaluation of the Palliative
Care Extended Packages at Home (PEACH) pilot.
BMJ Supportive & Palliative Care.
doi:10.1136/bmjspcare-2012-000361
Dr Nikki McCaffrey Apr 2015
PEACH pilot
PICO
Dr Nikki McCaffrey Apr 2015
Treatment arm
PEACH (n=23)
Usual care (n=8)
Increment
Total number of
days at home
13.09
(8.52, 17.65)
12.13
(5.88, 18.38)
0.96
(-6.79, 8.64)
Proportion of home
deaths (%)
56.25
(31.25, 80.00)
80.00
(33.33, 100)
-23.75
(-63.16, 25.00)
Outcome
Cost (AUS$)
PEACH package
$3,489
($2,170, $4,943)
0
$3,489
($2,170, $4,943)
Inpatient stay
$2,603
($1,205, $4,147)
$5,053
($2,084, $8,139)
-$2,450
(-$5,843, $957)
$6,452
($4,469, $8,586)
$5,425
($2,404, $8,531)
$1,027
(-$2,612, $4,738)
Total costs
Threshold value above which the mean INB becomes
positive
$1,068
(-$6,627, $6,578)
Results: Summary of incremental costs &
outcomes framed from a utility perspective
Results: Incremental costs & outcomes
framed from a disutility perspective
Model of care (95% CI)
Increment (95% CI)
Outcome
PEACH (n=23)
UC (n=8)
PEACH
UC
Number of
inpatient days
14.91
(10.35, 19.48)
15.88
(9.63, 22.13)
0#
(0, 6.78)
0.96#
(0, 8.64)
Proportion of
inpatient deaths
(%)
43.75
(20.00, 68.75)
20.00
(0,66.67)
23.75#
(0, 63.16)
0#
(0, 25.00)
Total costs,
mean
$6,452
$5,425
($4,469, $8,586) ($2,404, $8,531)
$1,027$
(0, $4,738)
$0$
(0, $2,612)
#
relative to the most effective model of care (DU1i - DU1*);
$ relative to the cheapest model of care (C – C )
i
*
Cost-disutility plane
Incremental cost relative to the least expensive
strategy (A)
$500
$450 D
$400
$350
$300
$250
$200
E
$150
$100
B
$50
C
$0
A
0
1
2
3
4
5
6
7
Incremental days WITH pain relative to the most effective strategy (D)
Adapted from Eckermann, Briggs & Willan 2008
8
Efficiency frontier
Threshold regions
threshold value per additional home
death (λ2)
$6,000
usual care is the preferred strategy
$4,000
$2,000
B
A
$1,068
$0
$0
$500
$1,000
$1,500
$2,000
$2,500
-$2,000
PEACH is the preferred strategy
-$4,000
- $4,324
-$6,000
threshold value for one extra day at home over 28 days (λ1)
McCaffrey et al. 2011, 2015
$3,000
Summary measures
 Threshold regions: combinations of values over
which strategies maximise ENB
- Expected Net Loss (ENL) planes
- ENL contour
- Cost Effectiveness Acceptability Plane (CEAP)
Bootstrapping the (∆DU1, ∆DU2, ∆C) distribution
Dr Nikki McCaffrey Apr 2015
Expected net loss (ENL) plane
McCaffrey et al. 2011, 2015
Expected net loss contour (ENLC)
Eckermann et al. 2007, 2008;
McCaffrey et al. 2011, 2015
Expected net loss (ENL) planes
McCaffrey et al. 2011, 2015
Expected net loss contour (ENLC)
Eckermann et al. 2007, 2008;
McCaffrey et al. 2011, 2015
The ENLC and EVPI
- The loss of $3,004 per participant could be avoided by
picking the optimal service model in each realisation
with perfect information.
- The expected value of perfect information is the loss
from a bad decision that could be avoided with
perfect, rather than current information
- Hence, the ENLC represents the EVPI per patient of
adopting the strategy minimising ENL given current
uncertainty as a function of threshold values for
multiple effects
Eckermann et al. 2007, 2008;
McCaffrey et al. 2011, 2015
Cost-effectiveness acceptability plane (CEAP)
Eckermann et al. 2008;
McCaffrey et al. 2011, 2015
Robust presentation & summary measures
 Threshold regions: combinations of values over
which strategies minimise NL (maximise NB)
 Expected Net Loss (ENL) planes: differences in
ENL
 ENL contour: strategy minimising ENL & EVPI
with current evidence
 Cost Effectiveness Acceptability Plane (CEAP):
probability the strategy minimises ENL
Conclusions
Limitations of current methods
– Cost-consequences analysis deterministic
– QALYs have limited coverage of palliative care
domains
Advantages of proposed approach
– Jointly allows for uncertainty & explicit weights
across multiple domains or outcomes
– ENL planes & contour identify optimal strategies
& value of information
Dr Nikki McCaffrey Apr 2015
Better informing decision making with
multiple outcome cost-effectiveness
analysis under uncertainty in
cost-disutility space
McCaffrey N, Agar M, Harlum J, Karnon J,
Currow D, Eckermann S
PLoS ONE 10(3):e0115544.
doi:10.1371/journal.pone.0115544
Dr Nikki McCaffrey May 2015
Dr Nikki McCaffrey
Palliative Care Clinical Studies
Collaborative (PaCCSC)
Flinders Health Economics Group
[email protected]
Flinders University
South Australia
References
Arrow, K. J. and R. C. Lind (1970). "Uncertainty and the Evaluation of Public Investment Decisions." The
American Economic Review 60(3): 364-378.
Berenstein, E. G. and Z. Ortiz (2005). "Megestrol acetate for the treatment of anorexia-cachexia
syndrome." Cochrane Database Syst Rev(2): CD004310.
Briggs, A. H. and A. M. Gray (1999). "Handling uncertainty when performing economic evaluation of
healthcare interventions." Health Technology Assessment 3(2): 1-134.
Briggs, A. H., D. E. Wonderling and C. Z. Mooney (1997). "Pulling cost-effectiveness analysis up by its
bootstraps: a non-parametric approach to confidence interval estimation." Health Economics 6(4): 327340.
Coelli, T. (1996) "A guide to DEAP Version 2.1: A data envelopment analysis (computer) program." CEPA
Working Papers No. 8/96.
Eckermann S, Coory M and Willan AR (2008). "Consistently estimating risk difference in a jurisdiction of
interest: odds solution to relative risk fallacies." Value in Health 11(6): A577-578.
Eckermann S and Willan A (2011). "Presenting evidence and summary measures to best inform societal
decisions when comparing multiple strategies." PharmacoEconomics 29(7): 563-577.
Eckermann, S., A. Briggs and A. Willan (2008). "Health technology assessment in the cost-disutility
plane." Medical Decision Making 28(2): 172-181.
Dr Nikki McCaffrey Apr 2015
Eckermann, S. and T. Coelli (2013). "Including quality attributes in efficiency measures consistent with
net benefit: creating incentives for evidence based medicine in practice." Soc Sci Med 76(1): 159-168.
Eckermann, S., M. Coory and A. R. Willan (2011). "Consistently Estimating Absolute Risk Difference
when Translating Evidence to Jurisdictions of Interest." PharmacoEconomics 29(2): 87-96
10.2165/11585910-000000000-000000000.
Eckermann, S., J. Karnon and A. R. Willan (2010). "The Value of Value of Information: Best Informing
Research Design and Prioritization Using Current Methods." PharmacoEconomics 28(9): 699-709.
Eckermann, S. and A. R. Willan (2007). "Expected value of information and decision making in HTA."
Health Economics 16(2): 195-209.
Fare, R. and S. Grosskopf (1998-2000). Reference Guide to OnFront. Lund, Sweden, Economic
Measurement and Quality in Lund Corporation.
Hearn J, Higginson IJ. “Outcome measures in palliative care for advanced cancer patients: a review.” J
Public Health Med. 1997 Jun;19(2):193-9.
Higginson IJ, Sen-Gupta GJA. “Place of Care in Advanced Cancer: A Qualitative Systematic Literature
Review of Patient Preferences.” Journal of Palliative Medicine. 2000;3(3):287-300.
Koerkamp, B. G., M. G. M. Hunink, T. Stijnen, J. K. Hammitt, K. M. Kuntz and M. C. Weinstein (2007).
"Limitations of acceptability curves for presenting uncertainty in cost-effectiveness analysis." Medical
Decision Making 27(2): 101(111).
Dr Nikki McCaffrey Apr 2015
Loprinzi, C. and J. Kugler (1999). "Randomized comparison of megestrol acetate versus
dexamethasone versus fluoxymesterone for the treatment of cancer anorexia/cachexia." Journal
of Clinical Oncology 17: 3299-3306.
Löthgren, M., Zethraeus, N (2000). "Definition, interpretation and calculation of costeffectiveness acceptability curves." Health Economics 9(7): 623-630.
McCaffrey N & Currow D, ‘Separated at birth?’, BMJ Supportive & Palliative Care. Published
Online 5 Mar 2015;5(1):2-3. DOI: 10.1136/bmjspcare-2015-000855
McCaffrey N, Agar M, Harlum J, Karnon J, Currow D, Eckermann S, 'Better informing decision
making with multiple outcome cost-effectiveness analysis under uncertainty in cost-disutility
space', PLoS ONE 10(3):e0115544. DOI: 10.1371/journal.pone.0115544
McCaffrey N, Karnon J, Currow DC and E. S (2011). There's life in the old dog yet: modelling joint
uncertainty in cost-consequences analysis. Adelaide, South Australia, Flinders Centre for Clinical
Change and Health Care Research.
McCaffrey N, Skuza P, Breaden K, Eckermann S, Hardy J, Oaten S, Briffa M and C. DC (2014).
"Preliminary Development and Validation of a New End-of-Life Patient-Reported Outcome
Measure Assessing the Ability of Patients to Finalise Their Affairs at the End of Life." PLoS ONE
9(4): e94316.
McCaffrey, N, Currow, DC & Eckermann, S 2009, 'Measuring Impacts of Value to Patients Is Crucial
When Evaluating Palliative Care', Journal of Pain and Symptom Management, vol. 37, no. 6, pp.
e7-e9.
Dr Nikki McCaffrey Apr 2015
McCaffrey, N, Agar, M, Harlum, J, Karnon, J, Currow, D & Eckermann, S 2013, 'Is home-based
palliative care cost-effective? An economic evaluation of the Palliative Care Extended Packages at
Home (PEACH) pilot', BMJ Supportive & Palliative Care, doi:10.1136/bmjspcare-2012-000361
McCaffrey, N., J. Karnon, D. Currow and S. Eckermann (2010). "The Old Dog & the Poor Relation:
Informing funding decisions in palliative care with cost-consequence analysis in the cost-disutility
plane." Palliative Medicine 24(4): S207.
Mauskopf JA, Paul JE, Grant DM, Stergachis A 1998, ‘The role of cost-consequence analysis in
healthcare decision-making’, Pharmacoeconomics, vol. 13, no. 3, pp 277-88.
Mooney, G., J. Coast, S. Jan, M. Ryan and V. Wiseman (2007). "Searching for a threshold - Not so
NICE." Journal of Health Services Research & Policy 12(3): 190-191.
Mooney GH, Blackwell SH 2004, ‘Whose health service is it anyway? Community values in
healthcare’, Medical Journal of Australia, vol 180, no. 2, pp. 76-8.
Stinnett, AA & Mullahy, J 1998, 'Net health benefits: a new framework for the analysis of
uncertainty in cost-effectiveness analysis', Medical Decision Making, vol. 18, no. 2 Suppl, pp. S6880.
Dr Nikki McCaffrey Apr 2015
Cost-consequences analysis
Variable
Efficacy
Survival without
relapse (months)
Capecitabine
Mayo Clinic
De Gramont
35.00
33.73
33.73
Grade 3/4 adverse event rates
Diarrhoea
11%
13%
6%
Nausea/vomiting
17%
0%
0%
Alopecia
2%
26%
11%
Adverse events
43.46
153.37
62.29
Total
3,697
10,635
7,266
Mean costs (Euros)
Douillard JY et al 2008
Farrell’s production possibilities frontier
with two inputs and one output
Adapted from Coelli 1996
Preferred model of care?
The preferred option is the model of care
minimising NL
Rearranging terms,
NLi = (Ci -C*) + λ1(DU1i - DU1*) + λ2(DU2i – DU2*)
NLPEACH = $? + λ1?+ λ2? = ?
NLUC = $? + λ1?+ λ2? = ?
McCaffrey et al. 2011, 2015
Results: Incremental costs & outcomes
framed from a disutility perspective
Model of care (95% CI)
Increment (95% CI)
PEACH (n=23)
UC (n=8)
PEACH
UC
Number of
inpatient days
14.0
(10.4, 19.4)
15.9
(9.5, 21.9)
0#
(0, 6.8)
0.96#
(0, 8.6)
Proportion of
inpatient deaths
(%)
43.8
(13.0, 52.2)
20.0
(0,63.2)
23.8#
(0, 63.2)
0#
(0, 25.0)
$1,027$
(0, $4,738)
$0$
(0, $2,612)
Outcome
$6,452
$5,425
($4,469, $8,586) ($2,404, $8,531)
Total costs,
mean
#
relative to the most effective model of care (DU1i - DU1*);
$ relative to the cheapest model of care (C – C )
i
*
Preferred model of care?
$1,027 + 0λ1 + 0.24λ2 = $1,027 + 0.24λ2
$0 + 0.96λ1 + 0λ2 = 0.96λ1
PEACH
Usual Care
0.96λ1 = $1,027 + 0.24λ2
λ1 < 0.25λ2 + $1,068
Usual care
λ1 > 0.25λ2 + $1,068
PEACH
McCaffrey et al. 2011, 2015
Efficiency frontier in cost-disutility space
megestrol
(6.8%, 12.8%, $291.50)
Incremental cost
relative to the
cheapest strategy
(AUS$)
Additional proportion of
patients without
appetite improvement
relative to the most
effective strategy (%)
dexamethasone
(0%, 17.0%, $25.20)
placebo
(52.2%, 0%, $0)
1ry - case study 2
Additional proportion of
patients with oedema
relative to the most
effective strategy (%)
Threshold regions