Economic evaluation of health
programmes
Department of Epidemiology, Biostatistics
and Occupational Health
Class no. 19: Economic Evaluation using
Patient-Level Data II
Nov 12, 2008
Plan of class
Collecting patient-level data alongside
RCTs – some issues
Analysis of uncertainty
Collecting economic data
alongside clinical trials
Issues with adding economic
component to an RCT
 Useful complement to efficacy data – very
common in UK; and, relatively cheap
 However:
 Is comparison condition relevant?
 Non-standard, more intensive measurement of
outcomes that could affect results
 Use of intermediate health outcomes
 Inadequate follow-up period or sample size
 Need to abstract from protocol-driven costs
 Artificial rules mandated by protocol (selection of
subjects, keeping them in study, adherence to Tx)
What to do?
‘Pragmatic’ trials:






Patients representative of typical caseload
Routine follow-up
Meaningful and wide range of outcomes
Longer follow-up if relevant
Usually larger sample size
Unblinded patients and physicians
Not often done, particularly having a
sample size powered for cost data
Data collection methods
Add sheets to case report forms in hospital
for hospital-based study
 More or less detailed depending on what is
being evaluated
Administrative data
Questionnaires and diaries
 Accuracy of patient recall is an issue
 Ideally monitor every 30 days
Statistical analysis of
economic data
Nature of economic data
 Skewed distribution (see examples next slide)
 Missing data
• Hopefully missing at random
 Censored data
 Difficulties with ICERs
Examples of skewed
cost data
For
discussion:
Why are
health care
cost data
typically
skewed?
4 problems with ICERs
1: Opposite
meanings,
same ICER
2: If one of 3 interventions (each
compared to a fourth one) is dominant,
and all 3 ICERs are negative, magnitude
of ICER is meaningless
Example:
A: (1 LY, - $2,000): ICER = -2,000 $/LY
B: (2 LY, - $2,000): ICER = -1,000 $/LY
C: (2 LY, - $1,000): ICER = - 500 $/LY
B is preferred yet is intermediate in value
3: X/0 = Infinity!
 If difference in effects can be zero, with non-zero
probability, then values will become impossible to
calculate.
 If confidence interval spans negative and positive
difference, ICER becomes discontinuous:
0
Difference in effects
4: Ratios don’t work well as
dependent variables in
regressions
Often interesting to try to identify factors
associated with cost-effectiveness!
Limits of hypothesis testing
 Test H0 : No difference in effects
 Even if cannot reject H0, this does not prove there is no
difference (power of test could be low – high risk of Type
II error).
 Point estimate remains best estimate of difference
 Mean incremental cost of £186 (95% CI, - £26 to £375)
 Mean incremental QALYs: 0.007 (-0.008 to 0.023)
 Decision-maker may be willing to accept higher risk than
0.05 of incorrectly rejecting null (may prefer to go with
intervention even if not in fact more effective)
Confidence region for costs
and effects
Cost-effectiveness
acceptability curves
Incremental cost-effectiveness
ratio
Average cost per person:
Experimental Tx (E)
Control group(C)
RCEI =
C E- C C
E E- E C
Average value of effectiveness measure:
Experimental group
Control group (C)
Using the bootstrap to obtain a
measure of the sampling
variability of the ICER
Suppose we have nEXP et nCON observations in the experimental
and control groups, respectively. One way to estimate the
uncertainty around an ICER is to:
1. Sample nCON cost-effect pairs from the control group, with
replacement
2. Sample nEXP cost-effect pairs from the experimental group,
with replacement
3. Calculate the ICER from those two new sets of cost-effect
pairs
4. Repeat steps 1 to 3 many times, e.g., 1000 times.
5. Plot the resulting 1,000 ICER values on the Costeffectiveness plane
See Drummond & McGuire, Eds., Economic evaluation in
health care, Oxford, 2001, p. 189
An illustration of step 1
(Note: These are made-up data)
Going over the next steps
again…
 Do exactly the same steps for data from the
experimental group, independently.
 Calculate the ICER from the 2 bootstrapped
samples
 Store this ICER in memory
 Repeat the steps all over again
 Of course, this is done by computer. Stata is
one program that can be used to do this fairly
readily.
Bootstrapped replications of an ICER
with 95% confidence interval
ource: Drummond & McGuire 2001, p. 189
Note: ellipses here
are derived using
Van Hout’s
method and are
too big; the
bootstrap gives
better results
Bootstrapped replications that fall
in all 4 quadrants
Source: Drummond & McGuire 2001, p. 193
A solution: the Cost-effectiveness
acceptability curve
• Strategy: We recognize that the decision-maker may
in fact have a ceiling ratio, or shadow price RC – a
maximum amount of $ per unit benefit he or she is
willing to pay
• So we will estimate, based on our bootstrapped
replications, the probability that the ICER is less than
or equal to the ceiling ratio, as a function of the
ceiling ratio
• If the ceiling ratio is $0, then the probability that the
ICER is less than or equal to 0 is the p-value of the
statistic from testing the null hypothesis that the costs
of the 2 groups are the same
 Recall that the p-value is the probability of observing the
difference in costs seen in the data set (or a smaller one) by
chance if the true difference is in fact 0.
Cost-effectiveness acceptability
curve (CEAC)
Source: Drummond & McGuire 2001, p. 195