Economic evaluation alongside
clinical trials (EEACTs)
Claire Hulme
Academic Unit of Health
Economics
Leeds Institute of Health Sciences
Introduction
• How does the NHS make decisions about
what to prescribe or to recommend for
patients?
• How do they decide which new
technologies, programmes or service
delivery models to adopt?
• How do we decide what represents value for
money?
• Economic evaluation provides a tool by
which to help make these decisions.
• In the UK economic evaluation constitutes
one of the major ways that health
economics contributes to the efficient
running of the NHS
Introduction
The objectives of this seminar are:
• To provide an introduction to economic
evaluation and EEACTs and how they
inform reimbursement decisions
• To demonstrate why, when and how
health economics should be considered in
clinical trials.
Introduction
• What is health economics?
• What is economic evaluation?
• What about EEACTs?
What is health economics?
• Health economics is a branch of economics concerned
specifically with choices in health and health care
• It is based on the same premise as economics: limited
resources and unlimited wants; choice in the face of budget
constraints
Key concepts in health economics
• Scarcity: In health economics we assume that there are limited resources
(time of a surgeon; specialised equipment; number of beds in a ward
etc) and unlimited wants (unlimited needs of patients)
• Choice: And that choices are made in the face of budget constraints
(NHS budget)
• Opportunity cost: The opportunity cost of undertaking an activity is
defined as the benefits that must be foregone by not allocating
resources to the next best activity
The concept of opportunity cost is
fundamental to health economics. It
is based upon the idea that scarcity of
resources means that use of
resources on one health care activity
inevitably means sacrificing activity
somewhere else
Why is health economics
important?
•
•
•
In the UK around 8-9% of GDP is spent on healthcare and this is set to
due to demographic changes, impact of new technology, rising labour
costs and economic growth, increased demand for health care
In England alone the 2015/16 budget is £116.4 billion
We need to make choices and decisions about how our resources are
used
‘No healthcare system, anywhere in the world,
has achieved levels of spending sufficient to
meet all its clients’ wants for healthcare’
(Morris et al, 2007:3)
Economic evaluation
• The most important issue when deciding
whether to provide a healthcare intervention,
service or programme is the extent to which it
improves health; but…. there is no such thing as
a free lunch!
• The more a healthcare intervention or
programme costs, the fewer resources are
available for other programmes or
interventions (opportunity cost)
• When we are concerned with population health
cost becomes the second most important issue
Introduction
• What is health economics?
• What is economic evaluation?
• What about EEACTs?
Economic evaluation
•
•
•
•
Economic Evaluation aims to provide robust information to inform choices
It aims to ensure the benefits of programmes that are implemented exceed
their opportunity costs
It is a structured approach to help decision makers choose between
alternative ways of using resources
Economic evaluation is “... the comparative analysis of alternative courses
of action in terms of both their costs and consequences”
(Drummond, Stoddart & Torrance, 1987)
Economic evaluation
Method
Cost
Outcome
Cost benefit analysis
Monetary value
Monetary value
Cost effectiveness analysis
Monetary value
Natural Units
(e.g. life years saved)
Cost utility analysis
Monetary value
Utility values
(e.g.QALYs)
Choice of use of cost effectiveness analysis or cost utility analysis is connected
to:
Who is asking the question (the perspective)?
Why the question is being asked?
Economic evaluation
Within cost-utility analysis interventions are compared in terms of cost
per quality adjusted life years (QALY) – these include not just the quantity
of life gained after an intervention but also the quality
“For the reference case, cost-effectiveness
(specifically cost–utility) analysis is the
preferred form of economic evaluation. This
seeks to establish whether differences in
costs between options can be justified in
terms of changes in health effects. Health
effects should be expressed in terms of
QALYs.” (NICE, 2008:33)
Costs
Health care system perspective:
• Costs born by others outside of the health care system need not be
considered; e.g. employer costs; family costs
NICE reference case:
• Health and social care provider
Societal perspective:
• Can include cost of absence from work , informal care, out of pocket
expenses, criminal justice system, housing
• Once the perspective is agreed: Identify; Measure; Value
Costs
• Increasingly health economics uses electronic data collected by the NHS
for use in economic evaluation
• HES is the national statistical data warehouse for England of the care
provided by NHS hospitals and for NHS hospital patients treated
elsewhere. HES records: in-patient, out-patient, maternity, A&E
• It doesn’t include primary care data or social care data
• For a societal perspective we need to ask the patients
For valuing cost items identified with the questionnaire:
• PSSRU Unit costs of health and social care
– http://www.pssru.ac.uk/project-pages/unit-costs/2012/
• NHS Reference costs
– https://www.gov.uk/government/publications/reference-costs-guidance-for-2012-13--2
Once we have identified, measured and valued cost and
outcomes, how do we analyse and present the results?
Cost effectiveness ratios
• Incremental Cost Effectiveness Ratio
ICER = (C1 – C2) / (E1 – E2) = ∆C/∆E
(new intervention less control)
• In the UK the decision rule is based on the threshold for costeffectiveness, also known as willingness-to-pay threshold:
• Adopt the new intervention if ICER < Threshold
• NICE Threshold between £20,000 - £30,000
Cost-effectiveness plan
∆Cost (£)
More costly, less effective
(Dominated)
More costly, more effective
∆Effect
Less costly, less effective
Less costly, more effective
(Dominates)
Treatment cost-effective in shaded region
Uncertainty; sensitivity analysis
• Every evaluation will contain some degree of uncertainty
• Allow for uncertainty by way of sensitivity analysis
– One-way; varying on variable at a time.
– Multi-way; varying more than one variable at a time
– Probabilistic; varying all parameters simultaneously based
on probability distributions
Cost effectiveness plane
Incremental cost
Incremental
effectiveness
(Fenwick & Byford, 2005)
Cost effectiveness acceptability
curves
Cost Effectiveness Acceptablility Curve
1
Scenario
20 year time horizon
ABN guidelines
0.9
0.8
The CEAC indicates
the probability that
the intervention is
cost-effective
compared with the
alternative
Probability
0.7
If the threshold is
£140k, there is a 60%
probability that
intervention is costeffective
0.6
0.5
0.4
0.3
0.2
0.1
0
£0
£20,000 £40,000 £60,000 £80,000 £100,000 £120,000 £140,000
Cost Effectiveness Threshold
If the threshold is
£40,000 , there is a
20% probability that
the intervention is
cost effective
What about the long term?
•For economic evaluation we often need
to consider the long term effectiveness of
a health care intervention; we need to
estimate the long term effects and costs
•Within all areas of economic evaluation
we may use decision analytical modelling
with data from a number of sources
• Economic evaluation requires; evidence
from a range of sources; comparison with
all relevant alternatives; appropriate time
horizon; quantification of uncertainty
•The need to satisfy these requirements
provides a strong rationale for decision
analytic modelling as a framework for
economic evaluation (Briggs, 2007)
Briggs A, Claxton C, Sculpher M. 2007. Decision modelling for health economic evaluation. OUP: Oxford
Model: Hall PS, Hulme C, McCabe C, Oluboyede Y, Round J, Cameron D. 2011 Updated cost-effectiveness
analysis of trastuzumab for early breast cancer: A UK perspective considering long-term toxicity and pattern of
recurrence. Pharmacoeconomics 29 (5);415-432
Long term cost effectiveness
• A good economic model should...
•be populated with the most appropriate and good quality clinical
data
• reflect a realistic picture of current clinical practice
• use the appropriate comparator(s)
• be run for an appropriate time period
• be valid, transparent and reproducible
Clinical effects;
health state
valuation
Resource use;
unit costs
• explore uncertainty
• be easily interpreted
Epidemiological
data; expert
opinion
Estimate of ICER
Introduction
• What is health economics?
• What is economic evaluation?
• What about EEACTs?
EEACTs
• Clinical trials can establish the efficacy and
effectiveness of health care interventions or
therapies
• Over the past 20 years there has been a growing
trend to collect data for an economic evaluation
(medical service use, cost and effect) in the
clinical trial
• Typically economic evaluations (CEA or CUA) are
incorporated into phase III and phase IV trials and sometimes into phase II
• In the UK MRC and NIHR routinely expect cost
effectiveness analysis to be incorporated in large
scale trials
• Many other countries require evidence of
economic value as part of their reimbursement
decision
EEACTs
• By mid 1990s growing trend for EEACTs with direct
observation of the impact of a therapy on costs and effect
• Short term economic impacts are directly observed over the
trial period
• Longer term impacts need to be modelled
• In conducting a EEACT we identify, measure and value costs;
and quantify the effect
• Calculate the ICER
ICER = (C1 – C2) / (E1 – E2) = ∆C/∆E
• Account for uncertainty
• Identify the population for who the results apply
What is it we want to do in an
EEACT?
• Quantify the cost and effect of care
• Assess whether, and by how much average cost and effect
differ by treatment groups
• Compare the magnitude of differences in cost and effect –
evaluate the value for the cost (ICER)
• Identify populations for whom the results apply
Glick et al., 2007
Six steps in designing EEACTs
• Preplanning in preparation for the trial? E.g. identifying length
of follow up for economic endpoints, identifying the types of
health and social care services used by participants, piloting
data collection forms
• What service use is measured? E.g. Limit to disease related
services; limit to delivery setting; limit participants from
whom economic data are collected
• In what form should the data be collected? At what level
should service use be aggregated e.g. micro costing, average
costs
Glick et al., 2007
Six steps in designing EEACTs
• Which price estimates should be used? E.g. national average,
Trust costs
• How naturalistic should the study design be? (sample
inclusion criteria; intention to treat; minimising loss to follow
up) protocol induced cost and effect – exclude protocol
induced services as they wouldn’t occur in usual practice? But
do they have an impact on care?
• What should we do if the full benefit and cost are not
expected to be observed over the trial period? Modelling
Glick et al., 2007
When don’t we need
an EEACT?
• If the trial design means that no
unbiased evidence of economic value
will be observable e.g. differential in
following participants based on outcome
• If it is believed that the result of the
EEACT will not affect the decision to use
the intervention e.g. a therapy may be so
effective people aren’t interested in cost
Making EEACTs relevant to
policy debate
• Comparator
• Adoption of a final outcome such as a QALY
• Representative sample of the patient
population who will use the therapy
• Sufficiently long period of follow up
• Sufficient sample size to power the trial
• Data on a sufficiently broad set of resources
• Collect data to assess/improve transferability
Summary
EEACTs facilitate direct observation of the
impact of a therapy on costs and effect over
the duration of observation – longer term
impacts will need to be modelled
– Considering health economics evidence is a
mandatory part of NICE clinical guidelines
and technology appraisals
– UK funding bodies consider health
economics an important component of
research and research proposals
Text of interest
Glick H, Doshi JA, Sonnad SS, Polsky D.
2007. Economic Evaluation in Clinical
Trials. OUP
Gray AM, Clarke PM, Wolstenholme JL,
Wordsworth S. 2011. Applied Methods
of Cost Effectiveness Analysis in Health
Care. OUP
Morris, S., Devlin, N., Parkin, D. 2007
Economic Analysis in Health Care. John
Wiley & Sons: Chichester
Drummond,
M.F.,
Sculpher,
M.J.,
Torrance, G.W. O’Brien, B., Stoddart,
G.L. 2005 Methods for the Economic
Evaluation of Health Care Programmes.
3rd ed. Oxford University Press: Oxford