Transforming the cost-effectiveness
threshold into a ‘value threshold’
Initial findings from a simulation model
Mike Paulden and Christopher McCabe
Problem
• The conventional cost-effectiveness (CE) threshold represents
“an estimate of health forgone as other [services] are displaced to
accommodate the additional costs of new technologies”
(Claxton et al. 2013)
• Plotted as a straight line on the CE plane (Drummond et al. 2005)
• Numerous limitations and assumptions:
• Assumes constant marginal returns and divisibility of technologies
• No account for aspects of ‘value’ beyond those considered by the QALY
• Impact of imperfect information is not explicitly considered, nor the
possibility that new interventions represent net disinvestments
• No account for multiple decision makers with conflicting objectives
• Recently, NICE has applied ‘modifiers’ to its baseline threshold to
account for aspects of ‘value’ beyond the QALY (NICE 2009, 2014)
• Resulted in inconsistencies in NICE’s methodology (Paulden et al. 2014)
Objective
• Our objective is to transform the conventional CE threshold
into a ‘value threshold’ of greater use to decision makers
• In doing so we aim to address the limitations previously described
• As a first step we have developed a simulation model in order to
understand how a ‘value threshold’ may differ from a CE threshold
• Of key interest are the implications of:
i. Relaxing conventional assumptions such as constant marginal
returns to scale and perfect divisibility of technologies
ii. Incorporating imperfect information and ‘value’ considerations within
a complex health system with multiple decision makers
iii. Extending the threshold so that it may be used for net disinvestments
Model schematic
Pool of initial technologies
The cost and effectiveness of each
technology is drawn from a distribution
Each technology is randomly assigned a
‘value’ attribute and a specific health
production function ‘shape’ (applies
only if marginal returns are diminishing)
Model schematic
Pool of initial technologies
The cost and effectiveness of each
technology is drawn from a distribution
Each technology is randomly assigned a
‘value’ attribute and a specific health
production function ‘shape’ (applies
only if marginal returns are diminishing)
Divisibility of technologies
Technologies in the pool are either
all divisible or all indivisible
Marginal returns to scale
Technologies in the pool either all
have constant marginal returns to
scale or all have diminishing
marginal returns to scale
Model schematic
Pool of initial technologies
The cost and effectiveness of each
technology is drawn from a distribution
Each technology is randomly assigned a
‘value’ attribute and a specific health
production function ‘shape’ (applies
only if marginal returns are diminishing)
Initial budget
Upon the establishment of the
health system, an initial budget
is assigned for purchasing
technologies from the pool
Divisibility of technologies
Technologies in the pool are either
all divisible or all indivisible
Marginal returns to scale
Technologies in the pool either all
have constant marginal returns to
scale or all have diminishing
marginal returns to scale
Model schematic
Pool of initial technologies
The cost and effectiveness of each
technology is drawn from a distribution
Each technology is randomly assigned a
‘value’ attribute and a specific health
production function ‘shape’ (applies
only if marginal returns are diminishing)
1. The initial allocator
purchases technologies
from the pool until the
initial budget is exhausted
Initial budget
Upon the establishment of the
health system, an initial budget
is assigned for purchasing
technologies from the pool
Divisibility of technologies
Technologies in the pool are either
all divisible or all indivisible
Marginal returns to scale
Technologies in the pool either all
have constant marginal returns to
scale or all have diminishing
marginal returns to scale
Imperfect information
Each decision maker has one of
four levels of information regarding
the effectiveness of technologies:
none, poor, good, or perfect
Other value considerations
Each decision maker assigns one
of four possible weights to ‘value’
considerations beyond the QALY:
none, small, medium, or large
Initial
allocator
Model schematic
Pool of initial technologies
The cost and effectiveness of each
technology is drawn from a distribution
Each technology is randomly assigned a
‘value’ attribute and a specific health
production function ‘shape’ (applies
only if marginal returns are diminishing)
1. The initial allocator
purchases technologies
from the pool until the
initial budget is exhausted
Initial budget
Upon the establishment of the
health system, an initial budget
is assigned for purchasing
technologies from the pool
Divisibility of technologies
Technologies in the pool are either
all divisible or all indivisible
Imperfect information
Each decision maker has one of
four levels of information regarding
the effectiveness of technologies:
none, poor, good, or perfect
Initial
allocator
Marginal returns to scale
Technologies in the pool either all
have constant marginal returns to
scale or all have diminishing
marginal returns to scale
Other value considerations
Each decision maker assigns one
of four possible weights to ‘value’
considerations beyond the QALY:
none, small, medium, or large
New intervention
Each new intervention represents either
a net investment or net disinvestment
Net investments impose costs on the
health system, requiring that resources
be released from other technologies
Net disinvestments release resources,
allowing these to be spend on other
technologies from the pool
Model schematic
Pool of initial technologies
The cost and effectiveness of each
technology is drawn from a distribution
Each technology is randomly assigned a
‘value’ attribute and a specific health
production function ‘shape’ (applies
only if marginal returns are diminishing)
1. The initial allocator
purchases technologies
from the pool until the
initial budget is exhausted
Divisibility of technologies
Technologies in the pool are either
all divisible or all indivisible
Imperfect information
Each decision maker has one of
four levels of information regarding
the effectiveness of technologies:
none, poor, good, or perfect
Initial
allocator
Marginal returns to scale
Technologies in the pool either all
have constant marginal returns to
scale or all have diminishing
marginal returns to scale
Other value considerations
Each decision maker assigns one
of four possible weights to ‘value’
considerations beyond the QALY:
none, small, medium, or large
Agent
New intervention
Each new intervention represents either
a net investment or net disinvestment
Net investments impose costs on the
health system, requiring that resources
be released from other technologies
Net disinvestments release resources,
allowing these to be spend on other
technologies from the pool
2. The agent recommends
the new intervention if its
expected value exceeds the
agent’s value threshold
Value threshold
Used by the agent to determine
whether or not to recommend
the new intervention
Initial budget
Upon the establishment of the
health system, an initial budget
is assigned for purchasing
technologies from the pool
Model schematic
Pool of initial technologies
The cost and effectiveness of each
technology is drawn from a distribution
Each technology is randomly assigned a
‘value’ attribute and a specific health
production function ‘shape’ (applies
only if marginal returns are diminishing)
1. The initial allocator
purchases technologies
from the pool until the
initial budget is exhausted
Divisibility of technologies
Technologies in the pool are either
all divisible or all indivisible
Imperfect information
Each decision maker has one of
four levels of information regarding
the effectiveness of technologies:
none, poor, good, or perfect
Initial
allocator
Marginal returns to scale
Technologies in the pool either all
have constant marginal returns to
scale or all have diminishing
marginal returns to scale
Other value considerations
Each decision maker assigns one
of four possible weights to ‘value’
considerations beyond the QALY:
none, small, medium, or large
Reallocator
Agent
New intervention
Each new intervention represents either
a net investment or net disinvestment
Net investments impose costs on the
health system, requiring that resources
be released from other technologies
Net disinvestments release resources,
allowing these to be spend on other
technologies from the pool
2. The agent recommends
the new intervention if its
expected value exceeds the
agent’s value threshold
Value threshold
Used by the agent to determine
whether or not to recommend
the new intervention
Initial budget
Upon the establishment of the
health system, an initial budget
is assigned for purchasing
technologies from the pool
Model schematic
Pool of initial technologies
The cost and effectiveness of each
technology is drawn from a distribution
Each technology is randomly assigned a
‘value’ attribute and a specific health
production function ‘shape’ (applies
only if marginal returns are diminishing)
1. The initial allocator
purchases technologies
from the pool until the
initial budget is exhausted
Divisibility of technologies
Technologies in the pool are either
all divisible or all indivisible
Imperfect information
Each decision maker has one of
four levels of information regarding
the effectiveness of technologies:
none, poor, good, or perfect
Initial
allocator
Marginal returns to scale
Technologies in the pool either all
have constant marginal returns to
scale or all have diminishing
marginal returns to scale
Other value considerations
Each decision maker assigns one
of four possible weights to ‘value’
considerations beyond the QALY:
none, small, medium, or large
Reallocator
Agent
3. If the agent recommends a net investment, the reallocator must contract
adopted NE/NW technologies and/or expand non-exhausted SE/SW technologies.
Alternatively, if the agent recommends a net disinvestment, the reallocator may
expand non-exhausted NE technologies and/or contract adopted SW technologies
Initial budget
Upon the establishment of the
health system, an initial budget
is assigned for purchasing
technologies from the pool
New intervention
Each new intervention represents either
a net investment or net disinvestment
Net investments impose costs on the
health system, requiring that resources
be released from other technologies
Net disinvestments release resources,
allowing these to be spend on other
technologies from the pool
2. The agent recommends
the new intervention if its
expected value exceeds the
agent’s value threshold
Value threshold
Used by the agent to determine
whether or not to recommend
the new intervention
Model schematic
Pool of initial technologies
The cost and effectiveness of each
technology is drawn from a distribution
Each technology is randomly assigned a
‘value’ attribute and a specific health
production function ‘shape’ (applies
only if marginal returns are diminishing)
1. The initial allocator
purchases technologies
from the pool until the
initial budget is exhausted
Divisibility of technologies
Technologies in the pool are either
all divisible or all indivisible
Imperfect information
Each decision maker has one of
four levels of information regarding
the effectiveness of technologies:
none, poor, good, or perfect
Initial
allocator
Marginal returns to scale
Technologies in the pool either all
have constant marginal returns to
scale or all have diminishing
marginal returns to scale
Other value considerations
Each decision maker assigns one
of four possible weights to ‘value’
considerations beyond the QALY:
none, small, medium, or large
Reallocator
Agent
3. If the agent recommends a net investment, the reallocator must contract
adopted NE/NW technologies and/or expand non-exhausted SE/SW technologies.
Alternatively, if the agent recommends a net disinvestment, the reallocator may
expand non-exhausted NE technologies and/or contract adopted SW technologies
New intervention
Each new intervention represents either
a net investment or net disinvestment
Net investments impose costs on the
health system, requiring that resources
be released from other technologies
Net disinvestments release resources,
allowing these to be spend on other
technologies from the pool
2. The agent recommends
the new intervention if its
expected value exceeds the
agent’s value threshold
Value threshold
Used by the agent to determine
whether or not to recommend
the new intervention
Initial budget
Upon the establishment of the
health system, an initial budget
is assigned for purchasing
technologies from the pool
Agent’s authority
Agent may have mandate to
consider reallocation and/or an
alternative to the intervention
Model schematic
Pool of initial technologies
The cost and effectiveness of each
technology is drawn from a distribution
Each technology is randomly assigned a
‘value’ attribute and a specific health
production function ‘shape’ (applies
only if marginal returns are diminishing)
1. The initial allocator
purchases technologies
from the pool until the
initial budget is exhausted
Divisibility of technologies
Technologies in the pool are either
all divisible or all indivisible
Imperfect information
Each decision maker has one of
four levels of information regarding
the effectiveness of technologies:
none, poor, good, or perfect
Initial
allocator
Marginal returns to scale
Technologies in the pool either all
have constant marginal returns to
scale or all have diminishing
marginal returns to scale
Other value considerations
Each decision maker assigns one
of four possible weights to ‘value’
considerations beyond the QALY:
none, small, medium, or large
Reallocator
Agent
3. If the agent recommends a net investment, the reallocator must contract
adopted NE/NW technologies and/or expand non-exhausted SE/SW technologies.
Alternatively, if the agent recommends a net disinvestment, the reallocator may
expand non-exhausted NE technologies and/or contract adopted SW technologies
Initial budget
Upon the establishment of the
health system, an initial budget
is assigned for purchasing
technologies from the pool
4. Prior to making its recommendation, the agent places its own valuations on both the new
intervention and the reallocator’s preferred reallocation. If the agent has the authority to
mandate a reallocation and/or propose an alternative to the new intervention then it also
places a valuation upon this. The optimal value threshold is that which ensures that a new
intervention is only recommended if doing so maximizes the expected value to the agent
New intervention
Each new intervention represents either
a net investment or net disinvestment
Net investments impose costs on the
health system, requiring that resources
be released from other technologies
Net disinvestments release resources,
allowing these to be spend on other
technologies from the pool
2. The agent recommends
the new intervention if its
expected value exceeds the
agent’s value threshold
Value threshold
Used by the agent to determine
whether or not to recommend
the new intervention
Agent’s authority
Agent may have mandate to
consider reallocation and/or an
alternative to the intervention
Conventional Assumptions
$50m
Higher
budget
Expenditure on new technology
$40m
$30m
Lower
budget
Threshold
'kinks'
$20m
$10m
$0m
-$10m
All decision makers have
perfect information
-$20m
All decision makers place
no weight on other
‘value’ considerations
Lower
budget
-$30m
-$40m
Technologies are divisible
Technologies exhibit
constant returns to scale
Higher
budget
-$50m
-2,000
-1,000
Agent cannot reallocate
0
1,000
2,000
Value of new technology (QALY equivalents)
Diminishing Returns to Scale
$50m
Higher
budget
Expenditure on new technology
$40m
$30m
Lower
budget
$20m
$10m
$0m
-$10m
All decision makers have
perfect information
-$20m
All decision makers place
no weight on other
‘value’ considerations
Lower
budget
-$30m
-$40m
-$50m
-2,000
Technologies are divisible
Technologies exhibit
diminishing returns to scale
Agent cannot reallocate
Higher
budget
-1,000
0
1,000
2,000
Value of new technology (QALY equivalents)
Indivisible Technologies
$50m
Higher
budget
Expenditure on new technology
$40m
$30m
Lower
budget
$20m
$10m
$0m
-$10m
All decision makers have
perfect information
-$20m
All decision makers place
no weight on other
‘value’ considerations
-$30m
Technologies are indivisible
Lower
budget
-$40m
-$50m
-2,000
Returns to scale irrelevant if
technologies are indivisible
Higher
budget
(overlap)
-1,000
0
Agent cannot reallocate
1,000
2,000
Value of new technology (QALY equivalents)
Imperfect Information and
Other ‘Value’ Considerations
$50m
Initial allocator and agent
have perfect information
Higher
and reallocator has
budget
poor information
Expenditure on new technology
$40m
Higher
budget
Initial allocator and agent
place small weight and
reallocator places
large weight on other
‘value’ considerations
$30m
$20m
Lower
budget
Lower
budget
Technologies are divisible
Technologies exhibit
diminishing returns to scale
$10m
Agent cannot reallocate
$0m
Reallocator and agent have
Threshold
perfect information
and initial allocator has
'kinks'
poor information
Threshold
'kinks'
-$10m
Reallocator and agent
place small weight and
initial allocator places
large weight on other
‘value’ considerations
-$20m
-$30m
Lower
budget
-$40m
-$50m
-2,000
Higher
budget
Lower
budget
Technologies are divisible
Higher
Technologies
exhibit
diminishing
budgetreturns to scale
Agent cannot reallocate
-1,000
0
1,000
2,000
Value of new technology (QALY equivalents)
Agent Has Authority to Reallocate
$50m
Initial allocator and agent
have perfect information
and reallocator has
poor information
Expenditure on new technology
$40m
Higher
budget
Initial allocator and agent
place small weight and
reallocator places
large weight on other
‘value’ considerations
$30m
$20m
Lower
budget
Technologies are divisible
Technologies exhibit
diminishing returns to scale
$10m
Agent can reallocate
$0m
-$10m
Threshold
'kink'
-$20m
Lower
budget
-$30m
-$40m
Higher
budget
-$50m
-2,000
-1,000
0
1,000
2,000
Value of new technology (QALY equivalents)
Conclusions
• The conventional ‘CE threshold’ model is merely a special case
among many approaches for determining a value threshold
• Departing from this special case allows for consideration of:
• Differences in the information available to, the values held by, and the
objectives pursued by, multiple interacting decision makers
• The specific value characteristics of each technology
• This has potentially significant implications for the appropriate
specification of value thresholds used for decision making
• Our findings provide insights for future theoretical work, as well
as a rich source of potential hypotheses for researchers
conducting empirical research in this area
Questions
1. Why should value considerations be accounted for within the
threshold used for CE analysis? Isn’t it sufficient to simply apply
weights to new technologies or to consider ‘values’ separately?
2. Why might differences in information, values and objectives
across multiple interacting decision makers result in:
a) Different thresholds for net investments and net disinvestments?
b) Thresholds that cross into the SE and NW quadrants of the CE plane?
3. Why is the threshold dependent upon the agent’s authority?
Are there any implications for the recommendations made by
CADTH or for the decisions of Canadian policy makers who
depend upon CADTH’s guidance?
References
• Claxton et al. (2013). Methods for the Estimation of the NICE Cost
Effectiveness Threshold. CHE Research Paper 81. York: University of York.
• Drummond et al. (2005). Methods for the Economic Evaluation of Health
Care Programmes. Third Edition. Oxford: Oxford University Press.
• Sendi et al. (2002). Opportunity costs and uncertainty in the economic
evaluation of health care interventions. Health Economics, 11(1), 23–31.
• National Institute for Health and Care Excellence (2009). Appraising lifeextending, end of life treatments. London: NICE.
• National Institute for Health and Care Excellence (2014). Consultation
Paper: Value Based Assessment of Health Technologies. London: NICE.
• Paulden et al. (2014). Some Inconsistencies in NICE’s Consideration of
Social Values. PharmacoEconomics. November 2014, 32(11), 1043-1053.