Pharmacoeconomics 2006; 24 (11): 1121-1131
1170-7690/06/0011-1121/$39.95/0
CONFERENCE PAPER
 2006 Adis Data Information BV. All rights reserved.
Information Created to Evade
Reality (ICER)
Things We Should Not Look to for Answers
Stephen Birch1,2 and Amiram Gafni1
1
2
McMaster University, Hamilton, Ontario, Canada
University of Manchester, Manchester, UK
Abstract
Cost-effectiveness analysis has been advocated in the health economics methods literature and adopted in a growing number of jurisdictions as an evidence
base for decision makers charged with maximising health gains from available
resources.
This paper critically appraises the information generated by cost-effectiveness
analysis, in particular the incremental cost-effectiveness ratio (ICER). It is shown
that this ratio is used as comparative information on what are non-comparable
options and hence evades the reality of the decision-maker’s problem. The
theoretical basis for the ICER approach is the simplification of theoretical
assumptions that have no relevance to the decision maker’s context. Although
alternative, well established methods can be used for addressing the decision
maker’s problem, faced with the increasing evidence of the theoretical and
empirical failures of the cost-effectiveness approach, some proponents of the
approach now propose changing the research question to suit the approach as
opposed to adopting a more appropriate method for the prevailing and continuing
problem.
As long as decision makers are concerned with making the best use of available
healthcare resources, cost-effectiveness analysis and the ICER should not be
where we look for answers.
Decision makers continue to struggle with the
challenges of how to allocate whatever resources are
made available for healthcare among the rapidly
increasing number and range of possible ways of
using those resources. Cost-effectiveness analysis
(CEA) has been advocated in the health economics
methods literature and adopted in a growing number
of jurisdictions as an evidence base for decision
makers charged with deciding whether to fund a
particular intervention in the context of an objective
of maximising health gains from available resources.[1-3]
In this paper we critically appraise the information that is created by CEA, the incremental cost-effectiveness ratio (ICER). We first consider the calculation of the ICER and how it relates to the
decision maker’s problem. We show that the ICER
represents an attempt to provide comparative information on what are non-comparable (and therefore
unreal) options and hence is irrelevant to, and
evades the reality of, the decision maker’s problem.
1122
We then consider how information created in a way
that evades the decision maker’s reality emerged as
the recommended source of evidence for healthcare
decision making. We show that the ICER approach
is based on simplifying theoretical assumptions that
have no relevance to the decision maker’s context.
We briefly explain the appropriate approach for
addressing the decision maker’s problem based on
well established methods that have been presented
in the literature over many years. Finally, we note
that faced with increasing evidence of the theoretical
and empirical failures of the CEA approach in addressing the basic decision-making problem, some
proponents of CEA now propose changing the nature of the problem (i.e. reality) to suit the CEA
method as opposed to adopting a more appropriate
method for the prevailing and continuing real problem.
1. The Information Generated by
Cost-Effectiveness Analysis
The ICER is the analytical tool used in CEA to
summarise information on the comparison of both
costs and effects of two different interventions (usually a new intervention and the current intervention)
for addressing a particular healthcare problem or
patient group.
Where the new intervention is more effective but
not more costly, or where it is less costly but no less
effective than the current intervention, no further
consideration is required to establish the efficiency
of allocating resources to the new intervention.
Common sense is sufficient to determine that replacing the current intervention with the new intervention will increase benefits generated from available
resources or release resources for other uses while
maintaining benefits – either of which represents an
unambiguous increase in efficiency. However,
where incremental costs and effects are in the same
direction (including the more typical case of new
interventions that produce greater effects but only
with an increase in costs) the implications for the
efficient use of resources are less clear.
1
Birch & Gafni
The ICER is given by the ratio of the betweenintervention difference in costs, i.e. incremental
costs (measured in monetary units), and the between-intervention difference in effects, i.e. incremental effects (often measured in QALYs), and is
expressed as the cost per additional unit of effect, for
instance cost per QALY. It is argued that by comparing the ICER, or price per additional QALY, of
the intervention under consideration with an ICER
threshold associated with their own subjective assessment, or in relation to the ICER of other interventions, decision makers concerned with maximising health gain from available resources can determine whether the proposed intervention represents
‘value for money’.[1-3]
Unfortunately, the ICER is of no use whatsoever
in determining whether implementing the new intervention represents an efficient use of resources.
Although dividing incremental costs by incremental
effects produces a price per QALY, this does not
mean that any number of QALYs (large or small)
can be purchased at this price. On the contrary, this
means that for an additional investment given by the
incremental cost, additional health gains equal to the
incremental effects could be produced. Thus, the
ICER simply represents the average cost (or price)
of these additional health gains. For example, Ubel
et al.[4] cites a US study that estimated the ICER for
sildenafil (the Viagra 1 drug) to be $US11 000 per
QALY. But this does not mean that a decision maker
can increase total health gain by one QALY for an
additional $US11 000 to be found from existing
expenditures, either within or outside the healthcare
system. On the contrary, introducing sildenafil to a
formulary would require substantially more resources (again to be found from existing expenditures either within or outside the healthcare system)
in order to cover the cost of the drug to all those
individuals eligible to receive it. In order to understand this better it is helpful to consider applying the
same cost-effectiveness reasoning to an example of
grocery shopping.
The use of trade names is for product identification purposes only and does not imply endorsement.
 2006 Adis Data Information BV. All rights reserved.
Pharmacoeconomics 2006; 24 (11)
ICER: Information Created to Evade Reality
2. Illustrating the Problem: Cost
Effectiveness in Grocery Shopping
The problem facing the decision maker corresponds to that of the shopper in the grocery store
wondering which cornflakes packet to purchase
from among the different sizes and brands of cornflakes available. Many grocery stores provide additional information in terms of a comparative price
per standard unit (e.g. 100g of cornflakes). Imagine
where this would leave the shopper if that were the
only information provided on the different products.
The temptation might be to choose the best ‘value
for money’ or lowest ‘cost per 100g of cornflakes’.
However, this leaves the shopper ignorant of how
much this product costs and hence what other groceries (e.g. eggs, milk, flour, bread, etc.) have to be
forgone to accommodate this purchase of cornflakes
into the grocery budget. Imagine the shopper’s surprise when, on arrival at the checkout, he discovers
that the cornflakes product, chosen because of its
particularly low price per 100g, involves a quantity
of 1000kg at a cost that represents more than the
entire household budget for groceries for a month.
Basing decisions only on the average price per
unit has a tendency to increase total expenditures on
that product. The shopper cannot make sensible
decisions without information on the total cost and
total content of each different cornflakes product (as
well as knowledge of the available budget for groceries and the total costs and contents of all other
grocery products on the shopping list). Dividing
costs by content to generate a comparative ‘value for
money’ indicator across products involves the loss
of important information to the shopper. Where the
price per package of the different cornflakes products differs, the different products are not directly
interchangeable and hence cannot be assessed in
terms of the average cost per 100g.
An alternative way of looking at this is to consider the inverse of the ICER, in other words, the
incremental effectiveness cost ratio or IECR. Instead of providing a cost per 100g of cornflakes, it
now provides a measure of quantity of cornflakes
per dollar expenditure. So products with the lowest
cost per 100g of cornflakes would produce the great 2006 Adis Data Information BV. All rights reserved.
1123
est quantity of cornflakes per dollar. In other words,
this represents a measure of the average rate of
return on investment (in this case, investment in
cornflakes purchases). As with the cost per 100g of
cornflakes measure, this provides a comparative
measure across different cornflakes products. But
since the different products are non-interchangeable,
i.e. they involve different levels of investment, the
comparative measure is no use in determining efficient investment decisions.
In a further example, how many of us have been
attracted by adverts in in-flight magazines for high
interest rates offered by offshore banks before the
small print reveals a required minimum investment
of say $US100 000? If the balance in my current onshore account is only $US20 000, a comparison of
interest rates (or average rates of return) between the
two investment options is meaningless – the plans
are not interchangeable. I would need to forgo far
more than my current on-shore account (e.g. also
sell my $US30 000 car and take out an extra
$US50 000 mortgage on my house) to be able to
invest at the high average rate of return in the
offshore account.
It should be clear by now that the comparison of
ICERs (or IECRs) as a basis for making investment
decisions aimed at maximising health (or cornflakes) gains is only valid where the interventions
being compared have identical total costs (i.e. the
alternatives are truly interchangeable). But if the
two alternatives have the same cost there is no need
to calculate a ratio – common sense is again sufficient to determine that the intervention with the
greater effectiveness represents the more efficient
use of resources. However, it is not simply that
calculating the ratio is unnecessary or unhelpful; in
some situations it leads to interventions being adopted that prevent the maximisation of health gain from
available resources occurring.[5]
3. Determining the Incremental
Cost-Effectiveness Ratio (ICER)
Threshold: The Silence of the λ
Notwithstanding the problem of ‘comparing the
non-comparable’ discussed in section 2, researchers
Pharmacoeconomics 2006; 24 (11)
1124
and decision makers have persisted in using the
ICER as an important tool in determining the efficiency of allocating resources to individual interventions by reference to a threshold ICER (commonly referred to as the λ value).[1-3,6,7] But if decision makers are to achieve their objectives of
maximising health benefits from available resources, consideration needs to be given to how any
chosen λ value (or range of values) relates to this
stated objective. Claxton et al.[8] note that the maximisation of health benefits from available resources
requires information on the shadow price of the
decision maker’s budget constraint, which represents the marginal opportunity cost of available resources. Remarkably little attention has been given
in the literature to how particular ICER thresholds
are selected and, more importantly, how they relate
to the opportunity cost considerations of the constrained maximisation problem facing the decision
maker.[9]
Consider the example of the UK National Institute for Health and Clinical Excellence (NICE),[2]
the most widely discussed example of the institutionalisation of economics as a basis for healthcare
decision making. The stated objective of the NICE
guidelines is the maximisation of health gain from
whatever resources are committed to the NHS in
England and Wales. NICE specifies two different λ
values, a lower threshold of £20 000 per QALY and
a higher threshold of £30 000 per QALY. Interventions with ICERs below the lower threshold are
judged primarily on their ICER value (i.e. other
considerations are irrelevant, as the ICER value is
sufficiently low to warrant that the intervention be
adopted). Interventions with ICERs between the two
thresholds are judged not exclusively on their ICER
value but also on other factors (uncertainty, the
innovative nature of the intervention, features of the
condition and populations being treated, and the
wider societal costs and benefits). In other words,
although the ICER is above the lower threshold,
these other factors might be sufficient to warrant
adoption of the intervention. Finally, interventions
with ICERs above the higher threshold require ‘increasingly strong’ justification beyond the ICER
 2006 Adis Data Information BV. All rights reserved.
Birch & Gafni
value. In other words, these interventions are unlikely to warrant adoption, because the high ICER value
would, except in very special circumstances, trump
all other factors.
No attempt is made to rationalise these chosen
thresholds with the maximisation of health benefits
from NHS resources. Instead, the approach of NICE
is to consider ICER values of interventions that are
currently funded by the NHS as ‘a legitimate reference’ for decision makers. Hence, if the NHS currently funds something with a higher ICER than the
new intervention under consideration, then funding
the new intervention would represent an improvement in efficiency, irrespective of how that new
intervention is to be funded.
The use by NICE of two different ICER thresholds further questions the ability of the approach to
satisfy the objectives of the exercise. It probably
represents more a case of a ‘sophisticated decision
rule’[10] in which the ICER values are only one input
into the decision-making process. Other inputs (uncertainty, innovative nature of the intervention, features of the condition and populations being treated,
and the wider societal costs and benefits) are important to the decision-making process. However, no
information is provided on how these considerations
are to be incorporated into the sophisticated decision
rule to be used to determine which interventions
increase total health benefits from available NHS
resources and which interventions do not.
In the absence of any clear theoretical and/or
empirical basis for the NICE thresholds, it is not
surprising that alternative algorithms have been suggested. For example Rawlins and Culyer[11] acknowledge the arbitrary nature of the thresholds and
argue that decisions are based on a case-by-case
basis. “As the incremental cost effectiveness ratio
increases, the likelihood of rejection on grounds of
cost effectiveness rises.” However, no empirical or
theoretical basis is provided for these authors’ assigned ranges of £5000–15 000 and £25 000–35 000
for the lower and upper thresholds. Williams[12]
acknowledges that there is no practical way to determine the threshold and suggests instead “a bit of
common sense.” He argues that in the UK there are
Pharmacoeconomics 2006; 24 (11)
ICER: Information Created to Evade Reality
£18 000 of real resources per citizen to provide for
all needs (e.g. food, shelter, transport) and suggests
adopting this figure as the ICER threshold because
“we could do it at the margin for few people without
imposing great hardships on the bulk of the population but we could not do it for many.” No attempt is
made to rationalise this threshold with the NICE
objective of maximising total health gains from
NHS resources, and no attention is given to dealing
with ‘cost-effective’ interventions that relate to
treating conditions with a prevalence that extends
beyond ‘a few people’.
Similar approaches in other jurisdictions also
fail to provide any justification for the chosen ICER
thresholds as means of maximising health gain from
available resources.[9,13] For example, in 1992
Laupacis et al.[14] recommended thresholds of
$Can20 000 per QALY and $Can100 000 per QALY for new interventions in Canada. When questioned on the basis of these thresholds, Laupacis
admitted “we made them up.”[15]
So, in the absence of applied research literature
offering any clear explanation of how the chosen
ICER thresholds help achieve the goal of benefit
maximisation from available resources, what does
the theoretical literature have to offer on what determines the ICER threshold for deciding whether a
new intervention represents an efficient use of available resources?
4. The Theoretical Foundation of the
Threshold ICER
The origin of the ICER as a tool for determining
whether an intervention represents an efficient use
of available resources is found in Weinstein and
Zeckhauser[16] (the WZ model). Using assumptions
of perfect divisibility and constant returns to scale
for all interventions, they show that total health
benefits are maximised where:
• all interventions are ranked from the lowest to the
highest ICER and selected in descending order
until available resources are exhausted (league
table approach); or
• specification of the ‘critical ratio’, λ, given by the
opportunity cost of the resources at the margin,
 2006 Adis Data Information BV. All rights reserved.
1125
and implementation of all interventions with an
ICER less than or equal to λ (threshold ICER
approach).
Note that for the league table approach, information on the incremental costs and effects of all
current and potential interventions is required. Similarly, for the threshold ICER approach, in order to
determine the critical ratio λ, the marginal intervention must be identified, which also requires information on the incremental costs and effects of all
current and potential interventions. In the real world,
information for healthcare decision makers is incomplete. Consequently, complete rankings of interventions cannot be produced and therefore λ cannot
be determined.[4,6,17,18] Restricted league tables
based on rankings of those interventions for which
information is available can be used, but such tables
cannot determine the opportunity cost of marginal
resources and hence cannot determine if a particular
intervention contributes to the maximisation of total
health benefits from available resources.
Suppose complete information was to be available to decision makers. Several important implications emerge from this theoretical model. First, the
marginal opportunity cost of resources, λ, depends
crucially on, inter alia, the quantity of available
resources (i.e. the size of the budget).[19] Hence,
communities with the same healthcare needs but
different budgets will have different values of λ
against which to judge the efficiency of interventions. Secondly, any change in healthcare resources
generates a change in λ, the ICER threshold below
which an intervention represents an efficient use of
resources.[20] Thirdly, because the costs and effects
of all interventions are subject to uncertainty, λ,
given by the marginal intervention funded, is also
uncertain.[21] Finally, as new interventions are funded and others replaced, the marginal funded intervention changes. Hence the value and distribution of
λ, given by the ICER of the marginal funded intervention, are dynamic.[21] As a result, no generalised
statement can be made about the efficiency of an
intervention in isolation from the precise setting in
which it is being considered. A simple, single λ
value, or range of values, cannot be specified for
Pharmacoeconomics 2006; 24 (11)
1126
determining the efficiency of interventions across
different settings and over time.[20] Even under the
assumptions of the WZ model, the λ value consistent with the maximisation of total health benefits
from available resources will be context specific and
continuously changing.[20]
Notwithstanding the non-generalisable nature of
λ, the WZ model from which it emerges is based on
assumptions (perfect divisibility and constant returns to scale of all interventions) that are highly
theoretical and not valid in most, if not all, decisionmaking settings.[19] Under perfect divisibility, all
interventions can be purchased in incremental units
(e.g. 1 minute of a dialysis machine) while constant
returns to scale require that the marginal health
benefits of an intervention are constant, no matter
how many increments are purchased (e.g. if one
dialysis machine is associated with producing an
additional 50 QALYs per year, then one-tenth of a
dialysis machine produces five additional QALYs
per year and 1000 machines produce 50 000 additional QALYs in the population). The empirical
studies on which ICER calculations are based normally compare a new intervention with a current
intervention based on a particular size of intervention (usually chosen for statistical analysis reasons).
Rarely, if ever, are the consequences of varying the
size of the intervention/patient group for the incremental costs and effects considered. Instead, the
ICER is used to create an average (and an implicitly
assumed constant) additional cost per unit QALY.
When the theoretical assumptions of the WZ
model do not hold, then the ICER threshold approach does not guarantee that available resources
are being used in ways that maximise health benefits[19,21,22] and the conclusions of the WZ model are
no longer valid (see Gafni and Birch[23] and Birch
and Gafni[13] for numerical examples). As Doubilet
et al.[24] note, “there is no theoretical justification for
asserting that the strategy with the lowest costeffectiveness ratio is the most desirable one”.
5. Pursuing Efficiency: Back to the Future
“To improve efficiency, decision-makers need information on what economists call opportunity cost
 2006 Adis Data Information BV. All rights reserved.
Birch & Gafni
– the benefits forgone when scarce resources are
used one way rather than another… In absence of
any information about opportunity cost, however,
they cannot attempt to achieve the efficient use of
resources.”[25]
As we have shown, the use of an ICER threshold
to determine the efficiency of a new intervention is
at best based on underlying assumptions that are
inconsistent with the decision maker’s real problem.
Notwithstanding the serious limitations introduced
by these assumptions, lack of information prohibits
determining the theoretically correct λ value or even
a range within which λ lies. As a result, in practice λ
is determined arbitrarily and without any explanation of how application of the λ value leads to the
efficient use of available resources.
Weinstein[26] describes the proper use and interpretation of ICERs where interventions compete for
limited resources. He acknowledges the underlying
assumptions of the WZ model but suggests that
these assumptions can be relaxed without abandoning the simple basic (cost-effectiveness) paradigm. However, unlike the WZ model, no mathematical proof is given for how the use of ICERs
achieves the efficient use of resources once the
theoretical assumptions are relaxed. He notes that if
there are ethical or technical constraints on providing an intervention to only a fraction of the population in need, then a generalised optimisation framework would be necessary to address the decision
maker’s problem. He directs the reader to our own
work[27] for solutions and explains that computer
programs exist to solve these optimisation problems.
It is worth noting that the recognition of the need
for a generalised optimisation framework to address
the decision maker’s real problem is not new. Torrance et al.[28] in 1972 described the use of a standard
mathematical programming model to solve the optimisation problem of maximisation of total health
benefits from available resources. Similarly in 1976,
Chen and Bush[29] provided a framework for maximising healthcare output subject to political and
administrative constraints using mathematical programming techniques. Drummond[30] noted that the
use of mathematical programming techniques was
Pharmacoeconomics 2006; 24 (11)
ICER: Information Created to Evade Reality
“the only approach” that could rank interventions
under a resource constraint while Drummond et
al.[31] and Mason et al.[32] acknowledged that the
information provided in ICER league tables is not
sufficient to determine the efficiency of individual
interventions and that such problems require the use
of mathematical programming techniques. Finally,
Birch and Donaldson[33] and Birch and Gafni[22]
presented integer programming solutions for dealing
with the problems of indivisibilities and non-constant returns to scale in the context of a constrained
maximisation problem, while Stinnett and Paltiel[34]
provided a flexible extension to integer programming.
These mathematical programming techniques require information on the costs and effects of all
current and potential new interventions, together
with the resources available for investment. Although these data requirements may be difficult to
satisfy, they reflect the complex nature of the decision maker’s problem. The use, instead, of a restricted league table of ICER values or an arbitrarily
determined ICER threshold might offer an intuitive
shortcut to resolving the decision maker’s problem,
but as Williams noted, “reality is horrendously complicated…the more complex the reality is, the more
dangerous it is to rely on intuitive short-cuts rather
than careful analysis.”[12]
Faced with inadequate information to solve the
problem of maximisation of health benefits from
available resources, we previously provided a less
data-intensive approach aimed at a modified objective of an unambiguous improvement in efficiency
(i.e. achieving an increase in total health benefits
from available resources).[22,35] This requires that the
increase in health generated by the new intervention
exceeds the increases in health generated by that
combination of interventions to be given up in order
to fund the new intervention. This approach does not
rely on any arbitrary threshold values to ascertain
the efficiency of the intervention, and is not dependent on unrealistic assumptions about perfect divisibility and constant returns to scale. Instead, the
source of the additional resource requirements of the
new intervention is identified directly and the impli 2006 Adis Data Information BV. All rights reserved.
1127
cations of cancelling these interventions (i.e. the
opportunity cost of the new intervention) to generate
these resources form an explicit part of the analysis.
The approach has been extended to deal with the
uncertain nature of costs and outcomes.[21]
6. Discussion
“The basic economic problem is how to allocate
scarce resources so as to best satisfy human wants.
This may be contrasted with the romantic point of
view that fails to recognise scarcity of resources and
is misled into confusing the real world with the
Garden of Eden.”[36]
Guidelines for economic evaluation based on
CEA and the use of ICERs have already become an
important part of healthcare decision making in several jurisdictions,[1-3] while calls continue to be made
for its adoption in jurisdictions that to date have
remained ‘guideline free’.[37] In both cases, the use
of CEA is seen by researchers and policy makers as
the long-awaited solution or at least a major contribution to addressing the challenge of how to maximize health gains from available healthcare resources. For example, in the UK, NICE uses CEA
for deciding which interventions to recommend for
funding under the NHS.[2] This has been described
by Williams as “the closest anyone has come to
fulfilling the economist’s dream,”[12] while Smith
has suggested that NICE may prove to be “one of
Britain’s greatest cultural exports, along with
Shakespeare, Newtonian physics, the Beatles, Harry
Potter and the Teletubbies.”[38] But does this confuse
the real world with the Garden of Eden?
We have shown that CEA and the use of ICER
thresholds cannot be used to determine whether a
new intervention represents an efficient use of available healthcare resources. Elsewhere we have
shown that applications of the ICER approach to
healthcare decision making have been associated
with increased expenditures on healthcare interventions and concerns about the sustainability of public
funding for those interventions without any evidence of increases in total health gains.[9,17,23] In
other words, instead of the analysis helping decision
makers produce ‘the biggest bang for the healthcare
Pharmacoeconomics 2006; 24 (11)
1128
bucks’ it has been associated with ‘bigger healthcare
bucks for the bang’.[13]
Some have suggested that one way of dealing
with the statistical problems arising from the calculation of the ICER (i.e. the assumptions involved in
dividing incremental costs by incremental effects) is
to use a net-benefits approach by converting the
incremental costs into a negative benefits expression
and determining if the intervention represents a net
increase in benefits.[39] Unfortunately, this does not
escape the problem of the ICER threshold since the
conversion of incremental costs to negative benefits
is performed using the ICER threshold and hence all
of the problems associated with the ICER threshold
discussed in the previous sections are reintroduced.
Other researchers have proposed the use of costeffectiveness acceptability curves[40,41] in which information is provided to decision makers on the
probability that an intervention has an ICER that is
less than or equal to a given level or range of λ
values. Again, λ (or the range of λ) is required as an
input, which undermines the approach as a means of
determining whether an intervention represents an
efficient use of resources.
Others have argued that despite the theoretical as
well as empirical limitations with the ICER approach, because decision makers are unable to establish what the opportunity costs of a new intervention will be, the question about the efficiency of a
new intervention can only be answered by establishing (implicitly or explicitly) “our generic willingness to pay (WTP) for an additional unit of effect.”[42] For example, Ubel et al.[4] argue that physicians’ scepticism about using CEA may be because
they feel that ICER thresholds used in decision
making are too low. They acknowledge that currently used thresholds are arbitrary but suggest the use
of $US200 000 or more per QALY as a new threshold, arguing that the actual threshold should be set
“by consensus process, such as that used by the US
Public Health Service when it established standards
for the field of cost-effectiveness in the mid 1990s.”
However, the cost-effectiveness standards referred
to[6] acknowledged that no absolute standard exists
for deciding whether a specific ICER value repre 2006 Adis Data Information BV. All rights reserved.
Birch & Gafni
sents a cost-effective use of resources or not. Moreover, despite calling for a consensus-based ICER
threshold, the authors[4] acknowledge that the use of
such a threshold would lead to continual increases in
per capita healthcare costs, an outcome that is inconsistent with the efficient use of available resources.
Suggestions for the use of a ‘generic’ ICER threshold that in some way measures society’s willingness
to pay for a QALY fail to consider how the use of
such a measure is consistent with the maximisation
of health benefits from available resources. As Claxton et al.[8] note in relation to the NICE threshold, “it
is not the societal valuation that is relevant…but the
shadow price of the…budget constraint.”
Despite these ‘fatal’ limitations of the ICER
threshold approach as a solution to the constrained
maximisation problem, considerable research attention has been given to dealing with issues of uncertainty, sample size requirements[43] and the determination of the value of additional information,[44,45]
all based on the ICER and the threshold ICER
approach. Hence, while they each may represent
important contributions to a theoretical model, they
offer no help to decision makers faced with choices
between different ways of allocating available resources.
Where the incremental cost of a new intervention
is positive, the question facing decision makers is
not simply the choice between the new intervention
and the old intervention it is intended to replace. The
new intervention involves an opportunity cost that
exceeds the health benefits of the old intervention.
Because the ICER approach is restricted to comparative information on the new and old interventions
(i.e. two interventions involving different resource
requirements), it implies that there is an indeterminate stream of additional resources available for
investment in new interventions at a constant marginal opportunity cost.[19,21,46] In reality, marginal opportunity costs of resources are likely to be increasing (the bigger the additional resource requirements
the further you have to go in cutting current
programmes, from least productive to most productive, in order to generate the required resources).
Moreover, differences in prevailing circumstances
Pharmacoeconomics 2006; 24 (11)
ICER: Information Created to Evade Reality
(e.g. overall health profiles of populations, healthcare budgets etc.) both between settings and over
time mean that the efficiency of a particular intervention will be context specific.[20]
Some authors have defended the use of CEA by
distancing themselves from the constrained maximisation problem. For example, Rawlins and Culyer[11] see the use of CEA by NICE as simply a way to
judge ‘value for money’; in other words, whether
something ought to be purchased from within the
resources made available while the total cost of all
interventions deemed suitable for purchase is a separate matter of ‘affordability’. But matters of efficiency cannot be separated from matters of affordability.[9,23] Because money represents only
command over resources, value for money is determined in relation to what it can purchase. Hence,
whether a particular intervention represents ‘value
for money’ is determined by what is forgone in order
to ‘afford it’. As Williams notes, if affordability
could be separated from efficiency there would be
no need for a threshold.[12]
Baltussen et al.[47] acknowledge that the comparison of interventions based on ICER values cannot
provide solutions to context-specific decisions and
that the solution of such problems requires “more
complex optimal resource allocation planning models.” As a result, they argue that “CEA can be most
useful with more modest goals.” However, they do
not explain what these more modest goals are that
CEA can address or whether these goals are compatible with the problem of constrained maximisation
facing the decision-making processes that CEA is
intended to inform.
Similarly, Sculpher et al.[48] suggest that economic evaluation should focus more on tackling the
needs of social decision making than on the underlying principles of welfare economics, implying that
the two are incompatible. They argue “for this research to be relevant to policy, it needs to be seen
less as economic evaluation and more as evaluation.” But if social decision makers are faced with
the problem of constrained maximisation and the
economics discipline provides the only approach for
solving such problems, how is a departure from
 2006 Adis Data Information BV. All rights reserved.
1129
economic principles and the concept of opportunity
cost ‘relevant to policy’?
7. Conclusion
In summary, the ICER represents Information
Created to Evade the decision maker’s Reality. No
justification exists for the ICER thresholds used (or
any other ICER threshold) being compatible with
the decision maker’s goal of maximising health benefits from available resources. Evidence from Ontario (Canada), England and Australia shows that the
adoption of the ICER approach has been associated
with substantial unplanned increases in healthcare
expenditures without any evidence of any increase
in total health benefit.[9,13] For example, the estimated additional resources absorbed by NICE-recommended technologies in the first 2.5 years after the
implementation of NICE exceeded £575 million.[49]
What would have happened to expenditures in these
jurisdictions in the absence of ICER-based approaches to decision making is unknown.
However, estimating the counterfactual fails to
address the stated objectives of the ICER-based
approaches – maximisation of health gains from
whatever level of resources is committed to healthcare interventions. Maynard et al.[50] have suggested
imposing a notional budget for the ICER-based
adoption of new interventions to encourage organisations to examine the effect of their decisions
on overall health gains in the system. But as we have
shown previously,[13] maximisation of benefits from
a given budget cannot be determined based on the
ICERs of possible interventions. Questions concerned with the maximisation of efficiency require
information on the total costs and total effects of all
interventions. More modest objectives of improving
efficiency require corresponding information on the
new interventions being considered for adoption and
the possible interventions to be forgone to provide
funding for the new interventions.[21] Hence, where
the questions that decision makers face are concerned with making the best use of available resources, we should not look to CEA, and the ICER
threshold approach on which it is based, for answers.
Pharmacoeconomics 2006; 24 (11)
1130
Birch & Gafni
Acknowledgements
No sources of funding were used to assist in the preparation of this article. The authors have no potential conflicts of
interest that are directly relevant to the contents of this article.
References
1. Henry D. Economic analysis as an aid to subsidisation decisions: the development of Australian guidelines for pharmaceuticals. Pharmacoeconomics 1992; 1: 54-67
2. National Institute for Clinical Excellence. Guide to the methods
of technology appraisal. London: National Institute for
Clinical Excellence, 2004
3. Canadian Agency for Drugs and Technologies in Health. Guidelines for the economic evaluation of health technologies. 3rd
ed. Ottawa (ON): Canadian Agency for Drugs and Technologies in Health, 2006
4. Ubel P, Hirth R, Chernew M, et al. What is the price of life and
why doesn’t it increase at the rate of inflation? Arch Intern
Med 2003; 163: 1637-41
5. Birch S, Gafni A. The ‘NICE’ approach to technology assessment: an economics perspective. Health Care Manag Sci 2004;
7: 35-41
6. Gold M, Siegel J, Russell L, et al. Cost-effectiveness in health
and medicine. New York: Oxford University Press, 1996
7. Drummond M, Sculpher M, Torrance G, et al. Methods for the
economic evaluation of health care programmes. Oxford: Oxford University Press, 2005
8. Claxton K, Sculpher M, Culyer A, et al. Discounting and costeffectiveness in NICE: stepping back to sort out a confusion.
Health Econ 2006; 15: 1-4
9. Gafni A, Birch S. Incremental cost effectiveness ratios (ICERs):
the silence of the lambda. Soc Sci Med 2006; 62: 2091-100
10. Culyer A. Ought NICE to have a cost-effectiveness threshold?
In: Towse A, Pritchard C, Devlin N, editors. Cost-effectiveness thresholds: economic and ethical issues. London: Kings
Fund and Office of Health Economics, 2002: 9-14
11. Rawlins M, Culyer A. National Institute for Clinical Excellence
and its value judgements. BMJ 2004; 329: 224-7
12. Williams A. What could be nicer than NICE? London: Office of
Health Economics, 2004
13. Birch S, Gafni A. The biggest bang for the buck or bigger bucks
for the bang: the fallacy of the cost-effectiveness threshold. J
Health Serv Res Policy 2006; 11: 46-51
14. Laupacis A, Feeny D, Detsky A, et al. How attractive does a
new technology have to be to warrant adoption and utilization?
Tentative guidelines for using clinical and economic evaluations. CMAJ 1992; 146: 473-81
15. Lorinc J. How much are drugs worth? A fledgling medical
science attempts an answer. University of Toronto Magazine
[online].
Available
from
URL:
http://www.magazine.utoronto.ca/06Spring/drugs.asp [Accessed 2006 Apr
25]
16. Weinstein M, Zeckhauser R. Critical ratios and efficient allocation. J Public Econ 1973; 2: 147-57
17. Gafni A, Birch S. Inclusion of drugs in provincial benefit
programs: should ‘reasonable decisions’ lead to uncontrolled
growth in expenditures? CMAJ 2003; 168: 849-51
18. Devlin N. An introduction to the use of cost-effectiveness
thresholds in decision making: what are the issues? In: Towse
A, Pritchard C, Devlin N, editors. Cost-effectiveness thresh-
 2006 Adis Data Information BV. All rights reserved.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
olds: economic and ethical issues. London: Kings Fund and
Office of Health Economics, 2002: 16-24
Birch S, Gafni A. Changing the problem to fit the solution:
Johannesson and Weinstein’s (mis)application of economics to
real world problems. J Health Econ 1993; 12: 469-76
Birch S, Gafni A. Economics and the evaluation of health care
programmes: generalisability of methods and implications for
generalisability of results. Health Policy 2003; 64: 207-19
Sendi P, Gafni A, Birch S. Opportunity costs and uncertainty in
the economic evaluation of health care interventions. Health
Econ 2002; 11: 23-32
Birch S, Gafni A. Cost-effectiveness/utility analyses: do current
decision rules lead us to where we want to be? J Health Econ
1992; 11: 279-96
Gafni A, Birch S. NICE methodological guidelines and decision
making in the National Health Service in England and Wales.
Pharmacoeconomics 2003; 21: 149-57
Doubilet P, Weinstein M, McNeil B. Use and misuse of the term
‘cost effective’ in medicine. N Engl J Med 1986; 314: 253-6
Cookson R, McDaid D, Maynard A. Wrong SIGN, NICE mess:
is national guidance distorting allocation of resources? BMJ
2001; 323: 743-5
Weinstein M. Decision rules for incremental cost-effectiveness
analysis. In: Jones A, editor. The Elgar companion to health
economics. Northampton: Edward Elgar, 2006: 469-78
Birch S, Gafni A. Decision rules in economic evaluation. In:
Jones A, editor. The Elgar companion to health economics.
Northampton: Edward Elgar, 2006: 492-502
Torrance G, Thomas W, Sackett D. A utility maximization
model for evaluation of health care programmes. Health Serv
Res 1972; 7: 118-33
Chen M, Bush J. Maximizing health system output with political
and administrative constraints using mathematical programming. Inquiry 1976; 13: 215-27
Drummond M. Principles of economic appraisal in health care.
Oxford: Oxford University Press, 1980
Drummond M, Torrance G, Mason J. Cost-effectiveness league
tables: more harm than good? Soc Sci Med 1993; 37: 33-40
Mason J, Drummond M, Torrance G. Some guidelines on the
use of cost effectiveness league tables. BMJ 1993; 306: 570-2
Birch S, Donaldson C. Applications of cost-benefit analysis to
health care: departures from welfare economic theory. J Health
Econ 1987; 6: 61-72
Stinnett A, Paltiel D. Mathematical programming for the efficient allocation of health care resources. J Health Econ 1996;
15: 641-53
Gafni A, Birch S. Guidelines for the adoption of new technology: a potential prescription for uncontrolled growth in expenditures and how to avoid it. CMAJ 1993; 148: 913-24
Fuchs V. Who shall live? Health, economics and social choice.
New York: Basic Books, 1974
Neumann P, Rosen A, Weinstein M. Medicare and cost-effectiveness analysis. N Engl J Med 2005; 353: 1516-22
Smith R. The triumph of NICE. BMJ 2004 [online]. Available
from URL: http://bmj.bmjjournals.com/cgi/content/full/329/
7459/0-g [Accessed 2006 May 5]
Stinnett A, Mullahy J. Net health benefit: a new framework for
the analysis of uncertainty in cost effectiveness analysis. Med
Decis Making 1998; 18: S68-80
Van Hout B, Al M, Gordon G, et al. Costs, effects and C/Eratios alongside a clinical trial. Health Econ 1994; 3: 309-19
Briggs A. A Bayesian approach to stochastic cost-effectiveness
analysis. Health Econ 1999; 8: 257-61
Pharmacoeconomics 2006; 24 (11)
ICER: Information Created to Evade Reality
42. Dowie J. Why cost effectiveness should trump (clinical) effectiveness: the ethical economics of the South West quadrant.
Health Econ 2004; 13: 453-60
43. Briggs A, O’Brien B, Blackhause G. Thinking outside the box:
recent advances in the analysis and presentation of uncertainty
in cost-effectiveness studies. Ann Rev Public Health 2002; 23:
377-401
44. Claxton K, Posnett J. An economic approach to clinical trial
design and research priority setting. Health Econ 1996; 5: 51324
45. Claxton K. The irrelevance of inference: a decision making
approach to the stochastic evaluation of health care technologies. J Health Econ 1999; 18: 342-64
46. Sendi P, Briggs A. Affordability and cost effectiveness: decision-making on the cost-effectiveness plane. Health Econ
2001; 10: 675-80
47. Baltussen R, Adam T, Tan-Torres Edejer T, et al. What is
generalized cost-effectiveness analysis? In: Tan-Torres Edejer
T, Baltussen R, Adam T, et al., editors. Making choices in
 2006 Adis Data Information BV. All rights reserved.
1131
health: WHO guide to cost-effectiveness analysis. Geneva:
World Health Organization, 2003: 3-15
48. Sculpher M, Claxton K, Akehurst R. Its just evaluation for
decision making: recent developments in, and challenges for,
cost-effectiveness research. In: Smith P, Ginelly L, Sculpher
M, editors. Health policy and economics: opportunities and
challenges. Milton Keynes: Open University Press, 2004: 8-41
49. Mayor S. News extra: NICE estimates that its recommendations
have cost the NHS £575m. BMJ 2002; 325: 924
50. Maynard A, Bloor K, Freemantle N. Challenges for the National
Institute for Clinical Excellence. BMJ 2004; 329: 227-9
Correspondence and offprints: Dr Stephen Birch, Centre for
Health Economics and Policy Analysis, McMaster University, 1200 Main Street West, Hamilton, Ontario, L8N 3Z5,
Canada.
E-mail: [email protected]
Pharmacoeconomics 2006; 24 (11)