Commentary
From Principle To Public Policy: Using Cost-Effectiveness Analysis
by Peter J. Neumann and Magnus Johannesson
Medical technology assessment, which traditionally has involved evaluating whether a technology was safe and effective, has been expanded in
recent years to include consideration of cost-effectiveness. While most
analysts and policymakers would agree that cost-effectiveness is, in principle, a worthy goal, actually using such a standard in coverage decisions is
much more difficult. This Commentary focuses on the challenges involved
for public payers and what we can realistically expect a cost-effectiveness
criterion to accomplish. First, we briefly discuss the cost-effectiveness principle. Second, we address some of the key issues in assessing medical
technologies. Third, we review how different public payers in the United
States and overseas are addressing these issues. Finally, we discuss both the
opportunities and limitations of cost-effectiveness analysis.
The Cost-Effectiveness Principle
Although formal methods for using cost-effectiveness analysis have been
promoted to the medical community since at least the 1970s, costs have
1
played a secondary role in technology assessment until recently. In the past
few years more explicit considerations of costs have come to the fore. There
has been an ongoing expansion of technology assessment’s focus from
questions of “Is it safe!” and “Does it work?” to those involving the previously inviolate notion of “Is it worth it?” Medical journals now routinely
publish cost-effectiveness analyses. Moreover, policymakers have begun
trying to make such analyses an explicit part of the coverage process for new
2
medical technology.
Cost-effectiveness analyses begin with the presumption that the objec3
tive of medical technology is to improve health. These analyses are inPeter Neumann, senior research director at Project HOPE’s Center for Health Affairs, is on leave
working as a health economist on health reform at the Office of Management and Budget. Magnus
Johannesson is an assistant professor of health economics at the Stockholm (Sweden) School of
Economics ad an adjunct assistant professor in the Department of Health Policy and Management,
Harvard School of Public Health.
COMMENTARY 207
tended to show the relationship between resources used (costs) and the
health benefits achieved (effects) for a given technology. The objective of
such analyses is to illustrate how we can maximize the health benefits of a
4
given amount of resources While this goal may sound rather innocuous,
operationalizing a cost-effectiveness standard is difficult. A host of theoretical and political issues await payers that attempt to use cost-effectiveness
analyses explicitly in making coverage decisions.
Issues In Cost-Effectiveness Analysis
Theoretical and methodological issues. A critical issue in costeffectiveness analysis involves the existing alternative to which a technology is compared. Presumably, a new technology will be compared with one
that it likely will replace, but this is not always known at the time of the
5
analysis. Take hypertension treatment as one example: A large number of
different drugs are used, and it is not clear what the relevant comparison
6
would be for a new drug.
Even when the competing alternative is known, judging whether a
technology is cost-effective can be difficult. A technology may be costeffective in some patient groups but not in others. Researchers have noted,
for example, that the cost-effectiveness of hypertension treatment varies
widely in different patient groups, depending on age, sex, initial blood
7
pressure, and the presence of other risk factors. This problem may be
possible to handle by good study design and the right indications for use.
Another problem is that technologies constantly change and improve,
making it difficult to rely on one-time assessments.
Another important issue involves how to define costs for purposes of
analysis. While it is relatively straightforward to estimate the direct costs of
using a technology (for example, the cost of equipment, supplies, physician
time, and so forth) and the direct costs of competing alternatives, technologies usually also involve indirect costs and benefits (such as their impact on
a person’s ability to work) as well as longer-term induced costs or savings,
8
all of which are more difficult to estimate. Moreover, existing cost9
effectiveness analyses differ widely in how they define costs.
An issue in any analysis is that costs and health benefits rarely occur at
the same time. A technology may involve high initial costs, for example,
but may yield health benefits (in terms of life years saved) far into the
future. Typically, analyses are conducted in present-value terms, weighting
10
future costs and benefits by a discount factor to make them comparable.
But there are no standards regarding the length of patient follow-up to be
11
considered. Moreover, analysts do not always agree on the precise discount rate to use or whether future health benefits such as life years should
208 HEALTH AFFAIRS | Summer 1994
12
be discounted, although these decisions can influence results. For example, studies of the cost-effectiveness of osteoporosis screening have illus13
trated that the discount rate used can greatly affect results.
Incorporating patient preferences into an analysis is one of the most
14
challenging issues in cost-effectiveness analysis. Analysts have long argued that such preferences should be considered because a year of life with,
say, a chronic disability, is valued by patients differently than a completely
healthy year is valued. In recent years, researchers have substantially improved our ability to measure outcomes such as functional status, degree of
15
disability, and emotional health. Many studies have attempted to consider
quality-of-life or utility measures, but their use remains controversial. Critics argue, for example, that persons often possess ambivalent or imprecise
preferences or that their attitudes and tastes change over time or depend on
16
how a question is framed.
Administrative and political issues. Beyond the methodological challenges, there is the daunting task of selecting technologies to evaluate in
the first place. This process generally involves three elements: health importance, economic importance, and an expectation that an assessment
17
will make a difference. In 1992 Congress stipulated that the Agency for
Health Care Policy and Research (AHCPR) would consider such factors in
18
setting priorities for their its own health care technology assessments.
Actually quantifying the factors is more difficult, however, although some
progress has been made. Charles Phelps and Stephen Parente, for example,
have constructed an “index of expected gain” from technology assessment,
which combines measures of resource use, variation in use rates across
regions, and the rate at which the incremental value of a medical interven19
tion changes as its rate of use changes. The index predicts that technology
assessment should have a greater expected benefit when (1) large numbers
of people are affected (health importance); (2) the per unit cost of the
intervention is large (economic importance); and (3) the level of uncertainty is high and the marginal value of the technology falls rapidly as its
20
use expands (assessment is likely to make a difference).
Another issue concerns the quality of the data. Data often are unavailable or incomplete. The source of the data is itself an important issue.
Potentially, manufacturers could submit cost data to the government
agency conducting the cost-effectiveness analysis, as pharmaceutical companies now submit clinical data to the Food and Drug Administration
(FDA). This possibility underscores the need for the government to standardize reporting requirements and to ensure adequate verification and
evaluation of submissions. Another thorny problem with some products
and services involves what price to use in analyses and whether manufacturers and providers should be bound by the analysis price in marketing and
COMMENTARY 209
21
selling their products and services.
Beyond methodological issues, the most formidable hurdle may be a
political one. Adding an explicit cost-effectiveness criterion invites heightened public scrutiny. An example can be seen in the experience of the
Health Care Financing Administration (HCFA), which proposed in 1989
to use an explicit cost-effectiveness criterion in approving medical technologies for Medicare. The regulation was withdrawn for further study in
22
the election year of 1992 and has yet to be finalized.
Experience Of Public Payers
United States: Medicare. HCFA’s proposed regulation to add costeffectiveness to technology assessment illustrates how a public payer in the
United States may approach the issue of cost-effectiveness. In its proposed
regulation, HCFA first defended its legal authority to use cost-effectiveness
with a broad interpretation of a clause in the original Medicare statute that
prohibited payment for expenses incurred for services “which were not
reasonable and necessary for the diagnosis or treatment of illness or injury
23
or to improve the functioning of a malformed body member” HCFA
added that it was also on firm policy ground because it believed that
“considerations of cost are relevant in deciding whether to expand or
continue coverage of technologies, particularly in the context of the cur24
rent explosion of high-cost medical technologies.” HCFA qualified its
stance, however, by explaining that “cost-effectiveness is one of several
potential factors to be weighed in a given situation” and by proposing “to
use a flexible definition of cost-effectiveness that encompasses a wide range
25
of impacts.” HCFA further noted that it would not necessarily consider
cost-effectiveness in every coverage decision and that it was “aware that
cost-effectiveness analysis is a complex field that suffers from data limitations and the inability to quantify some costs.”
Lastly, HCFA proposed the analytic steps it would follow, noting that
these steps are “well-accepted among economists.” The steps included (1)
considering relevant alternative technologies to which current interventions will be compared; (2) identifying all relevant costs expected from the
intervention; and (3) considering nonquantifiable factors. HCFA stated
that it would study further how to collect and analyze primary and secon26
dary data for cost-effectiveness.
Australia: drug guidelines. Since January 1993 Australia has required
economic evaluations in applications for new drugs to be included in the
national drug reimbursement scheme. For prescription drugs to be reimbursed in Australia, they must be included on the Pharmaceutical Benefits
Scheme (PBS), operated by the federal government. If a drug is to be listed
210 HEALTH AFFAIRS | Summer 1994
on the PBS, an application must be made to the Pharmaceutical Benefits
Advisory Committee (PBAC), which makes recommendations to the Australian government about listing new drugs.
The new requirement has its roots in a 1987 amendment to the National
Health Act that required the PBAC to consider both costs and effectiveness in recommendations about listing new drugs in the PBS. After the
legislative change, a group of experts was engaged to assist in the implementation of the new policy. This resulted in draft guidelines, issued by the
27
federal government, for the preparation of submissions to the PBAC. The
guidelines were revised once, and the inclusion of an economic evaluation
in all submissions to the PBAC was made mandatory in January 1993.
Economic evaluations are required only for new drugs. When the PBAC
recommends a listing, it also passes on satisfactory economic evaluations to
a separate pricing authority (the Pharmaceutical Benefits Pricing Authority, or the PBPA).
28
The Australian guidelines contain a number of important features.
First, they state that the reimbursement decision and thus the economic
evaluations should be based on a societal perspective-that is, taking into
account all costs and benefits regardless of to whom they accrue. Second,
they state that new drugs should be compared with the therapies that they
are most likely to replace in practice. Third, they suggest that results of the
economic evaluation should be presented both with and without indirect
costs (production losses). Fourth, they note that there is inadequate experience to recommend utility measures in policy formulation, although they
give cautious support to the continued development of such measures and
suggest that the number of quality-adjusted life years (QALYs) gained
should be used as the measure of effectiveness. Fifth, they encourage the use
of intermediate effectiveness measures such as change in blood pressure for
antihypertensive drugs, Finally, they recommend that the PBAC develop
“yardsticks” to compare the cost-effectiveness of different drugs and to
determine whether a cost-effectiveness ratio should be considered “high” or
“low” for purposes of decision making.
Canada: drug guidelines. Economic evaluations are not yet required in
Canada, although policymakers have been working to develop guidelines.
In 1991 guidelines were issued in Ontario proposing economic evaluation
requirements as part of submissions to the Drug Quality and Therapeutics
29
Committee (DQTC). A revised version has since been issued for all of
30
Canada (that is, all of the provincial drug programs).
In Canada each province has a drug benefit program that subsidizes the
31
use of drugs by its residents. The drugs that are reimbursed are listed on
different formularies in each province. In Ontario the minister of health
decides which drugs should be included in the Ontario Drug Benefit For-
COMMENTARY 211
mulary, after advice from the DQTC, which considers effectiveness, safety,
and cost in its recommendations.
As is true for the Australian guidelines, the Canadian guidelines adopt a
32
societal perspective. Analysts also are encouraged, however, to conduct
separate analyses showing results when only those costs funded by the
provinces are considered. The guidelines state that new drugs should be
compared with the cheapest alternative treatment for the same patients,
but that comparisons with other treatment strategies also may be appropriate. The guidelines propose using QALYs to measure effectiveness, although they encourage experimentation with other approaches such as
cost/ benefit analysis where benefits are based on persons’ willingness to pay.
The Canadian guidelines include a system to assess the quality of the
economic evaluations. A checklist of questions is provided against which
the economic evaluations are to be assessed. These questions address the
quality of the evidence and the independence between the investigators
conducting the economic evaluations and the sponsoring drug company.
The guidelines also recognize that if the cost per QALY is measured, it will
be necessary to determine society’s willingness to pay per QALY gained in
33
order to reach a recommendation based on the economic evaluation.
Tentative thresholds for this ju d gm ent-that is, whether a cost-effectiveness ratio is considered “high” or “low”–were published in the initial
version of the guidelines but were removed from revisions.
Guideline and coverage comparisons. From a methodological viewpoint, there are many similarities between the Australian and the Canadian
guidelines, but also some important differences. The Canadian guidelines
incorporate quality assessment of the economic evaluations, whereas the
Australian guidelines are silent on this point, Also, the Australian guidelines encourage the use of intermediate effectiveness measures, while the
Canadian guidelines do not.
From a policy perspective, the Australian and Canadian guidelines are
similar. In both countries the registration, reimbursement, and pricing of
drugs are separated; economic evaluations are intended as an aid to decisions about reimbursement. In Canada, however, prices are set before the
decision about reimbursement, whereas in Australia the reimbursement
decision precedes the pricing decision. Moreover, reimbursement decisions
are made on the provincial level in Canada and on the national level in
Australia.
Europe: drug reimbursement. According to a recent survey of health
economists in Europe, economic evaluations are now used in a number of
34
European countries as an input into decisions about drug reimbursement.
An example involves reimbursement decisions for simvastatin and pravastatin in the Netherlands. Based on an economic evaluation, these drugs
212 HEALTH AFFAIRS | Summer 1994
were restricted to persons with high cholesterol levels who had one or more
35
additional cardiovascular risk factors. However, formal requirements for
economic evaluations as a basis for reimbursement decisions do not yet
exist in any European country, although some attempts have been made in
36
recent years to add standards for economic evaluations in Europe.
Discussion
Policymakers have long debated whether new medical technology is the
37
culprit behind rising health care costs. Although they do not agree on the
magnitude involved, experts generally concur that new technology is at
least partly to blame; its contribution to rising health spending is usually put
38
at between 10 percent and 40 percent. But this debate has been misguided.
First, technologies by themselves do not raise costs. The more meaning
ful question involves the health system in which technologies are used and
whether the system encourages the adoption of important new technologies. Second, whatever the profile of the system, the problem is not whether
costs are rising per se, but what is being achieved (in terms of health
benefits) for the resources consumed. It is important to keep these points in
mind when considering what a cost-effectiveness criterion can and cannot
accomplish.
Ideally, a cost-effectiveness criterion can play an important role by
guiding more sensibly the march of new technologies into the population.
Implementing an economic criterion also will create an incentive for producers to develop cost-effective products in the first place. But ultimately,
the underlying system and the incentives it embodies will largely determine
the extent to which new technologies are developed, marketed, and used in
clinical settings, regardless of whether an explicit cost-effectiveness provision exists in the coverage process. Moreover, a finding that a new technology is cost-effective will not assure use or reimbursement. How a technology is coded, classified, and priced by the system once a coverage decision is
rendered will be critical factors.
It should be emphasized that cost-effectiveness analysis cannot resolve
39
the question of whether society should pay for a new technology. It is only
possible to compare cost-effectiveness ratios across technologies. To determine whether a technology should be covered, it will be necessary to
determine how much society is willing to pay for the health benefits derived
from the technology. Conducting cost-effectiveness analyses will not remove the need for difficult resource allocation decisions. But explicitly
illuminating the trade-offs involved should help the process; thus, it represents a prudent step forward in public policy.
COMMENTARY 213
NOTES
1. See, for example, M.C. Weinstein and W.B. Stason, “Foundations of Cost-Effectiveness Analysis for Health and Medical Practices,” The New England Journal of Medicine
296 (1977): 716-721; and D.M. Eddy, “What Do We Do about Costs?” Journal of the
American Medical Association 264 (1990): 1161-1170.
2. Examples are “Medicare Program: Criteria and Procedures for Making Medical Services
Coverage Decisions That Relate to Health Care Technology,” Federal Register 54 (30
January 1989): 4304; AS. Detsky, “Guidelines for Economic Analysis of Pharmaceutical Products: A Draft Document for Ontario and Canada,” PharmacoEconomics 3
(1993): 354-361; and D. Henry, “Economic Analysis as an Aid to Subsidisation
Decisions: The Development of Australian Guidelines for Pharmaceuticals,” PharmacoEconomics 1 (1992): 54-67.
3. We define technology broadly to encompass drugs, devices, and medical and surgical
procedures.
4. This description is based on the discussion in P.J. Neumann and M.C. Weinstein, “The
Diffusion of New Technology: Costs and Benefits to Health Care,” in Medical Innovation at the Crossroads, Vol.. 2: The Changing Economics of Medical Technology, ed. A.C.
Gelijns and E.A. Halm (Washington: National Academy Press, 1991).
5. M. Johannesson, “The Australian Guidelines for Subsidisation of Pharmaceuticals: The
Road to Cost-Effective Drug Prescribing?” PhamoEconomics 2 (1992): 355-362.
6. J.T. Edelson et al., “Long-Term Cost-Effectiveness of Various Initial Monotherapies
for Mild to Moderate Hypertension,” Journal of the American Medical Association 263
(1990): 407-413; and I. Kawachi and L.A Malcolm, “The Cost-Effectiveness of
Treating Mild to Moderate Hypertension: A Reappraisal,” Journal of Hypertension 340
(1991): 148-149.
7. W.B. Stason and M.C. Weinstein, “Allocation of Resources to Manage Hypertension,”
The New England Journal of Medicine 296 (1977): 732-739; and M. Johannesson and
B. Jonsson, “Cost-Effectiveness Analysis of Hypertension Treatment: A Review of
Methodological Issues,” Health Policy 19 (1991): 55-78.
8. M.C. Weinstein and H.V. Fineberg, Clinical Decision Analysis (Philadelphia: Saunders,
1980), 241-242.
9. Johannesson and Jonsson, “Cost-Effectiveness Analysis of Hypertension Treatment.”
10. Weinstein and Stason, “Foundations of Cost-Effectiveness Analysis for Health and
Medical Practices.”
11. M.F. Drummond, “Basing Prescription Drug Payment on Economic Analysis: The Case
of Australia,” Health Affairs (Winter 1992): 191-196.
12. Editorial, “Discounting Health Care: Only a Matter of Timing?” The Lancet 340 (1992):
148-149; and J. Lipscomb, “Time Preferences for Health in Cost-Effectiveness Analysis,” Medical Care (1989 Supplement): 233-253.
13. A.N. Tosteson et al., “Cost-Effectiveness of Screening Perimenopausal White Women
for Osteoporosis: Bone Densitometry and Hormone Replacement Therapy,” Annals of
Internal Medicine 113 (1990): 594-603.
14. M.C. Weinstein, “Challenges for Cost-Effectiveness Research,” Medical Decision
Making 6 (1986): 194-198.
15. A.R. Tarlov et al., “The Medical Outcomes Study: An Application of Methods for
Monitoring the Results of Medical Care,” Journal of the American Medical Association
262 (1989): 925-930.
16. P. Anand, “Are the Preferences Axioms Really Rational?” Theory and Decision 23
(1987): 189-214; D. Kahneman and A. Tversky, “Prospect Theory: An Analysis of
Decision under Risk,” Econometrica 47 (1979): 262-290, cited in P.J. Neumann and
P.E. Politser, “Risk and Optimality,” in Risk-Taking Behavior, ed. J.F. Yates (New York:
214 HEALTH AFFAIRS | Summer 1994
John Wiley and Sons, 1992), 27-47.
17. D.M. Eddy, “Selecting Technologies for Assessment,” International Journal of Technology Assessment 5 (1989): 485-501.
18. U.S. Congress, P.L. 102-410, Agency for Health Care Policy and Research Reauthorization Act of 1992.
19. C.E. Phelps and S.T. Parente, “Priority Setting in Medical Technology and Medical
Practice Assessment,” Medical Care 28 (1990): 703-723.
20. Ibid., 707.
2 1. Drummond, “Basing Prescription Drug Payment on Economic Analysis.”
22. D. Durenberger and S.B. Foote, “Medical Technology Meets Managed Competition,”
Journal of American Health Policy (May/ June 1993): 23-31.
23. Federal Register 54, 4304.
24. Ibid., 4308-4309.
25. Ibid., 4309.
26. Ibid.
27. Commonwealth of Australia, Draft Guidelines for the Phameceutical Industry on Preparation of Submissions to the Pharmaceutical Benefits Advisory Committee, including Submissions Involving Economic Analyses (Canberra: Department of Health, Housing, and
Community Services, 1990); and D. Evans et al., The Use of Economic Analysis as a Basis
for Inclusion of Pharmaceutical Products on the Pharmaceutical Benefits Scheme (Canberra:
Department of Health, Housing, and Community Services, 1990).
28. Henry, “Economic Analysis as an Aid to Subsidisation Decisions.”
29. Ontario Ministry of Health, Guidelines for Preparation of Economic Analysis to Be Included
in Submission to the Drug Programs Branch for Listing in the Ontario Drug Benefit
Formulary/ Comparative Drug Index (Toronto: Drug Programme Branch, Ministry of
Health, 1991).
30. Detsky, “Guidelines for Economic Analysis of Pharmaceutical Products.”
31. Ibid.
32. Ibid.
33. Ontario Ministry of Health, Guidelines for Preparation of Economic Analysis.
34. M. Drummond et al., “Economic Evaluation of Pharmaceuticals: A European Perspective,” PharmacoEconomics 4 (1993): 173-186.
35. B. van Hout and M. Rutten, “Economic Appraisal of Health Technology in the
European Community,” in Proceedings of a Canadian Collaborative Workshop on Pharmacoeconomics, ed. F. Schubert (Princeton: Excerpta Medica, 1993), 8-13.
36. J. Rovira, “Standardizing Economic Appraisal of Health Technology in the European
Community” (Paper presented at the European workshop, “From Results to Action:
The Role of Economic Appraisal in Developing Policy for Health Technology,” Crete,
1992).
37. See, for example, S.H. Altman and R.J. Blendon, eds., Medical Technology: The Culprit
behind Health Care Costs! (Proceedings of the 1977 Sun Valley Forum on National
Health), DHEW Pub. no. 79-3216 (Washington: Department of Health, Education,
and Welfare, 1979); and Neumann and Weinstein, “The Diffusion of New TechnolI,
38. Neumann and Weinstein, “The Diffusion of New Technology.”
39. M. Johannesson and B. Jonsson, “Economic Evaluation in Health Care: Is There a Role
for Cost-Benefit Analysis ?” Health Policy 17 (1991): l-23.