Embedding Health Economics into
Pharmacy Research: a three-part guide
PART ONE: A basic guide to economic evaluation for primary care pharmacists
By David Wright, PhD, MRPharmS, and Tracey Sach, PhD
Contents and quick links to main topic areas
PART ONE
About the Guide
1
Political Direction
2
Economic Evaluation
3
Cost-Minimisation Analysis
4
Measuring Quality of Life
7
NICE Criteria
8
Cost-Benefit Analysis
10
Summary, Author Details and References
11
PART TWO
Introduction
12
Types of Sensitivity Analysis
13
Producing a CEAC
14
PART THREE
Introduction
17
Suitability of the Control
18
Measuring Outcomes
19
Collecting the Right Cost Data
20
Calculating Knock-On Costs Accurately
21
Scrutinising Economic Evaluations
22
Help for Conducting Future Research
23
Summary, References
24
Appendix A: Recommended Critical Appraisal Data
25
Appendix B: Recommended Further Reading
26
About the guide
Within a resource-constrained NHS, it is not justifiable to commission a new service or add a new
medicine to a formulary just because it produces the desired outcomes. Primary care commissioners
need to know that the cost of achieving those outcomes is reasonable given what could be purchased
instead with the same money.
Economists call this notion “opportunity cost” — eg, the quantity of service A that must be sacrificed to
obtain another unit of service B — and use it to identify which services use resources most efficiently.
Health economics is a relatively new discipline that has, in part, been developed to enable healthcare
commissioners to make informed decisions about health interventions by considering both costs and
outcomes.
A lack of understanding of this topic by pharmacists involved in formulary decisions provides an
advantage for the pharmaceutical industry. Furthermore, failing to use economic principles to evaluate
new pharmacy services will limit the quality of evidence that can be generated to support such services.
It is therefore important for pharmacists who are responsible for prescribing decisions and service
commissioning to:
Understand the fundamental concepts of health economics
Know how to critique evidence that purports to demonstrate economic evidence for a medicine
or service
This three-part guide provides a general overview of health economics and its language so that
those pharmacists who read papers that include economic evaluations can assess the quality of the
information provided. It will not provide the ability to perform an economic evaluation but, hopefully,
will enable readers to appreciate the value of engaging with a health economist prior to introducing
any new pharmacy service.
Disclaimer
The views put forward in this guide are the views of the authors and not necessarily those of the PCPA. Please note that any unauthorised
copying,printing or dissemination of this guide is strictly prohibited. © PCPA March 2011
1
CONTENTS PAGE | PART ONE | PART TWO | PART THREE
Political Direction
Under the previous Government’s agenda for world-class, practice-based commissioning, healthcare
providers were expected to produce robust business cases for setting up new NHS services. These
business cases would cover: [ 1 ]
A description of the service to be provided
An outline of the benefits for patients
Any expected improvements to the efficiency or effectiveness of patient care
An outline of the management resources required
The costs required to implement the service and an estimate of how long it was expected to take
to recover them
On receipt of such business cases, commissioners of NHS services were expected to assess them using
the following criteria:
Evidence-based clinical effectiveness
Clinical safety, quality and governance
Justification/evidence that resources could be released by implementing the new service
Affordability within current and future budgets
Under the new Government’s proposals, commissioning will move to GP-based consortia but the
principles will stay the same. [ 2 ] Having a good understanding of how costs and outcomes can be used
to make decisions about resource allocation is important within this process.
Primary care pharmacists are involved in both formulary decisions and the development of new pharmacy
services. Therefore, to be involved in making decisions about which service or drug treatments to fund,
or to develop an argument for introducing a new service, they need at least a basic understanding of
health economics.
2
CONTENTS PAGE | PART ONE | PART TWO | PART THREE
Economic Evaluation
Frequently, pharmacy research authors report to have undertaken an economic evaluation as part of
their research. In reality, they have often done nothing more than a simple cost analysis. For example,
let us consider a researcher who is assessing the cost of swapping patients who are taking proton-pump
inhibitors to H2-receptor antagonists.
The researcher might determine the cost of implementing the change simply by calculating the
pharmacist or technician time required to carry out the switch — and comparing it with the savings
generated within the drugs budget.
This “cost analysis” is, however, far too simplistic; it does not consider the outcomes of the swap (eg,
the effect on patients’ health-related quality of life or symptom relapse rates) nor does it consider
other, wider NHS costs (eg, readmission to hospital or increased GP visits). To conduct a comprehensive
economic evaluation, wider costs and outcomes should be considered.
The analysis also doesn’t include patient costs such as work days lost. The question of whether patient
costs should be included in an economic evaluation is discussed in Part 3 of this guide.
Four types of economic evaluation have been developed to help purchasers and providers to make sense
of these different variables:
Cost-minimisation
Cost-effectiveness
Cost-utility
Cost-benefit
Before selecting which type of evaluation is most appropriate, we need to consider what type of
question the evaluation needs to answer. So, do we want to know:
Which is the best service/treatment to achieve a specific objective
(eg, what is the best medicine to treat hypertension)?
Or
How can resources best be distributed for the common good
(eg, which primary care services should we purchase to improve public health)?
The first question is what health economists call a “technical efficiency question” and can usually
be answered using intervention-specific outcomes (eg, change in blood pressure). The second is an
3
CONTENTS PAGE | PART ONE | PART TWO | PART THREE
Cost-minimisation analysis
“allocative efficiency question” and requires generic outcomes, which can be compared between
different services (eg, health-related quality of life), to make a decision.
A cost-minimisation analysis is performed when the outcomes of different interventions are the same.
Therefore, choosing the intervention that costs the least is all that is required. This type of analysis
is mainly used to answer simple technical efficiency questions. For example, let us consider the best
drug choice to treat hypertension. If the dose required to cause a 10mmHg reduction in systolic blood
pressure was known for several different medicines, the acquisition costs of the medicines could be
calculated and the cheapest one identified.
Unfortunately health outcomes are usually not that simplistic. Different medicines tend to achieve
different magnitudes of the therapeutic outcome, have different adverse event profiles (hence monitoring
requirements) and different levels of patient acceptability. For these reasons, cost-minimisation analyses
are rarely the analysis of choice (unless there is strong prior evidence of equivalent outcomes) and only
appropriate for making resource allocation decisions if differences in other health-related outcomes or
elements of care are ignored.
Cost-effectiveness analysis
The term “cost effective” is one of the most overused and inappropriately applied components of a
healthcare professional’s vocabulary. A medicine or service should only be described as cost effective if
it has been proven so by economic analysis.
A cost-effectiveness analysis is used when the costs and outcomes of different services/treatments
are compared using an outcome that is bespoke to the intervention. Table 1 provides examples of
pharmacy-service-specific outcomes that could be used for a cost-effectiveness analysis.
Table 1: Examples of effectiveness outcomes for enhanced pharmacy services
SERVICE
MEASURE OF OUTCOME
Anticoagulant monitoring
Reduction in adverse events
Screening service
Number of additional diagnoses
Medication review
Adherence to a prescribing quality indicator
Asthma management service
Improvement in forced expiratory volume
Older person medicines use review
Number of falls prevented
Smoking cessation
Reduction in failed carbon monoxide tests
Consequently cost-effectiveness analyses can only be used to answer technical efficiency questions
where the treatments being compared share a common outcome. Let us once again consider which
medicines should be used to treat hypertension. It might be deduced that drug A causes a 10mmHg
drop in blood pressure and costs £120 per year, while drug B causes a 15mmHg drop in blood pressure
but costs £180 per year. We cannot use a cost-minimisation analysis in this instance because the
4
outcome achieved is different.
An incremental cost-effectiveness plane can be used to consider such situations (see Figure 1).
Figure 1: An incremental cost-effectiveness plane
If drug B costs less than Drug A and is more effective, the comparison would lie in the bottom-right
(South-East) quadrant of the plane. Consequently, drug B would always be preferred to drug A. If Drug
B was more costly and less effective (North-West quadrant) it would never be preferred. The difficulty
arises when a new drug is either more effective and more costly, or less effective and less costly. In
these circumstances, purchasers need to consider costs and effects in terms of an incremental costeffectiveness ratio (ICER), which is calculated using the following formulae:
ICER =
(CostDrug B – CostDrug A)
(EffectDrug B – EffectDrug A)
Taking the example figures provided above:
5
ICER =
£180 – £120 = £60 = £12 per 1mmHg decrease
15mm/Hg – 10mmHg 5
This tells us that for an extra £60 per year per patient, we get a further drop in blood pressure of
5mmHg, which is equivalent to £12 per 1mmHg decrease. Therefore the purchaser has to decide
whether this additional effect is worth the additional cost.
For such technical efficiency questions there is no guidance as to what is acceptable and what is not.
Many decisions made by purchasers involve comparisons that lie within the North-East quadrant. Due
to the lack of comparability of outcomes for different interventions purchasers need to make value
judgments based on the amount of resources available and, often, local politics.
The world-class commissioning agenda pragmatically accepts this reality by not requiring decisions
to be made purely on the basis of economic evidence. The availability of an economic evaluation will,
however, always strengthen the argument for the acceptance of any new service or treatment.
The example provided above is extremely simple and it is unlikely that data would be presented in such
a basic manner. Furthermore, the costs and effects of the medicines or services presented in economic
analyses are average (mean per patient) values obtained from study samples. Therefore they are only
estimates of the actual or “real world” value.
Health economists advocate using a sensitivity analysis to confirm the likelihood of study results being
correct ( see Part 2 of this series — “Conducting sensitivity analyses” ).
Cost-utility analysis
Cost-effectiveness analyses can only answer technical efficiency questions because the outcomes they
use are specific to the condition being analysed. They do not allow comparisons of medicines or services
for which success is measured using different outcomes. For example, knowing that £500 can prevent
a fall while £200 can reduce pain by 50% does not allow a commissioner to assess which intervention
is more valuable.
Generic outcome “quality of life” can be used to assess the effectiveness of interventions for different
conditions. If used appropriately it can answer allocative efficiency questions. Cost-utility analysis is a
form of cost-effectiveness analysis that uses change in quality of life as the measure of effect.
Measuring impact on quality of life alone is not sufficient as we also need to know the duration of
the impact. Increasing quality of life by 50% for 1 day is less valuable than increasing it by 10% for
40 years. The preferred outcome for cost-utility analysis is the quality-adjusted life year (QALY). This
combines a measurement of health-related quality of life with length of life. The use of QALYs in the
6
CONTENTS PAGE | PART ONE | PART TWO | PART THREE
Measuring quality of life
field of cardiac medicine provided the seminal piece of research that helped to cement the role of health
economics as a valuable academic discipline. [ 3 ]
Health-related quality of life can be measured using many tools but the National Institute for Health
and Clinical Excellence uses one called “EuroQol EQ-5D”. This tool can be applied to a wide range of
health conditions and provides a simple descriptive profile and a single index value. It consists of five
questions (otherwise known as dimensions):
Mobility
Self-care
Usual activities
Pain/discomfort
Anxiety/depression
For each dimension, patients are asked to rate themselves as one of three severity levels: no problems
(coded as 1); some problems (coded as 2); and extreme problems (coded as 3). Since the tool consists of
only five questions with three tick boxes as choices, it is quickly completed and requires little additional
workload for patients.
Patients in perfect health will be coded with 1 for all five questions (ie, “11111”) while those with the
worst possible health are coded as “33333”. There are 243 possible health states (plus an extra one for
when patients are unconsciousness) each of which has been rated, by a large public panel, on a scale of
0 to 1. 11111 is valued as 1 (ie, perfect health) while 0 is the expected value for death. That said, several
health states are rated worse than death (eg, 33333 has been given a rating of -0.594).
More information on Euroqol EQ-5D can be found at www.euroqol.org
Incorporating quantity with quality
One QALY equates to a gain of one year in perfect health (or a gain of two years living at 50% of
perfect health, etc). Simply described, researchers determine the difference in health-related quality of
life and life expectancy achieved by an intervention and calculate the number of QALYs associated with
it. Commissioners can then determine whether the intervention is worth funding.
By deciding how much you are willing to pay for one QALY, you can determine cost-effectiveness for
those interventions that lie in the North-East or South-West quadrants of the cost effectiveness plane
7
discussed earlier. This is shown in Figure 2. Cost effectiveness can then be determined from whether an
intervention’s ICER is above or below the diagonal line.
Figure 2: An incremental cost-effectiveness plane showing a ceiling ratio
NICE criteria
Within England and Wales, NICE takes a case-by-case approach to deciding which medicines or services
are recommended for NHS funding. It does not set explicit thresholds for costs per QALY. However, its
guidance suggests: [ 4 ]
Interventions that cost less than £20,000 per QALY are likely to be funded.
Interventions that cost more than £20,000 per QALY usually require mitigating circumstances to
support their funding - eg, where only a small number of people will benefit, where there is a lack of
alternatives or considerable uncertainty in the cost per QALY value calculated.
Interventions costing more than £30,000 per QALY require strong extenuating circumstances for NICE
to recommend they are adopted into practice. [ 5 ]
There is some debate as to whether NICE’s implicit thresholds for cost effectiveness are too high for
local PCTs. Supporters of this argument suggest that it fails to identify what is affordable to the local
8
area and could result in more cost-effective services losing funding just to ensure that interventions
recommended by NICE are implemented.
Disparate pharmacy data
There is currently extremely limited evidence for the cost per QALY of pharmacist services. As a
profession, we need to integrate the potential for a cost-utility analysis within any new pharmacy
service that is set up.
9
CONTENTS PAGE | PART ONE | PART TWO | PART THREE
Cost-benefit analysis
Organisations that can provide help in setting up high-quality economic analyses are discussed in
Part 3 of this series (“How to critically appraise an economic analysis”).
Cost-utility analysis is frequently criticised for its narrow focus on health outcomes. Health services or
treatments could improve patient satisfaction, widen access to services or improve outcomes in other
sectors of the economy. However, none of these would be measured by such an analysis.
Within cost-benefit analysis the main outcome is patients’ perceived value of a service or medicine
measured as their willingness to pay for it. This is a generic outcome that answers allocative efficiency
questions and overcomes some of the limitations of cost-utility analysis.
Although widely used in other sectors of the economy by the Government, such analyses are not
routinely used by health commissioners. This is in part because the method for measuring benefit by
10
CONTENTS PAGE | PART ONE | PART TWO | PART THREE
Summary
patients is still being developed for healthcare interventions. As a result, cost-benefit analysis currently
has little relevance for primary care pharmacists.
Cost-effectiveness analyses and cost-utility analyses are the two types of economic evaluation that are
currently most useful within healthcare. Where single-dimension outcomes enable us to make decisions
on how to provide products or services with the same outcome measure, cost-effectiveness analyses
should be used. Cost-utility analyses can help commissioners to decide which products or services
should be purchased with a finite budget.
An unwritten ceiling of £20,000 per QALY (according to NICE) provides the current yardstick both
determining whether new healthcare services and medicines should be widely available through
the NHS.
Author details
David Wright is Professor of Pharmacy Practice, School of Pharmacy, University of East Anglia.
Email: [email protected]
Tracey Sach is Senior Lecturer of Health Economics, School of Medicine, Health Policy & Practice,
University of East Anglia
References
Department of Health. Practice based commissioning: practical implementation. What does this
mean for practice? November 2006. www.dh.gov.uk
(gateway reference: 7444; accessed 14 October 2010). Return to reference in the guide >
[1]
Department of Health. Equity and excellence: Liberating the NHS. 12 July 2010. www.dh.gov.uk
(accessed 14 October 2010). Return to reference in the guide >
[2]
Williams A. The economics of coronary artery bypass grafting. British Medical Journal 1985; 291:
325–9. Return to reference in the guide >
[3]
National Institute for Health and Clinical Excellence. Guide to the methods of technology appraisal.
June 2008. www.nice.org.uk
(assessed 14 October 2010). Return to reference in the guide >
[4]
Appleby J, Devlin N, Parkin D, et al. Searching for cost effectiveness thresholds in the NHS.
Health Policy 2009;91:239–45. Return to reference in the guide >
[5]
11
Embedding Health Economics into
Pharmacy Research
PART TWO: Sensitivity analysis - a tool for quantifying decision uncertainty
By David Wright, PhD, MRPharmS, and Tracey Sach, PhD
CONTENTS PAGE | PART ONE | PART TWO | PART THREE
Introduction
With Part One we demonstrated how an ICER plane could be used to make decisions regarding whether
a new intervention should be purchased or not. Where a new intervention e.g. new drug was shown to
be more expensive and less effective (North West Quadrant) it should never be accepted and conversely
if it is found to be less expensive and more effective (South East Quadrant) it should always be accepted.
Difficulties arise when products are more expensive and more effective or less expensive and less effective
and consequently either a threshold of acceptance is required (Drawn by a straight line through zero) or
local decisions based on a number of factors have to be made.
NICE provide a threshold of acceptance of between £20,000 and £30,000 per QALY for cost-utility analysis
(which is a form of cost-effectiveness analysis). The position of the line is not definitive as factors such
as the variability in the accuracy of the cost-utility estimate need to be taken into account. To health
economists this potential variability in estimates is an important factor in all economic decision making.
The size of effect of an intervention on an outcome is usually presented as a mean score and this is an
estimate of the mean of the population from where the data came e.g. if we state that the mean change
in pain score for a sample of patients with a certain condition is 5.0, we are informing the reader that we
believe the mean change in pain score for all patients with that condition (population) who received the
intervention would be somewhere close to 5.0. We accept however that the population mean is likely to
be different to this because if we took another sample we would very likely achieve a slightly different
result. If our sample is based on a very spread out data set i.e. large standard deviation, then the sample
mean estimate is likely to not be too close to the population mean i.e. it is not a very precise estimate.
Using standard formula which include the sample mean, standard deviation and size we can estimate with
95% confidence the boundaries within which the population mean is likely to lay and these are called 95%
confidence intervals e.g. we may state that the mean (95% CI) for our sample is 5.0 (4.0, 6.0). This would
be informing the reader that the mean change for our sample was 5.0, however we are 95% confident that
the population mean change is somewhere between 4 & 6.
If it was found that it cost £120,000 to produce a mean pain change of 5 then the cost per 1 point change
would be £24,000. If however we took into account the possible extreme values for the population mean
i.e. 4 & 6 then this result could range from £20,000 per point change (£120,000 divided by 6) to £30,000
per point change (£120,000 divided by 4) i.e. the possible cost per 1 point change could be anywhere from
£20,000 to £30,000 and our original estimate of £24,000 is only accurate to ±£6000.
Variation doesn’t however just exist around the outcome, it also exists around the estimated cost of the
intervention and this is likely to be greater as it is frequently made up from a large number of estimated
means e.g. mean cost of hospitalisations saved, GP time saved, time to provide the intervention and
medicines usage by the patient. Consequently plotting a single point on an ICER plane, as shown in the
first guide, is overly simplistic. When plotting an ICER it is necessary to take into account the variation in
the accuracy of both the estimated costs and outcomes.
Therefore to test the robustness of results (in case of any uncertainties around these assumptions), or to
test other features of the analysis (eg, generalisability, methodological controversies), economists undertake
what is called sensitivity analyses. The methods used for the sensitivity analysis and the results of these
analyses should always be included in research papers that involve an economic evaluation.
12
CONTENTS PAGE | PART ONE | PART TWO | PART THREE
Types of sensitivity analysis
The most basic type of sensitivity analysis, called “one-way sensitivity analysis”, involves substituting one
variable at a time with a range of other possible values. Should the overall conclusion drawn from the
results not change, confidence in the conclusion is increased. Other types of sensitivity analyses include:
Multivariate (multi-way) analysis - several variables are changed simultaneously to look for
interaction effects
Threshold analysis - this identifies the critical value for each variable at which the conclusion of the
study results changes
Analysis of extremes - a base case analysis is presented alongside analyses of extreme cases to provide
an estimated range of the incremental cost-effectiveness ratio
A more advanced type of analysis that is being used increasingly for economic evaluations is called
“probabilistic sensitivity analysis”. The most common version of this is known as “non-parametric
bootstrapping”. This involves taking a large, random sample of cost and outcome pairs from the
intervention and control groups, and recalculating the mean cost and outcome for each of these pairs.
This is similar to calculating 95% confidence intervals around the data. The ‘bootstrapping’ approach
is taken because the formula for calculating 95% confidence intervals rely on the data being Gaussiannormal in its distribution i.e. parametric and this assumption cannot usually be made regarding cost and
outcomes data. The range of values obtained from bootstrapping can be plotted on a cost-effectiveness
plane to show the distribution. Figure 1 provides an example of this.
Figure 1: An example of an ICER plane with non-parametric bootstrapping
Change in QALY
Change in cost
13
CONTENTS PAGE | PART ONE | PART TWO | PART THREE
Producing a CEAC
If a commissioner was willing to pay £20,000 per QALY, we could plot a line on the cost-effectiveness
plane to indicate this (see Figure 2). All the points below the line are considered cost-effective whilst
those above the line are not.
It can be seen that approximately 30% of the points fall below the line and therefore there is a 30%
chance that the service is cost-effective.
Figure 2: Adding a threshold to a non-parametric bootstrapping chart
Change in QALY
Change in cost
Another approach to presenting such data is to consider the likelihood of an intervention being costeffective if the threshold cost was changed i.e. if we move the threshold upwards the number of points
below the line increase and therefore we are more confident that the intervention is cost-effective.
Producing a CEAC
The percentage of points which are cost-effective for different thresholds is plotted as a costeffectiveness acceptability curves (CEACs). This illustrates the probability of an intervention being costeffective (compared with the control group) for different threshold levels. To produce a CEAC we
calculate the proportion of points on each side of the line and therefore estimate the probability of
cost-effectiveness for that threshold. If we repeat this calculation several times using different levels of
willingness to pay, we can produce a CEAC as shown in Figure 3.
14
Figure 3: An example of a cost-effectiveness acceptability curve
Figure 3 shows an example CEAC for a healthcare intervention. A cost-utility analysis for the intervention
had already indicated that it was unlikely to be considered cost-effective. The CEAC quantifies the
uncertainty surrounding this conclusion. In this example, at a cost-effectiveness threshold of £20,000
to £30,000 (as often used by NICE) we see that the intervention has a much lower probability of being
cost effective than the control.
If commissioners could afford £40,000 to gain one QALY, the intervention is now slightly more likely to
be cost effective than the control. However, it requires much greater levels of affordability (i.e. £60,000
per QALY) for the intervention’s probability of the cost effectiveness to be substantially higher than the
control’s.
In Figure 3, the area of greatest uncertainty is near the point where the two lines cross (at a costeffectiveness threshold of about £35,000 per QALY. Commissioners who could afford an intervention
at this cost might wish to undertake further research to help inform future decisions about funding.
In reality, the shape of CEACs can vary dramatically — eg, some do not cross at all. As a result, careful
interpretation is essential.
15
Author details
David Wright is Professor of Pharmacy Practice, School of Pharmacy, University of East Anglia.
Email: [email protected]
Tracey Sach is Senior Lecturer of Health Economics, School of Medicine, Health Policy & Practice,
University of East Anglia
16
Embedding Health Economics into
Pharmacy Research
PART THREE: How to critically appraise an economic evaluation
By David Wright, PhD, MRPharmS, and Tracey Sach, PhD
CONTENTS PAGE | PART ONE | PART TWO | PART THREE
Introduction
Having read about the types of economic evaluation (see Part 1 of this three-part guide), it becomes
apparent that for a new service to be appraised accurately, a control group is required.
Traditional service evaluations, which compare what happened in the past with what happens as a
result of a new intervention (ie, a “before-and-after study”) are not considered by health economists to
be the best way to determine the costs and effects of new interventions.
Although such studies are better than nothing, they often fail to provide accurate results due to wellknown design problems (eg, patient deterioration, the inability to exclude other influencing factors).
An economic evaluation is only as good as the data on which it is based; good quality data is only
achieved through good study design.
17
CONTENTS PAGE | PART ONE | PART TWO | PART THREE
Suitability of the control
Control groups for economic evaluations can be called “alternatives” by health economists because they
should be the next best choice to the intervention being investigated.
In some instances there may not be a next best so the alternative is “usual care” or “do nothing”.
For new medicines, the control should be the current gold standard for the condition being treated.
This is important when considering the quality of evidence for new treatments since the comparators
selected by the pharmaceutical industry are often not the gold standard and are sometimes selected to
increase the likelihood of positive results for a new drug.
18
CONTENTS PAGE | PART ONE | PART TWO | PART THREE
Measuring outcomes
For the economic evidence supporting a proposed new pharmacy service to be regarded as high quality,
health-related quality of life data (eg, EQ-5D) and an intervention-specific outcome should have been
measured and then compared with a control arm.
These provide the basic data necessary for a cost-utility and a cost-effectiveness analysis, respectively.
New service providers can be unwilling to undertake a randomised controlled trial since this involves a
subset of the study population not receiving the service.
The easy solution to this issue is to provide a delayed intervention (ie, everyone accesses the service but
one group gets access later than the other — thus acting as a control while not receiving it).
19
CONTENTS PAGE | PART ONE | PART TWO | PART THREE
Collecting the right cost data
The other data required to complete an economic evaluation is a determination of costs. This has great
potential for data manipulation and always requires careful consideration when critically appraising an
economic evaluation. There are three elements to the costing process:
Identification
Measurement
Valuation
Identifying costs is dependent on the perspective of the analysis. From a commissioner’s perspective,
the only costs that need to be included are those that the commissioning body has to bear. So, for
example, it does not pay for patients’ lost work time so would not require this to be included.
Most UK research is undertaken from the NHS and Personal Social Services (PSS) perspective so only
NHS and PSS costs are considered.
A major deficiency in many pharmacy studies is that the only cost considered is the cost of medicines. A
good quality economic evaluation includes the costs of setting up the intervention (capital), delivering
it (operational costs) as well as the cost of any knock-on consequences of the intervention. The latter
includes:
The number of GP or other healthcare professional consultations that patients require
Hospital admissions
Calls made to helplines
These data must be collected for patients in both the intervention and the control group for any new
pharmacy service that is being evaluated.
20
CONTENTS PAGE | PART ONE | PART TWO | PART THREE
Calculating knock-on costs accurately
The next question relates to how these knock-on consequences should be measured. For example,
how can the time involvement of healthcare professionals or the use of hospital services be measured
accurately?
The most accurate approach is to record everything that happens to every patient in the study (microcosting) but this is an onerous task. A more pragmatic approach is to record what happens to a sample
of patients and use these data to estimate the average time involved for all patients. If sampling is
used, the potential for variability in results should gauge the size of the sample required; the larger the
variation, the bigger the sample required.
A good economic analysis considers the potential inaccuracy of sampling as part of its sensitivity
analysis.
Finally the valuation of non-drug costs (eg, cost per day of hospital admission, cost of pharmacist per
hour) can affect the accuracy of the final analysis. It is important that researchers make clear how they
have made these valuations. Ideally a national database (eg, the NHS Payment by Results tariff) will
have been used with each cost calculated using values taken from the same year and using the same
currency.
If an intervention is evaluated over more than 12 months, be aware that commissioners are more
interested in its impact on costs and outcomes “now” rather than in future years. A good economic
analysis will take this into account by discounting — calculating the reduction in value of an outcome
over future years.
High-quality economic evaluations identify the time period over which costs and outcomes have been
measured; where the period is greater than 12 months the authors should state the rate of discount
that has been applied.
21
CONTENTS PAGE | PART ONE | PART TWO | PART THREE
Scrutinising economic evaluations
When assessing evidence from an economic evaluation, pharmacists should ensure that the information
provided is robust and withstands detailed scrutiny. It is a good idea to use a checklist to ensure that all
appropriate elements have been included.
A checklist developed by eminent health economics professor Michael Drummond is commonly used by
health economists. [ 1 ] It has been adapted for use by primary care pharmacists (see Appendix A) and can
be used whether you are appraising economic evaluations of new medicines or new pharmacy services.
There are certain elements that must be included for a paper to “qualify” as an economic analysis the essential criteria. There are other elements that only weaken the quality of the research if they are
absent or done incorrectly - the desirable criteria.
For new medicines, perhaps the most pertinent questions from the checklist are:
Is the control suitable (ie, would you choose it as the best alternative)?
What is the perspective of the analysis?
Which costs have the authors included and have any important costs been excluded?
A bad example
In 2003, US researchers published a “cost-benefit” analysis of a clinical pharmacy service. [ 2 ] Scrutiny
of their method shows they did nothing more than calculate the cost of service provision and the drug
costs saved (ie, a simple and superficial cost analysis). The researchers then go on to report how many
dollars are saved for each dollar invested, when actually all they need do is subtract the service cost
from the amount saved and report the total saving.
Whenever a cost-benefit analysis is presented it is important to determine whether perceived value or
willingness to pay has been measured - and, indeed, whether a controlled trial has been performed.
If either or both of these elements are missing then the research is unlikely to be a true cost-benefit
analysis.
22
CONTENTS PAGE | PART ONE | PART TWO | PART THREE
Help for conducting future research
Once the necessary data have been collated, significant expertise is required to calculate ICERs accurately
and undertake discounting where appropriate. Guidance can be provided by a health economist
although, ideally, it should initially be obtained before a new service is set up. It is generally pointless
to set up a new service, deliver it and then ask someone to belatedly value it to create a case for service
continuation. The evidence generated from such a research process will, typically, not stand up to the
required scrutiny.
The National Institute for Health Research provides a research design service that employs statisticians
and health economists who can be approached for help. More information on the service can be found
at www.nihr-ccf.org.uk (visit the RDS section of the “Programmes” menu).
Additionally, the National Prescribing Centre has recently published an educational tool on health
economics which could prove beneficial for primary care pharmacists. It can be accessed at the following
web address:
www.npci.org.uk/ldm/public/e_learning/health_economics.html
Other further reading that might be of interested can be found in Appendix B.
23
CONTENTS PAGE | PART ONE | PART TWO | PART THREE
Summary
It is standard for NICE to use cost-utility analyses to decide which medicines the NHS can or cannot
afford. Nonetheless, the use of such analyses to determine the costs and outcomes of new pharmacy
services remains extremely rare.
To raise the quality of evidence advocating the cost effectiveness of pharmacy services, health
economic principles need to be embedded into pharmacy practice research methods. This will take
time and require closer working between academia and service providers. It is, however, essential in an
environment where cost and outcomes are the main drivers for service commissioning.
Author details
David Wright is Professor of Pharmacy Practice, School of Pharmacy, University of East Anglia.
Email: [email protected]
Tracey Sach is Senior Lecturer of Health Economics, School of Medicine, Health Policy & Practice,
University of East Anglia
References
Drummond MF, Jefferson TO. Guidelines for authors and peers reviewers of economic submissions
to the BMJ. BMJ 1996; 313:275–83. Return to reference in the guide >
[1]
Chisholm MA, Vollenweider LJ, Mulloy Ll, et al. Cost-benefit analysis of a clinical pharmacist-managed
medication assistance program in a renal transplant clinic. Clinical Transplantation 2003;14:304–7.
Return to reference in the guide >
[2]
24
CONTENTS PAGE | PART ONE | PART TWO | PART THREE
Appendix A: Recommended critical appraisal criteria
(adapted from Drummond et al [ 1 ] )
ESSENTIAL
Does the study examine both the costs and outcomes of an intervention?
Are all costs associated with the service (from the stated perspective) included?
Does the study include a control group (an alternative)?
DESIRABLE
Is the alternative a reasonable choice (ie, is it the next best or gold standard)?
Are the outcomes appropriate for the intervention?
Is the evidence obtained from a randomised, controlled trial?
Is the economic analysis evaluated from an NHS commissioners’ or a societal perspective?
Are costs and outcomes measured appropriately?
Are costs and outcomes valued credibly with clearly identified sources?
Are any future costs or outcomes discounted to their current value to take account for differential
timing? Is the discount rate stated and justified?
Is an incremental cost-effectiveness ratio calculated?
Is allowance made for uncertainty in the estimation of costs and outcomes?
Does the presentation and discussion of the study result include all issues of concern?
Are the results interpreted intelligently within the discussion?
Were the results compared with other studies using the same question? Are allowances made for
different methodologies?
Does the study discuss the generalisability of the results with respect to other patient groups
and settings and does it also take into account any important issues such as ethics?
Does the study discuss the issues of implementing the new service or intervention, such as
financial constraints or whether freed resources could be used elsewhere?
25
CONTENTS PAGE | PART ONE | PART TWO | PART THREE
Appendix B: Recommended further reading
Raftery J. Economics notes: Economic evaluation: an introduction.
BMJ 1998;316:1013–4.
Kernick DP. Economic evaluation in health: a thumb nail sketch.
BMJ 1998;316:1663–5.
Palmer S, Byford S, Raftery J. Economics Notes: Types of economic evaluation.
BMJ 1999;318:1349.
Briggs A. Economics notes: Handling uncertainty in economic evaluation.
BMJ 1999;319:120.
Briggs A, Gray A. Economics notes: Using cost effectiveness information.
BMJ 2000;320:246.
Coast J. Is economic evaluation in touch with society’s health values?
BMJ 2004;329:1233–6.
Hayward Medical Communications. What is…? www.whatisseries.co.uk/whatis
(accessed 14 October 2010).
26