Optimal Drug Development Programs and
Efficient Licensing and Reimbursement
Regimens
Neil Hawkins
Karl Claxton
CENTRE FOR HEALTH ECONOMICS
Overview
• Societal and commercial value of Information
• Decision rules incorporating value of information
• Challenges
The Quantitative Estimate of the Value of
Sample Information
The value of additional sample information is the value of
the increased likelihood of selecting the optimum
treatment arising from the reduction in uncertainty
regarding treatment effects (and costs).
What is the optimum treatment?
The treatment with greatest expected net benefit in terms
of costs and effects- Bayesian Decision Rule.
Decision Uncertainty
Net Benefit(tx) - Net Benefit(placebo)
Favours Treatment
Probability
Favours Placebo
-20000
-10000
0
Net Benefit(€)
10000
20000
Expected Net Value of Sample Information (ENVSI)
Expectation over potential future samples of:
Net benefit from optimum decision made including the additional sample data
—
Net benefit from optimum decision based on existing data
—
Cost of collecting sample
Note: ENVSI < 0, if the optimum decision does not change due to extra sample
data
Bayesian Simulation of Future Samples for
Binomial Parameter
Sample P from current posterior distribution:
Pcurrent ~ beta(a,b)
Simulate Trial Data
rT ~ bin(n, PTx )
Calculate new posterior distribution
Pnew ~ beta(a+r,b+n-r)
Small Sample (n=1)
Net Benefit(tx) - Net Benefit(placebo)
Favours Placebo
Favours Treatment
Probability
Current Sample
Future Samples
-20000
-10000
0
Net Benefit(€)
10000
20000
Large Sample (n=2000)
Net Benefit(tx) - Net Benefit(placebo)
Favours Placebo
Favours Treatment
Probability
Current Sample
Future Samples
-20000
-10000
0
Net Benefit(€)
10000
20000
Example Phase II Trial Results
Response
Deaths
Total
Tx
16
1
49
Placebo
8
1
45
χ2 Test: 2.0037, 1 df, p = 0.1569
Decision Analytic Model
Net Benefit =
P(Resp) x QALYs Gained | Resp x Monetary Value of a QALY
P(Death) x QALYs Lost | Death) x Monetary Value of a QALY
Treatment Cost
Example Parameter Estimates
QALYs Gained | Response : ~ N(0.7,0.12) x 4
QALYs Lost | Death :
~ N(0.7,0.12) x 4
Value of a QALY:
£30,000
Treatment Cost Per Course :(<10%  Response) £12,000
(≥10% & < 20%  Response) £14,000
(≥20%  Response) £16,000
Treatment Population:
20,000
Production Costs:
£150,000,000
Trial Costs:
(Fixed) £10,000,000
(per Patient) £20,000
Based on Current Data
• New treatment is cost-effective
• New treatment would not get approval based on a
frequentist hypothesis test
Societal Value of Sample Information (Efficacy Trial)
15000
10000
5000
0
Value of Information
20000
25000
Societal Value of Information
Efficacy Endpoint
0
100
200
300
Sample Size
400
500
Commercial Value of Sample Information
The value of increased sales due to the increased
probability of regulatory and reimbursement approval
arising from the extra information
ICH E9: Guidance on Statistical Principles for
Clinical Trials
Using the usual method for determining the appropriate
sample size, the following items should be specified:
• probability of erroneously rejecting the null hypothesis
• probability of erroneously failing to reject the null
hypothesis
ICH E1A: The Extent of Population Exposure to
Assess Clinical Safety
• 100 patients exposed for a minimum of one-year is
considered to be acceptable to include as part of the
safety data base.
• It is anticipated that the total number of individuals
treated with the investigational drug, including shortterm exposure, will be about 1500.
Probability of Approval
(Efficacy Endpoint – Current Regulatory Regimen)
0.4
0.2
0.0
Probability of Approval
0.6
0.8
Probability of Regulatory Approval
Efficacy Endpoint
0
100
200
300
Sample Size
400
500
Value of Sample Information
(Efficacy Endpoint - Current Regulatory Regimen)
10000
20000
30000
40000
Commercial
Societal
0
Value of Information
50000
60000
70000
Societal and Commercial Value of Information
Efficacy Endpoint
0
100
200
300
Sample Size
400
500
What happens if we just use a Bayesian CE
decision rule?
Value of Sample Information
(Efficacy Endpoint - Bayesian CE Decision Rule)
60000
70000
Value of Information (Bayesian Decision Rule)
Efficacy Endpoint
40000
30000
20000
10000
0
Value of Information
50000
Commercial
Societal
0
100
200
300
Sample Size
400
500
Societal Value of Sample Information
(Utility Study)
600
400
200
0
Value of Information
800
1000
Societal Value of Information
Utility Endpoint
0
20
40
60
Sample Size
80
100
Value of Sample Information
(Utility Study – Current Regulatory Regimen)
60000
20000
40000
Commercial
Societal
0
Value of Information
80000
Value of Information (Bayesian Decision Rule)
Utility Endpoint
0
20
40
60
Sample Size
80
100
Implications
• Under current regulatory system we might expect a lack of
outcomes and long-term data
• We need to consider uncertainty and resulting VOI when making
decisions, not just expectations based on current data
• How should we do this?
Maybe we shouldn’t abandon frequentist hypothesis testing just yet?
The FDA view
“...A reasonable basis for a claim [of cost-effectiveness]
depends on a number of factors relevant to the benefits and
costs of substantiating a particular claim. These factors
include: the type of product, the consequences of a false
claim, the benefits of a truthful claim, the costs of
developing substantiation for the claim ...”
Potential Industry Responses to Approval
based on Value of Information
Reduce cost of uncertainty by research or price reduction
Trade-off between:
•Additional research
-Cost, delay and uncertain outcome
-Entry and free rider
•Price reduction
-Reduces EVI (for payoffs > 0) but reduces revenues
Approval Based on Expected Net Value of
Sample Information
Approve new (more expensive?) treatment if expected net
benefit of treatment is greater than existing treatment
and expected net value of further sample Information is
zero
Approval Based on Expected Net Value of
Sample Information
• Hard to define set of endpoints, study designs and sample space
over which we calculate value of sample information
• ENVSI is uncertain and will change as data become available.
When is ENVSI defined?
• Many of the parameters required to estimate ENVSI are
uncertain and may not be transparent
• Non-financial capacity restraints on further research
• What decision do we make in the interim? - Sunk costs,
irreversibility and option value
Approval based on Expected Value of Perfect
Information
Population EVPI for pricing decisions
£5,000,000
£4,500,000
Price = £24
£4,000,000
Price = £12
Population EVPI
£3,500,000
£3,000,000
£2,500,000
£2,000,000
£1,500,000
£1,000,000
£500,000
£0
£0
£10,000
£20,000
£30,000
£40,000
£50,000
£60,000
Cost-effectiveness threshold
£70,000
£80,000
£90,000
£100,000
Summary results of the NICE pilot study
Case Study
Patient Group
Population EVPI
AMD Screening
Visual acuity 20/40
Visual acuity 20/80
£6.2m
£15.3m
Quality of life with and without
PDT (£3,370,000 for 20/40)
Glycoprotein IIb/IIIa
Acute treatment following
non-ST-elevation acute
coronary syndrome
(scenario 2)
£171m
Relative risk of death for non
acute PCI for GPA as medical
management and for Clopidogrel
(£85,041,000, and £68,137,000
respectively)
Clopidogrel and
dipyridamole for
secondary prevention
Stroke
Transient Ischaemic Attack
Myocardial Infarction
Peripheral Arterial Disease
(scenario 2)
£865m
£250m
£710m
£240m
Relative risks of vascular and non
vascular death
(£780m for ASA-MR-dipridamole
compared to clopidogrel in the
stroke subgroup)
Neurominidase inhibitors
Otherwise healthy adults
not at elevated risk of
complications
£66.7m
Quality of life with influenza, the
effect of oselatimivir and
amantadine (£44.3m, £0.43m and
£0.23m respectively)
Liquid Based Cytology
Women aged 18 to 64
years (scenario 3)
Disease modifying
therapies for multiple
sclerosis
Relapsing remitting and
primary progressive
multiple sclerosis (scenario
2)
£20m
£86.2m
EVPI for parameters
Specificity (£3.6m)
Relative risk of progression for
copaxone, Betaferon and rebif
(22mg) (£14m, £13.6m and £7m
respectively)
Also the cost of care, costs of
relapse and quality of life (£10m,
£7m and £6m respectively)
Approval based on Expected Value of Perfect
Information
Approve new therapy if Expected Value of Perfect
Information is below a given threshold at an acceptable
cost-effectiveness threshold
• Requires an arbritary EVPI threshold for approval
• Parameters still uncertain. For example; relevant time
horizon, future technological change.
Approval based on Decision Uncertainty
Cost-effectiveness acceptability curves for pricing decisions
1
0.9
Probability cost-effective
0.8
0.7
0.6
0.5
0.4
0.3
Price = £24
0.2
Price = £12
0.1
0
£0
£10,000
£20,000
£30,000
£40,000
£50,000
£60,000
Cost-effectiveness threshold
£70,000
£80,000
£90,000
£100,000
Approval based on Decision Uncertainty
Approve new therapy if decision uncertainty is below a
given threshold at an acceptable cost-effectiveness
threshold
• Requires an arbritary uncertainty threshold for approval
Some Challenges
• How we consider uncertainty when decision making will
influence the availability of evidence
• How do we frame explicit decision rules incorporating
uncertainty?