Centre for Health Economics
The determinants of PBAC recommendations to fund
drugs in Australia: an empirical analysis 1994-2004
Presented by: Anthony Harris
Centre for Health Economics, Monash University
Geoff Chin and Jing Jing Li
Centre for Health Economics, Monash University
Suzanne Hill, WHO
Emily Walkom, University of Newcastle
Funded by NHMRC project grant
www.buseco.monash.edu.au/centres/che
Context to pharmaceutical subsidy
decisions in Australia
•
National Health Act 1953 that set up the PBAC sets framework
– “For the purpose of deciding whether to recommend to the
Minister that a drug … be made available as pharmaceutical
benefits …., the Committee shall give consideration to the
effectiveness and cost of therapy involving the use of the drug,
… by comparing the effectiveness and cost of that therapy with
that of alternative therapies, whether or not involving the use
of other drugs or preparations.
– The Minister for Health and Ageing cannot list a medicine on the
PBS that has not been recommended by the PBAC.
– Since 1954 the PBAC has considered comparative effectiveness of
a drug prior to subsidy under PBS
– Since 1992 it has also formally considered value for money
•
•
1947: 139 life-saving drugs at cost A$300,000.
2005: >650 drugs at cost $5.8 billion
2
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
PHARMACEUTICAL EXPENDITURE
• 80% OF THE COST is DIRECTED
TOWARDS CONCESSIONAL
CARDHOLDERS
• PATIENT CONTRIBUTIONS % OF TOTAL
COSTS
–
–
–
–
19.5% in 90/91,
22.2% in 94/95
17.0% in 01/02
14.2 %in 02/03
3
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
Australian Govt Health Expenditure ($29.7bn)
2002-2003
Other
health
PBS
16.1%
27.5%
29.5%
26.9%
Public
hospitals
MBS
4
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
Government expenditure and volume of medical services and
pharmaceuticals in Australia 1993/4 to 2002/3
Medical services government expenditure $ million
Pharmaceutical government expenditure $ million
Medicare services volume
Pharmaceutical subsidised scripts
Millions of Scripts/services
$'000 million
9
250
8
200
7
6
150
5
4
100
3
2
50
1
Centre for Health Economics
5
0
1993/1994
1994/1995
1995/1996
1996/1997
1997/1998
1998/1999
1999/2000 2000/2001 2001/2002 2002/2003
www.buseco.monash.edu.au/centres/che
0
Aims of paper
1. To quantify the factors influential in decisions
to reimburse drugs on the Pharmaceutical
Benefits Scheme in Australia
2. To assess the extent to which confidence in the
strength of evidence on effectiveness and costs
modifies the influence of cost and cost
effectiveness
3. Assess the relevance of other factors not in
stated objective
4. Focus on drugs considered on the basis of
incremental cost per QALY
6
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
Previous Australian studies on the impact of
economics on decision making in drugs
• George et al (Pharmacoeconomics 2001)
showed that PBAC in the first few years the
PBAC had been influenced by economic data
• Hill Mitchell and Henry in JAMA 2000
emphasized the intensity of the review process
and shown that there is considerable
uncertainty introduced by the quality of the
clinical and economic evidence
7
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
Pharmaceutical Benefits Scheme approval
process in Australia
8
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
Data
• There were 858 major submissions to the
PBAC between Feb 1994 and Dec 2004.
• There were 182 (21%)major submissions
that calculate either a cost per QALY or cost
life year gained (50% of which 2001-2004)
• Of all major submissions Feb 1994 to Dec
2003 37% were recommended by the PBAC
to be listed on PBS at the price proposed.
Of those with a cost per QALY or LYG 31%
9
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
PBAC recommendations of list at price proposed
1994-2004 Cost per QALY
Cost per Quality Adjusted Life Years
Gained $A
99000
89000
79000
Accept
69000
Reject
59000
49000
39000
29000
19000
9000
-1000
Submissions ranked by cost per QALY
10
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
Influences beyond cost effectiveness:
confidence in estimates
•
Evidence of differential effectiveness
1. Size of difference in effect
2. Strength of evidence
i. Precision of effect;
ii. Level of evidence;
iii. Quality of evidence (bias)
•
3. Relevance of evidence (population and comparator)
Modeling of costs and effects beyond experimental evidence
1. Relevance of model structure
2. Measurement and extrapolation of outcomes beyond
experimental evidence
3. Measurement and extrapolation costs beyond experimental
evidence
4. Precision of net benefits
11
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
Other potential modifying influences on decision
making
• Severity of condition
– Life threatening
• Availability of alternative therapies
• Cost to patient
• Cost to government
– Impact on budget
– Financial risk
12
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
Statistical modeling the decision to fund
•
Test the a priori hypothesis that probability of “ a recommendation that a drug
be listed on the PBS at the price proposed” determined by the PBAC view
of
–
–
–
–
–
Incremental cost per QALY
Average utility gain of patients
Annual predicted cost to government
Highest cost per QALY in sensitivity analysis
Severity of condition:
> Life threatening illness or not (expected survival <5 years); or
> (QALY baseline commentary or HoDAR utility baseline scores)
– Availability of effective alternatives last line treatment with placebo as accepted
comparator
– Confidence in clinical claim – strength of evidence
> Clinical significance, precision, level, quality, relevance
– Confidence in economic claim
> Model validity, outcome measurement, cost measurement
– Resubmission
•
Using probit regression of P.recommend list on these exogenous variables
13
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
Data extraction process
• Double data extraction on the variables in the
statistical model by two teams of experienced
evaluators
• Primary data - PBAC minutes
– Secondary data from Economics Sub-committee
advice to PBAC
– If not definitive then last from Pharmaceutical
Evaluation Section Commentary on the submission
• Consensus achieved on values of items
14
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
Categories of confidence – clinical
Criterion
Items
Levels
Size of effect
1
2 (1= clinically
meaningful 0=
not)
Precision of effect
0.05<p>0.05
2 (1=sig. 0=not sig.)
Level of evidence
1
3 (3= head to head
RCT;2= indirect
RCT; 1 =non
randomised)
Quality of evidence
12
3 (3=high;
2=moderate;
1=low)
Relevance of
evidence
2 comparator &
population
2 (1=relevant 0=
not)
Clinical
15
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
Categories of confidence –Economic
Economic
Criterion
Items
Levels
Model validity
1
3 (1=plausible,
2=non critical, 3=
critical flaws)
Final health outcome
measured accurately
1
3 (1=plausible,
2=non critical, 3=
critical flaws)
Cost accuracy
1
3 (1=plausible,
2=non critical, 3=
critical flaws)
ICER precision
1
Highest value
16
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
12-item Quality of evidence
1
All relevant studies included? Selection bias
Yes/No
2
Randomisation (concealment)
Yes/No/NA
3
Blinding (patients, investigators, assessors)
Yes/No/NA
4
Patient follow-up (ITT), drop-outs properly accounted
5
Baseline characteristics well balanced
6
Dosage regimen appropriate
Yes/No
7
Duration of follow-up sufficient
Yes/No
8
Sample size adequate
Yes/No
9
Acceptable (surrogate/composite) outcomes
Yes/No
10
Method of analysis appropriate
Yes/No
11
Subgroup analysis appropriate
Yes/No/NA
12
Re-analysis of data (post hoc) appropriate
Yes/No/NA
Yes/No
Yes/No/NA
OVERALL QUALITY
High/Moderate/Low
17
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
Rule
For an overall quality index to be rated
“high”:
Item
2 Randomisation
3 Blinding
4 Follow-up
and
and
18
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
Results of probit model of factors influencing PBAC recommended
listing at price 1994-2004 full model cost per QALY n=103
Mean
Marginal
effect
p
Recommended at price
0.29
[.20,.38]
ICER $'0,000
4.64
-0.05
(0.00)
Cost to PBS $m
17.28
-0.01
(0.02)
Clinically significant
0.50
0.44
(0.00)
Precision of effect
0.75
-0.12
(0.22)
Level of evidence
2.79
0.05
(0.46)
Quality of studies
2.39
0.04
(0.53)
Relevance of evidence
0.56
0.13
(0.09)
Life threatening
0.17
0.37
(0.01)
Model validity
1.58
0.02
(0.72)
Modelled outcome
1.61
0.09
(0.08)
Modelled cost
2.19
-0.09
(0.08)
Last line
0.33
-0.05
(0.59)
Highest ICER $'0000
19.45
0.00
(0.05)
Resubmission
0.63
0.25
(0.00)
Resubmission x Effectiveness
0.33
-0.17
(0.06)
Effectiveness x Life threatening
0.12
-0.22
(0.01)
(*) dF/dx is for discrete change of dummy variable from 0 to 1; P>|z| correspond to the test of the underlying coefficient being 0;
Robust CI with clustering on drug name
Probit model for decision to list
Sensitivity Pr( +| Yes)
64.52%
Specificity Pr( -|no)
86.49%
Positive predictive value Pr(Yes| +)
66.67%
Negative predictive value Pr(no| -)
85.33%
20
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
Prob. of PBAC recommendation by incremental cost
per QALY by severity of condition and confidence in
Probability
the evidence
1
0.9
0.8
0.7
P
r
o
0.6
0.5
0.4
0.3
Mean
values of factors
0.2
0.1
0
0
50000
100000
21
150000
Centre for Health Economics
ICERwww.buseco.monash.edu.au/centres/che
$
Probability
1
Prob. of PBAC recommendation by incremental
cost per QALY by severity of condition and
confidence in the evidence
0.9
0.8
P
r
o
0.7
0.6
0.5
Resubmission in non life threatening disease w ith
confidence in effectiveness
0.4
0.3
Mean
values of factors
0.2
0.1
0
0
50000
100000
ICER $
22
150000
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
Probability
1
Prob. of PBAC recommendation by incremental
cost per QALY by severity of condition and
confidence in the evidence
0.9
0.8
0.7
P
r
o
Resubmission in life threatening disease w ith confidence in effectiveness
0.6
0.5
Resubmission in non life threatening disease w ith
confidence in effectiveness
0.4
0.3
Mean
values of factors
0.2
0.1
0
0
50000
100000
ICER $
23
150000
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
Conclusions
•
•
•
Cost effectiveness is a significant factor in decisions
Cost to government is also significant
The credibility and relevance of the clinical effectiveness evidence
–
•
•
Similar results for submissions with cost per life year gained
Other aspects of evidence (quality of studies) do not appear to be significant
–
•
Possible co- linearity with clinical effectiveness for validity;
Timing?
Severity of the underlying condition as measured by threat to life can be
decisive
–
–
•
not that the PBAC thinks them unimportant - there may be selection or co- linearity
effects
Model less significant factor
–
–
•
Uncertainty is more than just precision on ICER but might be structural ( not easily resolved by Monte
Carlo techniques)
Formal rule of rescue factor introduced in 2002 although rarely used
Availability of (effective) alternatives not significant (or interaction)
Resubmissions are more likely to be recommended ceteris paribus
– Either the committee is persuaded by the weight of evidence, or they are
overwhelmed by the weight?
24
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
Cost per life year gained
DECISION TO LIST COST
PER LIFE YEAR GAINED
N=105
MEAN
VALUE
LYG
DY/DX
ICER $’0000
10.33
-0.016
(-0.031 0.001)
COST TO PBS PER YEAR $
‘000,000
15.324
0.003
(-0.001
0.007)
CLINICALLY SIGNIFICANT
EFFECT
.552
0.372
(0.205
0.540)**
LIFE THREATENING
.448
P[Y=1|X=1] LESS
P[Y=1|X=0)
25
ROBUST 95%
CI
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
Strength and quality of clinical evidence in cost
per QALY or cost LYG submissions 1994-2004
Level of evidence
Quality
7.51 14.6
77.9
44.8
Appropriate population
18.4
36.8
67
Yes
No
33
High
Comparator
83.4
Precision
16.6
69.5
Clinical significance
0%
20%
Low
30.5
51
Moderate
49
40%
26
60%
80%
100%
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
12-item Quality Checklist (1994-2004)
Re-analysed data
Subgroup
10
36
(4%)
(15%)
201
(81%)
20
39
188
(8%)
(16%)
(76%)
Analysis
197
50
(80%)
Outcome
182
66
(73%)
Sample size
221
26
219
28
(89%)
Duration
(89%)
Dosage
215
32
(87%)
Baseline
203
(82%
)
Followup
17
222
27
25
(90%)
Blinding
166
8
73
(67%)
Randomisation
206
8
33
(83%)
Study inclusion
222
25
(90%)
0
50
100
150
200
No. of submissions
Yes
No
N/A
27
Centre for Health Economics
www.buseco.monash.edu.au/centres/che
250
List of items with most quality problems
(ratio of no:yes)
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
Randomisation
(LEAST problems)
Blinding
Baseline comparability
Follow-up & study inclusion
Sample size
Trial duration
Dosage
Method of analysis
Outcome (eg surrogate, composite)
Subgroup
Reanalysed data
(MOST problems)
28
Centre for Health Economics
www.buseco.monash.edu.au/centres/che