Is it time to ban PSA?
In support of OFSA…
Stirling Bryan, PhD
Disclosures and Acknowledgements
• I am not aware of any actual or potential conflicts of
interest in relation to this presentation.
• Some of my relevant current activities:
– Chair, CADTH’s Health Technology Expert Review Panel
– Member, CADTH’s Economic Evaluation Guidelines Working
Group
– Member, BC’s Health Technology Assessment Committee
– Director, VCH’s Centre for Clinical Epidemiology & Evaluation
• Collaborators on this topic
– Drs. Pelham Barton and Lazaros Andronis (University of
Birmingham, UK)
2
3
£1,000
£800
Incremental Cost
£600
£400
£200
-0.10
-0.05
£0
0.00
0.05
0.10
0.15
0.20
-£200
-£400
Incremental QALYs
1
0.9
Probability of being cost effective
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
£0
£1 0,000
£20,000
£30,000
Willingness to pay per additional QALY
£40,000
£50,000
£60,000
4
5
6
Definitions
• Sensitivity analysis (SA):
– Analysis that aims to assess and determine the influence
of input parameters on the outcomes of an economic
evaluation
• Probabilistic sensitivity analysis (PSA):
– A form of SA in which probability distributions are applied
to input parameters and parameter values drawn at
random from these distributions to generate an empirical
distribution of the relevant measure of cost-effectiveness
• Old-fashioned sensitivity analysis (OFSA):
– A form of sensitivity analysis in which the input
parameters are varied using alternative plausible values
7
Bernard et al (2011)
8
The argument…
• The growing influence of economic analyses (at
CADTH, NICE…) emphasises the importance of
methodological rigour
• The appropriate characterisation of uncertainty is
an essential component in economic analysis
• And so…
– Probabilistic analysis (PA) should be the base case
analysis, and so let’s ban PSA!
– OFSA has great value and should be based on PA
9
Review of NICE policy documents
• Many instances where a range of results from
sensitivity analysis was reported
– SA results cited in policy documents come mainly
from OFSAs
• Explicit reference is made in some documents
to the usefulness of the sensitivity analysis,
particularly OFSA
10
Interviews: NICE committee members
• “The most useful thing about the economic analysis is
it focuses the discussion on what actually matters…
What is it that can actually switch a decision…?” [3]
• “The other area… people do struggle with is
uncertainty. And I don’t think anybody quite knows
how to handle cost-effectiveness acceptability curves...
For some people it’s a technical problem of
understanding what they are. For those who
understand what they are, they still don’t understand
what they should do with it.” [4]
11
Conclusions
• Both PA and OFSA have their place
• PSA has no place…
• Effective communication between analysts
and policy makers remains a challenge
12