Do Evidence Review
Groups Bias NICE
Decisions?
Lorraine Versoza,
ISPOR Milan, Monday November 9th, 2015
Copyright © 2015 Context Matters, Inc. ALL RIGHTS RESERVED. CONFIDENTIAL.
NICE Single Technology Appraisal (STA) Process
Manufacturer submits evidence
ERG may ask for additional data
ERG reviews evidence and
produces a report for NICE
Consultees (clinical and patient
experts) make submissions to
NICE
NICE meets to consider all evidence and
develop a final appraisal document
Their evidence includes company’s submission,
ERG report, and submissions from consultees
Copyright © 2015 Context Matters, Inc. ALL RIGHTS RESERVED. CONFIDENTIAL.
2
Background
Evidence Review Groups (ERGs)
Aberdeen Health Technology Assessment (HTA) Group
ERGs are independent,
academic organizations.
They systematically
review the clinical efficacy
and cost-effectiveness of
products based on
manufacturer-submitted
dossiers.
BMJ-Technology Assessment Group (BMJ-TAG)
Kleijnen Systematic Reviews
Liverpool Reviews and Implementation Group (LRiG)
Peninsula Technology Assessment Group (PenTAG)
School of Health and Related Research (ScHARR)
Southampton Health Technology Assessments Centre
(SHTAC)
University of York
Warwick Evidence
West Midlands HTA Collaboration
Copyright © 2015 Context Matters, Inc. ALL RIGHTS RESERVED. CONFIDENTIAL.
3
Objective
This presentation explores how the different ERGs behave:
1. Trends in how ERGs are commissioned by NICE
2. How ERGs are related to NICE’s reimbursement decisions
3. How ERGs affect NICE’s cost-effectiveness assessments
This evaluation is important from policy and industry perspectives, as it sheds some light on
the factors that might influence NICE’s technology appraisal process.
Copyright © 2015 Context Matters, Inc. ALL RIGHTS RESERVED. CONFIDENTIAL.
4
How Do
ERGs
Behave?
Description of data sample
Copyright © 2015 Context Matters, Inc. ALL RIGHTS RESERVED. CONFIDENTIAL.
Methods
HTAs
STAs from Context Matters,
comprising 72% of the total STAs
published by NICE, published from
2005 to March 2015
Copyright © 2015 Context Matters, Inc. ALL RIGHTS RESERVED. CONFIDENTIAL.
178 reimbursement events were included
in the data set.
Exclusions
Terminated appraisals,
MTAs, Draft guidances
ERGs
Frequency
Therapeutic areas
Year of publication
6
Number of
Assessments by ERG
There was an unequal distribution in ERG frequency.
30
28
26
24
22
20
20
16
16
10
11
9
6
0
LRiG
SHTAC
Univ. of
York
Aberdeen ScHARR
HTA group
Copyright © 2015 Context Matters, Inc. ALL RIGHTS RESERVED. CONFIDENTIAL.
PenTAG
West
Midlands
HTA
BMJ-TAG
Warwick Kleijnen
Evidence Systematic
Reviews
7
Number of
Assessments by
Therapeutic Area
Oncology drugs were the most frequently assessed.
100
80
80
60
40
40
28
20
12
11
4
3
Opthalmology
Infectious
diseases
Other
0
Oncology
Cardiovascular/ Inflammation/ Neuroscience/
Metabolic
Immunology
Pain
Copyright © 2015 Context Matters, Inc. ALL RIGHTS RESERVED. CONFIDENTIAL.
8
ERGs by
Therapeutic Area
There was a difference in the therapeutic areas each
ERG evaluated.
30
Other
1
25
4
Infectious diseases
4
2
20
Ophthalmology
6
2
4
6
15
4
7
23
5
10
Neuroscience/Pain
1
1
Inflammation/ Immunology
Cardiovascular/Metabolic
2
8
5
Oncology
2
15
14
5
9
9
9
8
1
0
LRiG
SHTAC
Univ. of York
Aberdeen
HTA Group
Copyright © 2015 Context Matters, Inc. ALL RIGHTS RESERVED. CONFIDENTIAL.
ScHARR
PenTAG
West
Midlands HTA
6
1
1
3
6
2
1
BMJ-TAG
Warwick
Evidence
2
4
Kleijnen
Systematic
Reviews
9
Cumulative ERG
Assessments by
Year
There was a difference in the years that ERGs were
commissioned.
30
20
LRiG
SHTAC
Univ. of York
Aberdeen HTA Group
ScHARR
PenTAG
West Midlands HTA*
BMJ-TAG
Warwick Evidence
Kleijnen Systematic Reviews
10
0
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
* The West Midlands HTA is no longer listed as an ERG on NICE’s website.
Copyright © 2015 Context Matters, Inc. ALL RIGHTS RESERVED. CONFIDENTIAL.
10
Decisions
and ERGs
The relationship between ERGs and NICE’s
reimbursement decisions
Copyright © 2015 Context Matters, Inc. ALL RIGHTS RESERVED. CONFIDENTIAL.
Methods
NICE decision
Definition
Decision classification
Recommend
Drug is recommended in line with the marketing
authorisation from the European Medicines
Agency
Positive
Optimised
Drug is recommended for a smaller subset of
patients than stated by the marketing
authorisation
Positive
Only in Research
Drug is recommended for use only in the context
of a research study (clinical trial)
Positive
Not recommended
Drug is not recommended
Negative
Copyright © 2015 Context Matters, Inc. ALL RIGHTS RESERVED. CONFIDENTIAL.
12
Reimbursement
Decisions
The majority of reimbursement decisions were positive.
Positive
decisions
32%
68%
Copyright © 2015 Context Matters, Inc. ALL RIGHTS RESERVED. CONFIDENTIAL.
Negative
decisions
13
Reimbursement
Decisions by ERG
There was a correlation between ERG and decision
(p=0.024).
Positive decisions
11%
15%
17%
19%
27%
Negative decisions
27%
35%
39%
42%
69%
89%
85%
83%
81%
73%
73%
65%
61%
58%
31%
Warwick
Evidence
(n=9)
SHTAC
(n=26)
Kleijnen
West
Aberdeen BMJ-TAG
Systematic Midlands HTA Group (n=11)
Reviews
HTA
(n=22)
(n=6)
(n=16)
Copyright © 2015 Context Matters, Inc. ALL RIGHTS RESERVED. CONFIDENTIAL.
ScHARR
(n=20)
LRiG
(n=28)
Univ. of
York
(n=24)
PenTAG
(n=16)
14
Decisions by ERG:
Controlling for
Therapeutic Area
In logistic regression, when controlled for therapeutic
area:
SHTAC: 8x odds of positive reimbursement decisions
West Midlands HTA: 10x odds of positive reimbursement
decisions
No difference was found for the 8 other ERGs.
Copyright © 2015 Context Matters, Inc. ALL RIGHTS RESERVED. CONFIDENTIAL.
15
ICER Analysis
Manufacturer base-case ICER compared to the
NICE most-plausible ICER
Copyright © 2015 Context Matters, Inc. ALL RIGHTS RESERVED. CONFIDENTIAL.
Methods
Determination of ICER difference:
NICE most-plausible ICER – Manufacturer base-case
ICER
Ex: Lung Cancer, Xalkori (Sept 2013)
NICE’s
Manufacture
mostComparator r base-case
plausible
ICER
ICER
ICER
difference
Docetaxel
£41,544
£100,001 £58,457
BSC
£35,455
£50,201
£14,746
Assumptions:
The difference between NICE’s most-plausible ICER
and the manufacturer’s base-case ICER can be
attributed to the ERG report.
A large ICER difference indicates a stricter ERG
assessment of the economic model.
A small ICER difference indicates ERG agreement with
the manufacturer’s economic analysis.
Copyright © 2015 Context Matters, Inc. ALL RIGHTS RESERVED. CONFIDENTIAL.
17
ICER Difference
On average, NICE’s most-plausible ICER was
£6,200/QALY more than the manufacturer’s base-case
ICER.
30,000
20,000
14404
10,000
£6,200
4622
0
-10,000
6322
5405
7201
7918
8171
9077
-1286
-8424
-20,000
-30,000
Kleijnen
Systematic
Reviews
(n=4)
ScHARR
(n=21)
Aberdeen
HTA Group
(n=16)
Warwick
Evidence
(n=10)
Copyright © 2015 Context Matters, Inc. ALL RIGHTS RESERVED. CONFIDENTIAL.
LRiG
(n=26)
SHTAC
(n=17)
West
Univ. of York
Midlands HTA
(n=32)
(n=17)
BMJ-TAG
(n=9)
PenTAG
(n=15)
18
Conclusions
Implications for the objectivity of the NICE approval
process.
NICE commissioned ERGs differently.
They were commissioned for different therapeutic areas.
They were commissioned during different years.
NICE’s decisions were different across ERGs.
When controlled for therapeutic area, SHTAC and West Midlands HTA were more likely to be associated with
positive decisions.
On average, manufacturers underestimated their drugs’ ICERs by £6,200/QALY.
There was wide range in ICER differences across ERGs.
Maximum: PenTAG (£14,000); Minimum: Kleijnen Systematic Reviews (-£8,424)
What do these differences mean for manufacturers?
Copyright © 2015 Context Matters, Inc. ALL RIGHTS RESERVED. CONFIDENTIAL.
19
Thank You.
Your Blueprint for Market Access™
For more information:
Lorraine Versoza
Director, Reimbursement Data
[email protected]
Copyright © 2015 Context Matters, Inc. ALL RIGHTS RESERVED. CONFIDENTIAL.