PCN9
FROM INDIRECT EVIDENCE TO DIRECT EVIDENCE: A REAL - WORLD EXAMPLE FOR THE VALUE
OF INDIRECT TREATMENT COMPARISONS IN SECOND-LINE NSCLC THERAPY
Poster Content: ISPOR 16 th Annual International
1 Meeting
Poster Session III; Wednesday, May 25, 2011 8:00 AM - 3:00 PM
Schwander B , Chouaid C2, Vergnenegre A 3, Bischoff H4, Nuijten M5, Siebert U 6, Walzer S1
AiM GmbH - Assessment in Medicine, Research and Consulting, Loerrach, Germany; 2Hôpital Saint Antoine, Paris, France; 3Hôpital du Cluzeau Centre Hospitalier Universitaire de Limoges, Limoges, France; 4Hospital of Thoracic Diseases, University Hospital Heidelberg, Heidelberg, Germany;
5
6
Ars Accessus Medica,In Amsterdam
(Jisp),
Netherlands;
UMIT trials
- University
for Health
table 1 the key
information
on these phase-III
is summarized
togetherSciences, Medical Informatics and Technology / ONCOTYROL - Center for Personalized Cancer Medicine, Hall i.T. / Innsbruck, Tyrol, Austria
1
with the overall survival outcomes used as the basis for performing the ITCs
ISPOR 17th Annual International Meeting
(Evidence WAVE 1 and Evidence WAVE 2) or for validating these ITC results
(Evidence WAVE 2 and Evidence WAVE 3).
Table 1: Overview of phase-III trials of docetaxel, pemetrexed and
• The ITCs performed are described below according to the different evidence ‘WAVES’:
Table
1: Overview
phase-III
trials of
docetaxel, subdivided
pemetrexed and into
erlotinibspecific
in 2L NSCLCevidence
in of2L
NSCLC
therapy
erlotinib
Introduction & Objective
• Often new treatment options lack comparisons to treatment options which are already on
the market and which were launched several years ago, due to the fact that during
setting-up a phase-III trial the ‘current’ regulatory standard therapy is usually selected as
the comparator.
• Especially in phase-III trials with a long observation period (e.g. in oncology trials focusing
on the potential primary endpoint overall survival) the ‘current’ standard comparator may
have changed by the time the phase-III trial is finalized. This does not usually impact
regulatory approval of a healthcare intervention.
• However, it is often a critical issue when facing the hurdle set by health care payers, as they
might like to see efficacy evidence in comparison to the ‘current’ standard therapy before
deciding on reimbursement. • In these cases when there is a lack of head-to-head evidence, and due to the growing
number of requests from health care payers, indirect treatment comparisons (ITC) are
increasingly being performed.
• Although ITC methods are widely accepted, the results are often interpreted with caution,
potentially because of their lack of external validity proven by real clinical studies.
• In order to start filling the data gap ITCs have been performed and validated by real clinical
studies using an example in second-line non-small cell lung cancer (NSCLC) therapy. In this
assessment we aim to compare the outcomes of the clinical evidence development over
time, moving from indirect to direct evidence, in order to deliver an example for the external
validation of ITC findings.
Underlying Clinical Data
• Currently there are only three second-line (2L) NSCLC therapy options available: The oral
epidermal growth factor receptor (EGFR) tyrosine kinase inhibitor (TKI) erlotinib and the two
intravenous chemotherapies docetaxel and pemetrexed [1;2]. In squamous cell NSCLC
there are only two second-line options available, namely erlotinib and docetaxel (as
pemetrexed is not licensed for this patient population).
• In the following context the evidence development for these three therapies is outlined,
subdivided into specific evidence ‘waves’ according to the phase-III randomized controlled
trial (RCT) data availability, as shown in figure 1.
• Evidence WAVE 1 (years 2000-2005): The first available pivotal phase-III trial for docetaxel
(TAX 317) [3] in 2L NSCLC included best supportive care (BSC) as a comparator, which was
also used as comparator in the erlotinib phase-III trial (BR.21) [4]. In contrast the
pemetrexed pivotal phase-III trial (JMEI) [5] provided direct evidence of pemetrexed vs.
docetaxel. • Evidence WAVE 2 (year 2010): Another phase-III trial (HORG) [6] was published comparing
erlotinib vs. pemetrexed in 2L NSCLC patients.
• Evidence WAVE 3 (year 2012): A further phase-III trial (TITAN) [7] was published directly
comparing erlotinib vs. docetaxel or pemetrexed.
Figure 1: Phase III RCTs according to the specific evidence waves
Evidence Wave
E V ID E N C E W AV E 1
Pub. Year
Phase-III RCTs
2004
2000
Evidence
Pub. Year
Trial Name
Study Arm
WAVE 1
2004
JMEI
2000
TAX 317
2005
BR.21
Evidence WAVE 1
WAVE 2
2010
HORG
WAVE 3
2012
TITAN
BSC
100
PEM
283
DOX
288
ERL
488
BSC
243
ERL
166
PEM
166
ERL
203
DOX/PEM
221
61
37-73
61
28-77
59
22-81
57
28-87
62
34-87
59
32-89
65
37-83
66
42-86
59
36-80
59
32-89
N
Male
Female
64
36
65
35
69
31
75
25
65
35
66
34
81
19
82
18
79
21
72
28
BR.21 Trial
• ITC 3: The ITC of erlotinib vs. docetaxel was re-performed (using pemetrexed as
efficacy connector) on the basis of the HORG and the JMEI trials, as shown in figure 2.
ECOG Performance status (% of patients)
0 or 1
2
3
75
25
0
75
25
0
89
11
0
88
12
0
66
25
9
69
23
9
89
11
0
81
19
0
81
19
0
79
21
0
100
0
100
0
51
49
50
50
54
46
61
39
100
0
100
0
WAVE 1 ITC
80
20
76
24
Overall Survival
Comparison
OS HR**
95%CI
p-value
Phase-III RCTs
DOX vs BSC
0.56
0.35-0.88
p=0.01
PEM vs DOX
0.97
0.81-1.15
NA
ERL vs BSC
0.70
0.58-0.85
p<0.001
ERL vs PEM
0.96***
0.77-1.21***
p=0.916
ERL vs DOX/PEM
0.96
0.78-1.19
p=0.73
Study arms
WAVE 2 ITC
WAVE 1 ITC
Erlotinib vs. DocetaxelPhase-III
(ITC 1) RCTs
• One key difference observed between the phase-III trials is the performance status
One key difference observed between the phase-III trials is the performance
proportion: the BR.21 study includes patients with a performance status (PS) 0-3, while the
Study
arms1 ITC
status proportion: the BR.21 study includes patients with a performance
WAVE
BR.21 trialstatus
TAX
317 trial RCTs
other trials focus on patients with a PS of 0-2. The HORG trialPhase-III
and the RCTs
JMEI study are very
Phase-III
(PS)as0-3,
other
trials
focus on
patients
a PS of 0-2. The HORG trial
similar,
bothwhile
studiesthe
include
a high
proportion
of patients
withwith
PS 0-1.
Erlotinib
• In addition, the percentage of patients who have already failed a second-line therapy is
considerably higher in BR.21 and HORG compared to TAX 317, JMEI and TITAN.
4
Methods
Indirect
treatment
Docetaxel
BSC
Study
arms
BSC
comparison
Phase-III
RCTs
BSC
Indirect treatment
comparison
Indirect treatment
Study arms
comparison
Erlotinib
Docetaxel
Indirect treatment
B S C (B est S upportivecomparison
Care)
is used as efficacy connector
• Using the phase-III trials’ overall survival hazard ratios (shown in table 1) indirect treatment
comparisons (ITC) have been performed for the different evidence ‘WAVES’ and compared
WAVE 2 ITC
WAVE 2 ITC
to the head-to-head outcomes from the HORG and the TITAN trials.
Erlotinib vs. Pemetrexed (ITC 3)
• The overall survival hazard ratios (OS HRs) and the related 95% confidence intervals
WAVE
2 ITC
HORG trial
JMEI trial RCTs
Phase-III
Phase-III
RCTseffect measure
(95%CIs) were selected as the preferred outcome for the ITC,
as this
accounts for censoring and incorporates time to event information [8].
Erlotinib
Study arms
Pemetrexed
• The indirect treatment comparison of OS outcomes uses the most widely applied indirect
comparison approach by Bucher et al 1997 [9]. The Canadian
Agency for Drugs and
Indirect treatment
Technologies in Health [10] and others [11;12] have identified
this method as the most
comparison
Erlotinib
suitable approach for performing indirect treatment comparisons of RCTs.
Docetaxel
Study Pemetrexed
arms
Phase-III RCTs
P emetrexed
Indirect treatment
Study arms
comparison
Docetaxel
Pemetrexed Indirect treatment
comparison
is used as efficacy connector
BR.21 trial
Erlotinib
TAX 317 trial
Docetaxel
BSC
BSC
Erlotinib
DOX
BSC
PEM
DOX
BSC
ERL
∏ HRA A
i
i =1
E V ID E N C E W AV E 2
Pub. Year
Phase-III RCTs
E V ID E N C E W AV E 3
2010
2012
H O R G Trial
T ITA N Trial
(
exp
k –1
Σ In(HRA A ) +Ζα/2
i =1
i
i+1
k –1
)
Σ Var (In (HRA A ) )
i =1
i
i+1
K treatments: A1, A2,... Ak; HRAiAi+1= hazard ratio for treatment Ai and Ai+1
Study Arms
ERL
PEM
ERL
DOX/PEM
DOX=docetaxel; BSC=best supportive care; PEM=pemetrexed; ERL=erlotinib
• In table 1 the key information on these phase-III trials is summarized together with
the overall survival outcomes used as the basis for performing the ITCs (Evidence WAVE 1
and Evidence WAVE 2) or for validating these ITC results (Evidence WAVE 2 and Evidence
WAVE 3). • In order to test the statistical significance, p-values have been calculated by means of a
two-sided z-test, using the methodology of Snedecor and Cochran 1989 [13]. The null
hypothesis that the efficacy outcomes of the compared therapy options are equal, is
rejected if p<0.05.
• All calculations have been performed in Excel 2010. The ITC calculations can be
re-performed using the ITC tool [14] available from the Canadian Agency for Drugs and
Technologies in Health, ensuring maximum transparency.
Docetaxel vs. pemetrexed (H O R G )
E rlotinib
D ocetaxel
Docetaxel
HORG trial
JMEI trial
Erlotinib
vs. Docetaxel(ITC
(ITC2)1)
Erlotinib
vs. Pemetrexed
Docetaxel Pemetrexed
TAX
317trial
trial
JMEI
Erlotinib vs. Docetaxel (ITC 1)
P emetrexed
Docetaxel
BSC
Erlotinib
BSC
Pemetrexed
Docetaxel
Docetaxel
BR.21 trial
TAX 317 trial
Erlotinib
Docetaxel
BSC
D ocetaxel
Docetaxel
BSC
Erlotinib
BSCPemetrexed
is used as efficacy connector
Erlotinib
Docetaxel
Pemetrexed
BSC
B S C (B est
S upportive Care)
Docetaxel
isErlotinib
used as efficacy connector
Docetaxel
B S C (B est S upportive Care)
is used as
connector
Erlotinib
vs.efficacy
Pemetrexed
(ITC 3)
HORG trial
Pemetrexed
Docetaxel
is used as efficacy connector
0.00
Erlotinib vs. Pemetrexed (ITC 2)
IT C 1 result
Erlotinib
Favors erlotinib
1.00
2.00
2.50
Favors chemotherapy
OS Hazard Ratio
Discussion
Pemetrexed Docetaxel
JMEI trial
•D ocetaxel
According to the findings
Pemetrexed Docetaxel
Docetaxel
Erlotinib
Erlotinib
of the indirect treatment comparisons performed there is no
significant difference observed when indirectly comparing the overall survival outcomes of
Pemetrexed
the oral EGFR
TKI erlotinib vs. the chemotherapies docetaxel and pemetrexed in 2L NSCLC
D ocetaxel
therapy.
Docetaxel
isErlotinib
used as efficacy connector
Pemetrexed
• These ITC findings were confirmed by recently published head-to-head outcomes [6,7].
Docetaxel
This head-to-head evidence shows a slight non-significant tendency in favor of erlotinib
•
is used as efficacy connector
Docetaxel Pemetrexed
JMEI trial
P emetrexed
Docetaxel
Pemetrexed
1.50
JMEI trial
Erlotinib
Docetaxel
IT C 1 result
Erlotinib vs. Pemetrexed (ITC 3)
Erlotinib
0.50
Erlotinib vs. Pemetrexed (ITC 2)
JMEI trial
Erlotinib
Pemetrexed
HORG trial
0.96 (0.78-1.19): p=0.730
E rlotinib vs. chemotherapy (T ITA N )
Erlotinib
Erlotinib vs. Pemetrexed (ITC 3)
Erlotinib
Pemetrexed
trial
ITBR.21
C 1 result
0.96 (0.77-1.21): p=0.916
Pemetrexed
E vidence W AV E 3
BSC
Pemetrexed
Docetaxel
P emetrexed
Pemetrexed
isErlotinib
used as efficacy connector
Docetaxel
i+1
Formula for indirect treatment comparison 95% confidence interval:
Evidence Wave
0.95 (0.71-1.28): p=0.736
JMEI trial
Docetaxel
k –1
Study Arms
1.26 (0.74-2.15): p=0.392
H ead-To-H ead E vidence
IT C 1 result
Pemetrexed
is used as efficacy connector
Formula for indirect treatment comparison hazard ratio (HR):
Erlotinib vs. pemetrexed (ITC 2)
Erlotinib vs. Pemetrexed (ITC 2)
B S C (B est S upportive Care)
is used as efficacy connector
which shows the highest median age and the highest
proportion
WAVE
1 ITC of males.
Study arms
Erlotinib vs. Docetaxel (ITC 1)
Erlotinib
• Looking
at theatpatient
characteristics
of the phase-III
trialsphase-III
the patients’
agethe
andpatients’
gender age and
Looking
the patient
characteristics
of the
trials
distribution
fairly comparable,
with comparable,
the exception ofwith
the HORG
trial which shows
theHORG trial
gender are
distribution
are fairly
the exception
of the
highest median age and the highest proportion of males.
1.25 (0.76-2.06): p=0.381
Erlotinib vs. docetaxel (ITC 3)
Indirect treatment
comparison
*In TAX 317 also DOX 100 mg/m² was investigated – but this dosage is not in the label and therefore
*Innot
TAX 317
also DOX
mg/m² was**
investigated
butin
thisfavor
dosage of
is not
in the
label and therefore
not taken
** OS
taken
into100account;
OS HR–is
the
treatment
named
firstinto
in account;
the ‘comparison’
row;
HRDOX=docetaxel;
is in favor of the treatment
named first
in the ‘comparison’
DOX=docetaxel; BSC=best
supportive care;ECOG=Eastern
PEM=pemetrexed; Cooperative
BSC=best
supportive
care;row;
PEM=pemetrexed;
ERL=erlotinib,
Oncology ECOG=Eastern
Group; OS=overall
HR=hazard
ratio;
95%CI=95%
confidence
ERL=erlotinib,
Cooperativesurvival;
Oncology Group;
OS=overall
survival;
HR=hazard ratio;
95%CI=95%interval;
confidence **HORG trial
HRs have
determined
by reading
outoutthe
published
Kaplan-Meier
HR based
interval;
***HORGbeen
trial HRs
have been determined
by reading
the published
Kaplan-Meier
curves ascurves
HR basedas
on patient
level on patient
level
have
not been published
data
havedata
not been
published
Erlotinib vs. docetaxel (ITC 1)
E vidence W AV E 2
Figure 2: ITC approach for Evidence WAVE 1 and Evidence WAVE 2
Prior Regimens (% of patients)
1
≥2
E vidence W AV E 1
Evidence WAVE 2
Gender (% of patients)
ITC HR (95%CI); p-value
Indirect E vidence
• ITC 2: The result of ‘ITC 1’ was used to compare erlotinib indirectly to pemetrexed
(using docetaxel as efficacy connector) on the basis of the JMEI trial, as shown in
figure 2.
Age in Years
Median
Range
OS HR of erlotinib vs. chemotherapy
• ITC 1: Erlotinib was indirectly compared to docetaxel (using best-supportive care as
efficacy connector) on the basis of the BR.21 and the TAX 317 trials.
DOX*
55
• The related ITC formulae applied are shown below:
2005
JM E I Trial
TA X 317 Trial
therapy subdivided into specific evidence ‘WAVES’
‘WAVES’
Figure 3: Indirect treatment comparison and head-to-head comparison results for Evidence WAVE 1, WAVE 2 and WAVE 3 in 2L NSCLC therapy
Results
• ITCs - Evidence WAVE 1: Comparing erlotinib vs. docetaxel (ITC 1) resulted in an ITC-OS HR
of 1.25 (95%CI: 0.76-2.06, p=0.381). Using these ITC results to compare erlotinib to
pemetrexed (ITC 2) resulted in an ITC-OS HR of 1.26 (95%CI: 0.74-2.15, p=0.392).
[6,7]. The quality of this evidence is high as two phase-III RCTs have independently
demonstrated comparable outcomes. 1 M
aione P, Rossi A, Bareschino MA, et al. Factors Driving the Choice of the Best Second-Line Treatment of
Advanced NSCLC. Rev Recent Clin Trials 2010 Sep 22.
2 S cagliotti GV. Beyond first-line treatment: expanding the options. 2nd International Thoracic Oncology
Congress Dresden (ITOCD); Session: Treatment strategies to extend survival in advanced NSCLC; 2010.
3 S hepherd FA, Dancey J, Ramlau R, et al. Prospective randomized trial of docetaxel versus best supportive
care in patients with non-small-cell lung cancer previously treated with platinum-based chemotherapy. J Clin
Oncol 2000 May;18(10):2095-103.
4 S hepherd FA, Rodrigues PJ, Ciuleanu T, et al. Erlotinib in previously treated non-small-cell lung cancer. N Engl
J Med 2005 Jul 14;353(2):123-32.
• These head-to-head findings have been confirmed by the ITC results for WAVE 2 (ITC 3),
whereas looking at the ITCs performed for Evidence WAVE 1 (ITC 1 & ITC 2) a nonsignificant tendency in favor of chemotherapy can be observed.
5 H
anna N, Shepherd FA, Fossella FV, et al. Randomized phase III trial of pemetrexed versus docetaxel in
patients with non-small-cell lung cancer previously treated with chemotherapy. J Clin Oncol 2004 May
1;22(9):1589-97.
• The reason for the divergence of results is most likely triggered by a mismatch of prognostic
patient characteristics between the phase-III trials used as the basis for the Evidence WAVE
1 ITCs. But this is not relevant since results are non-significant.
6 V amvakas L, Agelaki S, Kentepozidis NK, et al. Pemetrexed (MTA) compared with erlotinib (ERL) in pretreated
patients with advanced non-small cell lung cancer (NSCLC): Results of a randomized phase III Hellenic
Oncology Research Group trial. 2010 Jun 4; American Society of Clinical Oncology (ASCO) Annual Meeting
2010 p. suppl. abstract # 7519.
• The BR.21 study included more patients with a worse prognosis (performance status 2 and
3) compared to the TAX 317 and the JMEI trials which might have led to underestimating
the efficacy of erlotinib in patients with a better prognosis.
• In contrast the HORG trial and the JMEI study, used for the Evidence WAVE 2 ITC (ITC 3),
show a good match of prognostic patient characteristics (both studies included a high
proportion of patients with a good prognosis); which might well explain why the results of
this ITC are fairly in line with the head-to-head findings.
Conclusions
• ITC - Evidence WAVE 2: Re-performing the ITC of erlotinib vs. docetaxel (ITC 3), on the basis
of the HORG and the JMEI trials, resulted in an ITC-OS HR of 0.95 (95%CI: 0.71-1.28,
p=0.736).
• In conclusion no significant difference has been observed when indirectly
comparing the overall survival outcomes of erlotinib vs. chemotherapy (docetaxel
and pemetrexed) in 2L NSCLC therapy.
• Real-world data - Evidence WAVE 2 and Evidence WAVE 3: The head-to-head evidence
validated these findings with an OS HR of 0.96 (95%CI: 0.77-1.21, p=0.916) and 0.96
(95%CI: 0.78-1.19, p=0.730), comparing erlotinib vs. pemetrexed [HORG trial] and
erlotinib vs. chemotherapy (pemetrexed or docetaxel) [TITAN trial], respectively.
• Although they do not lead to significant differences, the prognostic patient
characteristics show a considerable influence on the ITC results; hence it is
recommended only to do an ITC using studies with a similar patient population.
• The ITC results as well as the outcomes of the head-to-head comparisons are shown in
figure 3.
References
• Recently published clinical head-to-head evidence [6,7] has confirmed the
appropriateness and validity of ITC findings in 2L NSCLC.
7 C
iuleanu T, Stelmakh L, Cicenas S, et al. Efficacy and safety of erlotinib versus chemotherapy in second-line
treatment of patients with advanced, non-small-cell lung cancer with poor prognosis (TITAN): a randomised
multicentre, open-label, phase 3 study. Lancet Oncol 2012 Mar;13(3):300-8.
8 W
oods BS, Hawkins N, Scott DA. Network meta-analysis on the log-hazard scale, combining count and
hazard ratio statistics accounting for multi-arm trials: a tutorial. BMC Med Res Methodol 2010;10:54.
9 B ucher HC, Guyatt GH, Griffith LE, Walter SD. The results of direct and indirect treatment comparisons in
meta-analysis of randomized controlled trials. J Clin Epidemiol 1997 Jun;50(6):683-91.
10 W
ells GA, Sultan SA, Chen L, et al. Indirect Evidence: Indirect Treatment Comparisons in Meta-Analysis.
Canadian Agency for Drugs and Technologies in Health 2009 [cited 2010 Jul 15]; Available from: URL: http://
www.cadth.ca/index.php/en/hta/reports-publications/search/publication/884
11 S ong F, Altman DG, Glenny AM, Deeks JJ. Validity of indirect comparison for estimating efficacy of competing
interventions: empirical evidence from published meta-analyses. BMJ 2003 Mar 1;326(7387):472.
12 T udur C, Williamson PR, Khan S, Best LY. The value of the aggregate data approach in meta-analysis with
time-to-event outcomes. Journal of the Royal Statistical Society 2002 Jan 6;164(2):357-70.
13 S nedecor GW, Cochran WG. Tests of hypotheses. Statistical Methods (8th Edition). Iowa, USA: Iowa State
University Press, 1989. p. 64-82.
14 W
ells GA, Sultan SA, Chen L, et al. Indirect treatment comparison software application (Version 1.0). Canadian
Agency for Drugs and Technologies in Health 2009 Available from: URL: http://www.cadth.ca/en/resources/
about-this-guide/download-software.