Challenges of Bridging Studies in
Biomarker Driven Clinical Trials
Szu-Yu Tang, Chang Xu, Bonnie LaFleur
May 19, 2015. MBSW Conference. Muncie, IN.
1

Bridging Studies in diagnostic device:
Bridge the “efficacy” in the subpopulation tested
from the clinical trial assay (CTA or old assay) to
the subpopulation tested by a companion in vitro
diagnostic device (CDx or new assay)
Clinical bridging studies : A bridging study is defined as a study performed
in the new region to provide pharmacodynamic or clinical data on efficacy,
safety, dosage and dose regimen in the new region that will allow
extrapolation of the foreign clinical data to the population in the new region
(ICH E5 guideline).
2
Outline
• Bridging study examples
• The impact of diagnostic accuracy to treatment
efficacy in enrichment trail
• Single assay
• Old assay to new assay
• Conclusion
3
Companion Diagnostic Test (CDx)
Companion diagnostic test (CDx): the use of a diagnostic assay
as a test, assay, or test system to screen, or select patients who
may be candidates for a specific drug therapy.
EARLY PHASE
ALGORITHM
DEVELOPMENT
EARLY RESEARCH
DISCOVERY
PRECLINICAL
CUT-OFF
DETERMINATION
PROTOTYPE &
DEVELOPMENT
Phase 1
VALIDATION
STUDIES
ANALYTICAL
VALIDATION
Phase 2
CLINICAL
VALIDATION
Phase 3
4
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
Bridging Study Examples
• No CDx available in clinical trial
• A laboratory-development test (LDT) is used in
the clinical trial instead of CDx
Note: The FDA defines a Laboratory Developed Test (LDT) as an in vitro
diagnostic test that is manufactured by and used within a single laboratory
(i.e. a laboratory with a single CLIA certificate). LDTs are also sometimes
called in-house developed tests, or “home brew” tests.
• Multiple CDx products
• Multiple (competing) CDx products can be driven
by improved efficiency, cost, novel/new
technologies, or updated techniques over time.
5
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
Bridging Study Challenges
Verification Bias:
• Lack of specimen material or lack of consent for
re-testing from patients
• Lack of efficacy outcome for negative group from
old assay in enrichment trial
Old Assay
New
Assay
+
-
+
a
c
-
b
d
No clinical
Outcome!!
6
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
Impact of diagnostic accuracy on treatment
efficacy (single assay)
Goal: establish quantitative relationship between
response rate and components of CDx accuracy in an
enrichment trial.
Enrichment Trial
Randomize
T
+
NT
All screened
patients
Stratified by
marker
Marker (+)
Marker (-)
Response
No Response
7
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
Impact of diagnostic accuracy on treatment
efficacy (single assay)
•
Analytical versus clinical accuracy
Analytical accuracy is sensitivity/ specificity but clinical accuracy needs
to link the assay performance to treatment efficacy.
•
Method comparison studies
Method comparison studies are used to compare the new assay with the
one currently in use to see whether their measurements are indeed
comparable.
•
Issues with method comparison studies:
Reference standard, non-identifiability, conditional independent
assumptions (CIA)…etc
8
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
The impact of diagnostic accuracy on
treatment efficacy (single assay)
We establish the relationship between response rate in an enrichment trial
(𝑟+ ) and CDx accuracy as follows:
𝑟+ = 𝑟 + 𝛿 ∗
𝜋𝑠
𝜋𝑠+(1−𝜋)(1−𝑐)
 Response rate (r)
 CDx test sensitivity (𝑠)
 CDx test specificity (𝑐)
 Prevalence of marker (+) patients (𝜋)
 Difference in efficacy between treated and control populations ()
 Response rate in marker (+) treated group adjust for device accuracy (𝑟+ )
Assumptions:
(1) Marker (+) patients are 1: 1 randomized to treatment and control group.
(2) Response rate of true marker (–) patients and also patients in the control group (𝑟)
(3) Response rate of true marker (+) patients (𝑟 + 𝛿)
(Ref[1]) Maitournam A, Simon R: On the efficiency of targeted clinical trials. Statistics in Medicine 2005; 24:329–339.
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
9
Simulation Results
•
Prevalence of marker (+) = 20%; response rate of marker (-) = 40%; response
rate of marker (+) = 60%
•
𝑟+ will be between 40% and 60%
•
Fix sensitivity at 80% and 90% and plot 𝑟+ vs. specificity; Fix specificity at 80%
and 90% and plot 𝑟+ vs. sensitivity
10
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
The impact of diagnostic accuracy on
treatment efficacy (single assay)
What we see
What it means
Dotted curves ( 𝑟+ versus
specificity for given sensitivity) are
highest when sensitivity or
specificity is over 80%
The upper limit of the response
rate is determined by specificity,
not sensitivity
Dotted curves are also higher than
the solid (RR versus specificity) at
these levels
Dotted curves have steeper slope
than solid curves when sensitivity
or specificity is large
[ When specificity = 100%, 𝑟+
reaches maximum regardless of
sensitivity. When sensitivity =
100%, 𝑟+ does not necessarily
reach maximum]
Specificity improves response rate
faster than sensitivity
11
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
Method comparison studies
The first step is to consider the “accuracy” of both assays in
establishing true marker positivity and negativity
True Biomarker Positive
True Biomarker Negative
Old Assay
Old Assay
New
Assay
+
-
+
a
c
-
b
d
New
Assay
+
-
+
a´
c´
-
b´
d´
Probability of patients(+) in the enrichment trial tested by new
assay is conditional on patients have been tested as (+/-) from old
assay:
P(new assay=+| old assay=+)
= (𝜋𝑆1 𝑆2 + 1 − 𝜋 1 − 𝐶1 1 − 𝐶2 )/(𝜋𝑆1 + 1 − 𝜋 1 − 𝐶1 )
P(new assay=+| old assay=-)
= (𝜋(1 − 𝑆1 )𝑆2 + 1 − 𝜋 𝐶1 1 − 𝐶2 )/(𝜋(1 − 𝑆1 ) + 1 − 𝜋 𝐶1 )
(Ref [2])
(Ref [2])
(Ref [2]) Lu Y, Dendukuri N, Schiller I, Joseph L: A Bayesian approach to simultaneously adjusting for verification and
reference standard bias in diagnostic test studies.Stat Med 2010, 29(24):2532-2543.
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
12
Link of assay performance to
efficacy
We establish the relationship between response rate in enrichment trial
using new assay (𝑟2+ ) and old assay accuracy as follows:
𝑖𝑛𝑑
𝑟2+|1+
𝑖𝑛𝑑
𝑟2+|1−
𝜋𝑆1 𝑆2
=𝑟+𝛿∗
𝜋𝑆1 𝑆2 + (1 − 𝜋)(1 − 𝐶1 )(1 − 𝐶2 )
𝜋(1 − 𝑆1 )𝑆2
=𝑟+𝛿∗
𝜋(1 − 𝑆1) 𝑆2 + 1 − 𝜋 𝐶1 1 − 𝐶2
 Old assay test sensitivity (𝑆1 ) / new assay test sensitivity (𝑆2 )
 Old assay test specificity (𝐶1 ) / new assay test specificity (𝐶2 )
 Prevalence of marker (+) patients (𝜋)
 Difference in efficacy between treated and control populations ()
Assumption: New assay and old assay are conditionally independent given the true
disease status.
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
13
Assumptions
Conditional independence Assumption (CIA)
The CIA means that the test and the standard are not
prone to misdiagnose the same patients:
𝑃 𝑜𝑙𝑑 𝑎𝑠𝑠𝑎𝑦, 𝑛𝑒𝑤 𝑎𝑠𝑠𝑎𝑦 𝑇𝑟𝑢𝑡ℎ = 𝑃 𝑜𝑙𝑑 𝑎𝑠𝑠𝑎𝑦 𝑇𝑟𝑢𝑡ℎ ∗
𝑃(𝑛𝑒𝑤 𝑎𝑠𝑠𝑎𝑦|𝑇𝑟𝑢𝑡ℎ)
True Positive
Old Assay
+
New
Assay
+
Example
-
When assumption holds
64
80
80
100
-
𝑃 𝑜𝑙𝑑 = + 𝐷 = + ∗ 𝑃(𝑛𝑒𝑤 = +|𝐷 = +)
= 80% * 80%
= 64% (if CIA is satisfied)
When assumption is violated
> 64%
(Ref [3]) Zhou XH, Obuchowski N, McLish D. Statistical Methods in Diagnostic Medicine, 2 nd ed. Hoboken, New
Jersey: John Wiley and Sons; 2011.
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
14
Assay performance affect
on efficacy
We establish the relationship in enrichment trial between
𝑐𝑜𝑟𝑟
response rate using new assay (𝑟2+
) and old assay
accuracy without CIA as follows:
𝑐𝑜𝑟𝑟
𝑟2+|1+
=𝑟+𝛿∗
𝑐𝑜𝑟𝑟
𝑟2+|1−
=𝑟+𝛿∗
𝜋(𝑆1 𝑆2 +𝑐𝑜𝑣𝑝)
𝜋(𝑆1 𝑆2 +𝑐𝑜𝑣𝑝)+(1−𝜋)( 1−𝐶1 1−𝐶2 +𝑐𝑜𝑣𝑛)
𝜋 (1−𝑆1 𝑆2 −𝑐𝑜𝑣𝑝)
𝜋((1−𝑆1) 𝑆2 −𝑐𝑜𝑣𝑝)+(1−𝜋)(𝐶1 (1−𝐶2 )−𝑐𝑜𝑣𝑛)
• Covp~U(0, min(S1, S2)-S1*S2)
• Covn~U(0, min(C1, C2)-C1*C2)
(Ref [2]) Lu Y, Dendukuri N, Schiller I, Joseph L: A Bayesian approach to simultaneously adjusting for verification and
reference standard bias in diagnostic test studies.Stat Med 2010, 29(24):2532-2543.
15
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
Simulation parameters
• Prevalence of marker (+) = 20%; response rate of marker (-)
and control group = 40%; response rate of marker (+) = 60%
• 𝑟2+|1+ / 𝑟2+|1− will be between 40% and 60%
• Fix sensitivity/specificity for old assay at 30%/ 50%/ 90% and
plot 𝑟2+|1+ / 𝑟2+|1− vs. specificity/ sensitivity 30%/ 50% / 90% for
old assay.
• The impact of 𝑟2+|1+ / 𝑟2+|1− also provides for different
combinations of (𝑆2 , 𝐶2 )
𝑐𝑜𝑟𝑟
𝑐𝑜𝑟𝑟
• For correlated scenario (𝑟2+|1+
/ 𝑟2+|1−
), 80% dependency is
defined as:
Covp=(min(S1, S2)-S1*S2)*0.8
Covn=(min(C1, C2)-C1*C2)*0.8
16
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
Simulation results
CIA only
•
𝑖𝑛𝑑
Conditional on the test result from old assay is positive: 𝑟2+|1+
17
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
Summary of results
CIA only
•
𝑖𝑛𝑑
Conditional on the test result from old assay is positive: 𝑟2+|1+
What we see
What it means
𝑖𝑛𝑑
• When S1, C1 are low, the 𝑟2+|1+
are • Low sensitivity or specificity of old
𝑖𝑛𝑑
assay reduces 𝑟2+|1+
.
low.
• The slope of specificity effect on
𝑖𝑛𝑑
𝑟2+|1+
is steeper than sensitivity
effect from old assay .
• The top three lines in each panel
are C2=0.9.
• The effect of old assay’s specificity
is larger than sensitivity.
• Specificity of new assay improves
𝑖𝑛𝑑
𝑟2+|1+
18
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
Simulation results
CIA only
•
𝑖𝑛𝑑
Conditional on the test result from old assay is negative: 𝑟2+|1−
19
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
Simulation results
CIA only
•
𝑖𝑛𝑑
Conditional on the test result from old assay is negative: 𝑟2+|1−
What we see
What it means
𝑖𝑛𝑑
• When S1, C1 are high, the 𝑟2+|1−
are • High sensitivity or specificity of old
𝑖𝑛𝑑
assay reduces 𝑟2+|1−
.
low.
• The slope of sensitivity effect
𝑖𝑛𝑑
on 𝑟2+|1−
is steeper than specificity
effect from old assay .
• The top three lines in each panel are
C2=0.9.
• The effect of old assay’s sensitivity is
larger than specificity.
• Specificity of new assay improves
𝑖𝑛𝑑
𝑟2+|1−
.
20
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
Simulation results
dependency of 80% compared to CIA
•
𝑐𝑜𝑟𝑟
𝑖𝑛𝑑
Conditional on the test result from old assay is positive: Diff ( 𝑟2+|1+
- 𝑟2+|1+
)
21
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
Simulation results
dependency of 80% compared to CIA
•
𝑐𝑜𝑟𝑟
𝑖𝑛𝑑
Conditional on the test result from old assay is positive: Diff ( 𝑟2+|1+
- 𝑟2+|1+
)
What we see
What it means
• When S1, C1 are low, the diff (
𝑐𝑜𝑟𝑟
𝑖𝑛𝑑
𝑟2+|1+
- 𝑟2+|1+
) are mostly
above 0 except S2=0.9.
• Compared to CIA, high dependency
between two assays enhances efficacy
when old assay’s sensitivity or
specificity is low except when the new
assay has high sensitivity.
• When S1, C1 are high, the diff (
𝑐𝑜𝑟𝑟
𝑖𝑛𝑑
𝑟2+|1+
- 𝑟2+|1+
) are mostly
below 0 except S2=0.3 and
C1=0.9.
• Compared to CIA, high dependency
between two assays reduces efficacy
when old assay’s sensitivity or
specificity is high except for the
scenario of low sensitivity in new
assay and high specificity in old assay.
22
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
Simulation results
dependency of 80% compared to CIA
•
𝑐𝑜𝑟𝑟
𝑖𝑛𝑑
Conditional on the test result from old assay is negative:diff ( 𝑟2+|1−
- 𝑟2+|1−
)
23
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
Simulation results
dependency of 80% compared to CIA
•
𝑐𝑜𝑟𝑟
𝑖𝑛𝑑
Conditional on the test result from old assay is negative:diff ( 𝑟2+|1−
- 𝑟2+|1−
)
What we see
What it means
• When S1, C1 are low, the diff (
𝑐𝑜𝑟𝑟
𝑖𝑛𝑑
𝑟2+|1−
- 𝑟2+|1−
) are mostly
above 0 except S2=0.3 and
S1=0.3.
• Compared to CIA, high dependency
between two assays enhances efficacy
when old assay’s sensitivity or
specificity is low except when both
assays have low sensitivity.
• When S1, C1 are high, the diff (
𝑐𝑜𝑟𝑟
𝑖𝑛𝑑
𝑟2+|1−
- 𝑟2+|1−
) are mostly
below 0 except S2=0.9 and
C1=0.9.
• Compared to CIA, high dependency
between two assays reduces efficacy
when old assay’s sensitivity or
specificity is high except for the
scenario of high sensitivity in new
assay and high specificity in old assay.
24
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
Conclusion
• We established a statistical framework that describes how assay accuracy affects
clinical efficacy in biomarker driven clinical trials
• Inaccurate CDx diminishes treatment efficacy but specificity is the driving factor to
improve response rate.
• We describe the processes for methods comparison studies between two assays
under the following conditions
• Conditional independence assumption is satisfied:
• Depending on positive or negative result of old assay, sensitivity or
specificity of old assay has opposite effect on new assay’s efficacy.
However, high specificity of new assay improves efficacy no matter what
test result of old assay is.
• Dependency is about 80%, compared with CIA:
• In general, when old assay’s sensitivity or specificity is low, high
dependency enhances new assay’s efficacy. On the contrary, high
dependency reduces the efficacy when old assay’s sensitivity or specificity
is high.
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
Future Work
• Generalize the simulation to cover the range of sensitivity/ specificity,
marker prevalence and dependency.
• Use measurement error models to incorporate impact of assay methods
comparison to efficacy evaluation
• Evaluate and compare impact of clinical study design on assay bridging
studies (e.g., all-comers versus enrichment studies)
26
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
References
[1] Maitournam A, Simon R: On the efficiency of targeted clinical trials.
Statistics in Medicine 2005; 24:329–339.
[2] Lu Y, Dendukuri N, Schiller I, Joseph L: A Bayesian approach to
simultaneously adjusting for verification and reference standard bias in
diagnostic test studies.Stat Med 2010, 29(24):2532-2543.
[3] Zhou XH, Obuchowski N, McLish D. Statistical Methods in Diagnostic
Medicine, 2 nd ed. Hoboken, New Jersey: John Wiley and Sons; 2011.
27
Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.
Doing now what patients need next