Branching tests in
clinical trials with
multiple objectives
Alex Dmitrienko
Eli Lilly and Company
Brian Wiens
Myogen, Inc
Outline
Gatekeeping procedures
Serial and parallel testing
Branching procedures
Multiple tests for clinical trials with
hierarchically ordered objectives
Extension of gatekeeping methods
Clinical trial examples
Trial with multiple endpoints and objectives
Dose-finding trial with multiple endpoints
[Slide 2]
Mickey Mouse problem
[Slide 3]
Multiple endpoints
Two co-primaries/one secondary
Primary
endpoint 1
Primary
endpoint 2
p≤0.025
p≤0.025
Secondary
endpoint
p≤0.05
[Slide 4]
Multiple endpoints
Two co-primaries/one secondary
Family 1:
Bonferroni
test
Primary
endpoint 1
Primary
endpoint 2
p≤0.025
p≤0.025
Secondary
endpoint
p≤0.05
[Slide 5]
Multiple endpoints
Two co-primaries/one secondary
Family 2:
Test if at
least one
primary
endpoint is
significant
Primary
endpoint 1
Primary
endpoint 2
p≤0.025
p≤0.025
Secondary
endpoint
p≤0.05
[Slide 6]
Multiple endpoints
Two co-primaries/one secondary
Primary
endpoint 1
Primary
endpoint 2
p≤0.025
p≤0.025
Secondary
endpoint
p≤0.05
Type I error rate is inflated
0.025+0.05>0.05
[Slide 7]
Gatekeeping methods
Gatekeeping procedures
Multiple testing procedures for sequential
families of null hypotheses
Serial gatekeeping methods, Westfall and
Krishen (2001)
Parallel gatekeeping methods, Dmitrienko,
Offen and Westfall (2003)
Parallel gatekeeping methods with logical
restrictions, Chen, Luo and Capizzi (2005)
General overview
Dmitrienko et al (2005, Chapter 2)
[Slide 8]
Gatekeeping methods
Serial versus parallel strategies
Serial strategy
(Rheum arthritis)
Endpoint 1:
Signs and
symptoms
Endpoint 2:
Disease
progression
Endpoint 3:
Physical function/
disability
Parallel strategy
(Acute lung injury)
Endpoint 1:
Lung
function
Endpoint 2:
Mortality
rate
Endpoint 3:
Quality
of life
Endpoint 4:
ICU-free
days
[Slide 9]
strategy
arthritis)
Branching methods
Extension of gatekeeping methods
Branching methods
oint 1:
s and
ptoms
Trial designs are becoming
increasingly more complex
Clinical researchers explore
complex testing strategies
oint 2:
ease
ession
Examples
oint 3:
function/
bility
Two- or three-dimensional
rather than simple
sequential strategies
Logical restrictions
Parallel s
(Acute lun
Endpoint 1:
Lung
function
Endpoint 3:
Quality
of life
[Slide 10]
Clinical trial examples
Hypertension trial
Design
Experimental drug versus active control
Four endpoints
Primary (P): Systolic blood pressure
Secondary (S1 and S2): Diastolic blood
pressure and proportion of patients with
controlled systolic/diastolic blood pressure
Tertiary (T): Average blood pressure based
on ambulatory blood pressure monitoring
Noninferiority vs superiority
[Slide 11]
Hypertension trial
Decision tree
P
Noninferiority
S1
Noninferiority
P
Superiority
S2
Noninferiority
S1
Superiority
T
Noninferiority
S2
Superiority
T
Superiority
P=Primary, S1 and S2=Secondary, T=Tertiary endpoints
[Slide 12]
Clinical trial examples
Type II diabetes trial
Design
Three doses (L, M and H) versus placebo (P)
Three endpoints
Primary (P): Hemoglobin A1c
Secondary (S1 and S2): Fasting serum
glucose and HDL cholesterol
Logical restrictions
[Slide 13]
Diabetes trial
Decision tree
P
L vs P
P
M vs P
P
H vs P
S1
L vs P
S1
M vs P
S1
H vs P
S2
L vs P
S2
M vs P
S2
H vs P
P=Primary, S1 and S2=Secondary endpoints
[Slide 14]
Branching framework
Closed testing principle
Marcus, Peritz and Gabriel (1976)
Define a branching procedure based on
Bonferroni test
Compute multiplicity-adjusted p-values
[Slide 15]
Gatekeeping sets
Gatekeeping sets
Gatekeepers specific to each null
hypothesis
Parallel gatekeeping and serial
gatekeeping sets for each null hypothesis
[Slide 16]
Serial gatekeeping set
P
L vs P
S1
L vs P
S2
L vs P
P
M vs P
P
H vs P
Null hypothesis H
S1
S1
Serial
M vsgatekeeping
P
H vs Pset:
All null hypotheses must be
rejected in this set to test H
S2
M vs P
S2
H vs P
[Slide 17]
Parallel gatekeeping set
P
Noninferiority
S1
Noninferiority
P
Superiority
S2
Noninferiority
S1
Superiority
T
Noninferiority
S2
Superiority
Parallel gatekeeping set for H:
T
Superiority
At least one null hypothesis
must be rejected in this
set to test H
[Slide 18]
Hypertension trial
Decision tree
H11
P, Noninf
H21
S1, Noninf
H23
P, Super
H22
S2, Noninf
H31
S1, Super
H33
T, Noninf
H32
S2, Super
H41
T, Super
[Slide 19]
Hypertension trial
Parallel gatekeeping sets
Null hypothesis
H11 (P, Noninf )
H21 (S1, Noninf )
H22 (S2, Noninf )
H23 (P, Super)
H31 (S1, Super)
H32 (S2, Super)
H33 (T, Noninf )
H41 (T, Super)
Serial gatekeeping sets are empty
Parallel set
NA
H11
H11
H11
H21
H22
H21, H22
H33
[Slide 20]
Hypertension trial
Multiplicity-adjusted p-values
P, Noninf
0.001 0.001
S1, Noninf
0.008 0.024
P, Super
0.003 0.009
S2, Noninf
0.026 0.078
S1, Super
0.208 0.624
T, Noninf
0.010 0.045
S1, Super
0.302 0.906
T, Super
0.578 0.906
Raw p-values
Multiplicity-adjusted p-values
[Slide 21]
Diabetes trial
Decision tree
L vs P
M vs P
H vs P
Endpoint P
H11
H12
H13
Endpoint S1
H21
H22
H23
Endpoint S2
H31
H32
H33
[Slide 22]
Diabetes trial
Serial gatekeeping sets
Null hypothesis
H11 (P, L vs P)
H12 (P, M vs P)
H13 (P, H vs P)
H21 (S1, L vs P)
H22 (S1, M vs P)
H23 (S1, H vs P)
H31 (S2, L vs P)
H32 (S2, M vs P)
H33 (S2, H vs P)
Parallel gatekeeping sets are empty
Serial set
NA
NA
NA
H11
H12
H13
H11, H21
H12, H22
H13, H23
[Slide 23]
Diabetes trial
Branching strategy
Logical restrictions
L vs P
M vs P
H vs P
Endpoint P
0.018
0.054
0.011
0.033
0.005
0.015
Endpoint S1
0.013
0.054
0.007
0.033
0.009
0.041
Endpoint S2
0.051
0.054
0.012
0.033
0.010
0.041
Raw p-values
Multiplicity-adjusted p-values
[Slide 24]
Diabetes trial
Parallel gatekeeping strategy
No logical restrictions
L vs P
M vs P
H vs P
Endpoint P
0.018
0.054
0.011
0.033
0.005
0.015
Endpoint S1
0.013
0.054
0.007
0.033
0.009
0.041
Endpoint S2
0.051
0.054
0.012
0.054
0.010
0.054
Raw p-values
Multiplicity-adjusted p-values
[Slide 25]
Extensions
Basic branching framework
Based on Bonferroni test
Account for correlation
Correlation among multiple endpoints
Correlation among multiple dose-control
comparisons
Account for correlation via resampling
(Westfall and Young, 1993)
[Slide 26]
Summary
Branching procedures
Efficient way to account for hierarchically
ordered multiple objectives in clinical trials
Extend serial and parallel gatekeeping
methods
Simple software implementation (SAS
macro)
Closed testing principle
Control the familywise error rate in the
strong sense
[Slide 27]
References
Chen, Luo, Capizzi. The application of enhanced
parallel gatekeeping strategies. Statistics in Medicine.
2005; 24:1385-1397.
Dmitrienko, Offen, Westfall. Gatekeeping strategies for
clinical trials that do not require all primary effects to
be significant. Statistics in Medicine. 2003; 22:23872400.
Dmitrienko, Molenberghs, Chuang-Stein, Offen.
Analysis of Clinical Trials Using SAS: A Practical Guide.
SAS Press: Cary, NC, 2005.
Marcus, Peritz, Gabriel. On closed testing procedures
with special reference to ordered analysis of variance.
Biometrika. 1976; 63:655-660.
[Slide 28]
References
Westfall, Krishen. Optimally weighted, fixed sequence
and gatekeeper multiple testing procedures. Journal of
Statistical Planning and Inference. 2001; 99:25-41.
Westfall, Young. Resampling-Based Multiple Testing:
Examples and Methods for P-Value Adjustment. New
York: Wiley, 1993.
[Slide 29]