Dynamic Treatment Regime, Sequentially
Randomized Designs, and Semi-parametric
Theory
Abdus S. Wahed, PhD
University of Pittsburgh
[email protected]
July 13, 2013
1 / 35
Table of Contents
1 Introduction
Background and Motivation
Objective
2 Estimation
IPW Estimator
Semiparametric Efficient Estimator
3 Simulation
Scenarios
Results of Simulation
4 Application
CALGB 8923 Trial
Results of Application
5 Conclusion
2 / 35
Dynamic Treatment Regime
An algorithm that chooses treatment for patients based on
the patient characteristics, previous treatment history and
response to previous treatments.
Examples:
Depression Sertraline for 8 weeks, if the patient does not respond, treat
the patient with Sertraline as well as with cognitive behavioral
therapy; if the patient responds, continue Sertraline
HIV Start on a ARV regimen, e.g., containing Efavirenz, switch to
second line regimen less than 8 weeks after confirmed virologic
failure
Cancer Treat with chemotherapy followed by cis-RA if there is no
disease progression
3 / 35
Dynamic Treatment Regime
An algorithm that chooses treatment for patients based on
the patient characteristics, previous treatment history and
response to previous treatments.
Examples:
Depression Sertraline for 8 weeks, if the patient does not respond, treat
the patient with Sertraline as well as with cognitive behavioral
therapy; if the patient responds, continue Sertraline
HIV Start on a ARV regimen, e.g., containing Efavirenz, switch to
second line regimen less than 8 weeks after confirmed virologic
failure
Cancer Treat with chemotherapy followed by cis-RA if there is no
disease progression
3 / 35
Dynamic Treatment Regime
An algorithm that chooses treatment for patients based on
the patient characteristics, previous treatment history and
response to previous treatments.
Examples:
Depression Sertraline for 8 weeks, if the patient does not respond, treat
the patient with Sertraline as well as with cognitive behavioral
therapy; if the patient responds, continue Sertraline
HIV Start on a ARV regimen, e.g., containing Efavirenz, switch to
second line regimen less than 8 weeks after confirmed virologic
failure
Cancer Treat with chemotherapy followed by cis-RA if there is no
disease progression
3 / 35
Dynamic Treatment Regime
An algorithm that chooses treatment for patients based on
the patient characteristics, previous treatment history and
response to previous treatments.
Examples:
Depression Sertraline for 8 weeks, if the patient does not respond, treat
the patient with Sertraline as well as with cognitive behavioral
therapy; if the patient responds, continue Sertraline
HIV Start on a ARV regimen, e.g., containing Efavirenz, switch to
second line regimen less than 8 weeks after confirmed virologic
failure
Cancer Treat with chemotherapy followed by cis-RA if there is no
disease progression
3 / 35
Dynamic Treatment Regime
Usual goal is
To develop optimal treatment regime to achive maximum
patient benefit.
To compare a fixed number of treatment regimes arising from
a sequence of treatments available at different stages of
therapy.
4 / 35
Sequentially Randomized Trials
Patients are randomized to treatment options sequentially as
they become eligible to receive treatments.
Useful for comparing treatment regimes with discrete
treatment options and intermediate responses
Has been emloyed in various reseach settings such as
cancer
depression
drug and alcohol addiction
education
5 / 35
CALGB 8923 Trial
Figure : Patient flow and study design for CALGB 8923 trial
Cytarabine
n=37
Consent
n=79
Cytarabine
+Mitoxantrone
n=42
Response
n=99
GM-CSF
n=193
No Consent
n=20
No Response
n=94
CALGB 8923
n=388
Cytarabine
n=45
Consent
n=90
Cytarabine
+Mitoxantrone
n=45
Response
n=106
Placebo
n=195
No Consent
n=16
No Response
n=89
6 / 35
Motivation: CALGB 8923 Trial
Previous analyses1 treated non-responders and non-consenters alike.
Cytarabine
n=37
Response and Consent
n=79
GM-CSF
n=193
Cytarabine
+Mitoxantrone
n=42
No Response or No Consent
n=114
CALGB 8923
n=388
Cytarabine
n=45
Response and Consent
n=90
Cytarabine
+Mitoxantrone
n=45
Placebo
n=195
No Response or No Consent
n=105
1
Lunceford et al. (2002), Wahed & Tsiatis (2006), and Lokhnygina & Helterbrand (2007)
7 / 35
Objective
However, the goal of such design is to determine which of the
treatment regimes result in longer survival.
Examples of regimes:
“Treat with GM-CSF followed by Cytarabine if respond”
“Treat with GM-CSF followed by Cytarabine+Mitoxantrone if respond”
“Treat with placebo followed by Cytarabine if respond”
“Treat with placebo followed by Cytarabine+Mitoxantrone if respond”
In general, let us define regime Aj Bk as “treat with Aj followed by
Bk if respond”.
8 / 35
Objective
But combining response and consent together does
not allow us to answer that question.
9 / 35
Evolution of a Paitent’s Status in CALGB Trial
D
X
NR
D
X
NR
R/NC
D
R/NC
R/C
R/C
R/C
R/C
R: responded
C: consented
Z: 2nd randomization
X
D
X
Z
Z
D: dead
X: censored
D
X
NC: not consented
NR: not responded
10 / 35
Evolution of a Paitent’s Status - dealt in previous analyses
D
X
NR
D
X
NR
R/C/Z
R/C/Z
R: responded
C: consented
Z: 2nd randomization
D
X
D: dead
X: censored
NC: not consented
NR: not responded
11 / 35
Goal of this work
Consistent and efficient estimation of mean survival under a
given treatment regime taking into account the actual data
structure. Specifically,
We will treat non-responders and non-consenters separately.
We will take into account the delay from becoming eligible for
second stage treatment to the assignment of treatment.
12 / 35
For the rest of the work, assume:
Two stage 1 treatments Aj , j = 1, 2
Two stage 2 treatments Bk , k = 1, 2 for responders
Consider data from patients who received treatment A1 .
13 / 35
For the rest of the work, assume:
Two stage 1 treatments Aj , j = 1, 2
Two stage 2 treatments Bk , k = 1, 2 for responders
Consider data from patients who received treatment A1 .
13 / 35
Counterfactual Variables
Each patient i has an associated set of random variables
Ri , (1 − Ri )TiNR , Ri TiR , Ri Ci , Ri (1 − Ci )TiNC , Ri Ci TiZ , Ri Ci T1i∗ , Ri Ci T2i∗
Ri : response status
TiNR : survival time for a patient who did not respond
TiR : time from initial treatment, A1 , to the time a patient responded to A1
Ci : consent status
TiNC : survival time for a patient who responded but did not consent to Bk
TiZ : time from initial treatment, A1 , to the initiation of second-stage treatment
T1i∗ : survival time if patient responded to A1 and received treatment B1
T2i∗ : survival time if patient responded to A1 and received treatment B2
14 / 35
Survival Time Under regime A1 Bk
In terms of counterfactual variables, the survival time under
regime A1 Bk can be expressed as
Tki = (1 − Ri )TiNR + Ri Tki∗ , k = 1, 2
The goal is to draw inference on the distribution of
Tk , k = 1, 2. For example, we may be interested in
µk = E [Tki ]
15 / 35
Observed Data
The observed data in the presence of right censoring can be
charaterized as the collection of i.i.d random vectors
h
i
Ri , Ri Ci , Ri TiR , Ri Ci TiZ , Ri Ci Zki , Ui , ∆i , {GiH (u), u 6 Ui } ,
i = 1, 2, . . . , n; k = 1, 2.
Ri , Ci , TiR , and TiZ are defined as before.
Z1i : the B1 treatment assignment indicator
Z2i : the B2 treatment assignment indicator
Ti : observed survival time
Xi : censoring time
Ui : min(Ti , Xi )
∆i : I (Ti < Xi )
GiH (u): data history collected on patient i prior to time u
16 / 35
Assumptions
In order to draw conclusion about counterfacters, we need to relate
them to the observed data, which is done through some
assumptions.
Consistency, or Stable Unit Treatment Value Assumption
(SUTVA)
Ti = (1 − Ri )TiNR + Ri (1 − Ci )TiNC + Ri Ci (Z1i T1i∗ + Z2i T2i∗ ),
where Z2i = 1 − Z1i .
No unmeasured confounding given data history, or Sequential
Randomization Assumption (SRA)
n
o
πk = P Zki = 1 Ri = 1, Ci = 1, GiH (TiZ ) = P {Zki = 1 |Ri = 1, Ci = 1 }
17 / 35
IPW Estimator
Given a treatment regime, A1 Bk , the
inverse-probability-weighted (IPW) estimator for mean
survival time is
µ̂IPW
= n−1
k
n
X
i=1
∆i
K (Ui )
1 − Ri +
Ri Ci Zki
πCi πk
Ui
Note that there are two nuisance parameters, K (.) and πCi .
K (.) is estimated by the Kaplan-Meier method.
πCi , the consent rate, we considered two estimators:
πCi is calculated as the proportion of responders
who
.P
Pn
n
∆i
i
consented, e.g. π̂Ci = i=1 K̂∆
R
C
i=1 K̂ (U ) Ri
(U ) i i
i
i
πCi is modeled by logistic regression on baseline and the 1st
stage covariate, e.g. logit(π̂Ci ) = α̂ + β̂ × TiR + γ̂ × Wi , where
Wi is the collection of baseline covariates.
18 / 35
Variance of IPW Estimator
When πCi is estimated empirically:
var
ˆ (µ̂IPW
)=
k
−
Ri Ci
− πCi
π̂R
Ê
1
n
(
n
1 X ∆i
n i=1 K̂ (Ui )
!
Ri Ci Zki
Ti
πC2 πk
i
"
!
Ri Ci Zki
Ui − µ̂IPW
k
π̂Ci πk
!
!#2
Ri Ci
Ri Ci Zki
+ (Ri − π̂R )Ê
Ê
Ti
πR2
πC2 πk
i
)
Z L
0
2
dN c (u)
Ê Li (u, π̂Ci )
+
0 K̂ (u)Y (u)
1 − Ri +
(1)
19 / 35
Variance of IPW Estimator
where
Ê
2
L0i (u, π̂Ci )
"(
)
n
1 X ∆i
Ri Ci Zki
=
1 − Ri +
Ui − Ĝ (u, π̂Ci )
n i=1 K̂ (Ui )
π̂Ci πk
!)
(
h
i
Ri Ci
Ri Ci
− Ĝ 0 (u, π̂R ) + Ri − Ĝ 00 (u) Ê
+
π̂R
πR2
!#2
Ri Ci Zki
×I (TiR ≥ u)Ê
Ti
I (Ti > u)
πC2 πk
i
Ĝ (u, π̂Ci ) =
Ĝ 0 (u, π̂R ) =
00
Ĝ (u) =
n
X
1
nŜ(u −1 )
1
nŜ(u −1 )
1
nŜ(u − )
i=1
n
X
i=1
∆i
K̂ (Ui )
∆R
i
(
1 − Ri +
Ri
K̂ (TiR ) π̂R
n
X
∆R
i
i=1
K̂ (TiR )
Ri Ci Zki
π̂Ci πk
)
Ui × I (Ui > u)
Ci × I (TiR ≥ u)
Ri × I (TiR ≥ u)
20 / 35
Variance of IPW Estimator
When πCi is modeled by logistic regression:
var
ˆ (µ̂IPW
)
k
(
−
∆R
i Ri
1
=
n
"
n
1 X ∆i
n i=1 K̂ (Ui )
Ci − π̂Ci
K (TiR ) π̂Ci (1 − π̂Ci )
)
Ê
(
Ri Ci Zki
1 − Ri +
π̂Ci πk
)
Ui − µ̂IPW
k
!2
Ri Ci Zki
Ê
Ti 
2
πC2 πk
πCi (1 − πCi )
i
#
Z L
2
dN c (u)
+
Ê L00
(u,
π̂
)
Ci
i
0 K̂ (u)Y (u)
(πCi − Ci )2
!−1
(2)
21 / 35
Variance of IPW Estimator
where
n
X
2
Ê L00
= n−1
i (u, π̂Ci )
∆i
K̂ (Ui )
!
Ri C i
Ê
πR2
"(
1 − Ri +
i=1
00
+Ĝ (u)Ê
Ri Ci Zki
Ti
πC2 πk
Ri Ci Zki
π̂Ci πk
!
I (TiR
)
Ui − Ĝ (u, π̂Ci )
#2
≥ u)
I (Ti > u)
i
22 / 35
Semiparametric Efficient Estimator
The IPW estimator is consistent, but not efficient1 .
Fails to take into account information from the censored
patients as well as patients who fail to receive the treatment
Bk due to randomization to other B-treatment or refusal to
take B-treatment.
1 Robins,
Rotnitzky, & Zhao (1994)
23 / 35
Semiparametric Efficient Estimator
Semi-parametric efficient estimator1 :
µ̂EFF
k
"
n
1X
∆i
=
n i=1 K̂ (Ui )
Ri Ci Zki
1 − Ri +
π̂Ci πk
−
1 Robins,
!
∆R
i Ri (Ci − π̂Ci )ν̂Ri
K̂ (UiR )π̂Ci
dMic (u) H
τ̂ Gi (u)
K (u)
0
#
∆C
i Ri Ci (Zki − πk )ν̂Ci
Z
Ui +
−
L
K̂ (UiC )π̂Ci πk
(3)
Rotnitzky, & Zhao (1994)
24 / 35
Semiparametric Efficient Estimator
τ (.), νRi , and νCi are estimated from the data based on suitable
regression models.
τ GiH (u) = E
o
Ti | Ri = 1, Zki = 1, TiR , GiH (TiR ) = α + βTiR
n
o
= E Ti | Ri = 1, Ci = 1, Zki = 1, TiZ , GiH (TiZ ) = α + β1 TiR + β2 (TiZ − TiR )
νRi = E
νC i
∆i
Ui GiH (u)
K (Ui )
n
where (TiZ − TiR ) is the delay of the second randomization
25 / 35
Variance of Semiparametric Efficient Estimator
var
ˆ (µ̂EFF
)=
k
"
(
)
Z L
n
dMic (u) H
1 X
∆i
Ri Ci Zki
τ̂ Gi (u)
1
−
R
+
U
+
i
i
2
n i=1 K̂ (Ui )
π̂Ci πk
K (u)
0
#2
R
C
∆i Ri (Ci − π̂Ci )ν̂Ri
∆i Ri Ci (Zki − πk )ν̂Ci
EFF
−
−
− µ̂k
(4)
K̂ (UiR )π̂Ci
K̂ (UiC )π̂Ci πk
26 / 35
Simulation Scenarios
Population:
R ∼ Bernoulli(πr );
T R ∼ EXP(αr ) truncated at br ;
T NR ∼ EXP(α0 ) truncated at b0 ;
T Z ∼ T R + Ur ; Ur ∼ uniform(0, θr );
Tk∗ ∼ T Z + (β1 + β2 × T Z )Uk ; Uk ∼ uniform(0, θk ), k = 1, 2;
Tk = (1 − R)T NR + RTk∗ , k = 1, 2;
Data generation:
C ∼ Bernoulli(πC ); logit(πC ) = α + β × T R ;
T NC ∼ T R + U NC ; U NC ∼ uniform(0, θNC );
Z1 ∼ Bernoulli(π1 ), Z2 = 1 − Z1 ;
T = min((1 − R)T NR + R(1 − C )T NC + RC {Z1 T1∗ + Z2 T2∗ }, L);
X ∼ uniform(0, ξ);
U = min(T , X );
27 / 35
Simulation Results
Table : Results of Monte-Carlo simulations for treatment regimes A1 B1 and A1 B2
under 50% response rate and 51%, and 25% censoring rate
Simulation=2000, n=500, 50% response
51% censoring
Estimator
IPW (emp1 )
IPW (mod2 )
Efficient (emp1 )
Efficient (mod2 )
A1 B1 (µ1 = 386)
A1 B2 (µ2 = 359)
µ̂1
SEMC (µ̂1 )
SE (µ̂1 )
CP%
µ̂2
SEMC (µ̂2 )
SE (µ̂2 )
CP%
386.3
385.9
389.2
386.4
29.3
29.6
18.5
19.4
30.1
30.2
19.6
20.8
95.4
95
96.4
97
360.1
360.7
362.0
359.9
27.4
28.1
18.0
19.2
28.3
28.6
19.2
20.5
95.7
95.4
96.7
96.9
1
empirical estimator for πC
2
logistic regression estimator for πC with covariates of TiR
28 / 35
Simulation Results
Table : Results of Monte-Carlo simulations for treatment regimes A1 B1 and A1 B2
under 50% response rate and 51%, and 25% censoring rate
Simulation=2000, n=500, 50% response
51% censoring
Estimator
IPW (emp1 )
IPW (mod2 )
Efficient (emp1 )
Efficient (mod2 )
A1 B1 (µ1 = 386)
SEMC (µ̂1 )
SE (µ̂1 )
CP%
µ̂2
SEMC (µ̂2 )
SE (µ̂2 )
CP%
386.3
385.9
389.2
386.4
29.3
29.6
18.5
19.4
30.1
30.2
19.6
20.8
95.4
95
96.4
97
360.1
360.7
362.0
359.9
27.4
28.1
18.0
19.2
28.3
28.6
19.2
20.5
95.7
95.4
96.7
96.9
µ̂1
SEMC (µ̂1 )
SE (µ̂1 )
CP%
µ̂2
SEMC (µ̂2 )
SE (µ̂2 )
CP%
386
386.7
388.2
386.7
20.8
21.0
12.4
12.7
21.3
21.1
12.8
13.1
95.5
95.3
95.0
95.3
358.2
359.7
361.6
360.2
18.9
19.3
12.1
12.5
19.8
19.6
12.3
12.7
96.0
95.4
94.9
95.1
25% censoring
Estimator
1
IPW (emp )
IPW (mod2 )
Efficient (emp1 )
Efficient (mod2 )
A1 B2 (µ2 = 359)
µ̂1
A1 B1 (µ1 = 386)
A2 B2 (µ2 = 359)
1
empirical estimator for πC
2
logistic regression estimator for πC with covariates of TiR
29 / 35
Simulation Results
Both IPW and efficient estimators consistently estimated the
mean survival time, but the efficient estimators improved the
efficiency significantly, in some cases by as large as 69%.
The Monte-Carlo and estimated variances increase with the
increase in censoring rate.
Decreasing the sample size from 500 to 250 and increasing
the response rate from 50% to 70% (results not shown here)
showed similar results.
When consent rate was estimated empirically, the precision of
the estimators was better than when consent rate was
estimated using logistic regression.
30 / 35
CALBG 8923 Trial
Double-blind, placebo-controlled two-stage trial;
The effects of infusions of granulocyte-macrophage
colony-stimulating factor (GM-CSF) after initial chemotherapy
in 388 elderly patients with acute myelogenous leukemia
(AML);
Patients were randomized initially to GM-CSF or placebo
following standard chemotherapy. Later, patients meeting the
criteria for complete remission were offered a second
randomization to one of two intensification treatments
(Cytarabine or Cytarabine+Mitoxantrone);
The treatment regimes: “treat with GM-CSF or placebo
followed by Cytarabine or Cytarabine+Mitoxantrone if
respond”
31 / 35
Results of Analysis of CALGB 8923 Trial
Table : Estimated mean survival time (days) and its variance for the CALGB 8923
study
A1 B1 3
Estimator
1
IPW (emp )
IPW (mod2 )
Efficient (emp1 )
Efficient (mod2 )
A 1 B2 4
A2 B 1 5
A2 B2 6
µ̂
SE (µ̂)
µ̂
SE (µ̂)
µ̂
SE (µ̂)
µ̂
SE (µ̂)
404.1
414.9
452.5
461.1
59.6
66.4
49.1
48.7
406.3
395.6
421.8
417.1
69.3
68.9
62.1
60.1
363.2
358.9
414.0
415.6
45.7
46.8
41.5
41.4
760.0
767.8
717.0
722.2
192.4
195.4
227.5
222.1
1
empirical estimator for πC
2
logistic regression estimator for πC with covariates of TiR , gender, race, age, and white blood cell counts
3
treatment regime “treat with GM-CSF followed by Cytarabine if respond”
4
treatment regime “treat with GM-CSF followed by Cytarabine+Mitoxantrone if respond”
5
treatment regime “treat with placebo followed by Cytarabine if respond”
6
treatment regime “treat with placebo followed by Cytarabine+Mitoxantrone if respond”
32 / 35
Results of Analysis of CALGB 8923 Trial
The IPW estimators tend to provide smaller estimates for the
mean survival time compared to the efficient estimators.
Standard errors of efficient estimators are smaller than that of
IPW estimators, as expected by theory.
Based on the estimates, it seems that treatment regime “treat
with placebo followed by Cytarabine + Mitoxantrone if
respond” provides the largest mean survival time among four
regimes. However, considering the large SE for the
corresponding estimator, a formal test of hypothesis might not
be able to show statistical significance.
Treatment regime “treat with placebo followed by Cytarabine
if respond” provides the shortest mean survival time among
four regimes.
Adding GM-CSF to chemotherapy does not improve regime
mean survival time.
33 / 35
Conclusion
We have presented several approaches to estimate mean
survival time and its variance in two-stage randomization
designs when some patients did not consent after responding
to the initial treatment.
The efficient (semi-parametric) estimator shows considerable
gain in efficiency over IPW estimator. This method allows us
to use information from auxiliary covariates.
We have demonstrated our methods through application of a
leukemia clinical trial data set.
In some cases, it might be of interest to estimate survival
probability at certain time point other than mean survival
time. In such case, an entirely similar strategy can be followed
by replacing Ti by I (Ti > t) for survival probability.
34 / 35
THANK YOU!
35 / 35