Randomized Controlled/Confounded Trials
(RCTs): Optimal methods for observing the
observed?
Ian Shrier MD, PhD, Dip Sport Med (FACSM)
•
•
Associate Professor, Dep’t of Fam Med, McGill University
Centre for Clinical Epidemiology, Jewish General Hospital and
McGill University
OBJECTIVES
• What is your question?
⇒ Intention-to-Treat, Per Protocol, As Treated
• Introduction to Complier Avg. Causal Effect
(CACE): Patient-oriented effects
⇒ Principal Stratification
⇒ Instrumental Variables
⇒ Concrete Examples
• Additional Topics
⇒ Defining Adherence: all or none?
⇒ Regression Discontinuity
(Shrier et al. Clin Trials 2014, Int J Biostats 2013)
REAL DATA EXAMPLE
• RCT in athletes with ankle sprain who
received usual rehab
(Hupperets BMJ 2009)
⇒Intervention: 8wks additional balance rehab (27%
adherent, 34% partially adherent, 39% non-adherent)
⇒Control: no access to balance rehab (5/269 took
extra rehab)
⇒Outcome: reinjury
(Hupperets et al. BMJ 2009)
EPIDEMIOLOGY 101
• Results/Interpretation = Data + Assumptions
• Equal prognosis at onset (randomization); Each group
treated equally except for exposure of interest
• Research Question: Does balance rehab reduce reinjury?
Common Causes
Randomization
Assigned Rx
Rx Received
Outcome
• Some participants do not adhere to their Rx assignment
“The perfect study exists only in the minds of those
who do no research.” (Tim Noakes)
WHAT IS YOUR QUESTION?
• Results/Interpretation = Data + Assumptions
• Epidemiology 101: Equal prognosis at onset (randomization);
Each group treated equally except for exposure of interest
• Research Question: Does balance rehab reduce reinjury?
Common Causes
Randomization
Assigned Rx
Rx Received
• Intention to Treat (ITT): treatment assignment
⇒ Regulatory Agency: avoids overestimation of effect
Outcome
ITT Biased Towards No Effect?
Truth
Ref. Rx (Plac.)
0
Truth
Novel Rx
0
Obs.
Obs.
Non-adherence
Obs.
Obs.
Non-adherence
Truth
Novel Rx
2
Truth
Ref Rx
2
WHAT IS YOUR QUESTION?
• Results/Interpretation = Data + Assumptions
• Epidemiology 101: Equal prognosis at onset (randomization);
Each group treated equally except for exposure of interest
• Research Question: Does balance rehab reduce reinjury?
Common Causes
Randomization
Assigned Rx
Rx Received
Outcome
• Intention to Treat (ITT): treatment assignment
⇒ Regulatory Agency: avoids overestimation of effect
⇒ Health Policy: requires % adherence (& reasons) = target population
• Patient wants measure of treatment effectiveness
WHAT IS YOUR QUESTION?
• Results/Interpretation = Data + Assumptions
• Epidemiology 101: Equal prognosis at onset (randomization);
Each group treated equally except for exposure of interest
• Research Question: Does balance rehab reduce reinjury?
Common Causes
Randomization
Assigned Rx
Rx Received
Outcome
• ITT measures effect of treatment assignment
⇒ Regulatory Agency: avoids overestimation of effect (vs. placebo…)
⇒ Health Policy: requires % adherence (& reasons) = target population
• Patient wants measure of treatment effectiveness
⇒ Analyses based on adherence-data have important assumptions
⇒ Analyses based on observational data have important assumptions
OBJECTIVES
• What is your question?
⇒ Intention-to-Treat, Per Protocol, As Treated
• Introduction to Complier Avg. Causal Effect
(CACE): Patient-oriented effects
⇒ Principal Stratification
⇒ Instrumental Variables
⇒ Concrete Examples
• Additional Topics
⇒ Defining Adherence: all or none?
⇒ Regression Discontinuity
(Shrier et al. Clin Trials 2014, Int J Biostats 2013)
Intention-To-Treat
Randomized Controlled Trial
Assigned Control
Received
Active Rx
Received
Control
Assigned Active Rx
Received
Active Rx
Received
Control
• Unbiased for Rx assignment IFF adherence in target
population is similar to study population………..
• Biased for Rx effectiveness at any other % adherence
Per Protocol
Randomized Controlled Trial
Assigned Control
Received
Active Rx
Received
Control
Assigned Active Rx
Received
Active Rx
Received
Control
• Biased for Rx Assignment
• Unbiased for Rx effectiveness IFF non-adherers respond
similar to adherers
As Treated
Randomized Controlled Trial
Assigned Control
Received
Active Rx
Assigned Active Rx
Received
Control
Received
Active Rx
Received
Control
Received
Active Rx
Received
Control
• Biased for Rx Assignment
• Unbiased for Rx effectiveness IFF non-adherers respond
similar to adherers
OBJECTIVES
• What is your question?
⇒ Intention-to-Treat, Per Protocol, As Treated
• Introduction to Complier Avg. Causal Effect
(CACE): Patient-oriented effects
⇒ Principal Stratification
⇒ Instrumental Variables
⇒ Concrete Examples
• Additional Topics
⇒ Defining Adherence: all or none?
⇒ Regression Discontinuity
(Shrier et al. Clin Trials 2014, Int J Biostats 2013)
EFFECT OF Rx RECEIVED
Baseline Common Causes
Confounders on
Assigned Exposure
Randomization
Assigned Rx
Rx Received
Outcome
• Assumptions
⇒
⇒
⇒
⇒
Randomization
Stable Unit Treatment Value (No interference, Consistency)
Exclusion restriction (Ass. Rx affects outcome only through Actual Rx)
Monotonicity (no one always takes opposite of Assigned Rx)
DEFINING PRINCIPAL STRATA
1.
2.
3.
4.
Baseline Common Causes
Randomization
Assigned Rx
Rx Received
Always Take Rx
Never Take Rx
Adhere to Rx Assignment
Defy Rx Assignment
Outcome
PS: RECEIVING ACTIVE RX
1.
2.
3.
4.
Baseline Common Causes
Randomization
Assigned Rx
Rx Received
Always Take Rx
Never Take Rx
Adhere to Rx Assignment
Defy Rx Assignment
Outcome
Randomized Controlled Trial
Assigned Active Rx
Assigned Control
Received
Active Rx
Always
Takers
Received
Control
Received
Active Rx
Baseline
Compliers
Received
Control
Always
Takers
(Shrier. Clin Trials 2014)
PS: RECEIVING CONTROL
1.
2.
3.
4.
Baseline Common Causes
Randomization
Assigned Rx
Rx Received
Always Take Rx
Never Take Rx
Adhere to Rx Assignment
Defy Rx Assignment
Outcome
Randomized Controlled Trial
Assigned Active Rx
Assigned Control
Received
Active Rx
Received
Control
Baseline
Compliers
Never
Takers
Received
Active Rx
Received
Control
Never
Takers
(Shrier. Clin Trials 2014)
PS: Baseline Compliers!
1.
2.
3.
4.
Baseline Common Causes
Randomization
Assigned Rx
Rx Received
Always Take Rx
Never Take Rx
Adhere to Rx Assignment
Defy Rx Assignment
Outcome
Randomized Controlled Trial
Assigned Active Rx
Assigned Control
Received
Active Rx
Received
Control
Baseline
Compliers
Never
Takers
Received
Active Rx
Baseline
Compliers
Always
Takers
Goal when estimating CACE…
Received
Control
PS: Baseline Compliers!
1.
2.
3.
4.
Baseline Common Causes
Randomization
Assigned Rx
Rx Received
Always Take Rx
Never Take Rx
Adhere to Rx Assignment
Defy Rx Assignment
Outcome
Randomized Controlled Trial
Assigned Active Rx
Assigned Control
Received
Active Rx
Received
Control
Baseline
Compliers
Received
Active Rx
Baseline
Compliers
Goal when estimating CACE…
Received
Control
PER PROTOCOL ANALYSIS
Randomized Controlled Trial
Assigned Active Rx
Assigned Control
Received
Active Rx
Always
Takers
Received
Active Rx
Received
Control
Baseline
Compliers
Never
Takers
Baseline
Compliers
Always
Takers
Received
Control
Never
Takers
AS TREATED ANALYSIS
Randomized Controlled Trial
Assigned Active Rx
Assigned Control
Received
Active Rx
Always
Takers
Received
Active Rx
Received
Control
Baseline
Compliers
Never
Takers
Baseline
Compliers
Always
Takers
Received
Control
Never
Takers
(Shrier. Clin Trials 2014; Shrier Int J Biostats 2013
PS ANALYSIS
Randomized Controlled Trial
Assigned Active Rx
Assigned Control
Received
Active Rx
Always
Takers
Equal ExpectationsReceived
for Outcome! Active Rx
Received
Control
Baseline
Compliers
• Principal Stratification
Never
Takers
Baseline
Compliers
Always
Takers
Received
Control
Never
Takers
PS ANALYSIS
Randomized Controlled Trial
Assigned Active Rx
Assigned Control
Received
Active Rx
Always
Takers
Received
Active Rx
Received
Control
Baseline
Compliers
Never
Takers
Baseline
Compliers
Always
Takers
Received
Control
Never
Takers
• Principal Stratification
1. MeanAss & Rec. Rx is wtd avg of (Baseline CompliersRx + Always Takers)
PS ANALYSIS
Randomized Controlled Trial
Assigned Active Rx
Assigned Control
Received
Active Rx
Always
Takers
Received
Active Rx
Received
Control
Baseline
Compliers
Never
Takers
Baseline
Compliers
Always
Takers
Received
Control
Never
Takers
• Principal Stratification
1. MeanAss & Rec. Rx is wtd avg of (Baseline CompliersRx + Always Takers)
2. Apply sensitivity analyses for Per Protocol Treatment: Baseline
CompliersRx / Always TakersRx = 0.5 or 1.0 (PP) or 1.5 or 2.0?
PS ANALYSIS
Randomized Controlled Trial
Assigned Active Rx
Assigned Control
Received
Active Rx
Always
Takers
Received
Active Rx
Received
Control
Baseline
Compliers
Never
Takers
Baseline
Compliers
Always
Takers
Received
Control
Never
Takers
• Principal Stratification
1. MeanAss & Rec. Rx is wtd avg of (Baseline CompliersRx + Always Takers)
2. Apply sensitivity analyses for Per Protocol Treatment: Baseline
CompliersRx / Always TakersRx = 0.5 or 1.0 (PP) or 1.5 or 2.0?
3. MeanAss & Rec. Ctrl is wtd avg of (Baseline CompliersCtrl + Never Takers)
4. Apply sensitivity analyses for Per Protocol Control: Never TakersCtrl /
Baseline CompliersCtrl = 0.5 or 1.0 (PP) or 1.5 or 2.0?
PS ANALYSIS
Randomized Controlled Trial
Assigned Active Rx
Assigned Control
Received
Active Rx
Always
Takers
Received
Active Rx
Received
Control
Baseline
Compliers
Never
Takers
Baseline
Compliers
Always
Takers
Received
Control
Never
Takers
• Principal Stratification
1. MeanAss & Rec. Rx is wtd avg of (Baseline CompliersRx + Always Takers)
2. Apply sensitivity analyses for Per Protocol Treatment: Baseline
CompliersRx / Always TakersRx = 0.5 or 1.0 (PP) or 1.5 or 2.0?
3. MeanAss & Rec. Ctrl is wtd avg of (Baseline CompliersCtrl + Never Takers)
4. Apply sensitivity analyses for Per Protocol Control: Never TakersCtrl /
Baseline CompliersCtrl = 0.5 or 1.0 (PP) or 1.5 or 2.0?
5. Compare MeanBaseline Compliers for different sensitivities
OBJECTIVES
• What is your question?
⇒ Intention-to-Treat, Per Protocol, As Treated
• Introduction to Complier Avg. Causal Effect
(CACE): Patient-oriented effects
⇒ Principal Stratification
⇒ Instrumental Variables
⇒ Concrete Examples
• Additional Topics
⇒ Defining Adherence: all or none?
⇒ Regression Discontinuity
(Shrier et al. Clin Trials 2014, Int J Biostats 2013)
Instrumental Variables
Baseline Common Causes
Randomization
Assigned Rx
Rx Received
Outcome
Pr[Outcome | Assigned Rx] = Pr[Outcome | Rx Received] * Pr[Rx Received | Assigned Rx]
Pr[Outcome | Assigned Rx]
= Pr[Outcome | Rx Received]
Pr[Rx Received | Assigned Rx]
(Greenland Int J Epid 2000)
Instrumental Variables
Baseline Common Causes
Randomization
Assigned Rx
Rx Received
Outcome
Pr[Outcome | Assigned Rx] = Pr[Outcome | Rx Received] * Pr[Rx Received | Assigned Rx]
Pr[Outcome | Assigned Rx]
= Pr[Outcome | Rx Received]
Pr[Rx Received | Assigned Rx]
Pr[Rx Received in Treatment Group]-Pr[Rx Received in Control Group]
(Greenland Int J Epid 2000)
OBJECTIVES
• What is your question?
⇒ Intention-to-Treat, Per Protocol, As Treated
• Introduction to Complier Avg. Causal Effect
(CACE): Patient-oriented effects
⇒ Principal Stratification
⇒ Instrumental Variables
⇒ Concrete Examples
• Additional Topics
⇒ Defining Adherence: all or none?
⇒ Regression Discontinuity
(Shrier et al. Clin Trials 2014, Int J Biostats 2013)
CONCRETE EXAMPLE
• RCT in athletes with ankle sprain who received usual rehab
(Hupperets BMJ 2009)
⇒ Intervention: 8wks additional balance rehab
⇒ Control: no access to balance rehab
⇒ Outcome: reinjury
Ankle Study
• Active Treatment Group
⇒ 27% fully adherent, 34% partially adherent, 39% non-adherent
• Control Group
⇒ 98% fully adherent, 2% partially adherent (extra rehab)
Risk Ratios
Partial = Full
ITT
Per Protocol
As Treated
Instrumental Variable
PS (NT/BC: 1.0)
0.65 (0.49 to 0.87)
0.48 (0.32 to 0.71)
0.48 (0.33 to 0.71)
0.50 (0.33 to 0.76)
0.48 (0.31 to 0.67)
Partial = None
Ankle Study
• Active Treatment Group
⇒ 27% fully adherent, 34% partially adherent, 39% non-adherent
• Control Group
⇒ 98% fully adherent, 2% partially adherent (extra rehab)
Risk Ratios
Partial = Full
Partial = None
ITT
Per Protocol
As Treated
Instrumental Variable
PS (NT/BC: 1.0)
PS (NT/BC: 2.0)
0.65 (0.49 to 0.87)
0.48 (0.32 to 0.71)
0.48 (0.33 to 0.71)
0.50 (0.33 to 0.76)
0.48 (0.31 to 0.67)
0.66 (0.42 to 0.96)
Ankle Study
• Active Treatment Group
⇒ 27% fully adherent, 34% partially adherent, 39% non-adherent
• Control Group
⇒ 98% fully adherent, 2% partially adherent (extra rehab)
Risk Ratios
Partial = Full
Partial = None
ITT
Per Protocol
As Treated
Instrumental Variable
PS (NT/BC: 1.0)
PS (NT/BC: 2.0)
0.65 (0.49 to 0.87)
0.48 (0.32 to 0.71)
0.48 (0.33 to 0.71)
0.50 (0.33 to 0.76)
0.48 (0.31 to 0.67)
0.66 (0.42 to 0.96)
0.65 (0.49 to 0.87)
0.17 (0.06 to 0.44)
0.17 (0.06 to 0.44)
0.19 (0.06 to 0.61)
Not applicable
0 events in 5 participants
CONCRETE EXAMPLE
• RCT in athletes with ankle sprain who received usual rehab
(Hupperets BMJ 2009)
⇒ Intervention: 8wks additional balance rehab
⇒ Control: no access to balance rehab
⇒ Outcome: reinjury
• Internet-based pragmatic RCT ≥18yrs, active over 12 weeks
(Jamtvedt Brit J Sports Med 2010)
⇒ Intervention: stretching pre/post exercise
⇒ Control group: no stretching (but have access to intervention)
⇒ Outcome: Injury
Stretching Study
• Active Treatment Group
⇒ 6% fully adherent, 92% partially adherent, 2% non-adherent
• Control Group
⇒ 56% fully adherent, 35% partially adherent, 9% non-adherent
Risk Ratios
Partial = Full
ITT
Per Protocol
As Treated
Instrumental Variable
PS (NT/BC & BC/AT: 1.0)
0.98 (0.85 to 1.13)
0.98 (0.84 to 1.14)
0.98 (0.85 to 1.13)
0.97 (0.84 to 1.13)
0.98 (0.85 to 1.14)
Partial = None
Stretching Study
• Active Treatment Group
⇒ 6% fully adherent, 92% partially adherent, 2% non-adherent
• Control Group
⇒ 56% fully adherent, 35% partially adherent, 9% non-adherent
Risk Ratios
Partial = Full
Partial = None
ITT
Per Protocol
As Treated
Instrumental Variable
PS (NT/BC & BC/AT: 1.0)
PS (NT/BC=1.0 & BC/AT=2.0)
0.98 (0.85 to 1.13)
0.98 (0.84 to 1.14)
0.98 (0.85 to 1.13)
0.97 (0.84 to 1.13)
0.98 (0.85 to 1.14)
1.03 (0.88 to 1.19)
Stretching Study
• Active Treatment Group
⇒ 6% fully adherent, 92% partially adherent, 2% non-adherent
• Control Group
⇒ 56% fully adherent, 35% partially adherent, 9% non-adherent
RiskRisk
Ratios
Ratios
Partial =
Partial
Full = Full Partial
Partial
= None
= None
ITT
0.98 (0.85
0.98
to 1.13)
(0.85 to 1.13)
Per Protocol
1.36 (0.96
0.98
to 1.93)
(0.84 to 1.14)
As Treated
1.07 (0.91
0.98
to 1.26)
(0.85 to 1.13)
Instrumental Variable
1.06 (0.73
0.97
to 1.56)
(0.84 to 1.13)
PS (NT/BC
(NT/BC:&1.0)
BC/AT: 1.0) 1.36 (0.93
0.98
to 1.90)
(0.85 to 1.14)
PS (NT/BC=1.0
(NT/BC: 2.0)& BC/AT=2.0)
0.66 (0.42
1.03
to 0.96)
(0.88 to 1.19)
0.98 (0.85 to 1.13)
1.36 (0.96 to 1.93)
1.07 (0.91 to 1.26)
1.06 (0.73 to 1.56)
1.36 (0.93 to 1.90)
Stretching Study
• Active Treatment Group
⇒ 6% fully adherent, 92% partially adherent, 2% non-adherent
• Control Group
⇒ 56% fully adherent, 35% partially adherent, 9% non-adherent
RiskRisk
Ratios
Ratios
Partial =
Partial
Full = Full Partial
Partial
= None
= None
ITT
0.98 (0.85
0.98
to 1.13)
(0.85 to 1.13)
Per Protocol
1.36 (0.96
0.98
to 1.93)
(0.84 to 1.14)
As Treated
1.07 (0.91
0.98
to 1.26)
(0.85 to 1.13)
Instrumental Variable
1.06 (0.73
0.97
to 1.56)
(0.84 to 1.13)
PS (NT/BC
(NT/BC:&1.0)
BC/AT: 1.0) 1.36 (0.93
0.98
to 1.90)
(0.85 to 1.14)
PS (NT/BC=1.0
(NT/BC: 2.0)& BC/AT=2.0)
0.66 (0.42
1.03
to 0.96)
(0.88 to 1.19)
Randomization
Assigned Rx
Rx Received
0.98 (0.85 to 1.13)
1.36 (0.96 to 1.93)
1.07 (0.91 to 1.26)
1.06 (0.73 to 1.56)
1.36 (0.93 to 1.90)
only 6% in active
Outcome
OBJECTIVES
• What is your question?
⇒ Intention-to-Treat, Per Protocol, As Treated
• Introduction to Complier Avg. Causal Effect
(CACE): Patient-oriented effects
⇒ Principal Stratification
⇒ Instrumental Variables
⇒ Concrete Examples
• Additional Topics
⇒ Defining Adherence: all or none?
⇒ Regression Discontinuity
(Shrier et al. Clin Trials 2014, Int J Biostats 2013)
PARTIAL ADHERENCE
• Dose-response
⇒ Some participants receive only a partial dose (e.g.
vaccine, 1 pill per day when prescribed 2 pills per day)
• Delayed partial adherence
⇒ Some participants only start the intervention after a
period of time has elapsed (traditional time-dependent
analyses appropriate)
• Post-treatment initiation adherence
⇒ Some participants may stop treatment after starting
(treatment affects adherence and traditional methods
inappropriate)
(Shrier et al. Under Review)
PARTIAL ADHERENCE
(Shrier et al. Under Review)
REGRESSION DISCONTINUITY
• Equal prognosis at onset: Randomization
• Known allocation mechanism (dependent or independent of
outcome
REGRESSION DISCONTINUITY
• Equal prognosis at onset: Randomization
• Known allocation mechanism (dependent or independent of
outcome
REGRESSION DISCONTINUITY
• Equal prognosis at onset: Randomization
• Known allocation mechanism (dependent or independent of
outcome
• Fuzzy cutoff: Students with marks just below cutoff artificially raised
• Linear vs. non-linear
REGRESSION DISCONTINUITY
• Equal prognosis at onset: Randomization
• Known allocation mechanism (dependent or independent of
outcome
Interaction
SUMMARY
• Review of commonly used analyses
⇒ ITT, PP and As Treated address different subgroups
• Introduction to Complier Avg. Causal Effect (CACE)
⇒ Same as PP when BC = Always Takers / Never Takers
⇒ ? Able to use sensitivity with Per Protocol (As Treated)
results instead of complicated published methods??
⇒ BUT: small sample sizes, target context changes BC
• Additional Topics
⇒ 3 types of partial adherence: Need to know why
⇒ Regression discontinuity powerful but with assumptions
REFERENCES
• Introduction to Causal Diagrams
⇒ Hernan et al. A Structural approach to selection bias. Epidemiology 2004;15:615628
⇒ Shrier & Platt. Reducing bais through directed acyclic graphs BMC Med Res
Methodol 2008;8:70
• Introduction to Complier Average Causal Effect
⇒ Shrier et al. Beyond intention-to-treat: What is the right question? Clin Trials
2014;11:28-37
⇒ Shrier et al. Principal stratification: A broader vision. Int J Biostats 2013;9:307-313.
⇒ Hernan et al. Beyond the intention-to-treat in comparative effectiveness research.
Clin Trials 2012;9:48-55
⇒ Baiocchi et al. Instrumental variable methods for causal inference. Stat Med
2014;33:2297-2340.
• Regression Discontinuity
⇒ Zuckerman et al. Application of regression-discontinuity analysis in pharmaceutical
health services research. Health Serv Res 2006;41:550-563
⇒ Cook. “Waiting for life to arrive”: A history of the regression-discontinuity design in
psychology, statistics and econcomics. J Econometrics 2008;142:636-654.