EFPSI-BBS Meeting on M&S, Basel 13 September 2012
Using M&S to develop a novel longitudinal modelbased approach for efficacy assessments, with an
application to biosimilars in rheumatoid arthritis
D Renard1, B Bieth1, F Mentré2, G Heimann1, I Demin1,
B Hamrén1, S Balser3
1Modeling
and Simulation, Novartis, Basel (Switzerland), 2 UMR 738, INSERM and Universite
Paris Diderot, Paris (France), 3 Sandoz Biopharmaceuticals, Holzkirchen (Germany)
Outline
 Background
 M&S approach
 Simulation models
 Statistical methodology
 Some results
 Next steps
2
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments
Problem that we attempt to address
 Question
• Can the size of a Phase III study be reduced by optimizing the
choice of analysis methodology, as compared to using the
standard approach (end-point analysis) ?
 An innovative model-based approach was developed
 Aim to maintain strict regulatory standards for phase III
• Sound statistical properties
• Fully pre-specified analysis
 Note: this is NOT how pharmacometric (M&S) analyses
are routinely applied in drug development!
3
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments
Application: Biosimilars in rheumatoid arthritis (RA)
 In RA studies, standard efficacy assessments rely on the
ACR20 after 24 weeks of treatment
(ACR20=American College of Rheumatology 20% response criterion)
 Objective is to demonstrate similar efficacy between the
reference and biosimilar products
 Formally achieved through statistical equivalence testing
 Standard method:
• Based on response rates (#ACR20 responders/#patients)
• Only uses data collected at Week 24
• Equivalence is inferred when the 95% confidence interval (CI) for the
treatment effect is included within the equivalence margins
4
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments
M&S approach
 Step 1. Establish simulation models
• Requirement: simulate realistic study outcomes for ACR20 (including compound
specific characteristics)
• Simulation models were built using a mix of literature (summary level) and internal
(patient level) data
• Models components: time course of response, different sources of variability, and
dropout characteristics
 Step 2: Development of statistical methodology
• Longitudinal model-based approach for equivalence testing
• Rooted to nonlinear mixed effect (NLME) modeling, entails pre-specifying different
candidate models and relies on model averaging techniques
 Step 3. Simulations and assessment of performance
• An extensive simulation program was conducted to assess performance of the
longitudinal model-based approach
• More is needed!
5
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments
Longitudinal model-based meta-analysis as basis
for setting up realistic simulation models
 9 anti-TNF agents, 37 studies
Demin et al, Clin Pharmacol Ther (2012)
ACR20 responses
to methotrexate
(MTX)
 Symbols = study
results (size
proportional to #
patients)
 Solid lines
connect points
from each study
 Broken lines =
smooth curves
for each patient
population
6
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments
Simulation models
 Different simulation models were considered to ensure
robustness of the conclusions
 Models were set up to
mimick different
compounds based on
meta-analytic
characterization
 Internal data were used
to inform variability
parameters in the
models
 An example of model is
given on slide 10
7
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments
Literature database was used to build assumptions
about dropout
Assumptions
 Dropout increases
proportionally over
time to reach 15% at
week 24
 Patients not
responding at the
previous visit were
twice more likely to
drop out compared to
patients who were
responders
 Dropout for non
responders assumed
to be due to lack of
efficacy
8
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments
Longitudinal model-based testing (1)
Pre-specified
candidate models
NLME models are pre-specified to describe transition
probabilities based on the Markov assumption linking
the entire longitudinal data.
Pr00
Pr10
0
Nonresponder
1
Responder
Pr01
Pr11
ACR20
Model averaging
Lacroix BD et al, Clin Pharmacol
Ther 2009, 86: 387-395.
1
Pr10
Pr10
Pr01
0
0
Equivalence
testing
9
2
4
Visits (week)
20
24
The response rate at week 24 is a function of transition
probabilities:
 PACR200 
1  Pr10 (tk ) 1  Pr11 (tk )  0 

  
 
 Pr (t )
 1
P

Pr
(
t
)
k
11 k
 10 k
 
 ACR201 
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments
Longitudinal model-based test (2)
Pre-specified
candidate models
Markov models are specified for two independent
transition probabilities, e.g.
 Pr10i (tk ) 
   0  1TRTi  (  2  3TRTi ) log( tk )  10i
log

1

Pr
(
t
)
10i k 

i= subject
k=study visit
 Pr11i (tk ) 
   4  5TRTi  (  6   7TRTi ) log( tk )  11i
log

1

Pr
(
t
)
11i k 

Model averaging
Equivalence
testing
• 10 candidate models are pre-specified, which differ
through constraints in parameters and function of time
• Range of simple up to more complex models
Estimating ACR20 response rates for a given model
• Individual response probabilities are derived from the
Markov property (previous slide) using modeled
transition probabilities
• Population response rates are derived by integrating
out the random effects (η10, η11)
10 | EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments
Longitudinal model-based test (3)
Pre-specified
candidate models
Model averaging
Equivalence
testing
Key concept: model averaging
• Used to estimate the response rates at week
24 by combining results from the different
candidate models
• Point estimate = weighted average of the
individual model estimates
p̂k  m1 wm p̂ k|m
M
k = treatments
m = models
• Weights are functions of a statistical criterion
(BIC)
• Larger weights are assigned to models that fit
the data better.
11 | EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments
Longitudinal model-based test (4)
Pre-specified
candidate models
Example: candidate models
10 candidate models
3 models shown with corresponding weights (w)
W=.18
W=.75
W=.00
Model averaging
Equivalence
testing
12 | EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments
Longitudinal model-based test (5)
Pre-specified
candidate models
Example: model averaging
10 candidate models
3 models shown with corresponding weights (w)
Model average estimate (thick black)
W=.18
W=.75
W=.00
Model averaging
Equivalence
testing
13 | EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments
Longitudinal model-based test (6)
Pre-specified
candidate models
Model averaging
• Bootstrap is used to derive a confidence
interval for the treatment difference at week
24.
• Bootstrap datasets are built by resampling
over subjects.
• The 95% CI is compared with the equivalence
margins for equivalence testing.
Equivalence
testing
14 | EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments
Model-based analysis versus classical test
 Model-based approach does not change the nature of the
Difference in response rates (%)
comparability testing and makes it more efficient!
Simulation #
Symbols correspond to point estimates and bars to 95% CI
10 simulation runs assuming strict equivalence (n=180/arm)
15 | EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments
Simulation results: Power
 40% reduction in sample size compared to the classical
test at power levels of 80 and 90%
Power assessed based on 1000 simulations
16 | EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments
Simulation results: Type 1 error
 Type I error rate is close to the 2.5% nominal level
Assessment based on 1000
simulations (bars represent
Monte-Carlo error)
17 | EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments
Summary of methodology
 The model-based analysis uses all data collected to derive an
estimate and its confidence interval of the treatment effect at the end
of the study (week 24)
 Model averaging is used to prevent against model misspecification
 Number of patients was reduced up to 40% with the longitudinal
model-based analysis – confirmed by additional simulation scenarios
and sensitivity analyses
 Extensions: the principles could be applied to other types of endpoints
in RA (e.g. DAS28), other therapeutic areas for biosimilarity
assessments, or for efficacy assessments in late stage clinical
development of new drugs
18 | EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments
Summary of health authority interactions
 Initial project feedback from EMA was negative
 Overall encouraging feedback obtained at EMA/EFPIA
workshop (Dec 2011)
• Absence of theoretical results to justify type I error control appears
to be a critical concern deserving careful consideration
 How can regulatory acceptance be gained ?
• Planned interaction with EMA through Innovation Task Force
process
• Perform large simulation study to evaluate type 1 error control
(ongoing)
• Use model-based approach as supportive analysis in future studies
• Present and discuss the method with the scientific community
19 | EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments