ISPOR 17th Annual European Congress, Amsterdam, November 2014 | PRM197
MULTI-LEVEL NETWORK META-ANALYSIS TO ACCOUNT FOR DOSE-RESPONSE
AND CLASS EFFECTS
Authors: Tim Reason1, Sofia Dias2, Nicky Welton2
1 IMS Health, London, United Kingdom.; 2 University of Bristol, Bristol, UK
U
Methods contd
Conducting a Network Meta-Analysis (NMA) involves
synthesising relative treatment effects from Randomised
Controlled Trials (RCTs) comparing several different
treatments1.
Grouping and splitting treatments (e.g by dose) is important
both from a methodological and a decision making
perspective2. An analysis conducted with interventions
grouped either at the ‘treatment’ or ‘class’ levels will
inevitably show more heterogeneity, but a network split too
thinly will be less powered to detect meaningful differences
between interventions and may not actually be connected.
The decision to group or split is often informed by the
decision making perspective. Clinicians may favour an
approach where interventions are grouped and compared at
the class level allowing flexibility to recommend specific
treatments to individual prescribing clinicians. Decision
makers making recommendations on the basis of costeffectiveness may prefer a splitting approach since individual
treatments and doses are associated with specific costs and
outcomes.
While arguments can be made for grouping and splitting,
each is associated with disadvantages. If doses are grouped
a comparison at the dose level is not possible and therefore
a fully informed decision cannot be made between competing
doses of the same intervention. Information regarding
heterogeneity that arises due to grouping of doses is also
lost.
Analyses conducted at the dose level make the assumption
that all treatment dosages have distinct effects and therefore
may lead to less precision in estimation of effect sizes since
doses of the same treatment will not borrow information
from each other.
Objectives
The objective of this study was to explore multi-level NMA
models where interventions could be compared at the
dose, treatment and class levels and the utility of these
models in explaining heterogeneity, improving model fit
and increasing precision of treatment effects
Methods
Standard NMA models were extended to account for dose response
and class effects, building on the methods proposed by Del Giovane
et al2. We adopt further notation so that xj,k tj,k and cj,k are used to
index the dose, treatment and class respectively for intervention j
in study k. One level models were fitted at each level, i.e:
𝑫𝒐𝒔𝒆 𝑳𝒆𝒗𝒆𝒍: 𝜹𝒋,𝒃,𝒌 ~𝑵 𝒅𝒙𝒋,𝒌,𝒕𝒋,𝒌 , 𝝈𝟐𝒅
‘Pain free at 2 hours’ was chosen as the main outcome of
interest. The network of RCT evidence is shown in figure 2
We fitted standard one level network meta-analysis models at
the dose, treatment and class levels simultaneously and
compared them to multi-level models accounting for doseresponse and/or class effects.
For multi-level models we built selected non-parametric
models proposed by Del Giovane et al2 and extended them to
account for class effects. We start with the standard definition
of a one level NMA as proposed by Cooper et al7.
𝒓𝒋𝒌~𝑩𝒊𝒏𝒐𝒎𝒊𝒂𝒍(𝒏𝒋𝒌, 𝒑𝒋𝒌)
𝝁𝒋𝒃
𝒍𝒐𝒈𝒊𝒕(𝒑𝒋𝒌 ) =
𝝁𝒋𝒃 + 𝜹𝒋𝒃𝒌
𝜹𝒋𝒃𝒌~𝑵(𝒅𝒃𝒌 , 𝝈𝟐 ),
𝒃 = 𝑨, 𝑩, 𝑪 𝒊𝒇 𝒌 = 𝒃
𝒊𝒇 𝒌 𝒂𝒍𝒑𝒉𝒂𝒃𝒆𝒕𝒊𝒄𝒂𝒍𝒍𝒚 𝒂𝒇𝒕𝒆𝒓 𝒃
𝒅𝒃𝒌 = 𝒅𝑨𝒌 − 𝒅𝑨𝑩
-rjk, pjk and njk denote the number of events, probability of
event and number at risk respectively in arm k of trial j
-jb is the log odds for treatment b in trial j
-jbk and dbk are the study specific and pooled log odds ratios
for treatment k relative to treatment b in trial j
200 mg
200 mg
(1-levD)
100 mg
𝑻𝒓𝒆𝒂𝒕𝒎𝒆𝒏𝒕 𝑳𝒆𝒗𝒆𝒍: 𝜹𝒋,𝒃,𝒌 ~𝑵 𝒅𝒕𝒋,𝒌 , 𝝈𝟐𝒕
100 mg
(1-levT)
85 mg
𝑪𝒍𝒂𝒔𝒔 𝒍𝒆𝒗𝒆𝒍: 𝜹𝒋,𝒃,𝒌 ~𝑵 𝒅𝒄𝒋,𝒌 , 𝝈𝟐𝒄
85 mg
(1-levC)
We also fitted a 3-level exchangeable variance model (3-levExch)
where, as well as doses being exchangeable within treatment,
effect sizes at the treatment level were also considered
exchangeable within class, i.e:
50 mg
50 mg
25 mg
25 mg
𝒅𝒙𝒋,𝒌 ,𝒕𝒋,𝒌 ~𝑵 𝒅𝒕𝒋,𝒌 , 𝜽𝟐
𝒅𝒕𝒋,𝒌 ~𝑵 𝒅𝒄𝒋,𝒌 , 𝜼𝟐
Where 2 is a common variance parameter for effect sizes across
the different treatments and 2 denotes a common variance
parameter for effect sizes across the different classes.
0
𝒅𝒙𝒋,𝒌,𝒕𝒋,𝒌 = 𝒅𝒙𝒋,𝒌−𝟏,𝒕𝒋,𝒌 + 𝒛𝒙𝒋,𝒌,𝒕𝒋,𝒌
𝒛𝒙𝒋,𝒌,𝒕𝒋,𝒌 ~𝑵(𝟎, 𝟐𝝈𝟐 )
Effect sizes at the lowest dose are then assumed exchangeable
within each treatment and these treatment level effects are
assumed exchangeable within classes in a similar way to the 3-levT
model. Effect sizes for each treatment and class were estimated by
calculating an inverse variance weighted average using postestimation to avoid the confounding bias associated with taking the
simple geometric mean.
1
2
Log odds ratio
3
0
1
2
Log odds ratio
3
Table 1- Model comparison
We also extended the monotonic non-parametric dose-response
model proposed by Del Giovane et al2. For this model (3-levMono)
we assume that the increments in effectiveness between doses of
the same treatment can be represented by a latent variable z which
is strictly >0, i.e:
Model
DIC
Residual deviance*
1-levD
1034
155.5
0.23 (0.14 , 0.32)
1-levT
1034
154
0.30 (0.22 , 0.39)
1-levC
1037
155
0.33 (0.26 , 0.42)
3-levExch
1031
156.2
0.24 (0.14 , 0.34)
3-levMono
1022
155
0.21 (0.13 , 0.30)
*Compared to 150 datapoints
Heterogeneity
(95% crI)
Figure 4- Modelled dose-response relationships vs placebo
Sumatriptan log-odds ratios by dose
2.3
2.1
1.9
Models were fitted using MCMC in OpenBUGS; flat normal priors
were given to all location parameters and uniform or half-normal
priors were given to all standard deviation parameters. All models
were run for 300,000 iterations with a 100,000 burn-in and
convergence was by visual inspection of plots. We compared all
fitted models in terms of DIC, posterior residual deviance and
heterogeneity.
We used four previously conducted Cochrane reviews3-6 in
aspirin, diclofenac, ibuprofen and sumatriptan for acute
pharmacological treatment of migraine as the data source.
Network meta-analysis models were developed to account for
‘dose’,
‘treatment’
and ‘class’
effects
simultaneously
and
distinct
and therefore
modifying
effects
of dose are
implicitly
applied
to the
data; a schematic
of This
the intervention
accounted
forcollected
in the treatment
definitions.
however,
hierarchy
modelled isbetween
shown intreatmen
Figure 1.
ignores similarities
Sumatriptan
1-levD
Sumatriptan
3-levMono
Log-odds ratio
Introduction
Figure 3- forest plots
1.7
1-levD
3-LevExch
3-levMono
1.5
1.3
1.1
Results
0.9
0
Statistics pertaining to model fit can be found in table 1. Treatment
effects for different doses of sumatriptan vs placebo with
associated credible intervals for 1-levD and 3-levMono can be seen
in figure 3. This is intended to show that using a multi-level
structure, i.e. allowing doses of the same intervention to borrow
information from each other leads to more precise estimates of
treatment effect. Estimated treatment effects from all the models
vs placebo for sumatriptan can be seen in figure 4. This is intended
to show how models making different assumptions around doseresponse will estimate treatment effects at different doses.
3-levMono was the best model in terms of DIC and heterogeneity
(Table 1). This model also produced the most precise treatment
effects (Figure 3). It can be seen from comparing the models that
the improvement in model DIC was due to a reduction in effective
parameters rather than a substantive improvement in fit as residual
deviances are very similar.
3-levExch had higher heterogeneity and poorer fit than both the 1levD and 3-levMono models and failed to capture the monotonic
nature of the dose-response. This is due to the fact that imposing
exchangeability causes all the effect sizes to be pulled towards the
overall mean. In general we expect dose-response to be
monotonic and therefore the exchangeability assumption is violated
since we know a-priori which effect sizes are likely to be higher.
This approach should therefore be used with caution and only if
there is very strong reason to believe a-priori that the doseresponse is flat.
3-levMono appeared to underestimate the effect size for
sumatriptan 200 mg; this is likely due to lack of data on the 200
mg dose and the fact that there is no information on intermediate
doses (i.e. 100-200 mg)
Figure 1- Three level intervention hierarchy
FOR FURTHER INFORMATION: Please contact Tim Reason ([email protected])
IMS HEALTH | 210 PENTONVILLE ROAD, LONDON N1 9JY, UNITED KINGDOM
©2014 IMS Health Incorporated and its affiliates. All rights reserved. Trademarks are registered in the United States and in various other countries.
50
100
150
Dose (mg)
200
250
Conclusions
Careful consideration should be given when making
assumptions about dose-response in NMA.
Grouping doses together for NMA can cause high amounts of
unexplained heterogeneity that could be explained by a more
flexible modeling method.
Allowing for dose-response in NMA leads to effect sizes with
higher precision but careful thought should be given to the
underlying assumptions and estimates of treatment effects at
different doses for the chosen model.
A more flexible parametric approach to multi-level doseresponse NMA would be desirable.
References
1 Caldwell DM, Ades AE, Higgins JPT: Simultaneous comparison of multiple treatments: combining direct and indirect
evidence. BMJ 2005, 331:897–900.
2 Giovane CD, Vacchi L, Mavridis D, Filippini G, Salanti G. Network meta-analysis models to account for variability in
treatment definitions: application to dose effects. Statistics in medicine 2013;32(1):25-39.
3 McCrory DC, Gray RN. Oral sumatriptan for acute migraine. Cochrane Database Syst Rev 2003;3(3).
4 Kirthi V, Derry S, Moore RA, McQuay HJ. Aspirin with or without an antiemetic for acute migraine headaches in adults.
Cochrane Database Syst Rev 2013;4.
5 Rabbie R, Derry S, Moore RA, McQuay HJ. Ibuprofen with or without an antiemetic for acute migraine headaches in adults.
Cochrane Database Syst Rev 2010;10(10).
6 Derry S, Rabbie R, Moore RA. Diclofenac with or without an antiemetic for acute migraine headaches in adults. Cochrane
Database Syst Rev 2013;4.
7 Cooper NJ, Sutton AJ, Morris D, et al. 2009. Addressing between-study heterogeneity and inconsistency in mixed
treatment comparisons: application to stroke prevention treatments in individuals with non-rheumatic atrial
fibrillation. Stat Med 28: 1861–1881.
Figure 2- Evidence network of RCTs