Optimal Design of New Research:
IVIG for Severe Sepsis
Nicky J Welton
16th
ISPOR
Annual European Conference
Dublin, Ireland. 4th-6th November 2013
School of Social and Community Medicine
Expected Value of Sample
Information (EVSI)
• EVSI measures the value (in terms of Net
Benefit) of reducing uncertainty by
running a study of a given design
• Can compare the benefits and costs of a
proposed study
• To see if a particular study design likely to be
a good use of resources
• To find the optimal study design
EVSI Can Help Assess
• Do we really need another study?
• What type of study (or studies)?
• RCT (# arms?)? Cohort? etc.
• What should the new study measure?
• Efficacy? Which interventions? Which outcomes?
• Economic data?
• Length of follow-up?
• Which patient population for inclusion?
• What sample size to use?
EVSI: Basic Idea
• A new study with given design provides data, D
• Reduces uncertainty in some model parameters
• Update model inputs (e.g. meta-analysis)
• Update the cost-effectiveness analysis
• If the optimal decision changes gain in NB from
using new optimal treatment
• If optimal decision unchanged, no gain in NB
• Average this gain in NB over future possible
datasets D to obtain EVSI
• Based on a prediction from existing evidence
Example: IVIG for Severe Sepsis
• Severe sepsis/septic shock
• Inflamatory response to infection
• Accounts for ≈ 30% adults admitted to ICU
• ≈15,000 deaths per year in England
• Intravenous Immunoglobulin (IVIG)
• Blood product (… CJD risk), used in other countries
• Commissioned by NIHR HTA
• Is it cost-effective to use IVIG in the UK?
• Is there value in a new trial?
IVIg vs Control
Note: heterogeneity
Explaining Heterogeneity
1. Some measure of risk of bias
• control (albumin or no treatment)
2. Treatment regimen covariate
• duration, dose, etc.
• … no clinical rationale
Choice of control: No Trt or Albumin
Duration of Treatment: 2,3,5, or 7 days
(no clinical rationale)
Relative Effect Summary
• CEA model requires prediction of relative effect
in target decision population
• EVSI requires prediction for relative effect
expect in new study
Model
Log Odds Ratios (95% CrI)
Fixed Effect Model
-0.30 (-0.55, -0.05)
(covariates: control=albumin;
duration=3 days)
Random Effects Model
-0.55 (-1.14, -0.05) RE Mean
(covariate: control=albumin) -0.56 (-1.86, 0.60) Pred. Distn
Cost-effectiveness model
• Inputs: Relative treatment effects (meta-analysis);
baseline mortality (ICNARC); Long-term mortality (UK
cohort); Long-term costs (Canadian cohort); Quality of
Life (US and Scottish Cohorts)
Value of Partial Information
Parameters
Population
EVPPI
(T=10 years)
1. Baseline mortality (short term)
£0
2. Relative treatment effect of IVIG
£173,736,363
3. Long term mortality
4. Long term costs
£0
£249,956,670
5. Quality of life
£7,919,499
Expected Net Benefit of
Sampling
Sensitivity to Efficacy Model
Model
ICER
Prob (CE) Max
ENBS
£20,000
threshold
Optimal
sample
size (n*)
Fixed Effect (with
covariates)
£20,850
0.505
£137m
1900
Random Effects
(Pred. Distn)
Random Effects
(Mean)
£16,177
0.597
£687m
1200
£15,488
0.721
£315m
1400
Summary
• IVIg borderline effective / cost-effective
• Value in a well-conducted multi-centre trial
• Basic science required first to learn more about
the mechanisms of action (e.g. dose-ranging
studies)
• Results are sensitive to efficacy model
• Issues with study quality
• Important to explore and reflect heterogeneity
appropriately
References
• Soares M, Welton NJ, Harrison DA, Peura P, Shankar
Hari M, Harvey SE, Madan J, Ades AE, Palmer SJ,
Rowan KM. An evaluation of the feasibility, cost and
value of information of a multicentre randomised
controlled trial of intravenous immunoglobulin for sepsis
(severe sepsis and septic shock): incorporating a
systematic review, meta-analysis and value of
information analysis. Health Technology Assessment
2012 16(7). http://www.hta.ac.uk/fullmono/mon1607.pdf