Some methodological issues in value of
information analysis:
an application of partial EVPI and EVSI to
an economic model of Zanamivir
Karl Claxton and Tony Ades
Partial EVPIs
Light at the end of the tunnel……
……..maybe it’s a train
A simple model of Zanamivir
pcz
pip
phz
0.018
1-phz
0.982
phz
0.018
1-phz
0.982
phs
0.025
1-phs
0.975
phs
0.025
1-phs
0.975
0.367
0.340
1-pcz
0.633
Zanamivir
pcs
1-pip
0.452
0.660
1-pcs
0.548
Cost-effectiveness plane
35.00
corr = -0.06
30.00
Incremental cost
25.00
20.00
15.00
10.00
5.00
-0.00040
-0.00020
0.00
0.00000
0.00020
0.00040
0.00060
0.00080
0.00100
Incremental effect (quality adjusted life years)
0.00120
0.00140
0.00160
Cost-effectiveness acceptability curve
1.0
0.9
Probability Zanamavir is cost-effective
0.8
0.7
0.6
0.5
P[inb > 0] = 0.461
0.4
0.3
0.2
0.1
ICER = £51,376
0.0
£0
£10,000
£20,000
£30,000
£40,000
£50,000
£60,000
Monetary value of health outcome
£70,000
£80,000
£90,000
£100,000
Distribution of inb
.026
Normal Distribution
Mean = (£0.51)
Std Dev = £12.52
.020
.013
.007
inb
.000
(£40.00)
(£20.00)
£0.00
£20.00
£40.00
EVPI for the decision
EVPI = EV(perfect information) - EV(current information)
Zan
pip
Pip
Zan
Std
EV(Perfect)
1-pip
EV(current)
Zan
1-pip
pip
Std
Std
1-pip
Expected value of perfect information (EVPI)
£6.00
£5.00
EVPI
£4.00
£3.00
£2.00
£1.00
£0.00
£0
£10,000
£20,000
£30,000
£40,000
£50,000
£60,000
Monetary value of health outcome
£70,000
£80,000
£90,000
£100,000
Partial EVPI
EVPIpip = EV(perfect information about pip) - EV(current information)
p(..)
…….
p(..)
…….
1-p(..)
…….
p(..)
……
1-p(..)
……
p(..)
……
1-p(..)
……
p(..)
……
1-p(..)
……
pip
Zan
1-p(..)
…….
pip
Zan
p(..)
……
1-pip
Std
1-p(..)
……
EV(Perfect)
EV(current)
p(..)
……
pip
Zan
1-p(..)
……
1-pip
Std
p(..)
……
Std
1-pip
1-p(..)
EV(optimal decision for a
particular resolution of pip)
……
-
EV(prior decision for the
same resolution of pip)
Expectation of this difference over all resolutions of pip
Partial EVPI
Some implications:
 information about an input is only valuable if it changes
our decision
 information is only valuable if pip does not resolve at its
expected value
General solution:
 linear and non linear models
 inputs can be (spuriously) correlated
Partial EVPI
£4.00
EVPIpip
£3.50
EVPIpcz
£3.00
EVPIphz
EVPIpcs
Partial EVPI
£2.50
EVPIphs
£2.00
EVPIupa
EVPIrsd
£1.50
£1.00
£0.50
£0.00
£20,000
£25,000
£30,000
£35,000
£40,000
£45,000
£50,000
£55,000
Monetary Value of Health Outcome
£60,000
£65,000
£70,000
Felli and Hazen (98) “short cut”
EVPIpip = EVPI when resolve all other inputs at their expected value
Appears counter intuitive:

we resolve all other uncertainties then ask what is the value of pip ie
“residual” EVPIpip ?
But:

resolving at EV does not give us any information
Correct if:


Partial EVPI
Felli and Hazen
linear relationship between inputs and net benefit
inputs are not correlated
pip
1.02039
1.02752
pcz
0.00688
0.00363
phz
0.53597
0.50388
pcs
0.00000
0.00000
phs
0.95540
0.92898
upa
1.81184
1.77514
rsd
3.54898
3.52854
So why different values?


The model is linear
The inputs are independent?
Spurious correlation
pip
pcz
phz
pcs
phs
upa
rsd
pip
pcz
phz
pcs
phs
upa
0.12
0.00
0.02
0.02
0.05
-0.06
-0.04
0.01
-0.03
0.00
0.02
0.08
0.02
0.06
0.00
0.08
-0.02
0.00
0.03
0.01
-0.01
rsd
“Residual” EVPI
EVPI when resolve all other inputs at each realisation ?



Partial EVPI
Residual EVPI
wrong current information position for partial EVPI
what is the value of resolving pip when we already have perfect
information about all other inputs?
Expect residual EVPIpip < partial EVPIpip
pip
1.02039
0.17985
pcz
0.00688
0.00201
phz
0.53597
0.06510
pcs
0.00000
0.00000
phs
0.95540
0.14866
upa
1.81184
0.49865
rsd
3.54898
1.98472
Thompson and Evans (96) and Thompson and Graham (96)




Felli and Hazen (98) used a similar approach
Thompson and Evans (96) is a linear model
emphasis on EVPI when set others to joint expected value
requires payoffs as a function of the input of interest
inb simplifies to:
inb =
Rearrange:
pip: inb =
pcz: inb =
phz: inb =
pcs: inb =
phs: inb =
upd: inb =
rsd: inb =
Reduction in cost of uncertainty
RCUE(pip) = EVPI - EVPI(pip resolved at expected value)


intuitive appeal
consistent with conditional probabilistic analysis
But
 pip may not resolve at E(pip) and prior decisions may change
 value of perfect information if forced to stick to the prior
decision ie the value of a reduction in variance
 Expect RCUE(pip) < partial EVPI
Partial EVPI
ROLE(pip)
pip
pcz
1.02039 0.00688
0.32380 -0.00035
phz
0.53597
0.03770
pcs
0.00000
0.00099
phs
0.95540
0.18698
upa
1.81184
0.56995
rsd
3.54898
2.13151
Reduction in cost of uncertainty
RCUpip = EVPI – Epip[EVPI(given realisation of pip)]
= [EV(perfect information) - EV(current information)] Epip[EV(perfect information, pip resolved) - EV(current information, pip resolved)]
Partial EVPI
ROLpip
pip
1.02039
1.16434
pcz
0.00688
0.00451
phz
0.53597
0.50858
pcs
0.00000
0.00060
phs
0.95540
0.99372
upa
1.81184
1.88313
rsd
3.54898
3.69577
spurious correlation again?
RCUpip = Epip[EVPI – EVPI(given realisation of pip)] = partial EVPI
Partial EVPI
E[ROLpip]
pip
1.02039
1.02039
pcz
0.00688
0.00688
phz
0.53597
0.53597
pcs
0.00000
0.00000
phs
0.95540
0.95540
upa
1.81184
1.81184
rsd
3.54898
3.54898
EVPI for strategies
Value of including a strategy?

EVPI with and without the strategy included
 demonstrates bias
 difference = EVPI associated with the strategy?

EV(perfect information, all included) –
EV(perfect information, excluded)
Eall inputs[Maxd(NBd|all inputs)] – Eall inputs[Maxd-1(NBd-1|all inputs)]
Conclusions on partials
Life is beautiful …… Hegel was right
……progress is a dialectic
Maths don’t lie ……
……but brute force empiricism can mislead
EVSI……
…… it may well be a train
Hegel’s right again!
……contradiction follows synthesis
EVSI for model inputs





generate a predictive distribution for sample of n
sample from the predictive and prior distributions to
form a preposterior
propagate the preposterior through the model
value of information for sample of n
find n* that maximises EVSI-cost sampling
EVSI for pip
Epidemiological study n






prior:
pip  Beta (, )
predicitive:
rip  Bin(pip, n)
preposterior:
pip’ =
(pip(+)+rip)/((++n)
as n increases var(rip*n) falls towards var(pip)
var(pip’) < var(pip) and falls with n
pip’ are the possible posterior means
EVSIpip
= reduction in the cost of uncertainty due to n obs on pip
= difference in partials (EVPIpip – EVPIpip’)
Epip[Eother[Maxd(NBd|other, pip)] - Maxd Eother(NBd|other, pip)] Epip’[Eother[Maxd(NBd|other, pip’)] - Maxd Eother(NBd|other, pip’)]
E(pip’) = E(pip)
Epip[Maxd Eother(NBd|other, pip)] = Epip’[Maxd Eother(NBd|other, pip’)]
pip’has smaller var so any realisation is less likely to change decision
Epip[Eother[Maxd(NBd|other, pip)] > Epip’[Eother[Maxd(NBd|other, pip’)]
Expected Value of Sample Information (EVSIpip)
1.00
Value of information
0.80
0.60
Posterior Partial EVPI
EVSIpip
Prior Partial EVPI
0.40
0.20
0.00
0
200
400
600
800
1000
Sample size (n)
1200
1400
1600
EVSIpip
Why not the difference in prior and preposterior EVPI?
 effect of pip’ only through var(NB)
 change decision for the realisation of pip’ once study is
completed
 difference in prior and preposterior EVPI will
underestimate EVSIpip
n
110
160
320
1750
EVSIpip
0.37790089 0.466544 0.635703 0.933119
EVPI-EVPI' 0.25157296 0.268576 0.292854 0.31551
Implications




EVSI for any input that is conjugate
 generate preposterior for log odds ratio for complication and
hospitalisation etc
trial design for individual endpoint (rsd)
trial designs with a number of endpoints (pcz, phz, upd, rsd)
 n for an endpoint will be uncertain (n_pcz = n*pip, etc)
 consider optimal n and allocation (search for n*)
combine different designs eg:
 obs study (pip) and trial (upd, rsd) or obs study (pip, upd),
trial (rsd)…. etc