Risk Analysis, Vol. 28, No. 4, 2008
DOI: 10.1111/j.1539-6924.2008.01078.x
Uncertainty and Sensitivity Analyses of a Decision Analytic
Model for Posteradication Polio Risk Management
Radboud J. Duintjer Tebbens,1 Mark A. Pallansch,2 Olen M. Kew,2 Roland W. Sutter,3
R. Bruce Aylward,3 Margaret Watkins,4 Howard Gary,4 James Alexander,2 Hamid Jafari,5
Stephen L. Cochi,4 and Kimberly M. Thompson1 ∗
Decision analytic modeling of polio risk management policies after eradication may help inform decisionmakers about the quantitative tradeoffs implied by various options. Given the
significant dynamic complexity and uncertainty involving posteradication decisions, this article aims to clarify the structure of a decision analytic model developed to help characterize
the risks, costs, and benefits of various options for polio risk management after eradication of
wild polioviruses and analyze the implications of different sources of uncertainty. We provide
an influence diagram of the model with a description of each component, explore the impact
of different assumptions about model inputs, and present probability distributions of model
outputs. The results show that choices made about surveillance, response, and containment
for different income groups and immunization policies play a major role in the expected final
costs and polio cases. While the overall policy implications of the model remain robust to the
variations of assumptions and input uncertainty we considered, the analyses suggest the need
for policymakers to carefully consider tradeoffs and for further studies to address the most
important knowledge gaps.
KEY WORDS: Decision analysis; polio eradication; risk management; sensitivity analysis; uncertainty
analysis
1. INTRODUCTION
With global polio eradication approaching,(1)
national, regional, and global policymakers must
prepare to make choices among available poliomyelitis (polio) risk management policies in the
posteradication era.(2) An existing decision analytic
model(3) integrates information about the available
set of policy options,(2) risks,(4) costs,(5) and outputs from a dynamic outbreak submodel(6) into
an overall framework that aims to inform policymakers about the quantitative tradeoffs at a relatively aggregate level. The overall model incorporates many of the significant complexities involved
in the decisions and runs multiple random iterations to quantify the impact of small outbreak probabilities and model input uncertainties. The overall model generates probabilistic output for a large
1
Kids Risk Project, Harvard School of Public Health, Boston,
MA, USA.
2 Centers for Disease Control and Prevention, National Center for
Immunization and Respiratory Diseases, Division of Viral Diseases, Atlanta, GA, USA.
3 World Health Organization, Polio Eradication Initiative,
Geneva, Switzerland.
4 Centers for Disease Control and Prevention, National Center for
Immunization and Respiratory Diseases, Global Immunization
Division, Atlanta, GA, USA .
5 National Polio Surveillance Project, World Health Organization,
New Delhi, India.
∗ Address correspondence to Kimberly M. Thompson, Kids Risk
Project, Harvard School of Public Health, 677 Huntington Ave.,
3rd Floor, Boston, MA 02115, USA; tel: 617-432-4285; fax: 617432-3699; [email protected].
855
C 2008 Society for Risk Analysis
0272-4332/08/0100-0855$22.00/1 856
number of permutations of policies and assumptions. To comprehensively evaluate the tradeoffs, decisionmakers should also consider uncertainty and
sensitivity analyses of the results,(7) which can help
to identify the sources of model output uncertainty
and target future research to reduce uncertainty regarding assumptions and model inputs.(8)
Complementing the base case results presented
elsewhere,(3) this article aims to provide a more comprehensive report of the model structure and associated uncertainties. Based on this analysis, we identify
areas for which policymakers might find value in conducting studies to address gaps in knowledge.
We define the base case as the simulation
that maintains the general assumptions while varying many uncertain inputs probabilistically, which
yielded the main results presented elsewhere.(3)
Section 2 presents an influence diagram of the overall
model to describe its mechanics and provides details
about the sensitivity and uncertainty analyses. Section 3 provides the main results in the form of model
output distributions for the base case and discusses
several departures from the base case assumptions.
It also provides results of a probabilistic sensitivity analysis of the base case, focused on a subset of
model outputs of high interest: (1) the incremental
net benefit of IPV versus OPV without SIAs, aggregated over the low- and middle-income groups and
(2) the incremental net benefit of no routine versus
OPV without SIAs, aggregated over the low- and
middle-income groups. Finally, Section 4 discusses
the limitations, insights, and policy implications of
the analyses presented. A technical appendix (available at http://www.kidsrisk.harvard.edu/) provides
additional results and details about the methods.
2. METHODS
2.1. Model Framework
The World Health Organization (WHO) is currently planning for the discontinuation of routine
OPV use as soon as possible after assurance of the
global interruption of wild poliovirus transmission to
avoid the on-going risk of circulating vaccine-derived
poliovirus (cVDPV) outbreaks(9) and to eliminate
vaccine-associated paralytic polio (VAPP). Consistent with the Strategic Plan 2004–2008 of the polio eradication initiative (PEI),(10) implementation
of posteradication risk management policies started
during the current endgame phase (e.g., first phases
of laboratory containment, establishment of a stockpile) and will continue until several years after the
Duintjer Tebbens et al.
certification of wild poliovirus eradication in all regions. For simplicity, we define the starting year T 0 of
the overall model as the point of implementation of
the full set of posteradication risk management policies. We assume that T 0 occurs after the reasonable
assurance of interruption of wild poliovirus transmission, implying no risk of continued undetected
circulation of wild polioviruses. The analytical time
horizon stretches through 20 years after T 0 . While
the true value of T 0 remains uncertain and an important question to resolve after the apparent completion of global eradication, when required we use
population data for the years 2010 through 2029. We
present all costs in U.S. dollars of the year 2002 (notation US$2002) adjusted using the Consumer Price
Index when necessary,(11) and discount all costs and
health outcomes to their T 0 net present value using a
fixed discount rate of 3%, unless otherwise noted.(12)
The overall model stratifies the world into four income groups (i.e., low, lower-middle, upper-middle,
and high income groups, based on the 2002 World
Bank classification).(13)
2.2. Components of the Model
Fig. 1 shows an influence diagram of the various components of the overall model. The rectangle
denotes a decision node, the oval a random event,
the diamond a submodel, and the rectangles with
rounded edges denote intermediate variables (white)
or outcomes (gray). While outbreaks represent random events, many rectangular nodes are a function
of the iteration, indicating that their values depend
on inputs sampled from probability distributions that
reflect input uncertainty and variability (see below).
The decisions represent the starting point of
the model and include multiple categories based on
prior research.(2) The routine immunization options
include continued routine trivalent oral poliovirus
vaccine (OPV) use, switching to (or continuing in
the high-income group) the enhanced-potency inactivated poliovirus vaccine (IPV), or cessation of all
routine poliovirus vaccinations. In the event of continued routine OPV immunization, countries may
elect to conduct periodic supplemental immunization activities (SIAs) or not, modeled as on average approximately one SIA of two rounds per
two years. The surveillance options include continued acute flaccid paralysis (AFP) surveillance(14) or
passive surveillance with or without a global environmental surveillance system. While (trivalent)
OPV or IPV in some circumstances remain potential vaccines for outbreak response,(2) we focus on
Polio Model Uncertainty and Sensitivity Analyses
857
Decisions
Risk inputs
f(income group)
routine immunization
SIAs
surveillance
response
containment
population immunity at T
T00
Fixed global and
country-level costs
f(decisions, income group,
year, iteration)
f(income group, iteration)
Population data
f(income group, year)
Population immunity
Sub-model
Submodel inputs
inputs
f(decisions, income group,
year, iteration)
f(decisions, income
group)
Outbreak Poisson rates
per 100 million people
Routine VAPP
VA PP
cases
f(decisions, income group, year,
iteration)
f(decisions, income
group, year, iteration)
Random cVDPV risk
case
f(income group, iteration)
Random outbreak
population size, R0,
and coverage
Population data
f(income group, year)
Random number of
outbreaks
f(income group, year,
iteration)
Dynamic outbreak
sub-model
submodel
f(decisions, income group,
year, iteration)
f(decisions, income group,
year)
Cost inputs
f(income group,
iteration)
OPV doses used in
response
OPV infections
f(decisions, income group,
year, iteration)
f(decisions, income group,
year, iteration)
VAPPP rates
VAP
rates
per OPV
infection
f(iteration)
Total costs
Total response costs
Aggregated over
all years
f(decisions, income
group, iteration)
Aggregated over
outbreaks
f(decisions, income group,
year, iteration)
Paralytic cases due to
outbreak
Total outbreak and
response VA
VAPP
PP cases
cases
f(decisions, income group,
year, iteration)
Aggregated over outbreaks
f(decisions, income group, year,
iteration)
Total cases
Aggregated over
all years
f(decisions, income
group, iteration)
Acronyms: cVDPV = circulating vaccine-derived poliovirus; iVDPV = immunodeficient vaccine-derived poliovirus; OPV = oral poliovirus
vaccine; SIAs = supplemental immunization activities; VAPP = vaccine-associated paralytic polio.
Fig. 1. Influence diagram of the overall model.
response strategies using monovalent OPV (mOPV)
of the outbreak serotype given current recommendations.(15) Based on evaluation of different response
scenarios, we model two mOPV response strategies
with 45 and 70 days, respectively, while fixing the assumptions about the number of rounds (3), coverage of each round (90%), interval between rounds
(30 days), duration of rounds (3 days), and target age
groups (all persons born since the last year with IPV
or OPV vaccination rounded to the next multiple of
5).(6,16) With elevated containment standards for IPV
manufacturing facilities and laboratories with (potentially) infectious materials representing a prerequisite for OPV cessation,(10,17) the decision for containment consists of whether or not to continue actively maintaining the guidelines in the long term.(2)
The final decision category refers to the intensity of immunization activities prior to T 0 . While
technically not within the analytical time horizon,
such activities would affect the population immunity
and risks after T 0 and come at real costs.(2,4,5) We
model the option to continue SIAs until T 0 in the low
and middle income groups (effectively equivalent to
conducting a global immunization day just prior to
T 0 ) yielding maximum population immunity (MPI),
or not, which leads to realistic population immunity
(RPI). In addition, we explore the impact of conducting targeted immunization activities (TIAs) in highrisk areas within the low and lower-middle income
groups. Given progress toward establishing a global
mOPV stockpile,(18) we do not model the options
of building national stockpiles or not establishing a
stockpile,(2) although some nations might consider
establishing a national stockpile. We do not model
the impact of management of long-term immunodeficient vaccine-derived poliovirus (iVDPV) excretors
858
because no specific policy currently exists other than
monitoring identified excretors.(2,4,5)
Any chosen set of decisions yields fixed costs,
which we estimate using income-group-dependent
cost inputs.(5) We characterize the major cost components related to vaccination and field AFP surveillance (i.e., not surveillance costs incurred to run the
global polio laboratory network (14,19,20) as countrylevel costs (even if external donors may contribute
a proportion of them). We view costs of laboratory
surveillance (including environmental surveillance),
containment, and the global stockpiles as global programmatic costs. We assume that the routine IPV
schedule for the high income group includes three
doses delivered in a combination vaccine, while low
and middle income countries would offer two doses
of IPV in a single antigen based on currently foreseeable vaccine formulation options.(21,22)
Continued routine OPV use implies a relatively
predictable number of VAPP cases, which we calculate by multiplying the typical number of primary
(i.e., successful vaccinations) and secondary OPV infections (which depend on the SIA policy and several other model inputs) by the VAPP rates per primary or secondary OPV infection, respectively.(4)
The decisions also imply different outbreak rates,
which follow from income-group-dependent risk inputs and vary with time.(4) We express risks in
terms of annual Poisson rates per 100 million people and assume independence of the rates between
geographical areas (i.e., we obtain aggregate rates
for an income group during a given year by multiplying the per 100 million rates by the aggregate
population in the income group in that year). Similarly, we assume independent probabilities for different outbreak types and focus on the aggregate
rates for the three types of outbreaks (i.e., cVDPV
outbreaks, iVDPV-induced outbreaks, and release
of transmissible poliovirus from a laboratory, IPV
manufacturing facility, or through an act of bioterrorism). For cVDPV outbreaks, we model different rates depending on the income group, routine,
and supplemental immunization policies, the population immunity at T 0 , and the cVDPV risk case.
The low-risk case bases the prediction of the initial
cVDPV outbreak rates on the observed frequency of
confirmed cVDPV outbreaks (i.e., 6 total outbreaks
during 1999–2005), and the high-risk case bases the
prediction on confirmed outbreaks and ambiguous
(aVDPV) events (i.e., 12 total outbreaks during
1999–2005).(4) We randomly sample the cVDPV
risk case assuming equal probability of each case
Duintjer Tebbens et al.
(Table I). For iVDPV outbreaks, we only model different rates depending on income group and routine
immunization policy. For (un)intentional poliovirus
releases, the rates depend on the routine and supplemental immunization and containment policies.
Using population data and the outbreak Poisson rates per 100 million people, each iteration of a
simulation generates a random number of outbreaks
for each permutation of decisions, income group,
and year in the time horizon. To ensure equivalent
stochastic draws for each decision option, we draw a
uniformly distributed random number u(i,j) for each
income group i and year j and then calculate the number of outbreaks for each set of decisions d in a given
income group and year by combining u(i,j) and the
Poisson rate λ(d,i,j) (see the technical appendix for
details). Similarly, we assure that for outbreaks occurring in the same income group and year we draw
random outbreak population sizes, basic reproductive numbers (R 0 ), and routine immunization coverage, based on the same random uniform numbers
for each decision. In addition, in the event of continued SIAs, we randomly draw for each outbreak a
reduction in population immunity from the average
to reflect different levels of heterogeneity in SIA frequency and quality (see below). We determine the
outbreak population size based on the distribution
of population sizes in the relevant year and income
group (see technical appendix)(13,23,24) and R 0 and
coverage from the distributions in Table I.
For computational reasons, we run the dynamic
outbreak submodel(6) a priori for all permutations
of decision options and conditions that we want to
consider. This includes a discrete number of possible outbreak population sizes, R 0 s, and routine immunization coverage levels. For a given randomly
drawn outbreak population size, we use a representative approximate discrete outbreak population
size in the submodel, which we limit to a maximum
of 100 million people in the base case. Moreover,
we run the submodel for only eight possible outbreak years, that is, T 0 , T 0 +1, T 0 +2, T 0 +3, T 0 +4,
T 0 +9, T 0 +14, T 0 +19, and linearly interpolate the
outputs for intermediate years. The outbreak model
uses a large number of submodel inputs and assumes an initial population immunity profile that
depends on the immunization policies, a randomly
drawn routine coverage level, population immunity
at T 0 , income group, and year, as described in Duintjer Tebbens et al. (2005).(6) The routine coverage
reflects the time period from the last year with
SIAs until the outbreak. Under the RPI scenario,
Polio Model Uncertainty and Sensitivity Analyses
859
Table I. List of Uncertain Inputs Modeled as Random Variables
No.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
Symbol
recipvapprate
contactvapprate
fracpopopvwithsias
Distribution
Description
Triangular(1.87,1.56,2.17)
Triangular (3.71,2.28,10.3)
Triangular(0.75,0.5,0.9)
Recipient VAPP cases per million primary OPV infections
Contact VAPP cases per million secondary OPV infections
Fraction of the population in LOW and LMI using SIAs during
1999–2005
rrcvdpv[3]
Triangular(0.1,0.05,0.5)
Relative risk of cVDPV outbreak in UMI vs. LOW or LMI
h[1,3]
Triangular(0.5,0.25,1)
Half-life of cVDPV outbreak rate with no routine in LOW, in years
h[2,3]
Triangular(0.4,0.25,1)
Half-life of cVDPV outbreak rate with no routine in LMI, in years
h[3,3]
Triangular(0.2,0.1,0.5)
Half-life of cVDPV outbreak rate with no routine in UMI, in years
relhalflifeipv
Triangular(2,1,3)
Relative half-life of cVDPV outbreak rate with IPV vs. no routine
ˆ
ˆ
lambdavdpv[4]
Lognormal(10(−6),0,10
(−5))
cVDPV outbreak rate in HIGH (with IPV) per year per 100 million
people
pisyears
Triangular(3,1,5)
Years without SIAs until cVDPV outbreak reaches OPV without SIAs
level, with MPI at T 0
dchron
Lognormal(11.5,10,21)
Average duration of excretion of chronic iVDPV excretors
dprol
Triangular(1.6,1,2)
Average duration of excretion of prolonged iVDPV excretors (LMI,
UMI, or HIGH)
dprollow
Triangular(0.5,0.1,1.5)
Average duration of excretion of prolonged iVDPV excretors in LOW
pobexcrbase
Lognormal(0.001,0,0.015)
Probability of outbreak given the presence of at least one long-term
excretor in a year (HIGH)
rrivdpv1
Triangular(1.5,1,3)
pobexcr in LOW divided by pobexcrbase
rrivdpv2
Triangular(5,3,7)
pobexcr in LMI divided by pobexcrbase
rrivdpv3
Triangular(8,5,10)
pobexcr in UMI divided by pobexcrbase
rrivdpvtend1
Lognormal(1,0.95,1.5)
pobexcr after 20 years with OPV divided by probexcrbase
rrivdpvtend2
Triangular(5,1,10)
pobexcr after 20 years with IPV divided by probexcrbase
rrivdpvtend3
Triangular(10,7.5,12.5)
pobexcr after 20 years with no routine divided by probexcrbase
reportedpertrueex cretors Triangular(1,0.08,1.92)
Fraction of long-term excretors to known long-term excretors
labrel[4or3]
Lognormal(0.001,0,0.01)
Rate of poliovirus releases from laboratories in HIGH or UMI per year
per 100 million people
labrel[2or1]
Lognormal (0.0001,0,0.001)
Rate of poliovirus releases from laboratories in LOW or LMI per year
per 100 million people
ipvrel[4or3]
Lognormal(0.01,0,0.05)
Rate of poliovirus releases from IPV production sites in HIGH or UMI
using IPV per year per 100 million people
ipvrel[2or1]
Lognormal(0.001,0,0.015)
Rate of poliovirus releases from IPV production sites in LOW or LMI
using IPV per year per 100 million people
ipvrelnoipv
Lognormal (10∧ (−6),0, 0.00015) Rate of polioviruses releases from IPV production sites in countries that
do not use IPV per year per 100 million people
biorel[1or2]
Lognormal(0.0001,0,0.0015)
Rate of intentional poliovirus releases, given continued routine OPV or
IPV use per year per 100 million people
biorel[3]
Lognormal(0.001,0,0.015)
Rate of intentional poliovirus releases, given no routine immunization
per year per 100 million people
rrcont
Triangular(5,1,10)
Relative risk of unintentional virus release if containment not
maintained vs. maintained
pobrelbase
Triangular (0.05,0.001,0.1)
Probability of outbreak given an (un)intentional release at T 0 in LOW
rrwild[2]
Triangular(0.625,0.5,1)
pobrel in LMI divided by pobrelbase
rrwild[3]
Triangular(0.25,0.1,1)
pobrel in UMI divided by pobrelbase
rrwildtend[1,1]
Triangular(1,0.8,1.2)
pobrel after 20 years with OPV and SIAs divided by pobrel at T 0
rrwildtend[1,2]
Triangular(2,1,3)
pobrel after 20 years with OPV but no SIAs divided by pobrel at T 0
rrwildtend[2,2]
Triangular(5,1,10)
pobrel after 20 years with IPV divided by pobrel at T 0
rrwildtend[3,2]
Triangular(10,1,20)
pobrel after 20 years with no routine divided by pobrel at T 0
rrwildhigh
Triangular(0.1,0,0.25)
pobrel HIGH divided by pobrelbase, any year
tiacoeff[1]
Triangular(0.75,0.5,1)
Initial cVDPV rate relative to rate with MPI and rate with RPI given
TIAs prior to T 0 in LOW1
tiacoeff[2]
Triangular(0.9,0.8,1)
Initial cVDPV rate relative to rate with MPI and rate with RPI given
TIAs prior to T 0 in LMI1
vpopv1
Triangular(0.0912,0.012,0.1)
Price of OPV in US$2002 per dose, LOW
vpipv1
Triangular(1,0.5,2)
Price of IPV in US$2002 per dose, LOW
nvcopv1
Triangular(2.08,0.5,4)
Nonvaccine cost in US$2002 per fully OPV-immunized child, LOW
(Continued).
Duintjer Tebbens et al.
860
Table I. (Continued).
No.
Symbol
Distribution
43
nvcipvsingle1
Triangular(3.25,1,10)
44
nvcipvcombo1
Triangular(0.71,0.5,3.25)
45
46
47
48
49
wastopv1
wastipv1
wastnid1
corrfactor1
rounds1
Triangular(0.2,0.05,0.5)
Triangular(0.1,0.05,0.5)
Triangular (0.15,0.05,0.25)
Triangular(2,1,3)
Triangular(0.67,0.4,2)
50
afpperchild1
Triangular (0.067,0.03,0.1)
51
52
53
54
vpopv2
vpipv2
nvcopv2
nvcipvsingle2
Triangular (0.0912,0.012,0.1)
Triangular(1.75,0.5,3)
Triangular(2.08,0.5,4)
Triangular(3.25,1,10)
55
nvcipvcombo2
Triangular(0.71,0.5,3.25)
56
57
58
59
60
wastopv2
wastipv2
wastnid2
corrfactor2
rounds2
Triangular(0.2,0.05,0.5)
Triangular(0.1,0.05,0.5)
Triangular(0.15,0.05,0.25)
Triangular(2,1,3)
Triangular(0.67,0.4,2)
61
afpperchild2
Triangular(0.087,0.03,0.12)
62
63
64
65
vpopv3
vpipv3
nvcopv3
nvcipvsingle3
Triangular(0.1,0.012,0.1)
Triangular(2.5,1,5)
Triangular(5.46,1,10)
Triangular(8.53,5,20)
66
nvcipvcombo3
Triangular(1.86,1,8.53)
67
68
69
70
71
wastopv3
wastipv3
wastnid3
corrfactor3
rounds3
Triangular(0.15,0.05,0.25)
Triangular(0.05,0.01,0.25)
Triangular(0.1,0.05,0.25)
Triangular(2,1,3)
Triangular(0.67,0.4,2)
72
afpperchild3
Triangular(0.176,0.1,0.2)
73
74
75
76
vpopv4
vpipv4
nvcipvsingle4
nvcipvcombo4
Triangular(5,2.5,15)
Triangular(10,5,30)
Triangular(48,10,100)
Triangular(10,5,48)
77
78
79
wastipv4
wastnid4
afpperchild4
Triangular(0.05,0.01,0.25)
Triangular(0.1,0.05,0.25)
Triangular (0.306,0.1,0.5)
80
81
82
gpln
envfactor
relnvccostresponse
Triangular (22M,15M,30M)
Triangular(0.15,0,0.5)
Triangular(1.5,1,3)
83
contperyear
Triangular(0.3M,0,1M)
Description
Nonvaccine cost in US$2002 per fully IPV-immunized child, single
antigen IPV, LOW
Nonvaccine cost in US$2002 per fully IPV-immunized child, IPV in
combination vaccine, LOW
Routine OPV wastage, LOW
Routine IPV wastage, LOW
NID (OPV) wastage, LOW
Actual divided by reported nonvaccine SIA costs per OPV dose, LOW
Average number of NID rounds per year given continued periodic SIAs,
LOW
Annual cost of field AFP surveillance in US$2002 per child younger than
15 years of age, LOW
Price of OPV in US$2002 per dose, LMI
Price of IPV in US$2002 per dose, LMI
Nonvaccine cost in US$2002 per fully OPV-immunized child, LMI
Nonvaccine cost in US$2002 per fully IPV-immunized child, single
antigen IPV, LMI
Nonvaccine cost in US$2002 per fully IPV-immunized child, IPV in
combination vaccine, LMI
Routine OPV wastage, LMI
Routine IPV wastage, LMI
NID (OPV) wastage, LMI
Actual divided by reported nonvaccine SIA costs per OPV dose, LMI
Average number of NID rounds per year given continued periodic SIAs,
LMI
Annual cost of field AFP surveillance in US$2002 per child younger than
15 years of age, LMI
Price of OPV in US$2002 per dose, UMI
Price of IPV in US$2002 per dose, UMI
Nonvaccine cost in US$2002 per fully OPV-immunized child, UMI
Nonvaccine cost in US$2002 per fully IPV-immunized child, single
antigen IPV, UMI
Nonvaccine cost in US$2002 per fully IPV-immunized child, IPV in
combination vaccine, UMI
Routine OPV wastage, UMI
Routine IPV wastage, UMI
NID (OPV) wastage, UMI
Actual divided by reported nonvaccine SIA costs per OPV dose, UMI
Average number of NID rounds per year given continued periodic SIAs,
UMI
Annual cost of field AFP surveillance in US$2002 per child younger than
15 years of age, UMI
Price of OPV in US$2002 per dose, HIGH
Price of IPV in US$2002 per dose, HIGH
Nonvaccine cost in US$2002 per fully OPV-immunized child, HIGH
Nonvaccine cost in US$2002 per fully IPV-immunized child, single
antigen IPV, HIGH
Routine IPV wastage, HIGH
NID (OPV) wastage, HIGH
Annual cost of field AFP surveillance in US$2002 per child younger than
15 years of age, HIGH
Cost of the global polio laboratory network in US$2002 per year
gpln with environmental surveillance divided by gpln
Nonvaccine cost per OPV dose during response compared to during regular SIAs
Cost of maintaining and enforcing laboratory and IPV production site
containment in US$2002 per year
(Continued).
Polio Model Uncertainty and Sensitivity Analyses
861
Table I. (Continued).
No.
Symbol
Distribution
84
85
NA
stockpilecost
nvcsiahigh
coverage case2
Triangular(325M,250M,500M)
Triangular(5,3,8)
Discrete3
NA
R 0 case
Discrete4
NA
NA
cVDPV risk-case
Heterogeneity level
Discrete5
Discrete
Description
One-time cost of establishing the global stockpile in US$2002
Nonvaccine cost in US$2002 per OPV dose during response, HIGH
Routine immunization coverage case that determines the
outbreak-population specific coverage in the submodel
R 0 case that determines the outbreak-population specific R 0 in the
submodel
cVDPV risk case that determines the baseline rate of cVDPV outbreaks
Heterogeneity level in the event of continued OPV with SIAs, reflecting
reductions from the average population immunity profiles for this
policy (see text)
use tiacoeff as follows: if λ mpi is the cVDPV rate with MPI and λ rpi the cVDPV rate with RPI, than the cVDPV rate with TIAs
becomes λ mpi +tiacoeff× (λ rpi -λ mpi ).
2 The input POL3 is distinct from the coverage case: it represents the income group average coverage that affects immunization costs,
routine VAPP cases, and probability of outbreaks in an income group, while the coverage case reflects the outbreak-population specific
routine coverage that we randomly sample for each outbreak.
3 P(coverage case = “projected averages”) = 0.8 and P(coverage case = “low”) = P(coverage case = “lowest”) = 0.1, where the projected
average case equals 0.68, 0.9, 0.92, and 0.94 in LOW, LMI, UMI, and HIGH, respectively, the low case equals 0.40, 0.70, 0.70, and 0.85 in
LOW, LMI, UMI, and HIGH, respectively, the lowest case equals 0.25, 0.50, 0.60, and 0.80 in LOW, LMI, UMI, and HIGH, respectively.
The coverage case represents the routine immunization coverage in the outbreak population since the last year of SIAs. This input is
distinct from the income group average immunization coverage for routine immunization and SIAs, which we model using income group
dependent constants that affect the aggregate immunization costs, routine VAPP cases, and probability of outbreaks, but not the size of
outbreaks.
4 For the base case, P(R case = “low-medium”) = P(R case = “high-medium”) = 0.5 and P(R case = “lowest”) = P(R case = “highest”)
0
0
0
0
= 0, where the lowest case equals 8, 6, 4, and 2 in LOW, LMI, UMI, and HIGH, respectively, the low-medium case equals 10, 8, 6, and 4
in LOW, LMI, UMI, and HIGH, respectively, the high-medium case equals 13, 11, 9, and 6 in LOW, LMI, UMI, and HIGH, respectively,
and the highest case equals 16, 14, 12, and 9 in LOW, LMI, UMI, and HIGH, respectively.
5 For the base case, P(cVDPV risk case = “low-risk”) = P(cVDPV risk case = “high-risk”) = 0.5, where the high-risk case bases the initial
cVDPV outbreak rate on historic occurrences of cVDPV and aVDPV events while the low-risk case bases the initial cVDPV outbreak rate
on historic occurrences of cVDPV events only. For the high income group, this distinction does not impact the cVDPV outbreak rate.
AFP = acute flaccid paralysis surveillance; cVDPV = circulating vaccine-derived poliovirus; HIGH = the high-income group; IPV =
(enhanced potency) inactivated poliovirus vaccine; iVDPV = immunodeficient vaccine-derived poliovirus; LOW = the low income group;
LMI = the lower-middle income group; MPI = maximum population immunity; NID = national immunization days; OPV = oral poliovirus
vaccine; RPI = realistic population immunity; SIAs = supplemental immunization activities; TIAs = targeted immunization activities;
UMI = the upper-middle income group; VAPP = vaccine-associated paralytic polio.
Triangular(x,y,z) indicates a triangular distribution on the interval y to z with mode at x, Lognormal(x,y,z) indicates a lognormal distribution
with mean x, minimum y, and 99th percentile z. Discrete indicates that we evaluated the model only at discrete values. Refer to other
publications for further details about each model input.(4−6)
1 We
we assume low-income countries did not conduct
SIAs for three years prior to T 0 , and lower-middle
or upper-middle income countries did not conduct
SIAs for five years prior to T 0 . For policies with continued SIAs, we assume maximum population immunity at the outset and typically very high population
immunity throughout the time horizon.(6) However,
to reflect possible variability in SIA quality and frequency (consistent with the use of a random variable for the number of rounds), we include different
heterogeneity levels to account for the possibility of
reduced population immunity compared to the average profile for SIAs. For low income countries, we
assumed medium heterogeneity implying probabilities of 0.6, 0.3, and 0.1 for the average profile, a 10%
reduction in partially infectibles, and a 25% reduction in partially infectibles, respectively (i.e., where
those reductions correspond to increases in the proportion of fully susceptibles). For the middle income
groups, we assume that higher routine immunization
coverage implies low heterogeneity and use probabilities of 0.9, 0.1, and 0 for the three levels of population immunity reduction. For high income countries, the routine immunization coverage applies for
all years since 1998, the approximate year when highincome countries switched to IPV on average.(25)
The population immunity profile determines the initial values for the set of differential equations of
the submodel,(6) with every outbreak starting with
the introduction of 1 infected fully susceptible in
Duintjer Tebbens et al.
862
age group 1 (the age group of the introduction does
not impact the results given the assumption of agehomogeneous mixing) at the beginning of the outbreak year. Depending on the surveillance policy,
outbreak detection occurs at onset of the first (AFP
surveillance) or fifth paralytic (polio) case (passive
surveillance) or at the 5,000th effective excretion (environmental surveillance).(6) Initiation of the mass
immunization response occurs relative to the detection day and depends on the response strategy, as
discussed. The target age groups apply to the entire outbreak population (i.e., we do not model geographically heterogeneous response immunization)
and we assume that the global stockpile contains
sufficient vaccine to cover the target population. We
also do not factor in seasonal or serotype variation in
virus transmissibility and instead we use inputs representing the “average serotype.”(6) We run the submodel over a 730-day time horizon and record the
number of paralytic cases, OPV doses used during
response rounds, and primary and secondary OPV
infections accumulated during each outbreak (note
that we discount these back to T 0 net present value
using the outbreak start year for outcomes occurring during both the first and second year of the
outbreak).
Using cost inputs and VAPP rates, we aggregate
the total response costs and total outbreak and response VAPP cases for all outbreaks of a given permutation of decisions, income group, and year. Finally, we aggregate the number of outbreak cases and
routine VAPP cases over all years to obtain the total
cases for a given iteration and set of decisions. Similarly, the total costs aggregate the fixed costs and response costs over all years.
A simulation consists of 10,000 iterations to obtain distributions for the total costs and cases for
each permutation. To compute the expected incremental cost-effectiveness ratios (ICERs; expressed in
US$2002 per prevented paralytic case) of an alternative decision versus a comparator, we use the formula
ICER(alternative vs. comparator)
= E[ccalt − cccomp − TC
notes the average treatment cost per paralytic case,
andE denotes the expected value operator. 6
In the overall model results presented elsewhere,(3) we also convert paralytic cases into
disability-adjusted life-years (DALYs)(26) to obtain
cost-effectiveness ratios in US$2002 per DALY
averted. Here, we focus on country-level costs per
paralytic case only, and we do not include the substantially lower global-level costs. Data on the treatment costs remain limited and vary strongly across
studies.(5,27–31) Consequently, we use a wide range
for the different income groups of 500 (low), 5,000
(lower-middle), 50,000 (upper-middle), and 500,000
(high) US$2002 per paralytic case. For some policy comparisons (e.g., no routine vs. OPV without
SIAs), our model yields either negative costs, or negative effectiveness, or both. We refer to policy comparisons with negative costs and positive effectiveness as “cost and life-saving,” those with negative
costs and negative effectiveness as “cost-saving but
life-costing,” and those with positive costs and negative effectiveness as “dominated.” Representing an
ICER as a single number is only meaningful for positive costs and effectiveness.(8) Given the changes in
sign we observed in many sensitivity analysis results,
we often choose to present the expected incremental
net benefit (INB) instead of ICER, which we defined
as
INB(alternative vs. comparator)
= (WTP + TC) × E[ppcomp − ppalt ]
−E[ccalt − cccomp ].
INB expresses the comparison on a monetary scale
and thus allows for meaningful quantitative interpretation of sensitivity analysis results. 7 Calculating
INB requires an estimate of the average willingness
to pay (WTP) to prevent one case of paralytic polio, which reflects indirect costs due to lost wages
and suffering. The estimation of this model input remains problematic, especially when used to reflect
foregone quality of life in different income groups.
Purely for purposes of expressing outcomes in INB,
we base estimates on the common interpretation that
public health interventions with an ICER of less
×(ppcomp − ppalt )]/E[ppcomp − ppalt ],
6 Note
where cc comp and cc alt denote the costs of the comparator and the alternative, respectively, pp comp and
pp alt denote the number of paralytic cases with the
comparator and the alternative, respectively, TC de-
that given the use of random variables for uncertain inputs
in the model, the ICER(alternative vs. comparator) as defined
in this study does not equal E[(cc alt – cc comp - TC×(pp comp –
pp alt ))/(pp comp - pp alt )](7).
7 Note that INB also equals E[(WTP+TC)×(pp
comp – pp alt ) –
(cc alt – cc comp )].
Polio Model Uncertainty and Sensitivity Analyses
than the per capita gross national income (GNI) per
DALY averted represent “very cost-effective” interventions.(32) Using population and income data(13,33)
and income-group-dependent DALY estimates per
paralytic case (without age-weighting), we derive
conceptual WTP estimates of 5,300, 17,000, 63,000,
and 340,000 US$2002 per paralytic case prevented
for the low, lower-middle, upper-middle, and high income groups, respectively (see technical appendix).
Thus, with these WTP estimates a net benefit of
greater than zero implies a “very cost-effective” alternative, while a negative net benefit indicates an
ICER of more than the per capita GNI per DALY
averted (or if the effectiveness were negative, the
same applies to comparator vs. alternative instead of
alternative vs. comparator).
2.3. Uncertainty and Sensitivity Analysis Methods
Table I lists all of the distributional assumptions
that we use to characterize variability and uncertainty in the model inputs, based on data collected
in prior studies and uncertainty ranges that reflect either different data sets or informal expert estimates
of the possible ranges of inputs.(4,5) Among the 85
inputs with continuous uncertainty distributions, we
distinguish those characterized by a best estimate
and a relatively narrow range of possible values from
those characterized by an estimated mean value and
very long tails. For the former, which includes all cost
inputs and some risk inputs, we assume triangular
uncertainty distributions with modes at the best estimate. For the latter, which includes inputs related
to many of the small outbreak probabilities, we assumed lognormal distributions with given means, and
tails defined by a minimum and a 99th percentile.
As noted above, the occurrences of outbreaks
represent random events drawn from a Poisson distribution whose parameter, λ, depends on uncertain
model inputs. We sample from each continuous input distribution independently for each iteration of
the model. The distributions for R 0 , the outbreak
population size, the heterogeneity level in the event
of continued SIAs, and the routine immunization
coverage represent outbreak variability and we resample these with every outbreak. Due to computational cost considerations, we could only run a limited number of permutations on those three inputs
through the outbreak submodel.(6) Consequently, we
characterize those inputs using discrete probability
distributions.
863
To quantify sensitivity to continuously distributed model inputs, we present (Pearson’s) product
moment correlation (PMC, equivalent to the standard regression coefficients(34) ), (Spearman’s) rank
correlation (RC), and the correlation ratio (CR) between model output and a given model input.(8) PMC
indicates a strong positive (negative) linear relationship between an input and the output if it is close to 1
(–1), while a PMC near 0 indicates a weak linear relationship. RC indicates a strong increasing (decreasing) monotonic relationship between an input and
the output if it is close to 1 (–1), while a RC near 0
indicates a nonmonotonic relationship. CR measures
the fraction of the output variance on a scale from 0
to 1 that relates to the uncertainty of a given input.
Given that RC does not penalize as strongly for outliers (which occur because of our choice of lognormal distributions for some inputs) as do PMC or CR,
we choose to determine importance rankings based
on the absolute RC values. Routine coverage, R 0 ,
and the cVDPV risk case represent outbreak-specific
variability. To evaluate sensitivity of the model to
those inputs, we can either fix their values for each
outbreak or run different simulations that each vary
the distributions of the three inputs separately, or we
can analyze the impact of the three inputs jointly, using main effects from design-of-experiment methods
between their lower and upper values in a separate
three-way sensitivity analysis.(8) Given the apparent
strong impact of R 0 on outcomes, we perform additional analyses in which we vary the outbreak-specific
distribution of R 0 . We also explore the impact of
varying some other model inputs one at a time. These
inputs represent decisions, preferences, or important
model assumptions that we want to keep fixed for
the base case. This includes the discount rate (3%
in the base case), WTP estimates (average per capita
GNI per DALY averted in the base case), response
delay (70 days in the base case), containment policy (maintained containment guidelines in the base
case), surveillance policy (passive surveillance in the
base case), and maximum possible outbreak population size (100 million people in the base case).
The correlation ratio provides a measure of the
variance attributable to an input based on an approximation of the best regression of an input on the output. 8 Mathematically, the CR of the model output
to a given input equals the squared product moment
8 In
contrast, the more widely known measure R-squared is based
on a linear regression of an input on the output.
Duintjer Tebbens et al.
864
correlation between the model output and the conditional expectation of the model output to that input.
The conditional expectation represents the best regression of the model output on the input, and the
squared correlation is the highest possible squared
correlation between the model output and any function of the input.(8,35) To compute the CR, we approximate the conditional expectation of the model
output to the given input by fitting a polynomial function of the input up to degree five and applying the
“early stopping” heuristic to prevent overfitting.(8,35)
We apply this heuristic both with the first and the second half-sample (i.e., of 5,000 iterations each) as the
training sample, yielding five polynomial functions
for each choice of training sample. After determining for each training sample which polynomial func-
tion yields the maximum squared correlation with
the full sample, we choose the one that yields the
lowest squared correlation among the two sets of
polynomials. This polynomial function provides our
approximation of the conditional expectation, and
the squared correlation between the model output
and the approximated conditional expectation provides our CR estimates.
3. RESULTS
3.1. Base Case Results
Fig. 2 shows the probability distributions of
the net benefit of IPV or no routine versus OPV
without SIAs, assuming the conceptual WTP of the
Fig. 2. Probability density functions (left panels) and cumulative distribution functions (right panels) of the aggregated INB of no routine
versus OPV without SIAs (solid line) and IPV versus OPV without SIAs (dashed line) for the base case (20-year time horizon, 3% discount
rate). The x-axes run from smallest 1st to largest 99th percentile among the two distributions in each panel. (INB = incremental net benefit;
IPV = (enhanced potency) inactivated poliovirus vaccine; OPV = oral poliovirus vaccine; SIAs = supplemental immunization activities.)
Polio Model Uncertainty and Sensitivity Analyses
income-group-dependent average per capita GNI
per DALY averted. Clearly, the net benefit typically
remains positive for no routine vs. OPV but negative
for IPV vs. OPV, although some probability exists of
different signs of the net benefit (consistent with the
patterns observed for prevented paralytic cases; see
technical appendix). Fig. 2 confirms the robustness
of the benefit of no routine over OPV without SIAs
and the lack of a benefit of IPV over OPV without
SIAs. Indeed, the probability that the net benefit of
IPV vs. OPV without SIAs becomes positive equals
6% or less in each income group, and the probability that the net benefit of no routine vs. OPV with
SIAs becomes negative equals 0.5% or less in each
income group. The probability that the net benefit of
IPV vs. no routine exceeds 0 remains very small (less
than 0.1%). This reflects the importance of the vaccination costs to the net benefit estimates; the valuation of paralytic cases using the per capita GNI per
DALY conversion implies that the number of paralytic cases prevented does not weigh heavily into the
net benefits. In fact, despite the superiority of no routine over IPV in terms of net benefits, the percentage of iterations in which no routine yielded more
cases than IPV ranged from 13% (upper-middle income group) to 90% (low income group). The percentage of iterations in which no paralytic cases occurred with either IPV or no routine ranged from 7%
(low income group) to 81% (upper-middle income
group). Moreover, the distributions of paralytic cases
prevented between immunization policies show important skewness, implying that the expected values
alone do not tell the full story. However, for the number of paralytic cases to substantially affect the sign
of the net benefits the valuation of paralytic cases
must exceed the GNI per capita by at least one order of magnitude.
865
sequently, the size of potential outbreaks substantially increases over time. Fig. 3b shows the results
of combining the decreasing outbreak probabilities
with the increasing consequences and highlights the
important short versus long-term tradeoffs in terms
of the effectiveness of different policies. In the low
income group, the expected annual disease burden
after OPV cessation initially decreases, but the small
probability of much larger outbreaks drives the expected burden up in the long term. However, the annual expected number of paralytic cases does not approach the expected burden attained with continued
OPV (without SIAs), which continues to increase
over time due to decreasing population immunity after T 0 and increasing total population. In the lowermiddle income group, the expected annual burden
with continued OPV remains much lower than in the
low income group (better routine coverage, lower
R 0 s, less total population). Consequently, the expected annual burden with no routine can eventually
approach that with OPV (without SIAs), although
the cumulative totals over the 20-year time horizon
remain strongly in favor of no routine (even without discounting). In the upper-middle income group,
the annual burden with IPV eventually even exceeds
that with OPV given the low expected number of paralytic cases (mostly VAPP) with OPV and the elevated risk of unintentional IPV production site releases. The latter risk also leads to the crossing of
the curves for IPV and no routine (we discuss the
removal of the assumption of this elevated risk below). In the high income group, the very small outbreak probabilities yield a fluctuating curve over time
(a smoother curve necessitates more iterations of the
model), and no year with an expected disease burden
of more than one paralytic case.
3.3. Departures from the Base Case Assumptions
3.2. Risks Over Time
While the previous section revealed the prominence of vaccination costs in the net benefit estimates, we observe important dynamics and risk versus risk tradeoffs at the level of expected disease
burden. Fig. 3a shows the expected annual number
of outbreaks over time by income group, suggesting
that most of the risk occurs during the first years after OPV cessation (T 0 ), while continued OPV would
lead to a sustained rate of outbreaks. However, population immunity will decrease over time after T 0
(unless SIAs continue) and most rapidly with no routine vaccination, followed by IPV and OPV. Con-
Although the implications from the net benefit
results (i.e., whether the net benefit remains positive
or negative) remain valid with a high probability for
the base case, the output distributions in Fig. 2 do
not reflect the impact of several dichotomous model
assumptions relating to preferences, policies, and implementation. Table II summarizes the impact of altering these assumptions on the expected net benefits. This table focuses on expected aggregate net
benefits and does not capture effects on paralytic
cases or costs of individual policies or on the shape
of the distribution (the technical appendix provides
further details).
Duintjer Tebbens et al.
866
(a) Expected annual number of outbreaks
OPV, SIAs
4.0
Low income group
OPV, no SIAs
IPV, no SIAs
3.0
Expected annual outbreaks
Expected annual outbreaks
4.0
no routine, no SIAs
2.0
1.0
3.0
2.0
1.0
0.0
0.0
0
2
4
0.12
6
8
10
12
Years after T0
14
16
18
0
2
4
6
0.0025
Upper-middle income group
Expected annual outbreaks
Expected annual outbreaks
Lower-middle income group
0.10
0.08
0.06
0.04
0.02
0.00
8
10
12
Years after T0
14
16
18
High income group
0.0020
0.0015
0.0010
0.0005
0.0000
0
2
4
6
8
10
12
Years after T0
14
16
18
0
2
4
6
8
10
12
Years after T0
14
16
18
7,000
OPV, SIAs
6,000
OPV, no SIAs
IPV, no SIAs
5,000
no routine, no SIAs
200
Low income group
4,000
3,000
2,000
1,000
160
140
120
100
80
60
40
20
0
0
0
2
30
4
6
8
10
12
Years after T0
14
16
18
0
2
4
6
0.18
Upper-middle income group
8
10
12
Years after T0
14
16
18
High income group
0.16
25
Expected annual cases
Expected annual cases
Lower-middle income group
180
Expected annual cases
Expected annual cases
(b) Expected annual disease burden
20
15
10
5
0.14
0.12
0.10
0.08
0.06
0.04
0.02
0
0.00
0
2
4
6
8
10
12
Years after T0
14
16
18
0
2
4
6
8
10
12
Years after T0
14
16
18
Fig. 3. Risks and consequences over time (both undiscounted) for the base case. (IPV = (enhanced potency) inactivated poliovirus vaccine;
OPV = oral poliovirus vaccine; SIAs = supplemental immunization activities.)
Polio Model Uncertainty and Sensitivity Analyses
867
Table II. Impact of Model Assumptions About Preferences, Model Implementation, and Additional Policies on the Expected Aggregate
Net Benefits
Change in Incremental Net
Benefit of No Routine vs.
OPV Without SIAS in
Billions US$2002 (% Change)
Departure from Base Case
(Assumption in the Base Case)1
Low Income
Group
Discount rate 0% (3%)
1.41
(46.7%)
−0.91
(−30.1%)
0.48
0.59
(33.8%)
−0.47
(−27%)
0.05
0.26
(33.8%)
−0.21
(−26.3%)
0.03
−0.17
(7.0%)
0.48
(−20.2%)
0.56
−0.56
(25.5%)
0.53
(−23.9%)
0.07
−0.40
(28.0%)
0.36
(−25.1%)
0.03
(16%)
0.76
(25.2%)
−0.19
(−6.4%)
−0.17
(−5.8%)
−0.02
(−0.6%)
−0.27
(−8.9%)
−0.10
(−3.2%)
(3.1%)
0.53
(30.3%)
0.00
(0.1%)
0.00
(0.1%)
−0.01
(−0.4%)
−0.37
(−21.3%)
−0.04
(−2.4%)
(4.3%)
0.00
(0.2%)
0.00
(0.5%)
0.00
(0.6%)
−0.02
(−2.3%)
−0.34
(−43.8%)
(−23.4%)
0.80
(−33.5%)
−0.22
(9.1%)
−0.21
(8.7%)
−0.07
(2.8%)
−0.30
(12.6%)
−0.11
(4.5%)
(−3.2%)
0.56
(−25.3%)
−0.01
(0.3%)
−0.01
(0.3%)
−0.02
(1.1%)
−0.40
(18.1%)
−0.05
(2%)
(−2.1%)
0.00
(−0.1%)
0.01
(−0.3%)
0.00
(−0.3%)
−0.06
(4%)
−0.34
(23.6%)
Discount rate 7% (3%)
WTP per DALY averted equals
3×per capita GNI (1 × per capita
GNI)
Maximum outbreak population 250
M (100 M)2
AFP surveillance (passive
surveillance)
Response delay 45 days (70 days)
Containment guidelines not
maintained (maintained)
Maximum population immunity at T 0
(realistic population immunity)
Targeted immunization activities
prior to T 0 (realistic population
immunity)
Lower-Middle
Income Group
Upper-Middle
Income Group
Change in Incremental Net
Benefit of IPV vs.
OPV Without SIAs in
Billions US$2002 (% Change)
NA3
Low Income
Group
Lower-Middle
Income Group
Upper-Middle
Income Group
NA
1 The
expected incremental net benefits in the base case equaled 3.02, 1.75, and 0.78 billion US$2002 in the low, lower-middle, and
upper-middle income groups, respectively, for no routine versus OPV without SIAs and −2.38, −2.21, and −1.44 billion US$2002 in the
low, lower-middle, and upper-middle income groups, respectively, for IPV versus OPV without SIAs.
2 In this alternative simulation, the outbreak model uses an outbreak population of 100 million people if the outbreak occurs in a country
with between 75 and 200 million people and uses an outbreak population of 250 million people if the outbreak occurs in a country with
over 200 million people.
3 NA = not applicable, since we assumed targeted immunization activities only for high-risk areas in low and lower-middle income
countries.
AFP = acute flaccid paralysis surveillance; DALY = disability-adjusted life year; GNI = gross national income; IPV = (enhanced potency)
inactivated poliovirus vaccine; OPV = oral poliovirus vaccine; SIAs = supplemental immunization activities; WTP = willingness to pay.
The discount rate affects both the costs differences and cases prevented between policies, but
its effect on the costs dominates in the net benefits, explaining the relative importance of this input.
Increasing the WTP value per paralytic case by a factor of 3 (i.e., valuing every averted DALY as three
times the per capita GNI) mostly affects the net benefits in the low income group given the large number
of cases prevented in that income group. However,
this does not change the sign of the expected net benefit for any of the policy comparisons in Table II, although a larger WTP implies a greater probability
of a different net benefit sign (see also the impact
of WTP on the output distribution in the technical
appendix).
Increasing the maximum possible outbreak population size from 100 million to 250 million has a relatively large impact on the outcomes, although not
as dramatic as one might expect given that for many
outbreaks the response prevents the outbreak from
reaching its full potential. Given that the outbreak
response includes the entire outbreak population, we
found that this assumption affects the costs to a similar degree as it affects the number of cases for individual policies, or even to a greater degree in the absence of routine immunization. With most outbreaks
occurring with the comparator program (continued
OPV), the net benefits of the alternatives typically
increase with continued OPV. However, the effect is
very small in the upper-middle income group because
Duintjer Tebbens et al.
868
outbreaks generally do not take off and do not require a response and because only one country (i.e.,
Brazil) becomes large enough to fall in the 250 million outbreak population category.
AFP surveillance reduces the time from initiating infection to outbreak detection and thus the timeliness of the response. This substantially reduces the
expected burden of cases of individual policies and
the expected number of cases prevented between
policies and it is crucial to the achievement of eradication, but it comes at a substantial cost. Our results
suggest that maintaining AFP surveillance long after eradication in its current form would not appear
“very cost effective” for any routine immunization
policy. When comparing alternatives (e.g., no routine with AFP vs. OPV (without SIAs) with AFP),
the AFP costs cancel out and the important effect on
the number of cases prevented remains buried under the vaccination costs due to the comparably low
WTP per paralytic case. Similarly, reducing the response delay greatly reduces the size of outbreaks,
but does not affect the net benefit comparisons
much. Nevertheless, we emphasize that AFP surveillance and rapid response remain important additional policies in maintaining a low expected disease
burden for any of the possible posteradication immunization options. Both AFP surveillance and a more
rapid response yielded up to almost 75% reduction in
the expected number of paralytic cases for the main
immunization options compared to the base case.
In addition, we anticipate that an integrated disease
surveillance system could eventually significantly reduce the costs that the current system incurs related
to maintaining its infrastructure.
We modeled a five-fold increase in the rate of
unintentional release of poliovirus from a laboratory
or IPV production site for a country that does not
actively maintain the containment guidelines. Not
maintaining containment mostly affects the burden
of disease in the long term and with routine IPV immunization. Given that we assumed a higher probability that upper-middle (and high) income countries would produce IPV domestically for their own
use than low or lower-middle income countries, we
see the largest impact of the failure to contain well
in the upper-middle income group looking at the
net benefit of IPV versus OPV without SIAs. The
base case yielded a slightly higher expected number
of paralytic cases with IPV than with no routine in
the upper-middle income group. However, a separate analysis that assumed the same IPV production
site release rate in the upper-middle income group
with IPV as in the lower two income groups found
a greater number of expected cases with no routine
vaccination than with IPV vaccination (see technical appendix). Thus, containment of IPV production
sites emerges as a relatively important priority in situations in which other risks remain small (e.g., in
the long term or in the upper-middle income group).
Changing the assumption of maintained containment
decreased the net benefit by approximately 4% given
the unchanged substantial costs of routine IPV immunization in the upper-middle income group. Nevertheless, the expected 57 million US$2002 difference in expected net benefits in the upper-middle income group alone already justifies the much smaller
investment in containment (estimated at 6.7 million
US$2002 globally(3) ).
While increasing population immunity at T 0
would not occur within the analytical time horizon
of our model, Table II includes its effect on the net
benefits. Although maximizing population immunity
reduces the expected number of paralytic cases by
up to 50% (see technical appendix), we found that
for all policy scenarios, the costs of maximizing population immunity substantially outweigh its benefits
in terms of the monetary equivalent of the paralytic
cases prevented. Even targeted immunization activities in high-risk areas yielded a negative impact on
the expected net benefits, although to a lesser extent
than a full global immunization day. We emphasize
that this reflects the high costs of these immunization activities compared to the willingness to pay for
prevented paralytic cases and not a failure of these
activities to prevent future outbreaks and paralytic
cases.
None of the departures from the base case assumptions in Table II altered the sign of the expected
net benefits of either IPV or no routine versus OPV
without SIAs.
3.4. Input Sensitivity
The base case includes four inputs that we modeled as discretely distributed random variables (R 0 ,
coverage, the cVDPV risk case, and heterogeneity
level in the event of continued SIAs) and 85 inputs
drawn from (independent) continuous distributions
(Table I). We separately analyzed the impact of the
main three discrete inputs using a simple experimental design and found that R 0 yielded the greatest
impact on the outcomes of all individual policies,
followed by coverage, and the cVDPV risk case. In
the base case, we vary R 0 and coverage not only
Polio Model Uncertainty and Sensitivity Analyses
869
12,000
Low income group
Lower-middle income group
150,000
Expected aggregate cases
120,000
IPV, no SIAs
no routine, no SIAs
90,000
60,000
8,000
6,000
4,000
30,000
2,000
0
0
500
25
Expected aggregate cases
Upper-middle income group
400
300
200
100
High income group
20
15
10
5
(0, 0
, 1, 0
)
.25)
4, 0.1
)
0.25,0
. 25, 0
(0.25
,
)
, 0.05
. 4 , 0.
(0.1,
0
, 0.45
0. 49,
0.01)
(0.05
, 0.45
0.5, 0
)
0. 49,
(0. 01
,
0.5,
= (0,
(0, 1
, 0, 0
)
base
1, 0)
(0, 0
,
.25)
4, 0.1
)
.25,0
0.25,0
(0.25
,
.4, 0.
(0. 1,
0
, 0)
.5, 0.5
(0, 0
0, 0)
base
=
(0, 1
,
, 0.01
)
(0.05
, 0.45
, 0. 4
5, 0.0
5)
0
0
, 0.49
Expected aggregate cases
10,000
OPV, no SIAs
(0.01
, 0.49
Expected aggregate cases
OPV, SIAs
Fig. 4. Impact of assumptions about the R 0 distribution on the expected aggregate number of paralytic cases.∗ (IPV = (enhanced potency) inactivated poliovirus vaccine; OPV = oral poliovirus vaccine; SIAs = supplemental immunization activities.) The notation indicates
the probability weight assigned to the lowest, low-medium, high-medium, and highest R 0 case. For example, (0,0.5,0.5,0) indicates 50%
probability of drawing low-medium or high-medium R 0 for a given outbreak, and 0% probability of drawing lowest or highest R 0 . The
income-group-dependent values for each R 0 case are 8, 6, 4, and 2 in the low (LOW), lower-middle (LMI), upper-middle (UMI), and high
(HIGH) income group, respectively, for the lowest case; 10, 8, 6, and 4 in LOW, LMI, UMI, and HIGH, respectively, for the low-medium
case; 13, 11, 9, and 6 in LOW, LMI, UMI, respectively for the high-medium case; and 16, 14, 12, and 9 in LOW, LMI, UMI, and HIGH,
respectively, for the highest case.
randomly for each iteration, but also for each outbreak, as described in the methods. To demonstrate
the impact of R 0 , in Fig. 4 we isolate the probability
distribution of the R 0 cases while keeping all other
inputs as in the base case. Clearly, attributing more
weight to the tails of the R 0 distribution (i.e., the lowest and highest cases) increases the expected number of paralytic cases. However, since this occurs for
all major routine immunization options, changing the
R 0 distribution in most scenarios does not alter the
rank order of expected cases. We found a much less
dramatic impact of the R 0 distribution on the net
benefits of no routine or IPV versus OPV without
SIAs than on the expected burden of paralytic cases
of individual policies shown in Fig. 4 (we observed a
maximum change of 42% in the net benefit of IPV vs.
OPV without SIAs in the lower-middle income group
for going from the low-medium R 0 s to the distribution that assigns equal probability to each R 0 level;
see technical appendix).
Duintjer Tebbens et al.
870
For the full probabilistic sensitivity analysis, we
focus on all 85 remaining inputs that follow continuous probability distributions as given in Table
I. Given the problems of interpretation of negative
ICERs, we focus on the INBs. To limit the amount
of results for this presentation, we focus on the main
questions of the post-OPV era (i.e., the net benefit of
IPV vs. OPV with SIAs and the net benefit of no routine vs. OPV without SIAs). We also aggregate over
the three income groups for which we modeled these
options (i.e., low, lower middle, and upper middle),
which allows us to capture the relative importance
of inputs in different income groups, noting that this
also depends on relative population sizes. The technical appendix includes results broken down by income
group.
Table III shows the 15 inputs that correlate most
strongly with the net benefit of IPV versus OPV without SIAs. While 14 of the 15 inputs relate to the
vaccination and response costs, only the sixth-ranked
input relates to the risks. This input (fracpopopvwithsias) represents the fraction of the OPV-using population living in low and lower-middle income countries that regularly conducted SIAs during 1999–2005
(best estimate 75%, range 50–90%; see Table I). This
affects our assessment of the initial rate of cVDPV
outbreaks per 100 million people with or without
SIAs that we base on the frequency of cVDPV outbreaks with or without SIAs during 1999–2005. If we
increase this fraction, then this implies a lower observed per capita frequency of cVDPV outbreaks on
a background of OPV with SIAs and a higher per
Rank∗
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
∗ Based
Input No.
43
41
54
52
42
3
65
53
57
82
46
64
63
48
59
Input Symbol
nvcipvsingle1
vpipv1
nvcipvsingle2
vpipv2
nvcopv1
fracpopopvwithsias
nvcipvsingle3
nvcopv2
wastipv2
relnvccostresponse
wastipv1
nvcopv3
vpipv3
corrfactor1
corrfactor2
capita frequency on a background of OPV without
SIAs. The price and other vaccination costs of IPV
in the low income group represent the most important inputs, followed by the same inputs in the less
populous lower-middle income group. Fig. 5 graphically illustrates the relationship between model output and the six most important inputs. The thick lines
represent an approximation of the best regression (or
conditional expectation)(8) of the model output to the
given input. Fig. 5 shows that while the relationship
between the net benefit and cost inputs remains close
to linear, the regression curve for fracpopopvwithsias
reveals a somewhat steeper increase in the net benefit as this input approaches its maximum of 90%.
Table IV shows the 15 inputs that correlate most
strongly with the net benefit of no routine versus
OPV without SIAs. While the nonvaccine costs of
routine OPV vaccination (i.e., administration, cold
chain costs, etc.) rank first, the second most important input (fracpopopvwithsias) relates to the risks.
Besides fracpopopvwithsias (which emerges as more
important here than in the IPV versus OPV comparison since no IPV-cost-related inputs influence the
results), several other inputs that affect the expected number of prevented paralytic cases rank among
the 15 most important inputs. These include the very
uncertain risk of an intentional poliovirus release
with no routine (biorel[3]), the baseline probability of an outbreak given an (un)intentional
release (pobrelbase), the relative risk of outbreaksgiven a release in the upper-middle income group
(rrwild[3]), and the baseline probability of an
Rank
Correlation
Product Moment
Correlation
Correlation
Ratio
−0.533
−0.332
−0.325
−0.298
0.270
0.266
−0.214
0.177
−0.168
0.143
−0.137
0.113
−0.107
0.072
0.061
−0.545
−0.342
−0.337
−0.310
0.277
0.271
−0.219
0.181
−0.175
0.152
−0.153
0.118
−0.112
0.074
0.062
0.297
0.117
0.114
0.096
0.077
0.083
0.048
0.034
0.031
0.023
0.025
0.014
0.013
0.005
0.004
on absolute values of the rank correlation.
Input descriptions are in Table I.
Table III. Results of the Probabilistic
Sensitivity Analysis: 15 Most Important
Continuously Distributed Inputs with
Respect to the Incremental Net Benefit
of Routine Inactivated Poliovirus
Vaccine Use Versus Oral Poliovirus
Vaccine Use (Without Supplemental
Immunization Activities) Aggregated
over the Low and
Middle-Income Groups
Polio Model Uncertainty and Sensitivity Analyses
871
Fig. 5. Scatter plots and approximation of the conditional expectation (or best regression) between the incremental net benefit (INB) of
routine inactivated poliovirus vaccine use versus oral poliovirus vaccine use (without supplemental immunization activities), aggregated
over the low and middle income groups, and the six most important continuously distributed inputs. Input descriptions are in Table I.
outbreak given the presence of an iVDPV excretor (pobexcrbase). Fig. 6 shows the relationship between the net benefit of no routine versus OPV
without SIAs for the six most important inputs.
As in Fig. 5, the cost-related inputs display a relatively linear relationship to the model output, while
fracpopopvwithsias yields a more nonlinear relationship. None of the regression lines cross the zero net
benefit line, indicating robustness of the positive net
benefit of no routine versus OPV without SIAs to
variations in these six most important inputs. However, with the lognormal distribution for the rate of
intentional releases with no routine (biorel[3]), we
found that the very rare instances with very low or
negative net benefits of no routine versus OPV without SIAs (12 of 10,000 iterations yielded a negative
net benefit, with the minimum as low as –32 billion US$2002) typically corresponded to values of
biorel[3] above the 99th percentile of 0.015 unintentional releases per year per 100 million people.
This observation explains the high product moment
correlation and correlation ratio for this input. If
we include only iterations for which the value of
biorel[3] equals less than its 99th percentile, the
Duintjer Tebbens et al.
872
Rank∗
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Input No.
42
3
53
82
64
48
59
40
28
51
56
30
32
45
14
Input Symbol
nvcopv1
fracpopopvwithsias
nvcopv2
relnvccostresponse
nvcopv3
corrfactor1
corrfactor2
vpopv1
biorel[3]
vpopv2
wastopv2
pobrelbase
rrwild[3]
wastopv1
pobexcrbase
Rank
Correlation
Product Moment
Correlation
0.560
0.465
0.339
0.258
0.227
0.123
0.090
0.082
−0.070
0.040
0.026
−0.025
−0.021
0.015
−0.015
0.500
0.434
0.301
0.249
0.198
0.121
0.093
0.070
−0.436
0.032
0.021
−0.026
−0.020
0.019
−0.037
Correlation
Ratio
0.250
0.215
0.090
0.062
0.039
0.015
0.009
0.005
0.190
0.001
0.000
0.001
0.000
0.001
0.001
Table IV. Results of the Probabilistic
Sensitivity Analysis: 15 Most Important
Inputs with Respect to the Incremental
Net Benefit of No Routine Versus Oral
Poliovirus Vaccine Use (Without
Supplemental Immunization Activities),
Aggregated over the Low and Middle
Income Groups
∗ Based
on absolute values of the rank correlation.
Input descriptions are in Table I.
product moment correlation increases from –0.436 to
–0.102 and the rank correlation only from –0.070 to
0.053, demonstrating that any high apparent sensitivity of the net benefit of no routine versus OPV without SIAs to this model input reflects only its impact
at very high values.
Based on the polynomial regression curve for
biorel[3], we find that with a rate of unintentional release of more than 0.096 per 100 million people per
year the expected net benefit of no routine versus
OPV without SIAs becomes negative. This clearly
exceeds the 99th percentile of 0.015 for this input and
would translate into what seems like an unrealistically high number of approximately 125 intentional
releases during the 20-year time horizon in the low
and middle income groups. Thus, the high correlation
between biorel[3] and the net benefit of no routine
versus OPV without SIAs appears an attribute of the
long tail in the lognormal distribution for biorel[3]
rather than an indication of truly high sensitivity of
the model to this input.
4. DISCUSSION
At the highest level, these results demonstrate the relative robustness of our prior results.(3)
Nonetheless, we find that changing several of the key
assumptions used in the modeling process can significantly impact the results. For example, changing
the analytical time horizon to a much longer time period and using the potential for large outbreaks as a
driving consideration might shift the decision toward
a preference for continued vaccination, depending
on the value attributed to future outcomes (i.e., on
the discount rate). In contrast, improving the capacity torespond fast to an outbreak (through intense
surveillance and rapid response mechanisms), maximizing population immunity prior to T 0 , and maintaining containment guidelines all represent policies
that would mitigate the long-term risks associated
with cessation of vaccination. However, with WTP
estimates that value one DALY averted at the per
capita GNI, the impacts of these policies on aggregate net benefits of alternatives versus continued
OPV vaccination remain small. For the number of
prevented paralytic cases to significantly affect the
net benefits, we must value DALYs averted as 10
times the per capita GNI or more.
Of the many uncertain inputs in our model, we
found that the distributions from which we sample
R 0 and the historic routine immunization coverage in
an outbreak population provide major drivers of the
expected disease burden for individual policies, with
R 0 clearly the more important of the two. However,
in terms of aggregate incremental net benefits, uncertain inputs related to the cost outcomes dominate
the importance rankings, with the price and nonvaccine costs for IPV and OPV emerging as the most
critical inputs. In terms of narrowing the distribution of possible future costs, ongoing efforts by the
WHO to systematically and repeatedly collect costs
associated with routine immunization in developing
countries(36) may prove helpful, although uncertainty
about the actual future price of IPV and the cost of
routine IPV immunization implementation will most
likely continue to exist. In this context, technological
Polio Model Uncertainty and Sensitivity Analyses
873
∗ For
purposes of presentation, these figures do not show the five samples with the lowest model output, which ranged from –32 to –6.2
billion US$2002
Fig. 6. Scatter plots and approximation of the conditional expectation (or best regression) between the incremental net benefit (INB) of
no routine versus oral poliovirus vaccine use (without supplemental immunization activities), aggregated over the low and middle income
groups, and the six most important continuously distributed inputs. Input descriptions are in Table I.
improvements could reduce the costs or increase the
benefits of IPV and enhance the attractiveness of
routine IPV vaccination. While inputs that affect the
number of paralytic cases do not typically impact
aggregate net benefits as much as the cost-related
inputs, the expected burden of disease remains an
extremely important consideration, and opportunities exist to reduce the uncertainty in disease burden
outcomes. R 0 represents an unobservable input (i.e.,
it cannot be measured directly and varies strongly
across populations) and, consequently we used relatively wide ranges. A formal expert judgment process
could summarize the opinions of different experts
into one distribution that would reflect the best state
of knowledge about this input.(37) While we reflected
outbreak variability in R 0 , the outbreak model nevertheless uses homogeneous R 0 within outbreak
populations. In reality, intercommunity variability of
874
hygienic and geographic circumstances implies that
polioviruses may transmit much better locally than
on average in the population, which may lead to
higher risks of OPV viruses persisting and becoming
VDPVs. While the cVDPV risks rely on previously
observed occurrences of outbreaks and thus implicitly account for this variability, the heterogeneity
issue (in R 0 as well as in population immunity) merits further study in the context of cVDPVs originating from OPV used for outbreak response as well
as the kinetics of outbreaks at the initial and final
stages. We did not analyze the impact of the relative potential of IPV vaccinees to participate in transmission in this study, but a prior study revealed that
this uncertainty may impact outbreak dynamics as
much as R 0 .(6) Further clinical or field studies measuring the participation of IPV vaccinees in transmission (especially in developing countries) could provide better resolution on this uncertainty. In terms of
inputs affecting the probabilities of outbreaks (i.e.,
not their sizes), we identified the fraction of the
OPV-using population that conducted SIAs during
1999–2005 as influential enough to significantly impact even the net benefits because it determines the
initial rate of cVDPV outbreaks (for both continued
OPV and alternative policy choices). Further resolution on how fast countries might reach population
immunity levels similar to those populations that experienced cVDPV outbreaks historically after eradication and cessation of SIAs may help reduce this uncertainty, although measuring population immunity
to polioviruses remains a very difficult task for which
currently only surrogate data exist (e.g., routine immunization coverage, data on last SIAs, date of last
wild poliovirus transmission, or limited small-scale
serology studies). In this context, incorporating more
serology testing into routine AFP surveillance may
provide additional insights on this risk. The cVDPV
outbreak rates in the model rely on the available data
on cVDPVs and aVDPVs during the period 1999–
2005 and defined as signals of possible outbreaks any
events involving one or more cases of polio associated with viruses diverged more than 1% of the nucleotides in the VP1 region from the original OPV
strains.(4) However, a large type 2 cVDPV outbreak
recently detected in Nigeria included multiple polio
cases associated with viruses diverging between 0.5
and 1% from the OPV virus.(38) Pending further investigation of this outbreak, this suggests that cVDPVs may arise even more quickly and frequently than
initially assumed, implying greater expected numbers
of outbreaks with each of the evaluated immuniza-
Duintjer Tebbens et al.
tion policies and, in particular, a permanent higher
risk of outbreaks with continued OPV. Stochastic
models of outbreak dynamics may prove useful in
addressing the uncertainty in the risk inputs related
to the conditional probabilities of outbreaks given
a virus introduction over time and could help determine the risk of continued undetected wild poliovirus circulation after apparent global interruption
of transmission.(39)
While the analyses in this study provide important insights into the possible tradeoffs among polio risk management policies after eradication, interpretation of the results requires understanding of
its limitations. First, this model addresses a global
decision problem at a very aggregate level (i.e., by
income group) and ignores many possible sources
of spatial, temporal, age, and serotype variability
and, consequently, the results lack accuracy for application to very specific settings. Furthermore, the
model assumes that outbreaks do not continue beyond 730 days and remain contained in the a priori defined outbreak populations. Thus, the need
to restart routine immunization after a large outbreak threatening other populations does not arise
in the model, although this may become a reality
in the event of a slow or otherwise ineffective outbreak response. Moreover, the model does not account for the possibility that the OPV viruses used
for the outbreak response mutate toward vaccinederived viruses and may become sources of cVDPV
outbreaks themselves. We emphasize that our model
also assumes successful wild poliovirus eradication
prior to T 0 , although in reality we recognize the
possibility that wild polioviruses could continue to
circulate undetected, especially in the context of
poor surveillance.(40) We further emphasize that our
model assumes coordinated cessation of OPV use,
which we view as a crucial requirement for cessation.
If some countries continue to use OPV while neighboring countries completely stop poliovirus vaccinations, the latter will face a much higher risk of
cVDPV outbreaks than estimated in this model. We
also highlight the fact that we assumed that the outbreak response in an IPV-using population would
target only children under five, regardless of the
time since OPV cessation. Prior work demonstrated
that targeting older children or adults may become
necessary in the longer term given the potential of
IPV-vaccinees to participate in transmission.(16) Targeting older IPV-vaccinees in an outbreak response
would limit the number of paralytic cases expected
with continued IPV and would thus improve the
Polio Model Uncertainty and Sensitivity Analyses
attractiveness of IPV in terms of expected number
of paralytic cases prevented. Future work balancing
these benefits against the costs of a larger response
may provide helpful direction to the development of
long-term response plans. We also did not address
the risks and consequences of an insufficiently large
stockpile, although we note that this model may provide a useful framework for determining the necessary stockpile size in the long term. While we did
not report the convergence of the results as a function of the number of iterations in detail, we found
robustness of the net benefits with 10,000 iterations,
although the expected number of paralytic cases in
the context of low-probability outbreaks (e.g., in high
income countries) remained more variable. Finally,
we did not incorporate any dependencies among
uncertain inputs other than the conditional dependence introduced through our stratification by income groups. While certainly some dependence remains plausible (e.g., between the distribution of R 0
and the vaccination coverage in an outbreak population), estimation of the dependence structure remains beyond the scope of this model.
In conclusion, although addressing limitations of
the model would provide more certainty about the
outcomes, this model suggests that for the first 20
years after T 0 , the option of no routine immunization
dominates that of continued OPV without SIAs for a
wide range of assumptions. Replacing OPV without
SIAs with IPV appears attractive only upon accepting a much higher cost to prevent morbidity and mortality than commonly associated with interventions
that would typically be considered cost effective.
ACKNOWLEDGMENTS
Drs. Thompson and Duintjer Tebbens acknowledge support for their work from the CDC under
grant numbers U50/CCU300860, TS–0675 and U01
IP000029. The contents of this manuscript are solely
the responsibility of the authors and do not necessarily represent the official views of the Centers for
Disease Control and Prevention or the World Health
Organization. We thank Victor Cáceres, Denise
Johnson, and Linda Venczel for providing helpful
comments.
REFERENCES
1. World Health Organization. (2006). Global Polio Eradication Initiative Polio Case Count. Available at http:// www.
polioeradication.org/casecount.asp. Accessed April 24, 2008.
875
2. Sangrujee, N., Duintjer Tebbens, R. J., Cáceres, V. M., &
Thompson, K. M. (2003). Policy decision options during the
first 5 years following certification of polio eradication. Medscape General Medicine, 5(4), 35.
3. Thompson, K. M., Duintjer Tebbens, R. J., Pallansch, M. A.,
Kew, O. M., Sutter, R. W., Aylward, R. B., Watkins, M., Gary,
H. E. Jr., Alexander, J. P. Jr., Jafari, H., & Cochi, S. L. (2008).
The risks, costs, and benefits of global policies for managing
polio after eradication (in press). American Journal of Public
Health, 98(7), 1322–1330.
4. Duintjer Tebbens, R. J., Pallansch, M. A., Kew, O. M.,
Cáceres, V. M., Jafari, H., Cochi, S. L., Aylward, R. B., &
Thompson, K. M. (2006). Risks of paralytic disease due to wild
or vaccine-derived poliovirus after eradication. Risk Analysis,
26(6), 1471–1505.
5. Duintjer Tebbens, R. J., Sangrujee, N., & Thompson, K. M.
(2006). The costs of polio risk management policies after eradication. Risk Analysis, 26(6), 1507–1531.
6. Duintjer Tebbens, R. J., Pallansch, M. A., Kew, O. M.,
Cáceres, V. M., Sutter, R. W., & Thompson, K. M. (2005). A
dynamic model of poliomyelitis outbreaks: Learning from the
past to help inform the future. American Journal of Epidemiology, 162(4), 358–372.
7. Thompson, K. M., & Graham, J. D. (1996). Going beyond
the single number: Using probabilistic risk assessment to improve risk management. Human and Ecological Risk Assessment, 2(4), 1008–1034.
8. Duintjer Tebbens, R. J., Thompson, K. M., Hunink, M., Mazzuchi, T. M., Lewandowski, D., Kurowicka, D., & Cooke, R.
M. (2008). Uncertainty and sensitivity analyses of a dynamic
economic evaluation model for vaccination programs. Medical
Decision Making, 28(2), 182–200.
9. World Health Organization. (2004). Conclusions and recommendations of the ad hoc advisory committee on poliomyelitis
eradication, Geneva, 21–22 September 2004. Weekly Epidemiological Record, 79(45), 401–408.
10. World Health Organization. (2004). Global Polio Eradication
Initiative: Strategic Plan 2004–2008. Geneva.
11. Bureau of Labor Statistics, U.S. Department of Labor. (2002).
Consumer Price Index. Available at ftp://ftp.bls.gov/pub/
special.requests/cpi/cpiai.txt. Accessed July 6, 2006.
12. Gold, M. R., Siegel, J. E., Russel, L. B., & Weinstein, M. C.
(1996). Cost-Effectiveness in Health and Medicine. New York:
Oxford University Press.
13. World Bank. (2002). World Bank List of Economies (July
2002). Available at http://www.csirwebistad.org/pdf/classi.pdf.
Accessed January 23, 2008.
14. Hull, B. P., & Dowdle, W. R. (1997). Poliovirus surveillance:
Building the global polio laboratory network. Journal of Infectious Diseases, 175(Suppl. 1), S113–S116.
15. World Health Organization. (2005). Advisory committee on
polio eradication—Standing recommendations for responding
to circulating polioviruses in polio-free areas. Weekly Epidemiological Record, 23(38), 330–331.
16. Thompson, K. M., Duintjer Tebbens, R. J., & Pallansch, M.
A. (2006). Evaluation of response scenarios to potential polio outbreaks using mathematical models. Risk Analysis, 26(6),
1541–1556.
17. World Health Organization. (2004). WHO Global Action Plan
for Laboratory Containment of Wild Polioviruses, 2nd ed.
Geneva: World Health Organization. 2004 January. Report
No.: WHO/V&B/03.11.
18. Global Alliance for Vaccines and Immunization. (2005).
Investment Case for the Polio Stockpile. Available at http://
www.vaccinealliance.org/resources/16brd 07. Polio Stockpile
investment case.pdf. Accessed January 7, 2006.
19. de Gourville, E. M., Sangrujee, N., Duintjer Tebbens, R. J.,
Pallansch, M. A., & Thompson, K. M. (2006). Global
surveillance and the value of information: The case of the
Duintjer Tebbens et al.
876
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
global polio laboratory network. Risk Analysis, 26(6), 1557–
1569.
Pinheiro, F. P., Kew, O. M., Hatch, M. H., da Silveira, C. M.,
& de Quadros, C. A. (1997). Eradication of wild poliovirus
from the Americas: Wild poliovirus surveillance—Laboratory
issues. Journal of Infectious Diseases, 175(Suppl. 1), S43–
S49.
World Health Organization. (2006). Inactivated poliovirus
vaccine following oral poliovirus cessation. Supplement to the
WHO position paper. Weekly Epidemiological Record, 81,
137–144.
World Health Organization. (2003). Introduction of inactivated poliovirus vaccine into oral poliovirus vaccine-using
countries. WHO position paper. Weekly Epidemiological
Record, 78, 241–250.
UN Population Division. (2003). World Population Prospects
Population Database: The 2002 Revision Population Database.
Available at http://esa.un.org/unpp/index.asp?panel=2. Accessed July 31, 2003.
UNICEF. (2003). The State of the World’ Children 2003. Available at http://www.unicef.org/sowc03. Accessed January 23,
2008.
Sutter, R. W., Cáceres, V. M., & Más Lago, P. (2004). The role
of routine immunization in the post-certification era. Bulletin
of the World Health Organization, 82(1), 31–38.
Murray, C. J., & Lopez, A. D. (1996). The Global Burden
of Disease: A Comprehensive Assessment of Mortality and
Disability from Diseases, Injuries, and Risk Factors in 1990
and Projected to 2020. Cambridge, MA: Harvard University
Press.
Musgrove, P. (1988). Is polio eradication in the Americas economically justified? Bulletin of the Pan American Health Organization, 22(1), 1–16.
Ministry of Health, Brazil. (1982). Memória Sobre Estimativa
de Custos dos Casos de Poliomielite no Brasil Em. Brasilia,
Brazil.
Miller, M. A., Sutter, R. W., Strebel, P. M., & Hadler,
S. C. (1996). Cost-effectiveness of incorporating inactivated
poliovirus vaccine into the routine childhood immuniza-
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
tion schedule. Journal of the American Medical Association,
276(12), 967–971.
Bart, K., Foulds, J., & Patriarca, P. (1996). Global eradication
of poliomyelitis: Benefit-cost analysis. Bulletin of the World
Health Organization, 74, 35-45.
Hatzandrieu, E. J., Palmer, C. S., & Halpern, M. T. (1994).
A Cost Benefit Analysis of the OPV Vaccine. Atlanta, Batelle:
Centers for Disease Control and Prevention.
Hutubessy, R., Chisholm, D., & Edejer, TT-T, WHOCHOICE. (2003). Generalized cost-effectiveness analysis for
national-level priority-setting in the health sector. Cost Effectiveness and Resource Allocation, 1(1), 8.
World Bank. (2006). World Development Indicators Data
Query. World Bank. Available at http://devdata.worldbank.
org/data-query/. Accessed May 8, 2006.
Saltelli, A., Tarantola, S., Campolongo, F., & Ratto, M. (2004).
Sensitivity Analysis in Practice: A Guide to Assessing Scientific
Models. Chichester, UK: John Wiley and Sons.
Lewandowski, D., Cooke, R. M., & Duintjer, Tebbens, R. J.
(2007). Sample-based estimation of correlation ratio with polynomial approximation (in press). Transactions on Modeling
and Computer Simulation, 18(1), Article 3: 1–17.
World Health Organization. (2005). Immunization Financing. Immunization, Vaccines and Biologicals. Available
at http://www.who.int/immunization financing/en/. Accessed
March 8, 2005.
Cooke, R. M. (1991). Experts in Uncertainty: Opinion and Subjective Probability in Science. New York. Oxford University
Press .
World Health Organization. (2007). Global update on vaccinederived polioviruses, January 2006–August 2007. Weekly Epidemiological Record, 82(39), 337–344.
Eichner, M., & Dietz, K. (1996). Eradication of poliomyelitis:
When can one be sure that polio virus transmission has been
terminated? American Journal of Epidemiology, 143(8), 816–
822.
World Health Organization. (2005). Progress towards poliomyelitis eradication–poliomyelitis outbreak in Sudan, 2004.
Weekly Epidemiological Record, 80(5), 42–46.