Simulations: What Level of Complexity is Appropriate?
Stephen Eubank
Virginia Bioinformatics Institute
Dec. 10, 2007
Advances in Modelisation for Infectious Diseases
F d i Mé
Fondation
Mérieux,
i
A
Annecy
Network Dynamics and Simulation Science Laboratory
Are Individual Based Simulations
Advances in Modelisation
for Control of Infectious Disease?
Stephen
p
Eubank
Virginia Bioinformatics Institute
Dec. 10, 2007
Advances in Modelisation for Infectious Diseases
Fondation Mérieux
Mérieux, Annecy
Network Dynamics and Simulation Science Laboratory
H0: Simulations Aren’t Science
• Not
ot reductionist
educt o st
– No symmetry assumptions
– No analytic solutions
• Not generalizable
– Specific solution to specific problem
– Don
Don’tt build intuition
Network Dynamics and Simulation Science Laboratory
Models of Complex Systems are
Either Complex or Idealized
What is the airspeed velocity of an unladen swallow?
1.
Consider an ideal (spherical) swallow …
2.
What do you mean? African or European swallow?
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Spherical Swallow South African Swallow
(Hi d physica)
(Hirundo
h i )
(Hi d spilodera)
(Hirundo
il d )
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
European Swallow
(Hi d rustica)
(Hirundo
i )
http://www.style.org/unladenswallow/
p
et al.,, Monty
y Python
y
and the Holy
y Grail,, Python
y
(Monty)
(
y) Pictures Ltd. (1975).
(
)
G. Chapman
Network Dynamics and Simulation Science Laboratory
H0’ : Simulations Aren’t Worth the Effort
• They cost too much
– Calibration, validation
• They yield too little
– Numbers,
N b
nott understanding
d t di
• They are hard to understand
– Onlyy experimental
p
sensitivity
y analysis
y
Network Dynamics and Simulation Science Laboratory
Calibration / Validation Is No More Difficult
Alvin Lucier, “Sitting in a Room” (1970)
Q1: What is the limit after many iterations?
Q2: How many are “many”?
Network Dynamics and Simulation Science Laboratory
A Simple
p Linear Model
A(f,t+1)
A(f
t+1) = (f) A(f,t)
A(f t)
Define R0  1 / 2, ratio of two largest ’s.
T=1
Network Dynamics and Simulation Science Laboratory
T=10
A Simple
p Linear Model
A(f,t+1)
A(f
t+1) = (f) A(f,t)
A(f t)
Define R0  1 / 2, ratio of two largest ’s.
Q1: What is the limit after many iterations?
A1: if R0 >1,
>1 f1 (+ harmonics)
Q2: How many are “many”?
many ?
A2: n >> log (A(f2,0)/A(f1,0) ) / log R0
Network Dynamics and Simulation Science Laboratory
Calibration & Validation are Not Easy
Even for a Simple Model
• Ca
Calibration:
b at o
– Measure frequency response
– Estimate R0
• Validation:
– Perform “Sitting…” and listen
– If  expectations after many iterations, model is
• Not valid? No, we know it is structurally correct
• Out of calibration, because system is non-stationary
Network Dynamics and Simulation Science Laboratory
Calibration & Validation are Not Easy
in a Nonstationary System
Calibrate here . . .
and y
you will be out of calibration here
Network Dynamics and Simulation Science Laboratory
All Interesting
g Systems
y
Are Non-stationaryy
“Twice
ce ou
our … e
efforts
o ts [to eliminate
e
ate measles]
eas es] have
a e fallen
a e flat.
at
The failures were due to
• faulty data on the level of immunity required for herd protection…
• models used did not take population heterogeneity into account”
– F.
F Black
Bl k and
dB
B. Si
Singer, “El
“Elaboration
b ti versus simplification
i lifi ti iin refining
fi i mathematical
th
ti l models
d l off
infectious disease,” Annual Reviews Microbiology 41:(1987) 677-701
Network Dynamics and Simulation Science Laboratory
A Simulation Works As Well
•
•
•
Different, but also structurally
y sound
May be easier to calibrate (architectural drawings vs measurements)
Validation is no more difficult, rejection may be easier
http://www.meyersound.com/support/papers/mapp_prediction/
p
y
pp p p
pp_p
Network Dynamics and Simulation Science Laboratory
Useful Acoustic Models Include Both
Dynamics & Architectural Environment
=
Network Dynamics and Simulation Science Laboratory
www.tm
ml.tkk.fi/~lass/publication
ns/thesis/Ray
y_tracing_M
Method.html
+
Useful Epidemiological Models Include
Both Biology & Social Environment
+
Transmission
rates
((biology)
gy)
=
Opportunities
for
transmission
(sociology)
Epidemiological
model
Network Dynamics and Simulation Science Laboratory
Simulations Allow Complete
p
Representations
p
"Much relevant work remains to be done in teasing apart the
social, genetic, age-related, and other complications that
are smoothed out in the usual mass action assumption.”
– May,
May S.R.,
S R Enhanced: Simple rules with complex dynamics,
dynamics Science 287 p.
p 601 (2000)
(2000).
Network Dynamics and Simulation Science Laboratory
Simulations Can Address Important
p
Questions
1. Control
Co t o o
of e
emerging
e g g infectious
ect ous d
disease
sease is
s app
applied
ed
– after an outbreak begins, (T > 0) but
– before it is over (T < ∞))
Network Dynamics and Simulation Science Laboratory
Simulations Provide Information
About Intermediate Times
T=1
T=10
T=3
Network Dynamics and Simulation Science Laboratory
Simulations Provide Information
About Intermediate Times
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
?
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
Network Dynamics and Simulation Science Laboratory
Simulations Can Address Important
p
Questions
1. Control
Co t o o
of e
emerging
e g g infectious
ect ous d
disease
sease is
s app
applied
ed
– after an outbreak begins, (T > 0) but
– before it is over (T < ∞))
2 Optimal controls are adaptive and targeted.
2.
targeted
Optimal allocation of limited resources requires knowing
– who is most vulnerable
– who is most critical
– and when
Network Dynamics and Simulation Science Laboratory
Simulations Permit New Characterizations
• Vulnerability  probability of infection in a set of people
• Criticality  change in vulnerability of others when a set of
people
p
p is removed
Both depend
p
on
• Who is in the set of people (superposition does not hold)
• Time
• Initial conditions (index case)
• Transmission dynamics
• Contact network
Network Dynamics and Simulation Science Laboratory
Vulnerabilityy of 2 People
p in a Population
p
of 150,000
,
Network Dynamics and Simulation Science Laboratory
Models Suggest Appropriate,
Self Consistent Level of Complexity
Self-Consistent
Given
G
e a quest
question,
o ,
• Note: simpler model often a special case of complex model
• Compare sensitivity in specification to model differences
• Does cost of improving specification outweigh benefits?
E g Compare deterministic model with Reed-Frost;
E.g.
Reed Frost;
compartmental with individual-based simulation;
simulations on scale-free
scale-free, small-world,
small-world & realistic graphs;
Network Dynamics and Simulation Science Laboratory
Example: How Much Detail Is
Necessary in Interaction Networks?
Does the degree
g
distribution suffice to estimate vulnerability?
y
• Under the null, vi(t+1) = {1 - [1 - wijjvj(t)]} [1-vj(t’)]
• Estimate v(t) for all t for vertex i and its neighbors
• Compare estimates with value under the null
E..g.: some people have < 1% of expected vulnerability
 Correlations among neighbors (global topology) important
Network Dynamics and Simulation Science Laboratory
Reams of Data Can Be Made Comprehensible
p
Impose conditions at time t to reduce variability significantly at t+
1. Cluster results, e.g. with Principle Components Analysis
2 Assign
2.
A i each
h run tto a cluster
l t
3. Find conditions determining which cluster a run is in
Network Dynamics and Simulation Science Laboratory
Stochasticityy is in the Eye
y of the Beholder
Quantify improvement using standard criteria:
• C
Compare iinformation
f
ti required
i d tto encode
d condition
diti
to information gained by reducing variability
• E.g.
g minimum description
p
length,
g , Akaike
information criterion
Network Dynamics and Simulation Science Laboratory
Simulations Enable Experimental Epidemiology
in a Controlled
Controlled, Stationary
Stationary, Virtual World
• The search for groups with extremes of risk
is an important focus of epidemiology:
“The study of health and disease of populations and
groups … The clinician deals with cases. The
epidemiologist deals with cases in their population.”
population ”
– J.N. Morris, “Uses of Epidemiology”, BMJ 1955 (2) pp. 395-401.
• It is made possible through controlled
experimentation in a stationary setting
Network Dynamics and Simulation Science Laboratory
Conclusion
Detailed simulations
• are not necessarily harder to calibrate and validate
• yield insights into adaptive, targeted control
• can suggest the appropriate level of complexity
• enable otherwise impossible analyses
Network Dynamics and Simulation Science Laboratory
Network Dynamics & Simulation Science Lab
C. Barrett, M. Marathe, S. Bedare, A. Feng, H. Mortveit, A. Vullikanti, B.
Lewis K
Lewis,
K. Adasi
Adasi, M
M. Macauley
Macauley, D
D. Chefakur
Chefakur, X
X. Feng
Feng, G
G. Hansen
Hansen, A
A. Aji , P
P.
Stretz, S. Harris, A. Marathe, S. Eubank, Y. Kidane, A. Apolloni, J. Randall
Not pictured: R. Beckman, J. Chen
Criticality
y of 2 People
p in a Population
p
of 34
Network Dynamics and Simulation Science Laboratory
What Constitutes a “Solution”?
• Probabilityy of seeing
g configuration
g
C at time t
–  joint probability of the state of every vertex at time t
– A linear (Markov) process: p(C’, t+1) = M * p(C, t),
where Mij = p(Ci(t+1) | Cj(t))
– M is specified by mN parameters: estimation not feasible
• Alternatively, consider update of each vertex’s state, si(t)
– Cannot be written as linear process in si,
p(si, t+1)  M * p(si, t))
– Marginal distributions are specified by m*N
m N parameters{
estimation feasible
Network Dynamics and Simulation Science Laboratory