Bounds for the expected value of one-step processes
Ádám BESENYEI
[email protected]
Department of Applied Analysis and Computational Mathematics,
Eötvös Loránd University, Budapest
&
Numerical Analysis and Large Networks Research Group,
Hungarian Academy of Sciences
Joint work with Benjamin
Armbruster & Péter L. Simon
10QTDE Szeged, July 3, 2015
Outline
1. Formulation of the problem
• One-step processes
• Mean-field approximation and accuracy
2. Towards the main result
• Moments of the process
• Approximating system
• Main result, tools and idea of the proof
3. Applications
• SIS epidemic propagation
• SIS epidemic with airborne infection
• Voter model
Ádám BESENYEI (ELTE)
Bounds for the expected value. . .
10QTDE Szeged, July 3, 2015
0 / 15
Setting
• Continuous-time Markov process X(t) with state space {0, 1, . . . , N }.
• One-step or Birth–Death process: transition from state k is possible
only to state k − 1 at rate ck and to state k + 1 at rate ak (and let
a−1 = aN = c0 = cN +1 = 0).
ak−1
ak−2
ak+1
aN −1
ak
a0
...
0
c1
k−1
ck−1
ck
...
k+1
k
ck+2
ck+1
N
cN
• Example: SIS epidemic or voter model.
• Probability distribution of X(t): pk (t) = P [X(t) = k].
⇓
• Time-evolution is described by Kolmogorov or master equation:
p0k = ak−1 pk−1 − (ak + ck )pk + ck+1 pk+1
Ádám BESENYEI (ELTE)
Bounds for the expected value. . .
(k = 0, . . . , N ).
10QTDE Szeged, July 3, 2015
1 / 15
Expected value of the process
Expected value
of the process
3
0
1
N≤
Numerical solution
of the master equation
Approximation
of the process
tic
o
pt
∞
ym
→
N
Mean-field
approximation
as
Ádám BESENYEI (ELTE)
Bounds for the expected value. . .
Monte Carlo
simulation
fo
ra
ll
N
Upper and
lower bounds
10QTDE Szeged, July 3, 2015
2 / 15
Accuracy of the mean-field approximation
• T. G. Kurtz (1970, 2005): density-dependent Markov processes,
stochastic convergence of the exact stochastic to the deterministic
mean-field model. (Tools: Trotter-type results, martingale theory.)
• A. Bátkai, I. Z. Kiss, E. Sikolya, P. L. Simon (2012): uniform
convergence of the expected value to the mean-field model, precisely,
if y1 (t) = E[X(t)/N ] is the expected value of the scaled process and
y(t) is the mean-field approximation, then
CT
|y1 (t) − y(t)| ≤
(t ∈ [0, T ]).
N
(Tools: operator semigroups.)
• B. Armbruster, E. Beck (2015): SIS epidemic model on a complete
graph, upper and lower bounds for y1 . (Tools: elementary ODE.)
• B. Armbruster, Á. B., P. L. Simon (2015): extension of the results of
Armbruster & Beck to a wider class of Markov chains.
Ádám BESENYEI (ELTE)
Bounds for the expected value. . .
10QTDE Szeged, July 3, 2015
3 / 15
Formulation of the main result: ODEs for the moments
• Transition rates are density dependent polynomials:
m
m
X
X
ak
ck
=
Aj (k/N )j and
=
Cj (k/N )j
N
N
j=0
j=0
such that A(1) = 0 and C(0) = 0 hold.
⇓
• ODEs for the scaled moments: yn = E[X(t)/N ] (n = 0, 1, 2, . . . ).
y10 =
m
X
Dj yj ,
j=0
m
X
yn0 = n
Dj yn+j−1 +
j=0
where Dj = Aj − Cj , 0 ≤ Rn ≤
Ádám BESENYEI (ELTE)
1
Rn
N
(n = 2, 3, . . . ),
m
X
n(n − 1)
c, and c =
(|Aj | + |Cj |).
2
j=0
Bounds for the expected value. . .
10QTDE Szeged, July 3, 2015
4 / 15
Formulation of the main result: approximations
• Mean-field equation: we approximate yn ≈ y1n , then by y10 =
y0 =
m
X
m
X
Dj yj ,
j=0
Dj y j ,
y(0) = y1 (0).
j=0
• ODEs for the powers of y:
(y n )0 = n
m
X
Dj (y n )
n+j−1
n
,
y n (0) = y1n (0)
(n = 2, 3, . . . ).
j=0
• Approximating system for the moments:
z10 =
m
X
D j zj ,
j=0
m
X
zn0 = n
z1 (0) = y1 (0),
n+j−1
n
Dj z n
j=0
+
n(n − 1)
c, zn (0) = y1n (0)
2N
(n = 2, 3, . . . , m),
where we let z0 = 1.
Ádám BESENYEI (ELTE)
Bounds for the expected value. . .
10QTDE Szeged, July 3, 2015
5 / 15
Main result
Theorem (B. Armbruster, Á. B., P. L. Simon, 2015)
Assume that
D0 ≥ 0, D1 ∈ R and Dj ≤ 0 for j ≥ 2,
and let y1 (0) = u ∈ (0, 1] be fixed. Then for the solutions y of the
mean-field equation and z1 of the approximating system, it holds that
z1 (t) ≤ y1 (t) ≤ y(t) for t ≥ 0
and for every T > 0 there exists a constant CT > 0 such that
CT
|z1 (t) − y(t)| ≤
in [0, T ].
N
Idea of the proof.
Smart application of three familiar inequalities.
Ádám BESENYEI (ELTE)
Bounds for the expected value. . .
10QTDE Szeged, July 3, 2015
6 / 15
Tools of the proof: familiar inequalities
• Comparison: If f : [0, T ] × [a, b] → R is Lipschitz continuous in its
second variable, then
)
x01 (t) = f (t, x1 (t)), x1 (0) = x0 ,
=⇒ x2 (t) ≤ x1 (t) for t ∈ [0, T ].
x02 (t) ≤ f (t, x2 (t)), x2 (0) ≤ x0
• Peano’s inequality: Assume that f1 , f2 : [0, T ] × [a, b] → R are
Lipschitz continuous in their second variable with Lipschitz constant
L and |f1 (t, x) − f2 (t, x)| ≤ M in [0, T ] × [a, b] with some M . Then
x01 (t) = f1 (t, x1 (t)), x1 (0) = x0 ,
)
x02 (t) = f2 (t, x2 (t)), x2 (0) = x0
M Lt
e − 1 for t ∈ [0, T ].
L
• Jensen’s inequality: If X is a random variable and ϕ : R → R is a
convex function, then
=⇒ |x1 (t) − x2 (t)| ≤
ϕ(E[X]) ≤ E[ϕ(X)].
Ádám BESENYEI (ELTE)
Bounds for the expected value. . .
10QTDE Szeged, July 3, 2015
7 / 15
Flow of the proof
Comparison
Step 1:
a priori bounds independent of N
Step 2:
comparison of y and y1
Step 3:
comparison of yn , y n and zn
Step 4:
comparison of y1 and z1
Ádám BESENYEI (ELTE)
Comparison
ison
Compar
Jensen
Peano
Comparison
Peano
Bounds for the expected value. . .
Lipschitz continuity
guaranteed
y1 (t) ≤ y(t) (t ∈ [0, T ])
yn (t), y n (t) ≤ zn (t)
(t ∈ [0, T ], n ≥ 2)
CT,n
N
(t ∈ [0, T ], n ≥ 2)
|y n (t) − zn (t)| ≤
z1 (t) ≤ y1 (t),
CT
|y1 (t) − z1 (t)| ≤
N
10QTDE Szeged, July 3, 2015
8 / 15
Application: SIS epidemic propagation
• Random d-regular graph with N nodes in two possible states:
S
S = Susceptible
I
I
S
I
S
I
S
I
= Infected
infection rate: τ
I
I
recovery rate: γ
I
S
• State space ≈ {0, 1, . . . , N } the number of infected nodes.
• Average number of SI edges is (N − k) · d Nk .
⇓
• Transition rates are ak = τ d(N − k)
Ádám BESENYEI (ELTE)
k
and ck = γk.
N
Bounds for the expected value. . .
10QTDE Szeged, July 3, 2015
9 / 15
Application: SIS epidemic propagation
• Mean-field equation:
y 0 = (τ d − γ)y − τ dy 2 ,
y(0) = i/N.
• Approximating system:
z10 = (τ d − γ)z1 − τ dz2 ,
z1 (0) = i/N,
2τ d + γ
3/2
,
z20 = 2(τ d − γ)z2 − 2τ dz2 +
N
z2 (0) = (i/N )2 .
expected value
0.6
0.5
Figure: SIS epidemic propagation.
Parameters: γ = 1, τ = 0.1, d = 30.
Curves:
y;
z1 for N = 106 ;
7
z1 for N = 10 .
0.4
0.3
0.2
0.1
0
0
1
2
3
4
5
6
time
Ádám BESENYEI (ELTE)
Bounds for the expected value. . .
10QTDE Szeged, July 3, 2015
10 / 15
Application: SIS epidemic with airborne infection
k
• Infection also due to external forcing: ak = τ d(N − k) + β(N − k).
N
• Mean-field equation:
y 0 = β + (τ d − β − γ)y − τ dy 2 ,
y(0) = i/N.
• Approximating system:
z10 = β + (τ d − β − γ)z1 − τ dz2 ,
z20 =
z1 (0) = i/N,
c
3/2
+ 2(τ d − β − γ)z2 − 2τ dz2 + , z2 (0) = (i/N )2 .
N
1/2
2βz2
c = β + |τ d − β| + τ d + γ
expected value
0.6
0.5
Figure: SIS with airborne infection.
Parameters:
N = 100, γ = 1, τ = 0.05, d = 20, β = 1.
Curves:
y;
z1 ;
y1 .
Inset: stationary part of the curves.
0.62
0.4
0.615
0.61
0.3
0.605
0.2
0.1
0
0.6
2
1
2.5
2
3
3.5
3
4
4
time
Ádám BESENYEI (ELTE)
Bounds for the expected value. . .
10QTDE Szeged, July 3, 2015
11 / 15
Application: voter model
• Two opinions, 0 and 1 are changing:
1
0
1
0
rate: τ
1
1
0
0
rate: γ
⇓
• Transition rates are ak = τ d(N − k)
• Mean-field equation:
k
N −k
and ck = γdk
.
N
N
y 0 = (τ d − γd)(y − y 2 ),
y(0) = i/N.
• Approximating system:
z10 = (τ d − γd)z1 − (τ d − γd)z2 ,
z1 (0) = i/N,
2τ d + 2γd
3/2
,
z20 = 2(τ d − γd)z2 − 2(τ d − γd)z2 +
N
Ádám BESENYEI (ELTE)
Bounds for the expected value. . .
z2 (0) = (i/N )2 .
10QTDE Szeged, July 3, 2015
12 / 15
Application: voter model
1
expected value
N = 107
0.8
0.6
N = 105
0.4
N = 104
Figure: Voter-like model: case D2 < 0.
Parameters: γ = 0.1, τ = 0.2, d = 10.
y;
z1 .
Curves:
0.2
0
0
2
4
6
8
10
time
0.1
−4
expected value
1
0.08
x 10
0.8
0.6
0.06
0.4
0.2
0.04
0
7
8
9
10
0.02
0
0
2
4
6
8
Figure: Voter-like model: case D2 > 0.
Parameters:
N = 200, γ = 0.2, τ = 0.1, d = 10.
Curves:
y;
z1 ;
y1 .
Inset: stationary part of the curves.
10
time
Ádám BESENYEI (ELTE)
Bounds for the expected value. . .
10QTDE Szeged, July 3, 2015
13 / 15
Further directions
• Relaxing the sign condition: convexity arguments?
• Improving the lower bound with modified approximating systems:
1. z10 = D0 + D1 z1 + D2 z2 .
1/2
2. z20 = 2D0 z2
expected value
0.6
3/2
+ 2D1 z2 + 2D2 z2
2τ d + γ
.
N
z22 /z1
Figure: Lower bounds of the SIS model.
Parameters:
N = 100, γ = 1, τ = 0.1, d = 30.
Curves:
y;
y1 ;
z1 in case 1;
z1 in case 2.
0.4
q=1
0.3
0.2
0
0
+
q = 0.5
0.5
0.1
q/2
z2 z12−q
q=2
1
2
3
4
5
time
Ádám BESENYEI (ELTE)
Bounds for the expected value. . .
10QTDE Szeged, July 3, 2015
14 / 15
References
B. Armbruster, E. Beck, An elementary proof of convergence to the
mean-field equations for an epidemic model, arXiv:1501.03250
B. Armbruster, Á. Besenyei, P. L. Simon, Bounds for the expected
value of one-step processes, arXiv:1505.00898
A. Bátkai, I. Z. Kiss, E. Sikolya, P. L. Simon, Differential equation
approximations of stochastic network processes: an operator
semigroup approach, Netw. Heter. Media, 7 (2012), 43–58.
S. N. Ethier, T. G. Kurtz, Markov Processes: Characterization and
Convergence, John Wiley & Sons Ltd, New York, 2005.
T. G. Kurtz, Solutions of ordinary differential equations as limits of
pure jump Markov processes, J. Appl. Prob., 7 (1970), 49–58.
Ádám BESENYEI (ELTE)
Bounds for the expected value. . .
10QTDE Szeged, July 3, 2015
15 / 15
Thank you for your attention!