73
Chapter 3
Reinforced Random Walks
The theory of reinforced random walks provides a natural framework for modelling
the movement of individuals. Continuum descriptions of biological behaviour, based
on average quantities such as concentrations and densities, apply to the macroscopic
behaviour of a large number of particles. As with all averaging processes, some
information is discarded for the sake of simplicity.
In the case of chemical substances, where the number of particles (in this case
molecules) is extremely large, a macroscopic description is sufficient. Reaction–
diffusion equations for chemical concentrations describe the chemical dynamics very
accurately. However, the smaller the number of individuals in the population, the
less accurate the macroscopic description becomes. For example, it would be inappropriate to use a reaction–diffusion equation for population density to model the
behaviour of a dozen birds.
A population of cells is intermediate, in terms of both the number and the size of the
individuals, between these two extremes. It is not unreasonable to use a macroscopic
description, but a glance at the complex structure of an in vivo capillary network
(such as that shown in Figure 1.10(a)) immediately shows that much important information is discarded by such an approach. Such a network clearly does not consist
of a continuous distribution of cells, but is a highly discrete arrangement. Of course,
one could claim the same to be true of molecules of a chemical compound, but the
scale required to observe such discrete behaviour is much smaller and its effect on
macroscopic behaviour much less. One could formulate a model in which, not only
the EC, but equally the molecules of the relevant chemicals, were treated as indi-
74
viduals. However, the disadvantages caused by the massive increase in complexity
of the model would not be matched by any significant improvement in accuracy. In
contrast, the behaviour of individual EC and their organisation into a hierarchy of
blood vessels is clearly an important feature of angiogenesis. The inclusion of such
individual behaviour into a mathematical model can provide information about the
microscopic properties of the emergent capillary network.
It is my opinion that studying both continuum and discrete models in tandem is the
best way to improve understanding of angiogenesis. The reinforced random walk
framework is particularly useful in this respect as it allows the continuum limit of
the discrete model to be rigorously derived. The transition probability function
for the discrete model survives this limiting process, thus providing a crucial link
between microscopic and macroscopic descriptions of cell behaviour.
The theory of reinforced random walks was developed by Davis [40] and first used
in a biological context by Othmer and Stevens [117]. Analytical work of this model
was performed in [88] (see section 2.3) and the framework of [88, 117] has since been
used to construct both continuum [87, 90] and discrete [151] models of angiogenesis.
In this chapter, the mathematical tools and techniques used to model angiogenesis
will be developed. In particular, attention will be focussed on the relationship
between the discrete and continuous forms of the random walk equations. In section
3.1, a definition of a reinforced random walk is given and then used to derive the
master equation of [117]. The continuum limit of the master equation will be given
in several cases in section 3.2. In section 3.3, a result regarding the large time
behaviour of the time-independent continuum limit equation will be proved. In
section 3.4, a generalisation of the model of [117] is presented. Section 3.5 examines
the aggregation and collapse properties of reinforced random walks. Finally, section
3.6 describes the method of simulation of cell movement.
3.1
The Reinforced Random Walk Master Equation
We begin with the definition of Davis [40].
Definition:
A reinforced random walk is a sequence of integer-valued random
variables, X = Xj : j ∈ Z≥0 , and a matrix of positive random variables, Ω =
75
ωn,j : n ∈ Z, j ∈ Z≥0 , such that
(i)
ωn,j+1 − ωn,j ≥ 0,
with equality if (Xj , Xj+1 ) is neither (n, n + 1) nor (n + 1, n).
(ii)
P (Xj+1 = n + 1|Xj = n) = 1 − P (Xj+1 = n − 1|Xj = n)
ωn,j
=
.
ωn,j + ωn−1,j
(3.1)
A reinforced random walk is therefore a discrete time and space jump process. The
variables Xj usually represent the position of the walker (or cell) after j steps, whilst
ωn,j may be thought of as the weight of the interval (n, n + 1) after j steps. The
greater an interval’s weight, the more likely it is to be crossed by the cell.
The random walk is called ‘reinforced’ because the weight of an interval increases
when it is crossed by the cell. The cell thus reinforces the intervals it crosses, making
them more likely to be crossed again in the future. Note that this definition reduces
to an ordinary, unbiased random walk in the case ωn,j = ω0 constant (i.e. the
weights of all the intervals are equal and no reinforcement occurs).
To close the system, a reinforcement scheme (i.e. rules about how the weight of an
interval changes when it is crossed) must be specified. The scheme may depend deterministically or stochastically on the movements of the cell. The simplest example
is called sequence type reinforcement [40]:
ψ(n,j)
ωn,j = ωn,0 +
X
ai ,
(3.2)
i=1
where ψ (n, j) is the number of times (n, n + 1) has been crossed after j steps and
the ai ≥ 0 are non-negative constants. This is a deterministic scheme and the
reinforcement sequence is independent of n.
Othmer and Stevens [117] used the ideas developed in [40] to formulate a framework
suitable for biological modelling, writing the position of the cell at time t = jk
as x = hXj (for constant space and time steps, h and k) and interpreting ω as
the concentration of some critical control substance. As in the chemotaxis models
76
described in section 2.3, this may be any substance that modulates the movement
of the cell, such as nutrient, oxygen, or TAF. The basic definition was extended to
allow P (Xj+1 = n|Xj = n) > 0 (i.e. there may be a positive probability of the cell
remaining at a grid point for more than one time step), and more general forms of
the transition probabilities were considered.
We therefore redefine the one-step transition probabilities:
±
P (Xj+1 = n ± 1|Xj = n) = Rn,j
,
(0)
P (Xj+1 = n|Xj = n) = Rn,j ,
for j ∈ Z≥0 , n ∈ Z, such that
(0)
+
−
Rn,j
+ Rn,j
+ Rn,j = 1.
It follows that
(0)
+
−
pn,j+1 = Rn−1,j
pn−1,j + Rn,j pn,j + Rn+1,j
pn+1,j ,
where pn,j ≡ P (Xj = n) is the probability that the walker is at point n at time step
j. Hence
pn,j+1 − pn,j =
+
Rn−1,j
pn−1,j
+
−
Rn+1,j
pn+1,j
+
−
− Rn,j + Rn,j pn,j .
(3.3)
We now move to a continuous time jump process by writing pn,j = pn (t) and
±
Rn,j
= Rn± (W (t)) (where t = jk). W (t) is the vector of control substance values:
W (t) = . . . , w− 1 (t) , w0 (t) , w 1 (t) , w1 (t) , . . . .
2
2
Dividing through by k in (3.3) and letting k → 0 gives the following differential
equation for pn (t) [117]:
where
∂pn
−
+
(W ) pn+1 − τ̂n+ (W ) + τ̂n− (W ) pn ,
(W ) pn−1 + τ̂n+1
= τ̂n−1
∂t
(3.4)
Rn± (W )
k→0
k
is the transition rate of movement from point n to point n ± 1.
τ̂n± (W ) = lim
Equation (3.4) is called the master equation and forms the basis for the simulations
of cell movement presented in this thesis. Specification of the reinforcement scheme
77
for the reinforced random walk is analogous to specification of the control substance
dynamics in the continuous time process. For example, the simple sequence type
reinforcement scheme (3.2) with ai = kβ (i = 1, 2, 3, . . .) is analogous to linear
growth of the control substance,
∂w
= βp.
∂t
Note, however, that in the discrete time and space process of [40], reinforcement
of an interval occurs only when it is crossed by a walker. In the continuous time
process, the walker modifies the control substance at the point it is occupying.
Furthermore, the walker may occupy the same point for an indefinite amount of
time.
3.2
The Continuum Limit
One of the advantages of the reinforced random walk approach is that it allows easy
transition between the discrete model and its continuum limit. The continuum limit
of the master equation (3.4) was derived in [117] for several choices of τ̂n± (W ), called
the local model, the barrier model (with and without normalisation), the nearest
neighbour model and the gradient model. Here, we are concerned mainly with the
first two of these.
3.2.1
Local Model
If the transition rates depend only on the local value of the control substance, wn :
τ̂n+ (W ) = τ̂n− (W ) = λτ (wn ) ,
for some constant, λ ≥ 0, and transition probability function, τ (w) ≥ 0, then,
writing p (x, t) = pn (t) and w (x, t) = wn (t) (where x = nh), and letting h →
0, λ → ∞, such that λh2 = D, the continuum limit of (3.4) is
∂p
∂2
= D 2 (pτ (w)) .
∂t
∂x
(3.5)
Strictly, this is a PDE for the probability density function, p (x, t), for the position,
X (t), of a single cell. However, for a non-interacting population of cells, each of
78
which moves independently according to (3.4), the sum of the probability densities
for all the cells will also satisfy the same continuum limit (3.5). For a large number
of cells, (3.5) may therefore be interpreted as a PDE for the cell density. The
derivation of a general continuum limit for interacting populations, however, is an
unresolved problem (see for example [154]).
The flux associated with equation (3.5) is
J = −Dτ (w)
∂p
∂w
+ χ (w) p
,
∂x
∂x
where the chemotactic sensitivity is χ (w) = −Dτ 0 (w). Note that, in common with
the chemotaxis model of [72] (see section 2.3), this does not allow a chemotactic term
to be retained in the case where the random diffusion coefficient is constant. This
is not surprising since the cells sense only the local control substance concentration,
with no directional discrimination.
3.2.2
Barrier Model
If the transition rates depend on the value of the control substance in the interval
to be crossed:
τ̂n± (W ) = λτ wn± 1 ,
(3.6)
2
then the continuum limit of (3.4) is
∂p
∂
=D
∂t
∂x
∂p
τ (w)
∂x
(3.7)
.
The flux associated with equation (3.7) is
J = −Dτ (w)
∂p
.
∂x
There is therefore no chemotaxis. Again, this is unsurprising because
−
(W ) = λτ wn+ 1 .
τ̂n+ (W ) = τ̂n+1
2
In other words, the transition rate from n to n + 1 is identical to the transition rate
from n + 1 to n. This is entirely analogous to the model of [72] in the case where
the length of steps taken by the cells is the same as the distance between receptors
at either end of the cell.
79
3.2.3
Barrier Model with Normalisation
In order to retain a chemotactic term, Othmer and Stevens [117] normalised the
transition rates so that the total rate of transition away from a given point is independent of W :
τ̂n+ (W ) + τ̂n− (W ) = 2λ.
They thus defined
2λτ wn± 1
2
.
τ̂n± (W ) = τ wn+ 1 + τ wn− 1
2
(3.8)
2
The control substance therefore affects the direction of movement, but not the overall
rate of movement: the decision ‘when to move’ is made independently of the decision
‘where to move’ and the mean waiting time of a cell at a point is constant,
1
2λ .
Note
that this is, in fact, the original form of transition probabilities used by Davis [40],
with τ (w) = w and wn+ 1 (t) in [117] equivalent to wn,j in [40].
2
Under the choice (3.8), the continuum limit of the master equation (3.4) is
∂
p
∂
∂p
p
ln
.
=D
∂t
∂x
∂x
τ (w)
(3.9)
This may be shown as follows. Suppressing the time argument and Taylor expanding
w about x = nh and τ about w = w (nh) gives
0
00 2
0
1 ± τ 2τwx h + 18 τ wτ xx + τ τwx h2 + O h3
∓
= λ
τ̂n±1
0
00 2
0
1 ± τ τwx h + 5(τ wxx8τ+τ wx ) h2 + O (h3 )
0
τ wxx τ 00 wx2
τ 02 wx2 h2
τ 0 wx
3
h+ −
−
+
+O h
= λ 1∓
.
2τ
τ
τ
τ2
2
Taylor expanding p about x = nh gives
p ((n ± 1) h) = p ± px h +
pxx 2
h + O(h3 ).
2
Therefore
∓
p ((n
τ̂n±1
where
Ux 2
τ0
3
± 1) h) = λ p ± px − p wx h +
,
h +O h
2τ
2
U = px − p
τ0
wx .
τ
80
Hence by (3.4),
∂pn
∂t
τ0
Ux 2
3
= λ p + px − p wx h +
h +O h
2τ
2
τ0
Ux 2
3
h +O h
− 2λp
+λ p − px − p wx h +
2τ
2
τ0
= λ
px − p wx h2 + O h3 .
τ
x
Letting h → 0, λ → ∞, such that λh2 = D gives equation (3.9) as required.
The above results have, for clarity, been given in one spatial dimension, but are
straightforward to generalise to higher dimensions. For example, the two-dimensional
version of the master equation (3.4) is
∂pn,m
∂t
H+
H−
V+
V−
= τ̂n−1,m
pn−1,m + τ̂n+1,m
pn+1,m + τ̂n,m−1
pn,m−1 + τ̂n,m+1
pn,m+1
H+
H−
V+
V−
pn,m ,
(3.10)
− τ̂n,m
+ τ̂n,m
+ τ̂n,m
+ τ̂n,m
where pn,m (t) is the cell probability density at point (n, m) at time t and the
superscripts H± and V ± indicate a transition from point (n, m) to point (n ± 1, m)
and (n, m ± 1) respectively.
Othmer and Stevens [117] wrote the transition rates as:
4λτ wn± 1 ,m
2
H±
, (3.11)
τ̂n,m
(W ) =
τ wn+ 1 ,m + τ wn− 1 ,m + τ wn,m+ 1 + τ wn,m− 1
2
2
2
2
4λτ wn,m± 1
V±
2
. (3.12)
τ̂n,m
(W ) =
τ wn+ 1 ,m + τ wn− 1 ,m + τ wn,m+ 1 + τ wn,m− 1
2
2
2
The continuum limit of the two-dimensional master equation (3.10) is
∂p
p
= D∇. p∇ ln
.
∂t
τ (w)
2
(3.13)
However, it should be noted that the form of the transition rates (3.11), (3.12) is
not the only choice that will lead to the continuum limit (3.13). By analogy with
the one-dimensional transition rates (3.8), one may equally define
2λτ wn± 1 ,m
2
H±
,
τ̂n,m
(W ) =
τ wn+ 1 ,m + τ wn− 1 ,m
2
2
(3.14)
81
V±
τ̂n,m
(W ) =
2λτ wn,m± 1
2
,
τ wn,m+ 1 + τ wn,m− 1
2
(3.15)
2
under which choice movement in the x direction is independent of movement in the
y direction.
The continuum limit in this case is
∂p
∂
∂
p
∂
∂
p
=D
p
ln
+D
p
ln
,
∂t
∂x
∂x
τ (w)
∂y
∂y
τ (w)
(3.16)
which is identical to (3.13). It seems somewhat artificial to separate the x and
y directions in the master equation in this way. We will, therefore, generally use
transition rates of the form (3.11), (3.12), but bear in mind that this choice is
non-unique.
The flux associated with equation (3.13) is
J = −D∇p + Dp
τ 0 (w)
∇w.
τ (w)
Hence we have a chemotactic flux that is independent of the random diffusive flux.
The relation between the chemotactic sensitivity, χ (w), and the transition probability function, τ (w), is given by
0
(w)
χ (w) = D ττ (w)
, ln (τ (w)) =
1
D
R
χ (w) dw.
(3.17)
Provided this relationship between τ (w) and χ (w) is satisfied, equation (3.13) is
exactly equivalent to the more familiar diffusion–chemotaxis equation (2.40). The
function τ (w) survives the transition from the discrete (3.10) to the continuous
(3.13) form. It therefore provides an important link between the macroscopic and
microscopic modelling approaches.
τ (w) may be thought of as the attractiveness to the cell of a control substance
concentration w. If τ 0 (w) > 0, χ (w) > 0: if higher concentrations of w are more
attractive than lower concentrations then, unsurprisingly, the chemotactic flux is
directed up the concentration gradient. The converse holds if τ 0 (w) < 0, whereas
if τ (w) is constant, the chemotactic flux is zero and (3.13) reduces to the heat
equation.
τ (w) may be used to generate the one-step transition probabilities of an individual
cell, and also to determine the associated PDE for cell density. Choosing a func-
82
tional form for τ (w) (or equivalently χ (w)) that faithfully reflects the governing
biology is a difficult step in mathematical modelling. Experimental data are needed
to correctly determine such functional forms for specific cell types and control substances.
The inclusion of multiple control substances, w = (w1 , . . . , wM ), in the master
equation (3.10) may be readily achieved by writing
τ (w) =
M
Y
τi (wi ).
i=1
The continuum limit equation (3.13) is unchanged and the extra flux terms simply
appear additively:
J = −D∇p + Dp
M
X
τ 0 (wi )
i
i=1
τi (wi )
∇wi .
Condition (3.17) becomes
τ 0 (w )
χi (wi ) = D τi(wii) , ln (τ (w1 , . . . , wM )) =
1
D
PM R
i=1
χi (wi ) dwi ,
(3.18)
where χi (wi ) is the chemotactic sensitivity of the cells to control substance i at
concentration wi .
3.3
Steady State Solution of the Time-Independent
Continuum Limit Equation
In this section, we prove that the continuum limit equation (3.13), in the case where
τ (w) is independent of t, has a unique steady state solution that is globally stable.
We assume that τ is explicitly known as a function of x: such a scenario could arise
when the control substances, w (x, t), are in steady state, or when w (x, t) tends to
a limiting distribution as t → ∞.
Theorem: Let Ω ⊂ RN be a connected, bounded domain with well-defined unit
outward normal, n̂ : ∂Ω → RN , such that
Z
Z
uxi (x) dx =
u (x) n̂i (x) dS,
Ω
∀u ∈ C 1 (Ω) , i = 1, . . . , N,
(3.19)
∂Ω
where dS is an element of ∂Ω. This is a form of the Gauss integral theorem [60].
83
Let D > 0 be a constant and let τ ∈ C 2 (Ω) be a real-valued function such that
R
τ (x) > 0, ∀x ∈ Ω and Ω τ (x) dx < ∞. Consider the following PDE, together with
initial and boundary conditions:
∂p
p
,
= D∇. p∇ ln
∂t
τ (x)
1 ∂τ
1 ∂p
=
,
x ∈ ∂Ω, t ≥ 0,
p ∂n
τ ∂n
p (x, 0) = p0 (x) ≥ 0,
x ∈ Ω.
x ∈ Ω, t ≥ 0,
(3.20)
(3.21)
(3.22)
The unique steady solution of this system is
R
p0 (x) dx
ps (x) = RΩ
τ (x) ,
Ω τ (x) dx
(3.23)
and this solution is globally stable (i.e. p (x, t) → ps (x) as t → ∞).
Proof: Let p (x, t) = f (x, t) τ (x). The transformed system is
1 ∂f
D ∂t
1 ∂f
f ∂n
= ∇2 f +
= 0,
f (x, 0) =
∇τ.∇f
,
τ
x ∈ Ω, t ≥ 0,
x ∈ ∂Ω, t ≥ 0,
p0 (x)
,
τ (x)
(3.24)
(3.25)
x ∈ Ω.
(3.26)
We now seek solutions in the form f (x, t) = X (x) T (t). The PDE (3.24) becomes
∇2 X
∇τ.∇X
T0
=
+
,
DT
X
τX
x ∈ Ω, t ≥ 0.
By separation of variables, we obtain the PDE:
∇2 X +
∇τ.∇X
τ
1 ∂X
X ∂n
= −σX,
= 0,
x ∈ Ω,
x ∈ ∂Ω,
where σ ∈ C is a separation constant.
We introduce the real Hilbert space,
Z
2
0
u (x) τ (x) dx < ∞ ,
H = u ∈ C (Ω) :
Ω
with inner product h·, ·i : H × H → R defined by:
Z
hu, vi =
u (x) v (x) τ (x) dx.
Ω
(3.27)
84
Let L : D → H be the Sturm–Liouville operator:
L=−
1
∇. (τ (x) ∇) ,
τ (x)
(3.28)
where the domain, D = u ∈ C 2 (Ω) : ∇u.n̂ = 0, x ∈ ∂Ω ⊆ H.
We thus have the classic eigenvalue problem
LX = σX,
X ∈ D, σ ∈ C.
(3.29)
Note that, by (3.19) and the boundary conditions on D,
Z X
N hLu, vi − hu, Lvi =
Ω i=1
Z
=
− (τ (x) uxi )xi v + (τ (x) vxi )xi u dx
τ (x)
∂Ω
N
X
(−uxi v + vxi u) n̂i dS
i=1
= 0,
∀u, v ∈ D.
D is dense in H, so L is a symmetric operator. It follows that all eigenvalues of L
are real [60].
Again by (3.19) and the boundary conditions on D, we have that
hLu, ui = −
Z X
N
Ω i=1
=
Z
τ (x)
Ω
≥ 0,
(τ (x) uxi )xi udx
N
X
u2xi dx
i=1
−
Z
τ (x) u
∂Ω
∀u ∈ D,
N
X
uxi n̂i dS
i=1
with equality if and only if u (x) ≡ k, for some k ∈ R.
Now let (λ, uλ ) be an eigenvalue/eigenfunction pair for the problem (3.29). We have
that
λ huλ , uλ i = hLuλ , uλ i ≥ 0,
so λ ≥ 0, with equality if and only if uλ (x) ≡ k.
By (3.27),
T (t) = e−σDt .
85
Hence, the only term in the solution for f (x, t) that does not tend to zero as t → ∞
is that for which σ = 0 and X (x) ≡ k. Returning to the original variable, we have:
!
X
p (x, t) = τ (x) k +
Aσ Xσ (x) e−σDt
σ>0
where the Aσ are constants.
Thus as t → ∞, p (x, t) → ps (x), where the value of k given in (3.23) is determined
by conservation of mass.
In summary, the stable steady solution of the continuum limit equation (3.13) is a
constant multiple of the transition probability function, τ (x). The steady solution
of diffusion–chemotaxis equations of the form (2.40) may also be obtained, by using
relationship (3.17).
3.4
The Reinforced Random Walk with Variable Mean
Waiting Time
The normalisation of the transition rates (3.8) was made in order to retain a chemotactic term. The assumption is effectively that the mean waiting time,
−1
,
τ̂n+ (W ) + τ̂n− (W )
is independent of W . In other words, the control substance affects the direction,
but not the rate, of movement.
This model may therefore be used for a control substance that has a chemotactic (gradient-driven) effect, but not one that has a chemokinetic (diffusion-driven)
effect. Conversely, the unnormalised barrier model (3.7) represents a control substance with a chemokinetic, but no chemotactic effect. In this section, we describe
two possible forms of the transition rates, τ̂n± (W ), that allow both chemotactic and
chemokinetic effects to be modelled independently.
86
3.4.1
Let
Variable Lambda
λ wn± 1 τ wn± 1
2
2 ,
τ̂n± (W ) = 2λ0 τ wn+ 1 + τ wn− 1
2
(3.30)
2
for some function, λ (w).
The mean waiting time at point n is then given by
+
τ
w
τ
w
1
1
n− 2
n+ 2
1
2λ0 λ w 1 τ w 1 + λ w 1 τ w
n+ 2
n+ 2
n− 12
n− 2
.
Taylor expanding w about x = nh and λ about w = w (nh) gives
h2
h
+ O h3 .
λ wn± 1 = λ ± λ0 wx + λ0 wxx + λ00 wx2
2
2
8
The master equation (3.4) is therefore
∂pn
τ0
τ0
2
0
= λ0 h λ px − p wx + λ wx px − p wx
+ O λ0 h3 .
∂t
τ
τ
x
(3.31)
(3.32)
Now letting h → 0, λ0 → ∞ such that λ0 h2 = D gives the continuum limit
∂p
∂
∂
p
∂
p
0
= Dλ (w)
p
ln
+ Dλ (w) wx p
ln
∂t
∂x
∂x
τ (w)
∂x
τ (w)
∂
∂
p
=
D (w) p
ln
,
(3.33)
∂x
∂x
τ (w)
where D (w) = Dλ (w).
The flux associated with equation (3.33) is
J = −D (w)
∂p
τ 0 (w) ∂w
+ D (w) p
.
∂x
τ (w) ∂x
The inclusion of variable mean waiting times, via the function λ (w), has therefore
given a variable diffusion coefficient, allowing chemokinetic effects to be modelled.
The chemotactic sensitivity is given by:
0
(w)
, ln (τ (w)) =
χ (w) = D (w) ττ (w)
R
χ(w)
D(w) dw.
(3.34)
87
In the case where there are multiple control substances, the equivalent relationship
is
M
X
1
∂
∂wi
χi (wi )
τ (w1 , . . . , wM ) =
.
τ (w1 , . . . , wM ) ∂x
D (w1 , . . . , wM ) ∂x
(3.35)
i=1
However, the right-hand side cannot be integrated without additional information
about the relationship between the wi and x. Usually, τ (w1 , . . . , wM ) will have to
be computed numerically.
Moreover, in the case where there is more than one control substance and more
than one spatial dimension, we must have
M
X
χi (wi )
1
∇τ (w1 , . . . , wM ) =
∇wi .
τ (w1 , . . . , wM )
D (w1 , . . . , wM )
(3.36)
i=1
Differentiating the first component of (3.36) with respect to y and the second component with respect to x, and demanding that
M
X
χi (wi )
i=1
∂2
∂x∂y
=
∂D ∂wi ∂D ∂wi
−
∂y ∂x
∂x ∂y
∂2
∂y∂x ,
gives
= 0.
For M > 1, this will not, in general, be true for all x and y. In this case there is no
well defined function, τ (w1 , . . . , wM ), for which the continuum limit of the master
equation (3.10), with transition rates of the form (3.30), is the diffusion–chemotaxis
equation:
3.4.2
∂p
= ∇. (D (w) ∇p) − ∇. (χ (w) p∇w) .
∂t
Combination of Normalised and Unnormalised Barrier Models
Recall that the unnormalised barrier model of [117] resulted in a continuum limit
PDE (3.7) with no chemotactic flux, but with a variable rate of random diffusive
flux. One may combine the unnormalised barrier model with the normalised barrier
model to include both diffusion-driven and gradient-driven stimuli. We therefore
take the transition rates to be the sum of the normalised (3.8) and the unnormalised
(3.6) transition rates, using two distinct transition probability functions, τ (w) and
D (w) respectively:

D wn± 1
2τ wn± 1
2
2
+
− 1 ,
τ̂n± (W ) = λ  D
0
τ wn+ 1 + τ wn− 1

2
2
(3.37)
88
for some constant D0 > 0.
The continuum limit h → 0, λ → ∞, such that λh2 = D0 of the master equation
(3.4) is simply the sum of the individual continuum limit PDEs (3.9), (3.7):
∂p
∂
p
∂
∂p
∂
p
ln
+
(D (w) − D0 )
(3.38)
= D0
∂t
∂x
∂x
τ (w)
∂x
∂x
τ 0 (w) ∂w
∂
∂p
− D0 p
=
D (w)
.
(3.39)
∂x
∂x
τ (w) ∂x
We have again succeeded in introducing a variable diffusion coefficient, D (w). Note
that this method avoids the problems associated with the method of section 3.4.1
of choosing the transition probability function, τ (w), because the variable diffusion
coefficient, D (w), does not appear in the chemotactic term. Hence the chemotactic
sensitivity is given by:
0
(w)
χ (w) = D0 ττ (w)
, ln (τ (w)) =
1
D0
R
χ (w) dw,
(3.40)
which is analogous with the constant mean waiting time case (3.17) and easily
extends to higher dimensions and multiple control substances. The disadvantage of
this approach, as opposed to the variable lambda technique, is the introduction of
the arbitrary parameter, D0 , in the transition rates (3.37), to which the continuum
limit PDE (3.39) (on making the choice (3.40)) is invariant.
3.5
Aggregation and Collapse in Discrete Time and Space
The behaviour of the continuum limit equations, and in particular the existence of
solutions that aggregate, blowup and collapse, was examined in [88] (see section 2.3)
and [117]. Here a corresponding result of [40] regarding the discrete jump process
is presented, together with a proof of the first part.
Theorem:
Consider a reinforced random walk X = (X0 , X1 , X2 , . . .) with ini-
tial weights, ωn,0 = 1, and sequence type reinforcement (3.2) with sequence a =
(a1 , a2 , . . .). Let
φ (a) =
∞
X
i=1
1+
i
X
k=1
ak
!−1
.
(3.41)
(i) If φ (a) is infinite then X is recurrent almost surely; if φ (a) is finite then X has
finite range almost surely.
89
(ii) In the finite range case, there are (random) integers, N and J, such that
j > J ⇒ Xj ∈ [N, N + 1] .
The theorem effectively states that localisation of the cell to two adjacent grid points
will occur if and only if the weights grow sufficiently rapidly. For example, if ai = a0
constant then φ (a) = ∞, so the walk is recurrent almost surely, meaning that, with
probability 1, all the integers are visited infinitely often (so the cell does not become
localised to a particular area). If, however, ai = i, then φ (a) is finite, so the cell only
visits a finite number of points almost surely, and will eventually alternate between
just two adjacent points. Recurrence of the walk corresponds to the collapse case
of the continuum model, whilst finite range is analogous to the aggregation case.
The following is a proof of part (i) (due to [40]). Let
α0 = 1, αi = 1 +
Pi
k=1 ak
(i = 1, 2, 3, . . .)
(3.42)
be the weight of an interval after it has been crossed i times and assume, without
loss of generality, that P (X0 = n) = 1 for some n ∈ Z.
Suppose first that φ (a) < ∞. For m > n, let
Tm = min {j : Xj = m}
be the time at which the cell first reaches point m.
Now at time Tm , the weights of the intervals (m − 1, m), (m, m + 1) and (m + 1, m + 2)
are
ωm−1,Tm = 1 + a1 , ωm,Tm = 1, ωm+1,Tm = 1.
Hence, conditioned on {Tm < ∞}, the probability that
(XTm , XTm +1 , XTm +2 , . . .) = (m, m + 1, m, . . .)
(i.e. that the cell, after arriving at point m, oscillates between m and m + 1) is
given by
p=
∞ Y
i=0
α2i
α2i + 1 + a1
α2i+1
.
α2i+1 + 1
φ (a) < ∞ so by (3.41) and (3.42), αj → ∞ as j → ∞. Therefore p > 0.
90
However, P (Tm+2 < ∞|Tm < ∞) ≤ 1 − p < 1, so P (Tm < ∞, ∀m > n) = 0. Thus
X is not recurrent. It can be shown [40] that
P (X is recurrent) + P (X has finite range) = 1,
so P (X has finite range) = 1.
Now suppose that φ (a) = ∞. For m ≥ n, let
v0 = min {j > 0 : Xj = m} ,
vi+1 = min {j > vi : Xj = m, Xk 6= m + 1, 0 ≤ k ≤ j} .
Then
{vi < ∞, ∀i ≥ 0} =
(
sup Xj = m, lim Xj = m
j→∞
j≥1
)
≡ Am .
If m > n, then at time vi , the weights of the intervals (m − 1, m) and (m, m + 1)
are
ωm−1,vi = α2i+1 , ωm,vi = 1,
since, if vi < ∞, (m − 1, m) is crossed exactly twice between vi−1 and vi .
Hence
P (vi+1 < ∞|vi < ∞) ≤ P (Xvi +1 = m − 1|vi < ∞) =
so
P (Am ) ≤ P (v0 < ∞)
∞
Y
i=0
φ (a) = ∞ so by (3.41) and (3.42), limi→∞
αi
αi+2
P (Am ) ≤ P (v0 < ∞) α1
α2i+1
,
1 + α2i+1
α2i+1
.
1 + α2i+1
≥ 1. Therefore
∞
Y
i=1
α2i+1
= 0.
1 + α2i−1
Similarly, P (An ) = 0, so P supj≥1 Xj = limj→∞ Xj = 0. It may also be shown
that P supj≥k Xj = limj→∞ Xj = 0 for each k ≥ 0, so X does not have finite
range. Hence P (X is recurrent) = 1, which completes the proof of (i).
This result relates the properties of the reinforced random walk to the reinforcement
scheme for the weights of the intervals. An equivalent result in the continuum case
91
would relate the behaviour of the cell density to the control substance dynamics
and the transition probability function, τ (w). For example, Levine and Sleeman
[88] studied the continuum limit equation (3.9) in the case
∂w
∂t
= βpw, τ (w) = w−χ0 ,
for constants, β ≥ 0 and χ0 (see section 2.3). The discrete time and space process of
[40] does not strictly correspond to a system of this form because of the differences
pointed out in section 3.1. Nevertheless, it is instructive to compare results in the
two cases.
The case χ0 = −1 corresponds to sequence type reinforcement (3.2), where the
weight of an interval after i reinforcements is
αi = eβki .
By (3.41),
φ (a) =
1
< ∞,
−1
eβk
so the walk has finite range.
This agrees with the results of [88], which say that, for χ0 = −1, there exist solutions
for which the cell density blows up in finite time. The other case treated in [88],
χ0 = 1, corresponds to taking the inverse of the interval weights:
αi = e−βki .
Note that, in addition to having positive waiting times at a grid point, this does
not conform to condition (i) of the definition of a reinforced random walk because
the weight of an interval decreases when it is crossed. Nevertheless, the definition
may be extended to include decreasing interval weights and the theorem still holds.
Clearly, φ (a) is infinite, so the walk is recurrent. Again, this is in agreement with
[88], which shows there are solutions for which the cell density collapses.
The theorem of [40] may easily be applied for any χ0 , and predicts finite range for
χ0 < 0 and recurrence for χ0 ≥ 0. It would be interesting to see if the equivalent
result (i.e. blowup for χ0 < 0 and collapse for χ0 ≥ 0) holds in the continuum case.
92
3.6
The Method of Simulation
This section contains a description of how the reinforced random walk framework,
as described above, is used to simulate cell movement. The method, first used by
Sleeman and Wallis [151], is described in the two-dimensional case, with constant
mean waiting times, but easily generalises to other cases.
Consider a cell at grid point (n, m) at time step j. For a given distribution of control
substances, w, and transition probability function, τ (w), the transition rates for that
cell are given by (3.11), (3.12). The one-step transition probabilities of moving to
the left, right, down and up are thus given by:
4λkτ wn± 1 ,m
H±
2
,
Rn,m
=
τ wn+ 1 ,m + τ wn− 1 ,m + τ wn,m+ 1 + τ wn,m− 1
2
2
2
2
4λkτ wn,m± 1
V±
2
.
Rn,m
=
τ wn+ 1 ,m + τ wn− 1 ,m + τ wn,m+ 1 + τ wn,m− 1
2
2
2
2
The probability of remaining at point (n, m) is given by:
(0)
Rn,m
= 1 − 4λk.
Having generated these probabilities, the method proceeds in the same way as in
[4, 7] (see section 2.5.2). A random number, r ∈ [0, 1), is generated1 .
If r ∈ 0, R(0) ,
If r ∈ R(0) , R(0) + RH− ,
If r ∈ R(0) + RH− , R(0) + RH− + RH+ ,
If r ∈ R(0) + RH− + RH+ , R(0) + RH− + RH+ + RV − ,
If r ∈ R(0) + RH− + RH+ + RV − , 1 ,
stay still.
move left.
move right.
move down.
move up.
This process is repeated for each cell before moving on to the next time step (see
Figure 3.1).
As we have seen above, not all PDE models conform to the strict definition of a
reinforced random walk, even if one allows a cell to remain at a grid point for more
than one time step. For example, inclusion of a natural decay term in the control
1
This is done in C, using the ‘ran0’ routine of [129].
93
Set up lattice, decide initial number
of cells, K, and total number of
time steps, J
Start
j=0
Initialise position of cell i:
n(i,0)=a(i), m(i,0)=b(i), Z(i)=-1
i=K?
Y
Initialise substrate concentrations
at each lattice node
N
i=i+1
i=0
j=j+1, i=0
i=i+1
N
Z(i)=-1?
Y
Compute transition probabilities
for cell i:
P(stay still) = R0
P(move left) = R1
P(move right) = R2
P(move down) = R3
P(move up) = R4
Generate random number: 0 < r < 1
Y
0 < r < R0 ?
n(i,j)=n(i,j-1), m(i,j)=m(i,j-1)
N
R0 < r < R0+R1 ?
Y
n(i,j)=n(i,j-1)-1, m(i,j)=m(i,j-1)
N
R0+R1 < r < R0+R1+R2 ?
Y
n(i,j)=n(i,j-1)+1, m(i,j)=m(i,j-1)
N
Y
R0+R1+R2 < r < R0+R1+R2+R3 ?
n(i,j)=n(i,j-1), m(i,j)=m(i,j-1)-1
N
n(i,j)=n(i,j-1), m(i,j)=m(i,j-1)+1
Y
Has cell anastomosed?
N
Has cell branched?
Z(k)=0
Y
Initialise new cell:
K=K+1, n(K,j-1)=n(i,j-1), m(K,j-1)=m(i,j-1), Z(K)=-1
N
N
i=K?
Y
Update substrate concentrations
at each node
N
j=J?
Y
Finish
Figure 3.1: A flowchart of the generic simulation algorithm. (n (i, j) , m (i, j)) are
the coordinates of cell i at time step j; (a (i) , b (i)) are the initial coordinates of
cell i; Z (i) is a flag that equals −1 if cell i is actively migrating and equals 0
otherwise. Methods for deciding whether a cell has anastomosed and/or branched
and for computing substrate dynamics will be defined for specific models.
94
substance dynamics means that the weight of an interval can change without it being
crossed by a cell. Incorporating control substance diffusion adds a spatial interaction
between interval weights. Furthermore, in the case where more than one cell is
present, the cells are not following independent walks, even if there are no direct cell–
cell interactions, because they are responding to a common set of interval weights,
which are (in most cases) modified by other cells. There will, therefore, always be
an implicit interaction between the cells via the reinforcement of the intervals they
cross. The continuum limit equations derived in sections 3.2 and 3.4 apply only for
non-interacting populations. Hence, these equations must, at present, be regarded as
an approximate macroscopic description for weakly interacting populations, which
is asymptotically accurate as the strength of the interactions tends to zero.
Nevertheless, the method described above may be used to simulate the movement of
one or more cells in response to one or more control substances. Provided that the
transition probability function, τ (w), is specified and that the values of the control
substances at each grid point and the positions of the cells are known, the method
can be used as a black box to determine the positions of the cells at the next time
step.
3.7
Summary
• Reinforced random walks provide a natural framework for modelling the movement of individual cells in response to one or more control substances.
• For non-interacting populations, the continuum limit of the reinforced random
walk master equation may be rigorously derived.
• To retain a chemotactic flux in the continuum limit, the transition rates must
be normalised, so the mean waiting time at a grid point is constant. In higher
dimensions, the choice of transition rates is non-unique.
• The transition probability function provides a link between the continuum
and discrete forms, and gives the stable steady solution of the continuum
limit PDE. Choosing an appropriate functional form is a non-trivial step in
constructing the model.
• Using a variable mean waiting time introduces a variable diffusion coefficient,
enabling chemotactic and chemokinetic effects to be modelled. There are two
95
ways of choosing the transition rates that achieve this goal, each with its own
advantages and disadvantages.
• Key references: Davis [40], Othmer and Stevens [117], Levine and Sleeman
[88], Sleeman and Wallis [151].