Master of Science in Mathematics
Numerical investigation of
solutions of Langevin equations
Nikki Vercauteren
Winter 2005-2006
supervised by Prof. Christof Schütte, Prof. John Maddocks and
Carsten Hartmann
Abstract
Two recent pieces of analysis predicted certain limiting behaviours of Langevin
equations. This thesis concerns the numerical investigation of the range of
validity of these two analyses. In the low-friction limit it has been suggested
that quasi-symplectic integrators could be appropriate to handle long time
integration. Based on two examples, it is concluded that quasi-symplectic
time-stepper are slightly more efficient than standard schemes. In the opposite context of high friction, the long time integration problem can be
addressed by approximating the second order Langevin equation by a first
order Smoluchowski equation, and recent research has provided an analytic
error estimator for this approximation. The numerical examples studied
here verify the accuracy of this error estimator.
Acknowledgement
It is my great pleasure to thank Prof. John H. Maddocks and Prof. Christof
Schütte for their mathematical guidance and for their many suggestions and
interesting discussions. I am also grateful for the opportunity to accomplish
my master thesis in the framework of an exchange program between the
Ecole Polytechnique Fédérale de Lausanne and the Freie Universität Berlin.
Special thanks go to Carsten Hartmann for his invaluable help throughout
the project, and to many members of the Bio Computing Group who all
welcomed me. Finally I would like to thank my parents and my friends for
their support during my studies.
Contents
1 Introduction
3
2 Background
2.1 Brownian motion and Langevin equation
2.2 The Fokker-Planck equation . . . . . . .
2.3 The fluctuation-dissipation relation . . .
2.4 The Smoluchowski time scale . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
3 Numerical integration of Langevin equation
3.1 Strong and weak schemes . . . . . . . . . . . . . . . . . .
3.2 Quasi-symplectic integrators . . . . . . . . . . . . . . . . .
3.3 Numerical example 1: a particle in a double-well potential
3.4 Numerical example 2: a stiff, 2 degree of freedom example
3.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . .
4 High-friction Langevin equations
4.1 Approximation by Smoluchowski equations . . . . .
4.2 Error indicator . . . . . . . . . . . . . . . . . . . . .
4.3 Numerical example 1: a 2 degree of freedom system
4.4 Numerical example 2: a linear chain of beads . . . .
4.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . .
2
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
4
4
5
7
8
.
.
.
.
.
9
9
10
15
20
24
.
.
.
.
.
25
25
26
28
32
37
Chapter 1
Introduction
This report is divided in two parts, both involving numerical implementations and experiments relating to recent analytical work on Langevin equations. Langevin equations are widely used in models from physics, chemistry
and biology. In particular they play a key role in molecular simulations. In
that context and in order to evaluate averages with respect to equilibrium
distribution for Langevin equations, one has to integrate a system over a
very long time, and this is the major difficulty. The two parts of this report address two different approaches to handling the long time integration
problem.
The first part is based on recent results in [1] on the computation of ergodic limits for Langevin equations, using a specialized integration scheme
called quasi-symplectic. In Chapter 2, we present the background and required standard results for Hamiltonian sytems and Langevin equations. In
the case of small dissipation the Langevin equations are close to Hamiltonian
systems. Therefore the authors of [1] propose that in such a situation, which
is rather typical, numerical methods based on symplectic integrators are the
most appropriate. Chapter 3 will provide numerical examples that resolve
whether or not the studied quasi-symplectic method is much more efficient
than traditional integration schemes. It is concluded that the new method
is only slightly more efficient than the standard schemes.
The second part deals with the approximation of Langevin equations in
the limit of high-friction by Smoluchowski equations; see the forthcoming
article [8]. It has been understood for a long time that Smoluchowski equations are a good approximation of Langevin equations in the asymptotic
limit of high friction. However, when is the friction high enough to use this
approximation? In the case where the Hamiltonian is linear, the authors in
[8] give an error indicator for the approximation. Chapter 4 describes this
indicator and provides numerical examples that show its accuracy.
3
Chapter 2
Background
2.1
Brownian motion and Langevin equation
Brownian motion was discovered in 1827 by the botanist Brown, see [10];
it describes the disordered motion of a particle in a fluid that experiences
collisions from the molecules surrounding it. The fluid particles are considered to move at random, and therefore the Brownian particle receives a
random number of impacts of random strength and from random directions
in any short period of time. Once in a while, the Brownian particle receives
more impacts from one direction, and this leads to small displacements. In
the case where an external force is present, the particle will have another
coherent movement that overlaps the previous one. In addition it will be
slowed down by friction from the fluid. Thus the fluid is both an accelerator
and a damper to the movement. If the velocity of the Brownian particle is
small compared to the one from the fluid particles, one can use the simplest
model, in which the friction force is proportional to the velocity. For this
Brownian particle, the fundamental equations of motion include an external
force, a friction force, and a fluctuating force:
dQ = P dt
∂U
dP = −
(Q) − νP + F(t),
∂q
q, p ∈ Rn .
(2.1)
This is called a Langevin equation. Considering that F(t) results from
numerous individual collisions, the central limit theorem suggests that its
distribution is asymptotically Gaussian. We also make the following assumption:
• The mean of F(t) is zero, i.e. hF(t)i = 0.
We will say more about the correlation properties in section 2.3.
4
2.2
The Fokker-Planck equation
An alternative way to study stochastic processes is by means of their probability distributions. A corollary of the Feynman-Kǎc formula (see [3] p.127)
enables us to link stochastic differential equations and Kolmogorov forward
equations, also called Fokker-Planck equations:
Theorem 1 Suppose that a, b and g are bounded smooth functions (a, b, g:
Rn 7→ Rn ). Let X ∈ Rn be the solution of the stochastic differential equation
dX(t) = a(t, X(t))dt + b(t, X(t))dW (t) and let
Z
u(x, t) = E[g(X(T ))|X(t) = x] =
g(y)ρ(y, T ; x, t)dy.
R
Then the density function, ρ, solves the Fokker-Planck equation,
n
n
X
∂
1 X ∂2
i
T ij
(a
ρ)
+
(bb
)
ρ
= 0,
−∂t ρ −
∂y i
2
∂y i y j
i=1
i,j=1
ρ(y, t; x, t) = δ(x − y),
where δ is the Dirac-delta measure concentrated at zero.
For simplicity of notation, we introduce the Hamiltonian of the system
(2.1):
1
H = p2 + U (q),
(2.2)
2
where the term 12 p2 is the kinetic energy of the system, and U (q) is the
potential energy. We introduce the following matrices:
0 1
0 0
0 0
J=
, Γ=
, Σ=
,
(2.3)
−1 0
0 ν
0 σ
where σ is the amplitude of the covariance of the random force F(t). It is
easily checked that equation (2.1) can be written into the vector SDE,
Q
d
= (J − Γ) ∇H(Q, P )dt + ΣdW (t),
(2.4)
P
where σ is the correlation of the random force F(t). The density function ρ
solves the Fokker-Planck equation:
1
−∂t ρ − ∇ · ((J − Γ) ∇H(Q, P )ρ) + ∇2 :
2
ΣΣT ρ = 0,
(2.5)
where ∇ is the gradient in Q and P , “ · ” is the usual scalar product, ∇ 2 is
the Hessian matrix, and “ : ” is the matrix inner product. This leads to
− (J − Γ) ∇H(Q, P ) · ∇ρ − ρ div ((J − Γ) ∇H(Q, P )) +
5
1
2
ΣΣT : ∇2 ρ
for the Kolmogorov forward operator. Now J∇H is a Hamiltonian vector
field, and since Hamiltonian systems preserve volume along the flow,
div(J∇H(Q, P )) = 0.
Furthermore we find
div(Γ∇H(Q, P )) = −Γ : ∇2 H(Q, P ),
which gives
1
− (J − Γ) ∇H(Q, P ) · ∇ρ + ρ Γ : ∇2 H(Q, P ) +
2
for the Kolmogorov forward operator.
We now seek conditions under which the density
ρ0 (p, q) =
ΣΣT : ∇2 ρ (2.6)
1 −βH(p,q)
e
,
Z
(2.7)
is a solution of (2.6). We first compute the gradient and the Hessian matrix
of (2.7):
∇ρ0 = −β∇Hρ0 ,
∇2 ρ0 =
−β∇2 H + β 2 ∇H∇H T ρ0 ,
Inserting (2.7) into (2.6), we find that
− (J − Γ) ∇H · β∇Hρ0 + ρ0 Γ : ∇2 H
1
+
ΣΣT : −β∇2 H + β 2 ∇H∇H T ρ0 = 0.
2
The term J∇H · ∇H cancels:
J∇H · ∇H =
∇p H
−∇q H
(2.8)
∇q H
·
= 0,
∇p H
Since the equality (2.8) must be satisfied for every value of the Hamiltonian
H(q, p), one obtains the condition
β2
T
T
βΓ∇H · ∇H + ΣΣ : ∇H∇H
ρ0 = 0,
2
β
Γ : ∇2 H − ΣΣT : ∇2 H ρ0 = 0,
2
and therefore
ΣΣT = 2Γ/β
(2.9)
Thus (2.9), or using (2.3), σσ T = 2ν/β, is both a necessary and sufficient
condition for e−βH(P,Q) to be a stationary density of the Fokker-Planck equation, or equivalently an equilibrium probability distribution for the SDE
(2.6).
6
2.3
The fluctuation-dissipation relation
We will now give some motivation for the equality (2.9). First we will make
a second assumption about the correlation function of the force F(t):
• The correlation is hF(t1 )F(t2 )i = σσ T δ(t1 − t2 ) where σσ T is a constant.
This force F(t) is a white noise. In order to determine the value of the
constant σσ T , we use the solution of (2.1) in the absence of external force,
Z t
P (t) = P0 e−νt +
e−ν(t−s) F(s)ds.
(2.10)
0
We compute the second moment:
Z t
hP (t)2 iP0 = P02 e−2νt + 2
P0 e−ν(2t−s) hF(s)iP0 ds
0
Z tZ t
0
e−ν(2t−s−s ) hF(s)F(s0 )idsds0 .
+
0
(2.11)
0
Using the assumptions made for F(t), we get
hP (t)2 iP0 = P02 e−2νt +
σσ T
(1 − e−2νt )
2ν
(2.12)
Finally we get the limit result from (2.12) for t → ∞
1
σσ T
hP (t)2 i =
,
t→∞ 2
4ν
lim
(2.13)
if we use the equality (2.9), i.e. σσ T = 2ν/β, we now get
1
1
hP (t)2 i =
.
t→∞ 2
2β
lim
(2.14)
Writing β = kB T −1 (kB being Boltzmann constant and T being the temperature), the equality 2.14 is the equipartition of energy. The equipartition
of energy is a common assumption in physics, and thus the result
σT σ =
2ν
β
(2.15)
is a natural result. This is the fluctuation-dissipation relation. Finally we
can write the Langevin equation with the fluctuation-dissipation relation:
dQ = P dt,
r
∂U
2ν
dP = −
(Q)dt − νP dt +
dW (t),
∂q
β
where hdW (t1 )dW (t2 )i = δ(t1 − t2 ).
7
(2.16)
2.4
The Smoluchowski time scale
From equation (2.12) we see that the characteristic time scale for decay of
memory of the particle’s initial velocity is τ ≈ 1/ν, in the absence of external
force. In the case where the viscosity ν is large in equation (2.16), the
particle will have hardly changed its configuration at times where velocities
have reached their equilibrium distribution. We shall briefly sketch the
transition from Langevin equation (2.16) with an external force (deriving
from a potential) to its high friction limit for a one dimensional system. To
this end, we scale the coefficients ν, σ in the following way:
√
ν 7→ ν/,
σ 7→ σ/ ,
and we will study the limit → 0. The scaling for σ comes from the
fluctuation-dissipation relation (2.15). We thus get the singularly perturbed
Langevin equation, written as a second order ordinary differential equation
for the sake of simplicity:
Q̈ = −
∂U
ν
σ
(Q ) − Q̇ + √ Ẇ (t).
∂q
(2.17)
Scaling time to the so-called diffusive time-scale according to t 7→ t/, the
equation becomes
2 Q̈ = −
∂U
(Q ) − ν Q̇ + σ Ẇ (t).
∂q
(2.18)
In a crude way, when → 0, we can assume that 2 Q̈ → 0 and obtain the
limit equation
∂U
(Q) + σ Ẇ (t),
(2.19)
ν Q̇ = −
∂q
which is the Smoluchowski equation. The white noise in the equation is
unbounded, and therefore we cannot be sure that 2 Q̈ goes to zero as goes
to zero. However it can be shown that the sample paths Q (t) converge with
probability one to Q(t) as defined by the Smoluchowski equation. Rigorous
computations can be found in [5] and [8].
8
Chapter 3
Numerical integration of
Langevin equation
In this chapter, we will test the quasi-symplectic numerical methods as presented in [1] for the integration of Langevin equations.
3.1
Strong and weak schemes
We will introduce in this section some material from [6]. Let us first consider
an Itô stochastic differential equation
dXt = a(Xt )dt + b(Xt )dWt .
(3.1)
The simplest time-discrete approximation of this equation is the stochastic
generalization of the Euler approximation,
Yn+1 = Yn + a(Yn )∆n + b(Yn )∆Wn ,
n = 0, 1, · · · , N − 1,
(3.2)
with initial value
Y0 = x 0 ,
(3.3)
∆n = tn+1 − tn = δ,
(3.4)
and with
where we consider an equidistant time discretization and
∆Wn = Wn+1 − Wn .
(3.5)
The random variables ∆Wn defined in (3.5) are independent normally distributed random variables of mean zero and variance ∆ n , N (0; ∆n ).
Depending on the problem, we will be either interested in approximations where the sample paths are close to those of the Itô process, or just in
some function of the value of the Itô process at a given final time T . The
first requirement is much stronger, and this suggests that we will distinguish
9
between a criterion involving some form of strong convergence, and a criterion involving a form of weak convergence. Mathematically, we consider the
absolute error at the final instant T , see [6].
(δ) = E (|XT − YN |) .
(3.6)
Definition 1 We say that the approximating process Y converges in the
strong sense with order γ ∈ (0, ∞] if there exists a finite constant K and a
positive constant δ0 such that
E (|XT − YN |) ≤ Kδ γ
(3.7)
for any time discretization with maximum step size δ ∈ (0, δ 0 ).
In the case where we are interested only in some function of the value of the
Itô process at a given final time T , it suffices to have a good approximation
of the probability distribution of the random variable X T rather than a close
approximation of sample paths. We then have the following criterion, see
[6].
Definition 2 We say that the approximating process Y converges in the
weak sense with order β ∈ (0, ∞] if for any polynomial g there exists a finite
constant K and a positive constant δ 0 such that
|E (g(XT )) − E (g(YN ))| ≤ Kδ β
(3.8)
for any time discretization with maximum step size δ ∈ (0, δ 0 ).
When we are interested only in weak approximations of the Itô process,
we have much greater liberty in choice of numerical approximation than
with strong approximations. For example, the random increments ∆W n of
the white noise can be replaced by some other approximations ∆ Ŵn which
have similar moment properties to the ∆W n , that is, same mean and same
variance. This means
we could use a two-point distributed random variable
√
taking values ± ∆n with equal probability,
p 1
P ∆Ŵn = ± ∆n = .
(3.9)
2
3.2
Quasi-symplectic integrators
The following theory can be found in more details in [1] and [2]. We consider
the Langevin equations (2.16) which we recover here with linear damping
and additive noise,
dQ = M −1 P dt,
dP = f (Q)dt − νΓP dt + σdW (t)
10
(3.10)
where P , Q, f are n-dimensional vectors, Γ is a n × n-dimensional constant
matrix, σ is a n-dimensional constant vector, W is a n-dimensional white
noise, M is a symmetric positive definite constant matrix, and
f i (Q) = −
∂U
(Q),
∂q i
i = 1, ..., n.
(3.11)
This system reduces to a deterministic Hamiltonian system in the absence
of friction, i.e. ν = 0, and noise, i.e. σ = 0. See [1, 2]
Hamiltonian systems possess the property of phase-volume preservation,
as well as the more important property of preserving symplectic structure
(symplecticness), which we will define in the following. Consider a domain with finite volume D0 ∈ Rd , d = 2n. The transformation (p, q) 7−→
(P (t; p, q), Q(t; p, q)) maps D0 into the domain Dt . The volume Vt of the
domain Dt is equal to:
Z
Vt =
dP 1 ...dP n dQ1 ...dQn
Dt
Z D(P 1 , .., P n , Q1 , ..., Qn ) 1
n 1
n
=
(3.12)
dp1 , ...dpn , dq 1 , ...dq n dp ...dp dq ...dq .
D0
Therefore, the volume preserving condition is the following equality:
D(P 1 , .., P n , Q1 , ..., Qn ) (3.13)
dp1 , ...dpn , dq 1 , ...dq n = 1.
Milstein et al. [2] consider this quantity for the Langevin equation with
linear damping and additive noise. We write equation (3.10) as the SDE
Q
P
0 0
d
=
dt +
(dW1 (t), dW2 (t)),
(3.14)
P
f (Q) − νΓP
0σ
where (W1 , W2 ) is a 2n-dimensional white noise. According to [2], we introduce the following matrix:

 1
∂Q
∂Q1 ∂Q1
∂Q1
.
.
.
.
.
.
n
n
1
1
∂q
∂p 
∂p
 ∂q.
..
.. . .
. 
..
.
Φ(t) = 
(3.15)
. .
. ..  .
.
 .
n
n
n
∂P
∂P
∂P
∂P n
∂q 1 . . . ∂q n ∂p1 . . . ∂pn
This matrix is a solution of the linear system
dΦ = AΦdt + BΦdW (t),
Φ(t0 ) = I,
(3.16)
where A and B are matrices with the components
∂ai
(t, Y 1 (t; t0 , y), · · · , Y 2n (t; t0 , y)),
∂y j
∂bi
(t, Y 1 (t; t0 , y), · · · , Y 2n (t; t0 , y)),
∂yj
11
0 0
with Y =
a=
and b =
[2]. The formula
0σ
for the determinant of the solution to this linear equation is
Z t
Z t
det Φ(t) = det Φ(t0 ) exp
tr(A)ds +
tr(B)dW (s) .
(3.17)
(Q, P )T ,
(P, f (Q) − νΓP )T
t0
t0
Since
D(P 1 , .., P n , Q1 , ..., Qn ) det Φ(t) = dp1 , ...dpn , dq 1 , ...dq n and
det Φ(t0 ) = 1,
we obtain the following theorem (Corollary 2.2 of [2]):
Theorem 2 The necessary and sufficient condition for volume preservation
of the system (3.14) are the equalities
1
div (a − Bb) = 0,
2
div b = 0.
(3.18)
The 12 Bb term is a consequence of the Itô formula (see [3] p. 33). For the
system (3.14), since σ is a constant vector, we get:
∂σ
∂σ
+ · · · + n = 0,
∂q 1
∂q
1
div (a − Bb) = div a
2
∂P 1
∂P n ∂(f (Q1 ) − νΓ11 p1 )
∂(f (Qn ) − νΓ11 pn )
=
+
·
·
·
+
+
+
·
·
·
+
∂q 1
∂q n
∂p1
∂pn
= −ν trΓ,
(3.19)
Pn
where trΓ = i=1 Γii is the trace of the matrix Γ. Replacing those values in
(3.17), we get the Jacobian of the system (3.10),
D(P 1 , .., P n , Q1 , ..., Qn ) = exp (−νt).
J =
(3.20)
dp1 , ...dpn , dq 1 , ...dq n div b =
Therefore when ν = 0, the volume of a domain is preserved along the flow.
If ν > 0, then phase volume contracts.
Let us now define the symplectic structure for a dynamical system. We
introduce an operation between differential 1-forms (see [4] p. 166)
Definition 3 Let ω1 , ω2 be a pair of 1-forms on Rn . The exterior product
ω1 ∧ ω2 on the pair of vectors ξ1 , ξ2 ∈ Rn is:
ω1 (ξ1 ) ω2 (ξ1 ) ,
(ω1 ∧ ω2 )(ξ1 , ξ2 ) = (3.21)
ω1 (ξ2 ) ω2 (ξ2 ) i.e. it is the oriented area of the image of the parallelogram with sides ω(ξ 1 )
and ω(ξ2 ) in the ω1 , ω2 -plane. This product is bilinear and skew-symmetric.
12
We can now define a symplectic structure on a vector space. (see [4] p. 201)
Definition 4 Consider the vector space R 2n . A symplectic structure on
R2n is a closed non-degenerate 2-form ω 2 on R2n :
dω 2 = 0
and
∀ξ 6= 0
∃η : ω 2 (ξ, η) 6= 0
(ξ, η ∈ T R2n )
We introduce the differential 2-form,
dp ∧ dq = dp1 ∧ dp1 + ... + dpn ∧ dq n .
(3.22)
This is a symplectic structure on R2n . For Hamiltonian systems, we have
the following theorem (see [4] p.204):
Theorem 3 A Hamiltonian phase flow preserves the symplectic structure:
dP ∧ dQ = dp ∧ dq.
(3.23)
Proofs and details can be found in [4]. We are interested in using numerical
methods that accurately mimic the continuous system: these should preserve
the symplectic structure when ν = 0, and for ν 6= 0, the methods should
reflect the phase volume contractivity given by the Jacobian condition (3.20).
Definition 5 The method based on the one-step approximation
P̄ = P̄ (t + h; t, p, q),
Q̄ = Q̄(t + h; t, p, q),
is called symplectic if
dP̄ ∧ dQ̄ = dp ∧ dq.
Definition 6 We call a method quasi-symplectic if it satisfies the two following conditions:
RL1 The method applied to (3.10)-(3.11) degenerates to a symplectic method
when ν = 0
RL2 The Jacobian
D(P̄ , Q̄)
J¯ =
D(p, q)
does not depend on p,q.
Milstein et al., in [1], consider a quasi-symplectic method based on a
splitting technique. Splitting techniques are fully introduced in [9]. With
splitting methods, one decomposes the vector field into integrable pieces and
treats them separately. Suppose a system ẏ = f (y), y ∈ R n can have its
vector field “split” as
ẏ = f 1 (y) + f 2 (y).
13
If, by chance, the exact flows ϕ1t and ϕ2t of the systems ẏ = f 1 (y) and
ẏ = f 2 (y) can be calculated explicitly, one can, from a given initial value
y0 , first solve the first system to obtain a value y 1/2 , and from this value
integrate the second system to obtain y 1 .
Such a method that we will study is the second order strong quasisymplectic method, introduced in [1]:
P0 = p,
P2,k
Pk+1
Q0 = q,
; Pk ,
τ
Q1,k = Qk + P1,k ,
2
2
= P1,k + τ f (Q1,k ) + τ 1/2 σξk ,
τ
τ
; P2,k ,
= PII
Qk+1 = Q1,k + P2,k ,
2
2
P1,k = PII
τ
k = 0, ..., N − 1,
(3.24)
where ξk are i.i.d random variables with normal distribution and P II is the
exact solution of
dPII
= −νPII ,
PII (0) = p,
(3.25)
dt
or
PII (t; p) = p exp(−νt).
(3.26)
Let us show that this method is indeed quasi-symplectic. For ν = 0, we can
rewrite the method as
P0 = p,
Q0 = q,
τ
Pk+1 = Pk + τ f (Qk + Pk ),
2
τ
Qk+1 = Qk + (Pk + Pk+1 ),
2
(3.27)
which is the well-known Verlet scheme. We then have
τ
dPk+1 ∧ dQk+1 = dPk+1 ∧ (dQk + (dPk + dPk+1 ))
2
τ
= dPk+1 ∧ dQk + dPk+1 ∧ dPk
2
τ
τ
0
= dPk + τ f (Qk + Pk )(dQk + dPk ) ∧ dQk
2
2
τ
τ
τ
0
dPk + τ f (Qk + Pk )(dQk + dPk ) ∧ dPk
+
2
2
2
τ2 0
τ
= dPk ∧ dQk + f (Qk + Pk )dPk ∧ dQk
2
2
τ2 0
τ
+ f (Qk + Pk )dQk ∧ dPk ,
2
2
and using the skew-symmetry property of the exterior product, we get
τ2 0
τ
τ2
τ
f (Qk + Pk )dPk ∧ dQk = − f 0 (Qk + Pk )dQk ∧ dPk ,
2
2
2
2
14
which leads to
dPk+1 ∧ dQk+1 = dPk ∧ dQk .
(3.28)
The method is thus symplectic for ν = 0.
The Jacobian for a numerical method can be computed using the formula,
D(Qk+1 , Pk+1 ) ∂Qk+1 ∂Pk+1 ∂Qk+1 ∂Pk+1
¯
=
J =
−
,
(3.29)
D(Qk , Pk ) ∂Qk ∂Pk
∂Pk ∂Qk
which gives:
J¯ = exp(−ντ ).
(3.30)
This is independant of p, q and therefore satisfies the requirement RL2, i.e.
the method is quasi-symplectic. Moreover the Jacobian of the numerical
method is equal to the Jacobian of the continuous system, and the phase
volume contractivity is respected during integration.
3.3
Numerical example 1: a particle in a doublewell potential
We consider the scalar Langevin equations for a particle in a double-well
potential:
dQ = P dt
dP = f (Q)dt − νP dt +
r
2ν
dW (t),
β
(3.31)
where ν > 0 is a friction parameter, β is the inverse temperature, W (t) is a
standard Wiener process, and f is the gradient of a double-well potential U
(figure 3.1), defined by:
f (Q) = −
∂U
(Q),
∂q
2
U (q) = (q − 1)2
(3.32)
The invariant density for such a system is known to be (see section 2.2):
ρ(p, q) =
1 −βH(p,q)
e
,
Z
where H is the Hamiltonian and Z is a normalization constant,
Z
1 2
H(p, q) = p + U (q),
Z=
e−βH(p,q) dpdq.
2
R2
(3.33)
(3.34)
We simulate (3.31)-(3.32) using different numerical schemes, comparing the
efficiency of the second order weak quasi-symplectic to an implicit quasisymplectic method (midpoint rule), and to non symplectic schemes (explicit
15
Runge-Kutta method and implicit trapezoidal rule). To estimate the accuracy of simulations, we use histograms of the distribution of the particle positions. As we know the invariant measure, we can compute the L 1
error between the distribution of the particles and the invariant density
ρ(q) = e−βU (q) /Zq :
Z
q = kρhist (q) − ρinv (q)kL1 = |ρhist (q) − ρinv (q)|dq.
(3.35)
Since
Z
|ρhist (q) − ρinv (q)|dq ≤
Z
|ρhist (q)|dq +
Z
|ρinv (q)|dq,
(3.36)
and ρhist and ρinv have norm 1, we have
q ≤ 2.
(3.37)
A second error estimation can be given by computing the average kinetic
energy,
Z ∞
β 2
1 2
1 1
hP i =
p2 e− 2 p dp
2
2 Zp −∞
√
2π 1
,
(3.38)
= √
2 βZp β
where
Zp =
Z
∞
e
− β2 p2
−∞
√
2π
dp = √ ,
β
(3.39)
such that known relation between temperature and average kinetic energy
appears:
1
1
= hP 2 i.
(3.40)
2β
2
Potential
9
8
7
6
5
4
3
2
1
0
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
Figure 3.1: Symmetric double-well potential
16
In figure 3.2, one can see the dynamics of the particle, with three different
friction values, from low (ν = 0.0004) to high friction (ν = 1). The histograms show the distribution of the particle in q, compared to the known
invariant distribution.
1.4
2
1.2
1.5
1
1
0.5
0.8
0
0.6
−0.5
0.4
−1
0.2
−1.5
−2
0
500
1000
1500
2000
2500
3000
0
−2.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
(a)
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
(b)
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
(c)
1
2
0.9
1.5
0.8
1
0.7
0.5
0.6
0
0.5
0.4
−0.5
0.3
−1
0.2
−1.5
−2
0.1
0
500
1000
1500
2000
2500
3000
0
−2.5
1
2
0.9
1.5
0.8
1
0.7
0.5
0.6
0
0.5
0.4
−0.5
0.3
−1
0.2
−1.5
−2
0.1
0
500
1000
1500
2000
2500
3000
0
−2.5
Figure 3.2: Position and distribution of the particle governed by the
Langevin equation (3.31). The line on the right hand graphs is the invariant density (3.33) - Quasi-symplectic method (a) ν=0.0004 - (b) ν=0.04
- (c) ν=1
The simulation’s inverse temperature is β = 3, the step size is set to
τ = 0.01, the values are recorded every 100 steps and the total number of
recorded steps is 300000. As one can see the simulations with low friction
need long integration time in order to sample the invariant measure, since
17
the jumps between the two wells are quite rare events. Indeed when ν 1,
the system (3.31) is close to a Hamiltonian system, where the particle would
oscillate in one single well,
pand the noise amplitude given by the fluctuationdissipation relation, σ = 2ν/β is very low. On the other hands, the system
gets stiffer with higher friction and thus needs smaller time steps to converge.
We integrate the system using the four methods (Quasi-symplectic, Midpoint rule, Runge-Kutta and trapezoidal rule) with the same set of parameters, in order to compare their efficiency. We compute the error q between
the histogram and the invariant measure. The results are shown on table
3.1. The trapezoidal rule diverges fast, therefore we do not show its results
in the table.
Method
Quasi-symplectic
Midpoint rule
Runge-Kutta
ν = 0.0004
0.2501
0.1361
0.3530
ν = 0.004
0.0327
0.0909
0.1807
ν = 0.04
0.0364
0.0389
0.0819
ν = 0.4
0.0244
0.0223
0.0290
ν=1
0.0280
0.0235
0.0232
Table 3.1: Error q between the distribution of the particles and the invariant
density
As one can see, the quasi-symplectic methods (quasi-symplectic and midpoint rule) lead to smaller error than the Runge-Kutta scheme when friction is low. Though, figure 3.2 showed that in the low friction cases, the
invariant measure is far from being sampled, and one thus needs longer integration in order to make reliable comparisons. Considering the running
time though, the implicit midpoint rule is very slow, and thus not satisfactory. The Runge-Kutta scheme includes twice more operations than the
quasi-symplectic scheme, and thus needs double time for the integration.
The trapezoidal rule, with its fast divergence, illustrates the advantage of a
volume preserving integration scheme. Indeed, the trapezoidal method,
τ
τ
(f (Qk ) + f (Qk+1 )) − ν(Pk + Pk+1 ) + σ∆W,
2
2
τ
= Qk + (Pk + Pk+1 ),
Q(0) = q,
2
Pk+1 = Pk +
Qk+1
P (0) = p,
(3.41)
has the following Jacobian (3.29):
τ
τ2
J¯ = 1 − ν − (−12Q2k + 4).
2
4
(3.42)
This Jacobian is not equal to the system’s Jacobian (3.26), and depends on
Qk . The dynamics of the particle is shown in figure 3.3, for ν = 0.0004,
τ = 0.01. In that case the Jacobian is strictly larger than one, when Q k gets
larger than 0.4. Consequently the system blows up.
18
The relation (3.40) between temperature and kinetic energy allows for
further comparisons. For a high friction value, where the three methods
seems to give equivalent results according to table 3.1, we get the temperature computed via kinetic energy in figure 3.4.
10
8
6
4
2
0
−2
−4
−6
−8
−10
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
Figure 3.3: Position of the particle governed by the Langevin equation
(3.31), Trapezoidal rule for ν = 0.0004
Temperature computed via kinetic energy
0.18
0.175
0.175
0.17
0.17
0.165
0.165
0.16
0.16
0.155
0.155
0.15
0
500
1000
1500
2000
Temperature computed via kinetic energy
0.18
2500
3000
(a)
0.15
0
500
1000
1500
2000
2500
3000
(b)
Temperature computed via kinetic energy
0.18
0.175
0.17
0.165
0.16
0.155
0.15
0
500
1000
1500
2000
2500
3000
(c)
Figure 3.4: Temperature computed via kinetic energy (relation (3.40)) for
ν=1 . The straight line is the real temperature: (a) Quasi-symplectic
method ; (b) Midpoint-rule ; (c) Runge-kutta
19
As the third plot in figure 3.4 illustrates, the Runge-Kutta method converges to the wrong temperature. This behaviour could be explained by the
fact that the method does not preserve symplectic structures.
In the previous simulations, the length of integration for small friction
values was not sufficient to sample the invariant measure, and thus did not
allow reliable comparisons between the numerical schemes. The implicit
midpoint rule is too costly, but we wish to make further comparisons between the Runge-Kutta scheme and the quasi-symplectic method in the low
friction case. To this end we integrated the system with ν = 0.0004 and
ν = 0.004 for much longer times (108 steps recorded every 100 steps, with
step size τ = 0.01, and 109 steps recorded every 100 steps, with τ = 0.001).
The results obtained showed that neither of these two methods is clearly
superior, but at least the quasi-symplectic method is twice as fast. Indeed
the problem of convergence of temperature computed via kinetic energy to
an incorrect value happened to appear with the quasi-symplectic method
as well. Nevertheless, it seems to appear more often with the Runge-Kutta
scheme: this conclusion is in accordance with Monte-Carlo simulations carried out in [1], in which for sufficiently many realizations, it turns out that
the quasi-symplectic method reproduces the correct temperature whereas
the Runge-Kutta scheme does not. For our example, we conclude that the
quasi-symplectic method gives slightly better results than the Runge-Kutta
scheme.
3.4
Numerical example 2: a stiff, 2 degree of freedom example
We study now the Langevin equation with a two-dimensional potential,
dQi = Pi dt
dPi = fi (Q)dt − νPi dt +
r
2ν
dWi (t),
β
i = 1, 2,
(3.43)
where ν > 0 is a friction parameter, β is the inverse temperature, W i (t) are
standard Wiener processes, and the force has the following expression:
f (Q) = −∇U (Q),
1
2
2
U (q) = (q1 − 1)2 + ω(q1 )2 q22 ,
ω(x) = 1 + 10e−200(q1 −1) , (3.44)
2
which is a double-well potential in the q 1 direction and a harmonic oscillator
in the q2 direction. Both are related through the factor ω(x) in the harmonic
oscillator. Figure 3.5 shows the form of ω and of the potential. The peak at
x = 1 introduces an additional barrier in the double-well potential.
The invariant measure is known to be
1
ρ(p, q) = e−βH(p,q) ,
(3.45)
Z
20
where as before H is the Hamiltonian and Z the normalization constant,
Z ∞
1 2
H(p, q) = p + U (q),
Z=
e−βH(p,q) dpdq.
(3.46)
2
−∞
We will consider the marginal density in the q 1 direction:
Z ∞
1
ρ(q1 ) =
e−βU (q1 ,q2 ) dq2
Zq −∞
Z
1 −β(q12 −1)2 ∞ − β ω(q1 )2 q22
=
e
e 2
dq2
Zq
−∞
r
1
2π 1 −β(q12 −1)2
=
e
.
ω(q1 ) β Zq
(3.47)
Again, we simulate (3.43)-(3.44) using the second order quasi-symplectic, an
implicit quasi-symplectic method (the midpoint rule), and a non symplectic
scheme (an explicit Runge-Kutta method). To estimate the accuracy of the
simulations, we use histograms of the marginal distribution of the particle
positions in the q1 direction, to be compared to the known marginal density
(3.47). Based on these, we compute the L 1 error. We get a second error
estimation by computing the average kinetic energy,
Z ∞
β
1 2
1 1
2
2
hP1 + P22 i =
(p21 + p22 )e− 2 (p1 +p2 ) dp1 dp2
2
2 Zp −∞
1
= .
(3.48)
β
In figure 3.6, one can see the dynamics of the particle, with three different
friction values, from low (ν = 0.04) to high friction (ν = 1). The histograms
show the distribution of the particle in q 1 , compared to the known invariant
distribution. As one can see the simulations with low friction need long
11
potential with dynamical barrier
2
10
1.5
9
1
8
0.5
y
7
6
0
−0.5
5
4
−1
3
−1.5
2
1
−2
−2
−1.5
−1
−0.5
0
0.5
1
1.5
−1.5
−1
−0.5
0
x
0.5
1
2
Figure 3.5: Form of ω = ω(x) and of the potential
21
1.5
integration time in order to sample the invariant measure, since the jumps
between the wells and over the barrier are quite rare events. Indeed when
ν 1, the system (3.43) is close to a Hamiltonian system, where the particle
would oscillate in one single well, and
p the noise amplitude given by the
fluctuation-dissipation relation, σ = 2ν/β is very low. On the other hands,
the system gets stiffer with higher friction and thus needs smaller time steps
to converge.
Position of the particle versus time
3
Distribution of particle positions
1.4
1.2
2
1
1
0.8
0
0.6
−1
0.4
−2
−3
0.2
0
100
200
300
400
500
600
700
800
900
1000
Position of the particle versus time
3
0
−2.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
(a)
1
1.5
2
2.5
(b)
1
1.5
2
2.5
(c)
Distribution of particle positions
1.4
1.2
2
1
1
0.8
0
0.6
−1
0.4
−2
−3
0.2
0
100
200
300
400
500
600
700
800
900
1000
Position of the particle versus time
3
0
−2.5
−2
−1.5
−1
−0.5
0
0.5
Distribution of particle positions
1.4
1.2
2
1
1
0.8
0
0.6
−1
0.4
−2
−3
0.2
0
100
200
300
400
500
600
700
800
900
1000
0
−2.5
−2
−1.5
−1
−0.5
0
0.5
Figure 3.6: Position and distribution of the particle governed by equation
(3.44). The line on the right hand graphs is the invariant marginal density
(3.47)- Quasi-symplectic method (a)ν=0.04 - (b)ν=0.4 - (c)ν=1
The simulation’s inverse temperature is β = 3, the step size is set to
22
τ = 10−4 , the values are recorded every 100 steps and the total number of
steps is 1000000. We integrate the system using the three methods with the
same set of parameters, in order to compare their efficiency. We compute
the error q between the histogram and the invariant measure. The results
are shown on table 3.2.
Method
Quasi-symplectic
Midpoint rule
Runge-Kutta
ν = 0, 04
0.1819
0.1297
0.0890
ν = 0, 4
0.0400
0.0492
0.0500
ν=1
0.0924
0.0864
0.0467
Table 3.2: Error q between the distribution of the particles and the invariant
measure
We make further comparisons using the relation (3.48). For a high friction value, where the Runge-Kutta method seems to give better results, we
get the temperature computed via kinetic energy on figure 3.7.
0.65
0.65
0.6
0.6
0.55
0.55
0.5
0.5
0.45
0.45
0.4
0.4
0.35
0.35
0.3
0
100
200
300
400
500
600
700
800
900
1000
(a)
0.3
0
100
200
300
400
500
600
700
800
900
1000
(b)
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
100
200
300
400
500
600
700
800
900
1000
(c)
Figure 3.7: Temperature computed via kinetic energy (relation (3.48)). The
straight line is the real temperature - ν=1 - (a) Quasi-symplectic method (b) Midpoint-rule - (c) Runge-kutta
So even though the Runge-Kutta method seems to sample the invariant
density better than the symplectic methods, it converges to a wrong value
23
of temperature. When we use a smaller time-step, this problem disappears,
but the computational effort is much higher. Therefore we can conclude
that the quasi-symplectic method allows us to use a slightly larger time step
than the Runge-Kutta scheme. This can turn out to have some importance
for time-expensive integration problems.
3.5
Conclusion
Based on two examples, we tested if the quasi-symplectic method proposed
by the authors of [1] is superior to non-quasi-symplectic schemes.
Our results were not as definitive as expected. Regarding the dynamics of
simple particles the quasi-symplectic method has proven to be only slightly
better than a Runge-Kutta scheme; see tables 3.1 and 3.2. In particular
we have observed that the Runge-Kutta integrator reproduces the wrong
temperature when computed via the kinetic energy; see figures 3.4 and 3.48.
This violation of the equipartition of energy principle seems to be a common
feature of Langevin integrators; see [11] and [12]. An attractive feature of
the quasi-symplectic integrator is that this effect was less severe.
Since from the viewpoint of numerical accuracy there is no clear favorite,
the question of numerical efficiency should tip the balance to one or another
integrator. Here the quasi-symplectic method is preferable; in the first example, the latter was twice as fast at the Runge-Kutta method.
We also saw that different problems arise depending on the value of the
friction parameter in the Langevin equation. If one is interested in sampling the invariant measure for a Langevin system, by numerically solving
the system (2.16), one is free to choose the value of the friction parameter
provided that the fluctuation-dissipation relation (2.15) is respected. Too
low a friction yields a necessarily very long integration time. On the other
hand too high a friction makes the system numerically stiff. In the context
of our model problems, a good value of ν seems to be of the order 10 −2 ,
independently of the choice of the numerical time-stepper.
24
Chapter 4
High-friction Langevin
equations
4.1
Approximation by Smoluchowski equations
In section 2.4, we have seen in a formal way that Langevin equations approach Smoluchowski equations when the friction is high. Therefore a natural question is the following: when is the friction high enough to approximate the Langevin equations by Smoluchowski equations? From a computational point of view, it is interesting to consider first order Smoluchowki
equations instead of the coupled second order Langevin equations, because
there is half less unknown in those. We will thus focus on error estimation
between trajectories of the two equations. For this purpose, we will consider
a rather general form of the Langevin equation, but with linear Hamiltonian
part H(q, p) = (q T Kq + pT M −1 p)/2:
dZ = Aτ Zdt + ΣdW,
(4.1)
where Z = (QT , P T )T ∈ R2n , W = (W1 , W2 )T is R2n -valued white noise,
and Aτ and Σ are 2n × 2n-matrices of the form
0 1
0 0
Aτ = (τ J − Γ)H,
J=
,
Σ=
,
(4.2)
−1 0
0σ
where σ ∈ Rn×n is a symmetric positive definite noise intensity matrix, Γ a
symmetric non-negative definite matrix, and H the linear Hamiltonian.
When choosing
K 0
0 0
H=
,
Γ
=
,
(4.3)
0 M −1
0γ
with symmetric, positive definite K, M −1 , and γ, we get back to the familiar
25
form of the Langevin equation:
dQ = τ M −1 P dt
(4.4)
dP = −τ KQdt − γM
−1
P dt + σdW2 ,
(4.5)
We have the following theorem from [8]:
Theorem 4 Suppose that τ > 0 is small enough. Then, there is a τ dependent, linear transformation of coordinates
Z = (QT , P T )T 7→ u = (uTQ , uTP )T ,
that brings equations (4.4) and (4.5) into the following decoupled form:
d
uQ = τ 2 D1 (τ )uQ + τ Z1 (τ )σdW2 ,
dt
d
uP = D2 (τ )uP + Z2 (τ )σdW2 ,
dt
(4.6)
(4.7)
where the matrices D1 , D2 , Z1 , Z2 are O(1) in τ such that one observes that
the dynamics of uQ happens on timescale O(1/τ 2 ), while that of uP happens
on timescale O(1). Moreover, the linear transformation can be shown to
satisfy
Q(t) = K −1 T uQ (t) + O(τ ),
P (t) = CuP (t) + O(τ ),
with some invertible, τ -independent transformation matrices T and C. Putting
these observations together one finally gets pathwise convergence of Q =
Q(τ 2 t) to the solution process X = X(t) of the Smoluchowski equation
γdX = −KXdt + σdW2 ,
(4.8)
for τ → 0. Likewise, we have pathwise convergence of P = P (t) to the
solution process V = V (t) of
dV = −γM −1 V dt + σdW2 .
4.2
(4.9)
Error indicator
It is also possible to compute an error indicator between the Langevin equation and the Smoluchowski equation. For this, we first consider the eigenvalue problem of the nonsymmetric matrix A τ :
Aτ R̂τ T̂ = R̂τ T̂ Λ̂τ ,
(4.10)
where R̂τ denotes the 2n × 2n-matrix of the right hand eigenvectors of A τ ,
Λ̂τ the diagonal matrix of the associated eigenvalues, and T̂ an invertible
26
transformation matrix that we will specify later. Let the matrix of the
associated left hand eigenvectors be denoted L̂τ . We have to learn about
the perturbation of the spectrum of A τ around τ = 0 for small τ > 0. For
τ = 0 the matrix A0 = −ΓH has n eigenvalues that are identical to 0, and
n others that are given by the negative eigenvalues of −γM −1 , but which
are bounded away from zero. The subspace associated with the vanishing
eigenvalues is the one associated with the position coordinates, Q. We are
mainly interested in the perturbation analysis of this subspace. Therefore,
we sometimes distinguish between the first and the second n components by
introducing the notation
Λτ 0
Lτ
Λ̂τ =
, R̂τ = (Rτ , Rτ ), L̂τ =
,
Lτ
0 λτ
where Λτ is the n × n diagonal matrix of the eigenvalues of A τ associated
with the n vanishing eigenvalues of A 0 , and λτ the n × n diagonal matrix of
the others (for τ > 0 small enough), R τ and Rτ are 2n × n-matrices, and
Lτ and Lτ are n × 2n matrices of eigenvectors.
Theorem 5 For τ > 0 being small enough, we find that
Λτ = −τ 2 T −1 Kγ −1 T + O(τ 3 )
K −1 −τ γ −1 C
R̂τ =
+ O(τ 2 )
−τ M γ −1
C
K
τ Kγ −1
L̂τ =
+ O(τ 2 ),
τ C −1 M γ −1 K C −1
where T is the invertible n × n matrix that diagonalizes Kγ −1 and C is the
invertible n × n matrix that diagonalizes γM −1 . It can be shown that T and
C exist. In addition, we have that
λτ = −C −1 γM −1 C + O(τ 2 )
Due to bi-orthonormality of left and right eigenvectors L̂τ R̂τ = Id, we
have T̂ −1 L̂τ Aτ R̂τ T̂ = Λ̂τ . When introducing new coordinates u = (u TQ , uTP )
by
z = R̂τ T̂ u,
we get by means of (4.1) that
du = Λ̂τ udt + T̂ −1 L̂τ ΣdW.
Using the results from Theorem 4 and Theorem 5, we have
−1
K T uQ − τ γ −1 CuP
Q
Z=
= R̂τ T̂ u =
+ O(τ 2 ).
−τ M γ −1 uQ + CuP
P
27
(4.11)
Moreover, we have from Theorem 4 that K −1 T uQ is an approximation of
the position component Q of the solution of the Langevin equation, and that
CuP is an approximation of the momentum component P . Thus we find
that the Langevin equation can be approximated up to second order by
Q − (K −1 T uQ − τ γ −1 CuP ) = O(τ 2 ),
which, with the solution V of the Smoluchowski equation, results in
Q − (K −1 T uQ − τ γ −1 V ) = O(τ 2 ).
The error indicator for the replacement of the position solution Q = Q(t) of
the Langevin equation by the solution X = X(s) = X(τ 2 t) of the Smoluchowski equation (4.8) is then
Q(t) − X(τ 2 t) = −τ γ −1 V (t) + O(τ 2 ).
4.3
(4.12)
Numerical example 1: a 2 degree of freedom
system
We consider the Langevin equation (4.4) and (4.5) with Q, P ∈ R 2 and the
following linear Hamiltonian and friction terms:
K 0
0 0
H=
,
Γ=
,
0 M −1
0γ
where
K=
3 0.2
0.2 2
,
M=
10
01
,
γ=
0.04 0.01
0.01 0.04
.
(4.13)
The noise value σ is given by the fluctuation dissipation relation
σσ T =
2γ
,
β
β = 3,
(4.14)
where β is the inverse temperature.
We wish to compare the solution of the Langevin equation to the solution
of the Smoluchowski equation. After fixing a realization of the noise process
dW2,t on the interval t ∈ [0, T ], we then integrate (4.4) and (4.5) using
the quasi-symplectic method (3.24). Additionally, we integrate (4.8) on the
interval s ∈ [0, τ 2 T ] and (4.9) on the interval t ∈ [0, T ]. The results for
the position and momentum solutions are shown on figure (4.1). On these
graphs, we chose τ = 10−3 and τ = 10−6 . The time step we use is h = 0.5
for (4.4), (4.5) and (4.9) and h = τ 2 .0.5 for (4.8).
The error indicator between the position solution of the Langevin equation on the interval [0, T ] and the position solution of the Smoluchowski
28
1.2
1.2
1
1
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0
0
−0.2
−0.2
−0.4
−0.4
−0.6
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
(a)
2.5
−0.6
2
2
1.5
1.5
1
1
0.5
0.5
0
0
−0.5
−0.5
−1
−1
−1.5
−1.5
−2
−2.5
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
(b)
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
(d)
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
(f)
0
0.5
1
1.5
2
2.5
0
2.5
−2
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
(c)
−2.5
0
−3
−3
x 10
2
2
1.5
1.5
1
1
0.5
0.5
0
0
−0.5
−0.5
−1
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
(e)
2.5
−1
x 10
2.5
2
2
1.5
1.5
1
1
0.5
0.5
0
0
−0.5
−0.5
−1
−1
−1.5
−1.5
−2
−2
−2.5
−2.5
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
−9
x 10
(g)
3
3.5
4
4.5
5
−9
x 10
(h)
Figure 4.1: Comparison of the position and momentum solutions for one
realization of the noise process - (a) Position, Langevin equation on [0, T ],
τ = 10−3 - (b) Position, Smoluchowski equation on [0, τ 2 T ], τ = 10−3 - (c)
Momentum (one coordinate), Langevin equation on [0, T ], τ = 10 −3 - (d)
Momentum (one coordinate), Smoluchowski equation on [0, τ 2 T ], τ = 10−3
- (e) Position, Langevin equation on [0, T ], τ = 10 −6 - (f) Position, Smoluchowski equation on [0, τ 2 T ], τ = 10−6 - (g) Momentum (one coordinate),
Langevin equation on [0, T ], τ = 10−629- (h) Momentum, Smoluchowski equation on [0, τ 2 T ], τ = 10−6
equation on the interval [0, τ 2 T ] is given by equation (4.12) and is linear
in τ . We thus compute both solutions for different values of τ (between
τ = 10−1 and τ = 10−6 ). For those solutions, we compute the following error between the position solution of the Langevin equation and the position
solution of the Smoluchowski equation,
LS = max
t∈[0,T ]
2
X
i=1
|Qi (t) − Xi (τ 2 t)|.
(4.15)
The results are shown on figure 4.2 for different values of τ ranging from
10−1 to 10−6 . As one can see, the error decreases linearly for τ smaller than
10−2 . For larger values of τ , the asymptotic regime is not reached yet. We
also compute the error estimator, and compare Q(t) − X(τ 2 t) with the error
estimator −τ γ −1 V (t). The result is shown on figure 4.3, for τ = 10 −3 and
τ = 10−6 . The table 4.1 shows the error of the estimator for different values
of τ ,
2
X
est = max
|Qi (t) − Xi (τ 2 t) + τ γ −1 Vi (t)|.
(4.16)
t∈[0,T ]
i=1
1
10
0
10
Position error
−1
10
−2
10
−3
10
−4
10
−6
10
−5
10
−4
10
−3
Tau
10
−2
10
−1
10
Figure 4.2: Error (4.15) versus τ between the Langevin solution and the
Smoluchowski solution
30
τ = 10−2
5.83.10−1
τ = 10−3
2.64.10−2
τ = 10−4
1.2696.10−4
τ = 10−5
5.8920.10−6
τ = 10−6
5.8515.10−7
Table 4.1: Error est (4.16) between the difference of Langevin solution and
Smoluchowski solution (right-hand side of (4.12)) and the estimator (lefthand side of (4.12))
0.01
0
−0.01
−0.02
−0.03
−0.04
1135
1140
1145
1150
1155
1160
1165
1170
(a)
−5
x 10
2
1
0
−1
−2
−3
−4
1135
1140
1145
1150
1155
1160
1165
1170
(b)
Figure 4.3: Pathwise error between the Langevin solution and the Smoluchowski solution and error estimator for one realization of the noise process
- Typical time segments from time series of length 10000 - (a) τ = 10 −3 -The
upper curve is the estimator - (b) τ = 10 −6 -The curves are indistinguishable
31
4.4
Numerical example 2: a linear chain of beads
We next consider an example with many more degrees of freedom. Specifically a linear chain of N +1 beads where the first and the last bead are fixed,
and where the interbead force is derived from a pairwise harmonic potential
with spring constant Ki , i = 1 . . . N . Denoting the beads coordinates by Q i ,
i = 0 . . . N , we define the interaction potential by
U (Q) =
N
X
1
i=1
2
Ki |Qi − Qi−1 |2 ,
(4.17)
so that each bead is attracted to its nearest neighbors, but the fixed extremities force the chain to be stretched.
Figure 4.4: Linear chain of beads
We will study this example in the context of high friction. Let us first
write this problem in the form given by equation (4.4) and (4.5). For simplicity, we consider the N + 1 beads as being of equal mass m. One obtains
the following quadratic Hamiltonian,
H(Q, P ) = (QT KQ + P T M −1 P )/2,
(4.18)
where Q = (Q0 . . . QN )T , the mass M of the system is set to M = Id, and


K1
−K1
0
 −K1 K1 + K2 −K2





..
K=
(4.19)

.




−KN −1 KN −1 + KN −KN
0
−KN
KN
We simulate the system using Euclidean coordinates (Q i = (xi , yi , zi )). For
simplicity again, we consider each spring constant K i as being of equal value,


100
Ki =  0 1 0  ,
i = 1 . . . N,
(4.20)
001
32
and the friction matrix is


γ0
0


..
γ=
,
.
0
γN


0.0004 0.0001 0.0001
γi =  0.0001 0.0004 0.0001  .
0.0001 0.0001 0.0004
(4.21)
The noise matrix σ is given by the fluctuation dissipation relation and is
σi σiT =
2γi
,
β
β = 3,
(4.22)
where β is the inverse temperature.
Once again, we wish to compare the solution of the Langevin equations
(4.4) and (4.5) to the solution of the Smoluchowski equations (4.8) and
(4.9), and this for different numbers of beads. Having fixed a realization of
a noise process dW2,t on the interval t ∈ [0, T ], we integrate (4.4) and (4.5)
using the quasi-symplectic method (3.24). We also integrate (4.8) on the
interval s ∈ [0, τ 2 T ] and (4.9) on the interval t ∈ [0, T ]. For both Langevin
and Smoluchowski equation, we keep the extremities of the chain fixed. We
carry out those simulations for chains of 5, 8 and 15 beads. The results
for the position solutions of the chain of length 15 are shown on figure 4.5.
Figure 4.6 shows the solutions for the x-coordinate of the middle bead. On
these graphs, we choose τ = 10−6 .
In order to check the linearity of the error between the Langevin solution
and the Smoluchowski solution given by the estimator (4.12), we compute
both solutions for different values of τ , ranging from 10 −4 to 10−10 . For
those solutions, we compute the following error between the position solution
of the Langevin equation and the position solution of the Smoluchowski
equation,
3(N +1)
X
|Qi (t) − Xi (τ 2 t)|.
(4.23)
LS = max
t∈[0,T ]
i=1
The results are shown on figure 4.7. We also compute another natural error
between the Langevin solution and the Smoluchowski solution:
Z
LS2 =
max
|Qi (t) − Xi (τ 2 t)|dt,
(4.24)
i=1...3(N +1)
consequently we obtain two errors for the estimator:
3(N +1)
est = max
t∈[0,T ]
and
est2 =
max
i=1...3(N +1)
X
i=1
Z
|Qi (t) − Xi (τ 2 t) + τ γ −1 Vi (t)|,
|Qi (t) − Xi (τ 2 t) + τ γ −1 Vi (t)|dt.
33
(4.25)
(4.26)
Langevin equation
0.1
0.08
0.06
0.04
0.02
0
−0.02
−0.04
−0.06
−0.08
0.2
0.15
0.1
0.05
0
0
2
4
6
8
10
12
14
(a)
Smoluchowski equation
0.1
0.08
0.06
0.04
0.02
0
−0.02
−0.04
−0.06
−0.08
0.2
0.15
0.1
0.05
0
0
2
4
6
8
10
12
14
(b)
Figure 4.5: Comparison of the position taken by the entire chain (length 15)
for one realization of the noise process - τ = 10 −6 (a) Langevin equation on
[0, T ] - (b) Smoluchowski equation on [0, τ 2 T ]
34
7.1
7.1
7.09
7.09
7.08
7.08
7.07
7.07
7.06
7.06
7.05
7.05
7.04
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
(a)
7.04
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
(b)
Figure 4.6: Comparison of the position solution of the middle bead in the
chain of length 15 (one coordinate) for one realization of the noise process
- τ = 10−6 (a) Langevin equation on [0, T ] - (b) Smoluchowski equation on
[0, τ 2 T ]
1
10
0
10
−1
10
−2
10
−3
10
−4
10
−5
10
−6
10
−10
10
−9
10
−8
10
−7
10
−6
10
−5
10
−4
10
Figure 4.7: Error LS (4.23) between the Langevin solution and the Smoluchowski solution versus τ for 15 beads
35
Number
of beads
5
10
15
30
τ = 10−4
2.47
4.96
7.60
14.66
τ = 10−6
2.85.10−2
4.82.10−2
7.54.10−2
1.55.10−1
τ = 10−8
2.85.10−4
4.82.10−4
7.54.10−4
1.55.10−3
τ = 10−10
2.85.10−6
4.82.10−6
7.54.10−6
1.55.10−5
Table 4.2: Table of error LS (4.23) between the Langevin solution and the
Smoluchowski solution
Number
of beads
5
10
15
30
τ = 10−4
1.32.103
1.31.103
1.41.103
1.40.103
τ = 10−6
14.13
14.25
14.88
14.85
τ = 10−8
1.41.10−1
1.43.10−1
1.49.10−1
1.48.10−1
τ = 10−10
1.42.10−3
1.42.10−3
1.49.10−3
1.48.10−3
Table 4.3: Table of error LS2 (4.24) between the Langevin solution and the
Smoluchowski solution
Number
of beads
5
10
15
30
τ = 10−4
1.30
2.85
4.75
8.59
τ = 10−6
9.91.10−6
2.39.10−5
5.28.10−5
9.62.10−5
τ = 10−8
1.53.10−8
3.16.10−8
4.88.10−8
9.24.10−8
Table 4.4: Table of error est (4.25) between the error between Langevin and
Smoluchowski solution and the estimator
Number
of beads
5
10
15
30
τ = 10−4
1.01.103
1.00.103
9.87.102
9.88.102
τ = 10−6
1.41.10−2
1.69.10−2
2.01.10−2
1.67.10−2
τ = 10−8
8.48.10−6
8.51.10−6
8.80.10−6
1.00.10−5
Table 4.5: Table of error est2 (4.26) between the error between Langevin
and Smoluchowski solution and the estimator
36
All the results are shown on tables 4.2, 4.3, 4.4 and 4.5), for chains of length
5, 10, 15 and 30.
From table 4.3, one can see that the maximal error over the degrees
of freedom between the Langevin solution and the Smoluchowski solution
hardly increases when the degrees of freedom are multiplied by almost 10
(from 3 free beads to 28 free beads). This is also true for the error of the
estimator, and can be seen on table 4.5. The linearity in τ given by the
analytical results (4.12) is very accurately fulfilled for values of τ smaller
than 10−6 .
4.5
Conclusion
We studied Langevin equations where the friction parameter ν was scaled
by a small factor τ , as ν 7→ ν/τ . From theorem 4, we saw that this scaling
between the friction and the other parameters of the Langevin equation induces a scale separation between the dynamics of Q (the position) and P (the
momentum). Nontrivial dynamics of Q happens on the timescale O(1/τ 2 ),
while those of P happens on the timescale O(1). The studied examples (a
two degrees of freedom system, and linear chains of beads of 9 to 84 degrees
of freedom) verified the asymptotic results of pathwise convergence of the
solution processes of the Langevin equation to the solution processes of a
corresponding Smoluchowski equation; see figure 4.3. Moreover, the error
estimator given by equation (4.12) for the case where the Hamiltonian part
is linear in the Langevin equation turned out to be accurate. Both examples
exhibited the property that the error of the approximation can be controlled
linearly by decreasing τ . In the example with two degrees of freedom, this
linearity in τ already starts being fulfilled with a scale separation of only
τ 2 = 10−4 between the slow variable (the position) and the fast variable (the
momentum); see figure 4.2. When going to problems with many more degrees of freedom, the growth of error between the Langevin equation by the
Smoluchowski equation remains surprisingly negligible, at least for values of
τ smaller than 10−6 ; see tables 4.3 and 4.5.
37
Bibliography
[1] G.N. Milstein and M.V Tretyakov: Computing ergodic limits for
Langevin equations, Journal of Computational Physics (2005)
[2] G.N. Milstein, Yu.M. Repin and M.V Tretyakov: Symplectic integration
of Hamiltonian systems with additive noise, SIAM Journal of Numerical
Analysis (2002)
[3] Øksendal.B.: Stochastic differential equations, Springer-Verlag, New
York(1998)
[4] V.I. Arnold: Mathematical methods of classical mechanics, SpringerVerlag, New York(1978)
[5] E. Nelson: Dynamical theories of Brownian motion, Princeton University Press (1967)
[6] P.E. Kloeden and E. Platen: Numerical solution of stochastic differential equations, XXIV-XXXII, Springer-Verlag, New York(1992)
[7] D. Givon, R. Kupferman and A.M. Stuart: Extracting macroscopic
dynamics:Model problems and algorithm
[8] C. Hartmann, J. Maddocks, C. Schütte: Diffusive limit of the Langevin
equation, under progress (2006)
[9] E. Hairer, C. Lubich, G. Wanner: Geometric numerical integration,
Springer-Verlag, Berlin Heidelberg (2002)
[10] W. T. Coffey, J. P. Kalmykov, J. T. Waldron: The Langevin equation,
World Scientific (1996)
[11] R.D. Skeel and J. Izaguirre: An impulse integrator for Langevin dynamics, Mol. Phys., 100, 3885-3891 (2002)
[12] W. Wang: Analysis of a few numerical integration methods for the
Langevin equation, M.S. Thesis, Department of Computer Science, University of Illinois at Urbana-Champaign (2001)
38