June, 2014
Fast iterative solvers for fractional differential
equations
Tobias Breiten, Valeria Simoncini, Martin Stoll
Numerical Linear Algebra for Dynamical Systems
Max Planck Institute for Dynamics of Complex Technical Systems
Magdeburg
Householder Symposium XIX
Belgium Spa
MAX PLANCK INSTITUTE
FOR DYNAMICS OF COMPLEX
TECHNICAL SYSTEMS
MAGDEBURG
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
1/27
What got me interested?
It all starts with my wife, Anke!
A conversation in the car
Anke: Hey, do you know how to solve fractional differential
equations?
Me (scratching my head): Um, oh. Not really but I think
Chebfun can do it.
Me (later that evening): No, Chebfun can’t do it. Let’s
google some more.
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
2/27
What got me interested?
It all starts with my wife, Anke!
A conversation in the car
Anke: Hey, do you know how to solve fractional differential
equations?
Me (scratching my head): Um, oh. Not really but I think
Chebfun can do it.
Me (later that evening): No, Chebfun can’t do it. Let’s
google some more.
I then found a code that got me interested (here is the crucial
snippet):
A=kron(I,L1)+kron(L2,I); x=A\b;
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
2/27
What got me interested?
It all starts with my wife, Anke!
A conversation in the car
Anke: Hey, do you know how to solve fractional differential
equations?
Me (scratching my head): Um, oh. Not really but I think
Chebfun can do it.
Me (later that evening): No, Chebfun can’t do it. Let’s
google some more.
I then found a code that got me interested (here is the crucial
snippet):
A=kron(I,L1)+kron(L2,I); x=A\b;
This was getting interesting.
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
2/27
Fractional differential equations: Why?
Fractional space derivatives are used to model anomalous diffusion or
dispersion, where a particle plume spreads at a rate inconsistent with the
classical Brownian motion model. (Meerschaert,Tadjeran 2004)
Applications
Groundwater flow
Hydrology
Protein surface modeling
Imaging
Material science, soft tissue modeling
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
3/27
Fractional differential equations: Why?
Fractional space derivatives are used to model anomalous diffusion or
Fractional
model atfora rate
viscoelastic
materials
dispersion,
where a Kelvin-Voigt
particle plume spreads
inconsistent
with the
classical Brownian motion model. (Meerschaert,Tadjeran 2004)
Applications
Groundwater flow
Hydrology
Protein surface modeling
Imaging
Material science, soft tissue modeling
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
3/27
Fractional derivatives
Caputo derivative of real order α with (n − 1 < α ≤ n) is defined
as
Z t
1
f (n) (s)ds
C α
D
f
(t)
=
,
a t
Γ(n − α) a (t − s)α−n+1
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
4/27
Fractional derivatives
Caputo derivative of real order α with (n − 1 < α ≤ n) is defined
as
Z t
1
f (n) (s)ds
C α
D
f
(t)
=
,
a t
Γ(n − α) a (t − s)α−n+1
(left-sided) Riemann-Liouville derivative of order α with
(n − 1 < α ≤ n) is defined as
n Z t
1
d
f (s)ds
RL α
,
a < t < b.
a Dt f (t) =
α−n+1
Γ(n − α) dt
a (t − s)
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
4/27
Fractional derivatives
Caputo derivative of real order α with (n − 1 < α ≤ n) is defined
as
Z t
1
f (n) (s)ds
C α
D
f
(t)
=
,
a t
Γ(n − α) a (t − s)α−n+1
(left-sided) Riemann-Liouville derivative of order α with
(n − 1 < α ≤ n) is defined as
n Z t
1
d
f (s)ds
RL α
,
a < t < b.
a Dt f (t) =
α−n+1
Γ(n − α) dt
a (t − s)
Symmetric Riesz derivative of order α, that is,
d α f (t)
1 RL α
α
RL α
=
D
f
(t)
=
D
f
(t)
+
D
f
(t)
.
t
t
R
b
d |t|α
2 a t
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
4/27
Some comments
Properties
Caputo and Riemann-Liouville have a switched order of
integration vs. differentiation.
Caputo is often preferred for time-fractional (0 < α ≤ 1) due
to conditions on the initial condition.
Symmetric Riesz is often used for space-fractional
(1 < α ≤ 2) derivatives. Sometimes asymmetrically.
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
5/27
Some comments
Properties
Caputo and Riemann-Liouville have a switched order of
integration vs. differentiation.
Caputo is often preferred for time-fractional (0 < α ≤ 1) due
to conditions on the initial condition.
Symmetric Riesz is often used for space-fractional
(1 < α ≤ 2) derivatives. Sometimes asymmetrically.
Further Reading
Caputo (1967), Gorenflo+Mainardi (1997), Metzler+Klafter
(2000), Meerschaert+Tadjeran (2004,2006,...), Podlubny
(1999,2009,...) plus many, many more
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
5/27
Model Problems
We consider four model problems:
du(x, t)
− x DRβ u(x, t) = f (x, t)
dt
Assume unit cube for the spatial domain, [0, T ] for the time
interval and zero initial as well as zero Dirichlet conditions.
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
6/27
Model Problems
We consider four model problems:
du(x, t)
− x DRβ u(x, t) = f (x, t)
dt
β
C α
0 Dt u(x, t) − x DR u(x, t) = f (x, t),
Assume unit cube for the spatial domain, [0, T ] for the time
interval and zero initial as well as zero Dirichlet conditions.
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
6/27
Model Problems
We consider four model problems:
du(x, t)
− x DRβ u(x, t) = f (x, t)
dt
β
C α
0 Dt u(x, t) − x DR u(x, t) = f (x, t),
du(x, y , t)
− x DRβ1 u(x, y , t) − y DRβ2 u(x, y , t) = f (x, y , t),
dt
Assume unit cube for the spatial domain, [0, T ] for the time
interval and zero initial as well as zero Dirichlet conditions.
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
6/27
Model Problems
We consider four model problems:
du(x, t)
− x DRβ u(x, t) = f (x, t)
dt
β
C α
0 Dt u(x, t) − x DR u(x, t) = f (x, t),
du(x, y , t)
− x DRβ1 u(x, y , t) − y DRβ2 u(x, y , t) = f (x, y , t),
dt
C α
D
u(x,
y , t) − x DRβ1 u(x, y , t) − y DRβ2 u(x, y , t) = f (x, y , t).
0 t
Assume unit cube for the spatial domain, [0, T ] for the time
interval and zero initial as well as zero Dirichlet conditions.
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
6/27
Discretization Problem 1
We employ finite-differences of Grünwald-Letnikov type based on
RL β
a Dx u(x, t)
M
1 X
= lim α
gβ,k f (x − (k − 1)h, t)
M→∞ h
k=0
which gives
RL β n+1
a Dx ui
≈
i+1
1 X
hxβ
n+1
gβ,k ui−k+1
k=0
and in matrix form








−β 
hx








gβ,1
gβ,2
gβ,3
.
.
.
.
.
.
.
.
.
gβ,nx
|
Max Planck Institute Magdeburg
gβ,0
gβ,1
gβ,2
.
.
.
.
.
.
0
gβ,0
gβ,1
gβ,0
.
.
gβ,2
.
.
.
gβ,nx −1
.
.
gβ,1
.
.
.
.
.
...
.
.
.
.
.
...
{z
n
T x
β
.
.
.
.
gβ,2
gβ,0
0
gβ,1
gβ,2
gβ,0
gβ,1


u1n+1


  u2n+1

  u n+1

3

.


.

.
  n+1
 u

nx −1

unn+1

x






.





}
Martin Stoll, Fast iterative solvers for fractional differential equations
7/27
Discretization Problem 1 ctd.
We now have
Inx − τ Lnβx un+1 = un + τ f n+1
for the discretization of
du(x, t)
− x DRβ u(x, t) = f (x, t)
dt
where
Inx − τ Lnβx
1
2
is an SPD Toeplitz matrix and Lβ =
Toeplitz matrix.
Max Planck Institute Magdeburg
Tβ + TT
β
a dense
Martin Stoll, Fast iterative solvers for fractional differential equations
8/27
Discretization Problem 2 and matrix equation
Again using Grünwald-Letnikov finite differences for Caputo and
Riemann-Liouville derivatives we obtain
(Tnαt ⊗ Inx ) − (Int ⊗ Lβnx ) u = f̃
for the discretization of
C α
0 Dt u(x, t)
− x DRβ u(x, t) = f (x, t).
Here, Tnαt and Lnβx are Toeplitz matrices .
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
9/27
The Toeplitz world
For any iterative solver it is important to efficiently perform the
matrix vector products, i.e, Ty uses circulant matrix
T B
y
B T
0
with
0
 t1−n


.

.
.
B=




t−2
t−1

tn−1
0
...
tn−1
t2
.
t1−n
0
.
t−2
.
.
...
.
.
.
.
.
t1−n
t1
t2
.
.
.
tn−1
0










 and T = 








t0
t1
.
.
.
t−1
t0
t1
...
t−1
t0
.
tn−2
tn−1
tn−2
.
.
...
t2−n
.
t1−n
t2−n
.
.
.
.
t1
t−1
t0
..
.
.





.




Now the FFT can be used.
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
10/27
Toeplitz matrices and preconditioning
We use the Strang-type approximation of the Toeplitz matrix










c0
c1
.
.
.
cn−1
c0
c1
...
cn−1
c0
.
cn−2
cn−1
cn−2
.
.
...
c2
..
.
.
.
.
c1
c1
c2
.
.
.

cn−1
c0









=C≈T=








t0
t1
.
.
.
t−1
t0
t1
...
t−1
t0
.
tn−2
tn−1
tn−2
.
.
...
t2−n
.
t1−n
t2−n
.
.
.
.
t1
t−1
t0
..
.
.










and the diagonals given by

0 ≤ j ≤ bn/2c
 tj ,
t , bn/2c < j < n
cj =
 j−n
cn+j 0 < −j < n
This can again be cheaply applied using the FFT.
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
11/27
Solving problem 1 (space-fractional)
Remember we obtained
Inx − τ Lnβx un+1 = un + τ f n+1 .
We can for every time-step apply the preconditioned conjugate
gradient method with circulant preconditioning.
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
12/27
Solving problem 1 (space-fractional)
Remember we obtained
Inx − τ Lnβx un+1 = un + τ f n+1 .
We can for every time-step apply the preconditioned conjugate
gradient method with circulant preconditioning.
Lemma
For 1 < β < 2, the spectrum of the matrix Lβ := 21 (Tβ + TT
β ) is
−β
contained in the open interval (−2h β, 0).
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
12/27
Solving problem 1 (space-fractional)
Remember we obtained
Inx − τ Lnβx un+1 = un + τ f n+1 .
We can for every time-step apply the preconditioned conjugate
gradient method with circulant preconditioning.
Lemma
For 1 < β < 2, the spectrum of the matrix Lβ := 21 (Tβ + TT
β ) is
−β
contained in the open interval (−2h β, 0).
Everything is implemented in a matrix-free way.
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
12/27
Problem 1: Results
Figure: Problem 1. Comparison of numerical solutions when using two
different values for β with zero initial condition and zero Dirichlet
boundary condition. (510 and 10 space and time points, resp.)
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
13/27
Problem 1: Results
DoF
32768
65536
131072
262144
524288
1048576
β = 1.3
avg. its (time)
β = 1.7
avg. its (time)
6(2.09)
6(3.69)
6(6.63)
6(47.26)
6(50.52)
6(150.36)
7(2.94)
7(3.45)
7(6.25)
8.3(56.33)
9(66.92)
9(197.3)
Table: Problem 1. Average PCG iteration numbers for 10 time steps and
two values of β, as the mesh is refined; in parenthesis is the total CPU
time (in secs).
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
14/27
Solving problem 2: Space- and time-fractional
We obtained a generalized Sylvester equation
(Tnαt ⊗ Inx ) − (Int ⊗ Lβnx ) u = f̃,
for which one can use a zoo of methods. Our focus: Low-rank
methods
U = [u1 , . . . , unt ] ≈ Ũ1 ŨT
2 with u = vec(U)
a structure that can be maintained throughout the algorithm.
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
15/27
KPIK: a very brief introduction
Consider the Sylvester equation
TU + UL = C.
We want U = VYWT for skinny V, W and some Y. Also,
C = C1 CT
2.
It is crucial to choose Range(V) and Range(W).
For KPIK we typically use:
Range(V) = Range([C1 , (T − σ1 I)−1 C1 , (T − σ2 I)−1 C1 , . . .])
Range(W) = Range([C2 , (LT − η1 I)−1 C2 , (LT − η2 I)−1 C2 , . . .])
with the choice σi , ηi ∈ {0, ∞} particularly effective.
The zero-choice corresponds to an approximation space using T, L,
L−1 and T−1 , where we inexactly evaluate the inverses.
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
16/27
Possible alternative solvers
There are other methods that one could use such as
ADI method
Sign function iteration
Low-rank Krylov solver with preconditioning (best
preconditioners?)
Multigrid
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
17/27
Problem 2: Numerical results
iKPIK
β = 1.3
β = 1.7
it(CPUtime) it(CPUtime)
#A/#P
#A/#P
nt
nx
1024
1024
10 (0.73)
108/80
8 (0.64 )
87/63
2048
2048
11 (1.73)
128/96
8 (1.25 )
90/66
4096
4096
11 (4.85)
146/110
8 (3.85 )
106/78
8192
8192
11 (21.25 )
146/110
9 (20.20 )
121/89
Table: Problem 2. Preconditioned low-rank bicg and ikpik, for a variety
of meshes and two different values of β. Shown are the iteration numbers
and the total CPU time, together with the number of matrix-vector
products (#A) and the number of preconditioner evaluations (#P).
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
18/27
Problem 3: Numerical results
2D space-fractional
ny
nx
256
256
256
1024
4096
512
1024
4096
8192
8192
β1 = 1.3, β2 = 1.9
aver.it/#A/#P
β1 = 1.7, β2 = 1.9
aver.it/#A/#P
2.25/59.00/44.00
2.25/59.00/44.00
2.25/59.00/44.00
2.00/28.75/20.75
2.00/26.00/18.00
2.50/108.12/84.62
3.00/135.37/104.37
3.37/155.00/120.50
2.87/104.62/81.12
2.25/66.12/50.12
Table: Problem 3. Performance of iKPIK as the mesh is refined for two
different values of (β1 , β2 ), in terms of average iteration numbers,
average number of matrix-vector products and number of preconditioner
evaluations.
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
19/27
Problem 4: The Tensor World
We consider
C α
0 Dt u(x, y , t)
− x DRβ1 u(x, y , t) − y DRβ2 u(x, y , t) = f (x, y , t),
which can be discretized as
n n
Tnαt ⊗ Inx ny − Int ⊗ Lβx1 ,βy2 u = f̃.
where
Lβ1 ,β2 =
Max Planck Institute Magdeburg
1 ny
1
n
I ⊗ Lnβx1 + Lβy2 ⊗ β Inx
h β1
h 2
.
Martin Stoll, Fast iterative solvers for fractional differential equations
20/27
Problem 4: Numerical Results
We use out of the (TT) box DMRG.
nt
ny
nx
512
256
1024
1024
256
512
512
1024
512
512
2048
4096
β1 = 1.3, β2 = 1.5
α = 0.3
it
β1 = 1.7, β2 = 1.9
α = 0.7
it
β1 = 1.9, β2 = 1.1
α = 0.2
it
6
6
6
6
6
7
7
7
6
6
11
11
Table: Problem 4. Performance of dmrg as the mesh is refined, for
different values of the differentiation orders.
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
21/27
An inverse problem
Consider the objective function
min J(y , u) :=
y ,u
1
γ
ky − yb k2Q1 + kuk2Q2
2
2
subject to
C α
0 Dt y (x, t)
Max Planck Institute Magdeburg
− x DRβ y (x, t) + u(x, t) = f (x, t).
Martin Stoll, Fast iterative solvers for fractional differential equations
22/27
An inverse problem
Consider the objective function
min J(y , u) :=
y ,u
1
γ
ky − yb k2Q1 + kuk2Q2
2
2
subject to
C α
0 Dt y (x, t)
− x DRβ y (x, t) + u(x, t) = f (x, t).
Discretization leads to the Lagrangian
L(y, u, p) = J(y, u) + pT ((Tα ⊗ In − Int ⊗ L) y − IM u − d) .
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
22/27
An inverse problem
We finally manage to get a saddle point problem.

I1
0

0
γI2
−Tα ⊗ In + Int ⊗ L I3

 

T
I1 ȳ
y
−TT
α ⊗ I n + I nt ⊗ L
 u  =  0 
I3
p
d
for which we can use a preconditioned low-rank method.
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
23/27
An inverse problem: Results
Computed State
Desired State
Computed Control
50
50
50
100
100
100
150
150
150
200
200
200
250
250
250
300
300
300
350
350
350
400
400
400
450
450
10
20
30
40
50
450
10
20
30
40
50
10
20
30
40
50
Figure: Computed state, desired state and computed control for the
optimization problem with γ = 1e − 1.
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
24/27
An inverse problem: Results
Computed State
Desired State
Computed Control
50
50
50
100
100
100
150
150
150
200
200
200
250
250
250
300
300
300
350
350
350
400
400
400
450
450
10
20
30
40
50
450
10
20
30
40
50
10
20
30
40
50
Figure: Computed state, desired state and computed control for the
optimization problem with γ = 1e − 4.
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
25/27
Conclusions
Fractional differential equations are interesting and relevant.
One obtains challenging matrix systems/equations.
Combination of many techniques: Toeplitz, Circulant, FFT,
Low-rank methods, Matrix equations, Tensors,...
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
26/27
Max Planck Institute Magdeburg
Martin Stoll, Fast iterative solvers for fractional differential equations
27/27