An Efficient Two-Level Preconditioner for
Multi-Frequency Wave Propagation Problems
M. Baumann∗,† and M.B. van Gijzen†
∗ Email:
[email protected]
Institute of Applied Mathematics
Delft University of Technology
Delft, The Netherlands
† Delft
May 19, 2017
M. Baumann
An optimal shift-and-invert preconditioner
1 / 27
Motivation (1/3)
Seismic exploration:
elastic wave equation
c geophysicsRocks!
in frequency-domain
0
Marmousi-II problem with 2,371,200 dofs
2560
z [m]
’only’ forward problem
Solve
18500
=b
for multiple frequencies ωk .
0
2000
x [m]
displacement at f = 6 Hz
1850
0
M. Baumann
4000 312
1e−9
1.8
z [m]
(K + iωk C −
ωk2 M)xk
An optimal shift-and-invert preconditioner
2000
x [m]
4000
−0.6
2 / 27
Motivation (1/3)
Seismic exploration:
elastic wave equation
c geophysicsRocks!
in frequency-domain
0
Marmousi-II problem with 2,371,200 dofs
2560
z [m]
’only’ forward problem
Solve
18500
=b
for multiple frequencies ωk .
K 0,
C 0,
0
2000
x [m]
displacement at f = 6 Hz
1e−9
1.8
M0
1850
0
M. Baumann
4000 312
z [m]
(K + iωk C −
ωk2 M)xk
An optimal shift-and-invert preconditioner
2000
x [m]
4000
−0.6
2 / 27
Motivation (2/3)
... (K + iωk C − ωk2 M)xk = b
Linearization:
b
iC K
M 0
ωk xk
=
,
− ωk
xk
0
I 0
0 I
k = 1, ..., Nω
Single preconditioner:
P(τ )
−1
=
I
=
0
M. Baumann
−1
K
M 0
−τ
0
0 I
τI I
0
0
I
0 (K + iτ C − τ 2 M)−1 I
iC
I
An optimal shift-and-invert preconditioner
I
−iC + τ M
3 / 27
Motivation (2/3)
... (K + iωk C − ωk2 M)xk = b
Linearization:
b
iC K
M 0
ωk xk
=
,
− ωk
xk
0
I 0
0 I
k = 1, ..., Nω
Single preconditioner:
P(τ )
−1
=
I
=
0
M. Baumann
−1
K
M 0
−τ
0
0 I
0
τI I
0
I
0 (K + iτ C − τ 2 M)−1 I
iC
I
An optimal shift-and-invert preconditioner
I
−iC + τ M
3 / 27
Motivation (3/3)
Convergence behavior for two different τ .
1
0
−1
−1
−3
−4
−5
−6
τ
−3
−4
−5
−6
−7
−8
−2
d
τ
−2
go
o
Relative residual norm
0
ba
d
Relative residual norm
1
−7
0
20
40
60
80
100
Number of matrix−vector multiplications
M. Baumann
120
140
−8
0
5
10
15
20
25
30
35
Number of matrix−vector multiplications
An optimal shift-and-invert preconditioner
4 / 27
Motivation (3/3)
Convergence behavior for two different τ .
1
0
−1
−1
−3
−4
−5
−6
τ
−3
−4
−5
−6
−7
−8
−2
d
τ
−2
go
o
Relative residual norm
0
ba
d
Relative residual norm
1
−7
0
20
40
60
80
100
Number of matrix−vector multiplications
M. Baumann
120
140
−8
0
5
10
15
20
25
30
35
Number of matrix−vector multiplications
An optimal shift-and-invert preconditioner
4 / 27
Outlook
M. Baumann
1
The shift-and-invert preconditioner within multi-shift GMRES
2
Optimization of seed frequency τ
3
Numerical experiments
Validations
Shifted Neumann preconditioner
Matrix equation with rotation
An optimal shift-and-invert preconditioner
5 / 27
Shift-and-invert preconditioner for GMRES
iC
A :=
I
K
,
0
M 0
B :=
0 I
Want to solve
(A − ωk B)xk = b,
k = 1, ..., Nω ,
with a single preconditioner P(τ ) = (A − τ B):
(A − ωk B)Pk−1 yk = b
M. Baumann
⇔
(A(A − τ B)−1 − ηk I )yk = b
An optimal shift-and-invert preconditioner
6 / 27
Shift-and-invert preconditioner for GMRES
iC
A :=
I
K
,
0
M 0
B :=
0 I
Want to solve
(A − ωk B)xk = b,
k = 1, ..., Nω ,
with a single preconditioner P(τ ) = (A − τ B):
(A − ωk B)Pk−1 yk = b
M. Baumann
⇔
(A(A − τ B)−1 − ηk I )yk = b
An optimal shift-and-invert preconditioner
6 / 27
Shift-and-invert preconditioner for GMRES
iC
A :=
I
K
,
0
M 0
B :=
0 I
Want to solve
(A − ωk B)xk = b,
k = 1, ..., Nω ,
with a single preconditioner P(τ ) = (A − τ B):
(A − ωk B)Pk−1 yk = b
M. Baumann
⇔
(
C
An optimal shift-and-invert preconditioner
− ηk I )yk = b
6 / 27
Multi-shift GMRES... Did you know?
For shifted problems,
(C − ηk I )yk = b,
k = 1, ..., Nω ,
Krylov spaces are shift-invariant
Km (C, b) ≡ Km (C − ηI , b)
∀η ∈ C.
Multi-shift GMRES:
(C − ηk I )Vm = Vm+1 (Hm − ηk I)
Reference
A. Frommer and U. Glässner. Restarted GMRES for Shifted Linear
Systems. SIAM J. Sci. Comput., 19(1), 15–26 (1998)
M. Baumann
An optimal shift-and-invert preconditioner
7 / 27
Multi-shift GMRES... Did you know?
For shifted problems,
(C − ηk I )yk = b,
k = 1, ..., Nω ,
Krylov spaces are shift-invariant
Km (C, b) ≡ Km (C − ηI , b)
∀η ∈ C.
Multi-shift GMRES:
(C − ηk I )Vm = Vm+1 (Hm − ηk I)
Reference
A. Frommer and U. Glässner. Restarted GMRES for Shifted Linear
Systems. SIAM J. Sci. Comput., 19(1), 15–26 (1998)
M. Baumann
An optimal shift-and-invert preconditioner
7 / 27
Optimization of seed frequency
A(A − τ B)−1 −
ωk
I
ωk − τ
Theorem: GMRES convergence bound
yk = b
[Saad, Iter. Methods]
Let the eigenvalues of a matrix be enclosed by a circle with
radius R and center c. Then the GMRES-residual norm after
i iterations kr(i) k satisfies,
kr(i) k
≤ c2 (X )
kr(0) k
R(τ )
|c(τ )|
i
,
where X is the matrix of eigenvectors, and c2 (X ) its condition
number in the 2-norm.
M. Baumann
An optimal shift-and-invert preconditioner
8 / 27
Optimization of seed frequency
A(A − τ B)−1 −
ωk
I
ωk − τ
yk = b
Theorem: msGMRES convergence bound
[Saad, Iter. Methods]
Let the eigenvalues of a matrix be enclosed by a circle with
radius Rk and center ck . Then the GMRES-residual norm after
(i)
i iterations krk k satisfies,
(i)
krk k
≤ c2 (X )
kr(0) k
Rk (τ )
|ck (τ )|
i
,
k = 1, ..., Nω ,
where X is the matrix of eigenvectors, and c2 (X ) its condition
number in the 2-norm.
M. Baumann
An optimal shift-and-invert preconditioner
8 / 27
The preconditioned spectra – no damping
Nω = 5
1
imag part
0.5
0
−0.5
−1
−1
−0.5
0
0.5
1
1.5
real part
M. Baumann
An optimal shift-and-invert preconditioner
9 / 27
The preconditioned spectra – no damping
Nω = 5
1
imag part
0.5
0
R
−0.5
≡
|
k
c
|
−1
−1
−0.5
0
0.5
1
1.5
real part
M. Baumann
An optimal shift-and-invert preconditioner
9 / 27
The preconditioned spectra – with damping > 0
ω̂k := (1 − i)ωk
2
imag part
1.5
1
0.5
0
−0.5
−1.5
−1
−0.5
0
0.5
1
1.5
real part
M. Baumann
An optimal shift-and-invert preconditioner
9 / 27
The preconditioned spectra – with damping > 0
ω̂k := (1 − i)ωk
R(τ )
2
ck+1
imag part
1.5
ck
R
1
ϕk
c
0.5
c1
0
−0.5
−1.5
−1
−0.5
0
0.5
1
1.5
real part
M. Baumann
An optimal shift-and-invert preconditioner
10 / 27
The preconditioned spectra
Lemma: Optimal seed shift for msGMRES
(i) For λ ∈
Λ[AB −1 ]
[B/vG, 2016]
it holds =(λ) ≥ 0.
(ii) The preconditioned spectra are enclosed by circles of radii
Rk and center points ck .
ω
(iii) The points {ck }N
k=1 ⊂ C described in statement (ii) lie on
a circle with center c and radius R.
(iv) Consider the preconditioner P(τ ∗ ) = A − τ ∗ B. An optimal
seed frequency τ ∗ for preconditioned multi-shift GMRES is
given by,
Rk (τ )
∗
τ () = min max
= ... =
τ ∈C k=1,..,Nω
|ck |
=
M. Baumann
2ω1 ωNω
−i
ω1 + ωNω
p
[2 (ω1 + ωNω )2 + (ωNω − ω1 )2 ] ω1 ωNω
ω1 + ωNω
An optimal shift-and-invert preconditioner
11 / 27
The preconditioned spectra – Proof (1/4)
Proof. (i) We have to show =(ω) ≥ 0 for,
iC K
M 0
x =ω
x
I 0
0 I
or, alternatively (λ = iω), consider the QEP,
(K + λC + λ2 M)v = 0.
<(λ) ≤ 0 ⇒ =(ω) ≥ 0
M. Baumann
An optimal shift-and-invert preconditioner
12 / 27
The preconditioned spectra – Proof (1/4)
Proof. (i) We have to show =(ω) ≥ 0 for,
iC K
M 0
x =ω
x
I 0
0 I
or, alternatively (λ = iω), consider the QEP,
(K + λC + λ2 M)v = 0.
<(λ) ≤ 0 ⇒ =(ω) ≥ 0
M. Baumann
An optimal shift-and-invert preconditioner
12 / 27
The preconditioned spectra – Proof (2/4)
(ii) The preconditioned spectra are enclosed by circles.
Factor out AB −1 ,
C − ηk I = A(A − τ B)−1 − ηk I = AB −1 (AB −1 − τ I )−1 − ηk I ,
and note that
Λ[AB −1 ] 3 λ 7→
is a Möbius transformation(∗) .
ωk
λ
−
,
λ−τ
ωk − τ
Reference
M.B. van Gijzen, Y.A. Erlangga, C. Vuik. Spectral Analysis of the Discrete
Helmholtz Operator Preconditioned with a Shifted Laplacian. SIAM J. Sci.
Comput., 29(5), 1942–1958 (2007)
(∗) : z 7→
M. Baumann
An optimal shift-and-invert preconditioner
az+b
cz+d
13 / 27
The preconditioned spectra – Proof (3/4)
(iii) Spectra are bounded by circles (ck , R). These center point
ω
{ck }N
k=1 lie on a ’big circle’ (c, R).
1. Construct center:
c = 0,
|τ |2
2=(τ )(=(τ ) + <(τ ))
∈C
2. A point ck has constant distance to c:
R2 = kck − ck22 =
M. Baumann
|τ |2 (2 + 1)
4(=(τ ) + <(τ ))2
An optimal shift-and-invert preconditioner
(independent of
ωk )
14 / 27
The preconditioned spectra – with damping > 0
ω̂k := (1 − i)ωk
R(τ )
2
ck+1
imag part
1.5
ck
R
1
ϕk
c
0.5
c1
0
−0.5
−1.5
−1
−0.5
0
0.5
1
1.5
real part
M. Baumann
An optimal shift-and-invert preconditioner
15 / 27
The preconditioned spectra – Proof (4/4)
1.00
(iv) Find optimal
τ ∗.
τ ∗ = min max
0.96
−1.0
0.92
−0.8
1
2
3
R
|ck |
|ck | = f (c, R, ϕk )
polar coordinates
∂τ
∂ϕ = 0 (optimize along ϕ)
0.88
imag part (relative)
τ ∈C k=1,..,Nω
−0.6
−0.4
0.84
0.80
τ∗
0.76
−0.2
0.72
ω̂min
0.0
0.0
M. Baumann
ω̂max
0.68
0.2
An optimal shift-and-invert preconditioner
0.4
0.6
real part (relative)
0.8
1.0
0.64
16 / 27
The preconditioned spectra – Proof (4/4)
1.00
(iv) Find optimal
τ ∗.
τ ∗ = min max
0.96
−1.0
0.92
−0.8
1
2
3
R
|ck |
|ck | = f (c, R, ϕk )
polar coordinates
∂τ
∂ϕ = 0 (optimize along ϕ)
0.88
imag part (relative)
τ ∈C k∈{1,Nω }
−0.6
−0.4
0.84
0.80
τ∗
0.76
−0.2
0.72
ω̂min
0.0
0.0
M. Baumann
ω̂max
0.68
0.2
An optimal shift-and-invert preconditioner
0.4
0.6
real part (relative)
0.8
1.0
0.64
16 / 27
The preconditioned spectra – Proof (4/4)
1.00
(iv) Find optimal
τ ∗.
τ ∗ = min max
0.96
−1.0
0.92
−0.8
1
2
3
R
|ck |
0.88
imag part (relative)
τ ∈C k∈{1,Nω }
|ck | = f (c, R, ϕk )
−0.6
−0.4
M. Baumann
=
0.80
τ∗
−0.2
0.72
ω̂min
0.0
0.0
2ω1 ωNω
ω1 +ωNω
0.84
0.76
polar coordinates
∂τ
∂ϕ = 0 (optimize along ϕ)
τ∗
ω̂max
−i
√
0.68
0.2
0.4
0.6
real part (relative)
0.8
1.0
0.64
[2 (ω1 +ωNω )2 +(ωNω −ω1 )2 ]ω1 ωNω
ω1 +ωNω
An optimal shift-and-invert preconditioner
16 / 27
M. Baumann
1
The shift-and-invert preconditioner within multi-shift GMRES
2
Optimization of seed frequency τ
3
Numerical experiments
Validations
Shifted Neumann preconditioner
Matrix equation with rotation
An optimal shift-and-invert preconditioner
17 / 27
The (damped) time-harmonic elastic wave equation
Continuous setting
Elastic wave equation
−ωk2 ρ(x)uk − ∇·σ(uk ) = s
with boundary conditions
iωk ρ(x)Buk + σ(uk )n̂ = 0,
σ(uk )n̂ = 0,
on ∂Ωa ∪· ∂Ωr .
Discrete setting
Solve
ω̂k = (1 − i)ωk
(K + iωk C − ωk2 M)uk = s
with FEM matrices
Z
Kij =
σ(ϕi ) : ∇ϕj dΩ,
ZΩ
Mij =
ρ(x)ϕi · ϕj dΩ,
ZΩ
Cij =
ρ(x)Bϕi · ϕj dΓ.
∂Ωa
nutils
M. Baumann
An optimal shift-and-invert preconditioner
18 / 27
The (damped) time-harmonic elastic wave equation
Continuous setting
Elastic wave equation
−ωk2 ρ(x)uk − ∇·σ(uk ) = s
with boundary conditions
iωk ρ(x)Buk + σ(uk )n̂ = 0,
σ(uk )n̂ = 0,
on ∂Ωa ∪· ∂Ωr .
Discrete setting
Solve
ω̂k = (1 − i)ωk
(K + iωk C − ωk2 M)uk = s
with FEM matrices
Z
Kij =
σ(ϕi ) : ∇ϕj dΩ,
ZΩ
Mij =
ρ(x)ϕi · ϕj dΩ,
ZΩ
Cij =
ρ(x)Bϕi · ϕj dΓ.
∂Ωa
nutils
M. Baumann
An optimal shift-and-invert preconditioner
18 / 27
Numerical experiments
Validations and second-level preconditioners
Set-up: An elastic wedge problem.
Density
2100
0
u_z at 16.0 Hz
1e−10
2.4
2070
0.8
1920
0.4
z [m]
z [m]
1950
1860
M. Baumann
300
x [m]
600
0.50
400
0.25
0.00
−0.25
0.0
800
−0.4
1830
1000
0
0.75
1.2
400
1890
800
1e−11
1.00
1.6
2010
1980
u_x at 24.0 Hz
2.0
2040
400
0
z [m]
0
1800 1000
0
−0.8
300
x [m]
600
An optimal shift-and-invert preconditioner
800
−0.50
−0.75
1000
0
300
x [m]
600
19 / 27
Numerical experiments (1/4)
Validations I
r
τ ∗ () =
p
i arctan −
ω1 ωNω (1 + 2 ) · e
ωmin /2π [Hz]
ωmax /2π [Hz]
1
5
1
10
Nω
5
10
20
5
10
20
2 (ω1 +ωN )2 +(ωN −ω1 )2
ω
ω
4ω1 ωN
ω
# iterations
106
106
106
251
252
252
!
CPU time [s]
45.6
48.7
47.3
205.1
223.7
243.5
damping factor = 0.05
#dofs = 48, 642 (Q1 finite elements)
M. Baumann
An optimal shift-and-invert preconditioner
20 / 27
Numerical experiments (2/4)
1
Nω = 5
0
Validations II
Relative residual norm
−1
Convergence behavior:
ωmin and ωmax converge
slowest,
smallest factor R/|ck |
yields fastest convergence,
−2
−3
−4
−5
−6
−7
−8
0
5
10
15
20
25
30
35
Number of matrix−vector multiplications
2
1.5
’inner’ frequencies for free.
1
0.5
0
−0.5
−2
M. Baumann
−1.5
An optimal shift-and-invert preconditioner
−1
−0.5
0
0.5
1
1.5
21 / 27
Numerical experiments (3/4)
Shifted Neumann preconditioner
140
140
120
120
100
100
Number of iterations
Number of iterations
Apply a Neumann polynomial preconditioner of degree n.
80
60
40
20
0
1.0
² = 0.5
² = 0.3
² = 0.1
² = 0.05
80
60
40
² = 0.5
² = 0.3
² = 0.1
² = 0.05
0.8
20
0.6
0.4
Seed shift (scaled imag part)
0.2
0.0
0
1.0
0.8
case n = 0
M. Baumann
An optimal shift-and-invert preconditioner
0.6
0.4
Seed shift (scaled imag part)
0.2
0.0
case n = 5
22 / 27
Numerical experiments (4/4)
Matrix equation with rotation
Solve matrix equation,
A(X) := K X + iC XΩ − MXΩ2 = B.
Spectrum of A ◦ P1−1
1.5
imag part
1.0
0.5
0.0
−0.5
−1.5
M. Baumann
−1.0
−0.5
0.0
real part
0.5
An optimal shift-and-invert preconditioner
1.0
1.5
23 / 27
Numerical experiments (4/4)
Matrix equation with rotation
Solve matrix equation,
A(X) := K X + iC XΩ − MXΩ2 = B.
Spectrum of A ◦ P1−1 ◦ P2−1
1.5
e−i(φ2−φ1)
imag part
1.0
0.5
0.0
−0.5
−1.5
M. Baumann
−1.0
−0.5
0.0
real part
0.5
An optimal shift-and-invert preconditioner
1.0
1.5
23 / 27
Summary
1
0
0
−1
−1
−4
−5
−6
−2
τ
−3
−3
ba
d
Relative residual norm
−2
−4
−5
−6
−7
−7
−4
−3
−2
−1
0
1
2
3
4
−8
5
0
20
40
60
80
100
120
140
30
35
Number of matrix−vector multiplications
1
0
2
1
0.5
τ
−2
−3
d
1.5
−4
go
o
Relative residual norm
−1
−5
−6
0
−7
−0.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
−8
0
5
10
15
20
25
Number of matrix−vector multiplications
M. Baumann
An optimal shift-and-invert preconditioner
24 / 27
Summary
1
0
0
−1
−1
−4
−5
−6
−2
τ
−3
−3
ba
d
Relative residual norm
−2
−4
−5
−6
−7
−7
−4
−3
−2
−1
0
1
2
3
4
−8
5
0
20
40
60
80
100
120
140
30
35
Number of matrix−vector multiplications
1
0
2
1
0.5
τ
−2
−3
d
1.5
−4
go
o
Relative residual norm
−1
−5
−6
0
−7
−0.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
−8
0
5
10
15
20
25
Number of matrix−vector multiplications
M. Baumann
An optimal shift-and-invert preconditioner
24 / 27
Conclusions
3 Optimal shift-and-invert preconditioner for msGMRES.
3 Second level: Shifted Neumann preconditioner & rotations
in matrix equation approach.
3 For multi-core CPUs: Splitting strategy of frequency range.
7 Optimality for = 0 only by continuity.
? Relation to pole selection in rational Krylov methods.
Thank you for your attention!
M. Baumann
An optimal shift-and-invert preconditioner
25 / 27
Conclusions
3 Optimal shift-and-invert preconditioner for msGMRES.
3 Second level: Shifted Neumann preconditioner & rotations
in matrix equation approach.
3 For multi-core CPUs: Splitting strategy of frequency range.
7 Optimality for = 0 only by continuity.
? Relation to pole selection in rational Krylov methods.
Thank you for your attention!
M. Baumann
An optimal shift-and-invert preconditioner
25 / 27
References
M. Baumann and M.B. van Gijzen. Nested Krylov methods for shifted
linear systems. SIAM J. Sci. Comput., 37(5), S90-S112 (2015).
M. Baumann, R. Astudillo, Y. Qiu, E. Ang, M.B. van Gijzen, and
R.-E. Plessix. An MSSS-Preconditioned Matrix Equation Approach for
the Time-Harmonic Elastic Wave Equation at Multiple Frequencies.
DIAM Technical Report 16-04, TU Delft [accepted].
M. Baumann and M.B. van Gijzen. Efficient iterative methods for
multi-frequency wave propagation problems: A comparison study.
Proceedings of ICCS 2017 [accepted].
M. Baumann and M.B. van Gijzen. An Efficient Two-Level
Preconditioner for Multi-Frequency Wave Propagation Problems.
DIAM Technical Report 17-03, TU Delft [under review].
Research funded by Shell.
M. Baumann
An optimal shift-and-invert preconditioner
26 / 27
Finite element discretization in Python
FEM discretization of stiffness matrix K ,
Z
σ(ϕi ) : ∇ϕj dΩ, where
Kij =
Ω
σ(ϕi ) = λ(x)div (ϕi )I3 + µ(x) ∇ϕi + (∇ϕi )T ,
becomes in nutils:
ndims = 3
phi
= domain.splinefunc(degree=2).vector(ndims)
stress = lambda u: lam*u.div(geom)[:,_,_]*function.eye(ndims)
+ mu*u.symgrad(geom)
elast = function.outer(
stress(phi), phi.grad(geom) ).sum([2,3])
K = domain.integrate(elast, geometry=geom, ischeme=’gauss2’)
Go back
M. Baumann
An optimal shift-and-invert preconditioner
27 / 27