On the residual inverse iteration
for nonlinear eigenvalue problems
Daniel Kressner
Chair of Numerical Algorithms and HPC
MATHICSE / SMA / SB
EPF Lausanne
http://anchp.epfl.ch
Householder Symposium XIX, Spa
Joint work with Cedric Effenberger.
Nonlinear eigenvalue problems
Mathematical formulation.
For analytic T : Ω → Cn×n , Ω ⊆ C, find eigenvalues λ ∈ C:
T (λ) x = 0,
x 6= 0
(NLEVP)
Important special cases:
I
linear eigenvalue problems: T (λ) = λI − A
I
quadratic eigenvalue problems: T (λ) = λ2 A2 + λA1 + A0
Reasons for more general nonlinearities in PDE eigenvalue problems:
I
frequency-dependent coefficients;
I
model reduction.
I
frequency-dependent boundary conditions;
I
discretization with frequency-dependent bases functions
(Trefftz-type methods);
I
time delays;
I
...
NLEVPs from wave propagation
Material with relative permittivity ε(ω)
in vacuum (ε0 ≡ 0)
T (ω) = −ω 2 A1 − ω 2 ε(ω)A2 + A3 .
Ω2
In practice:
I
Engineers model frequency dependency
of ε(ω) with rational functions.
I
Popular: Lorentz model
ε(ω) = ε∞ +
#terms
X
`=1
Ω1
α`
1 + iβ` ω − γ` ω 2
with parameters αk , βk , γk fitted to experimental data.
I
#terms typically between 1 and 7.
I
See, e.g., [Szabó, Kádár, and Volk. Band gaps in photonic
crystals with dispersion. COMPEL, 2005].
NLEVPs from BEM
3D Laplace eigenvalue problem
−∆u = λ2 u
in Ω ∈ R3 ,
with Dirichlet b.c.
boundary integral equation
Z
∂Ω
eiλkξ−ηk
s(ξ)t(η) dS(η) dS(ξ) = 0
4πkξ − ηk
Z
∂Ω
BEM
nonlinear eigenvalue problem
T (λ)x = 0 with
Z
Z
[T (λ)]ij =
4i
I
4j
eiλkξ−ηk
dS(η) dS(ξ)
4πkξ − ηk
3D, sparse matrix, linear
→ 2D, full matrix (low-rank structure), nonlinear
[Unger/Steinbach’2009], [Effenberger/DK’2012]
1
∀s ∈ H − 2 (∂Ω)
NLEVPs
Mathematical formulation.
For analytic T : Ω → Cn×n , Ω ⊆ C, find eigenvalues λ ∈ C:
T (λ) x = 0,
x 6= 0
(NLEVP)
Several exciting recent algorithmic developments:
I nonlinear Arnoldi [Voss’2004]
I rational linearization [Su/Bai’2011]
I block Newton [K.’2009]
I infinite Arnoldi [Jarlebring/Meerbergen/Michiels’2010/2012]
I contour integral method [Asakura/Sakurai/Tadano/Ikegami/Kimura’2009, Beyn’2012, . . .]
I interpolation based approaches [Effenberger/K.’2012, Van
Beeumen/Meerbergen/Michiels’2013, K./Roman’2013, Güttel/Van
Beeumen/Meerbergen/Michiels’2013]
I Jacobi-Davidson with eigenvalue locking [Effenberger’2013]
I ...
Talks @ Householder:
I
D. Bindel (Thursday morning)
I
E. Jarlebring (Thursday afternoon)
I
R. Van Beeumen (Thursday afternoon)
Goal of this talk
I
Understand convergence behavior of a simple method applied to
nonlinear eigenvalue problems from discretized PDEs.
I
Gain insight into use of preconditioners, etc.
From Newton to residual inverse iteration
Plain Newton applied to nonlinear system of equations
T (λ)x = 0,
w ∗x = 1
yields:
yj+1 = T (λj )−1 T 0 (λj )vj
λj+1 = λj − 1/w ∗ yj+1
vj+1 = yj+1 /w ∗ yj+1
From Newton to residual inverse iteration
Plain Newton applied to nonlinear system of equations
T (λ)x = 0,
w ∗x = 1
yields:
yj+1 = T (λj )−1 T 0 (λj )vj
λj+1 = λj − 1/w ∗ yj+1
vj+1 = yj+1 /w ∗ yj+1
Step 1: Replace eigenvalue update by nonlinear equivalent of
Rayleigh quotient.
From Newton to residual inverse iteration
(Almost) Newton applied to nonlinear system of equations
T (λ)x = 0,
w ∗x = 1
yields:
Compute solution λj+1 of vj∗ T (λ)vj = 0.
yj+1 = T (λj )−1 T 0 (λj )vj
vj+1 = yj+1 /w ∗ yj+1
Step 2: Replace T 0 (λj ) by finite difference.
From Newton to residual inverse iteration
Quasi-Newton applied to nonlinear system of equations
T (λ)x = 0,
w ∗x = 1
yields:
Compute solution λj+1 of vj∗ T (λ)vj = 0.
1
(T (λj+1 ) − T (λj ))vj
λj+1 − λj
= yj+1 /w ∗ yj+1
yj+1 = T (λj )−1
vj+1
From Newton to residual inverse iteration
Quasi-Newton applied to nonlinear system of equations
T (λ)x = 0,
w ∗x = 1
yields:
Compute solution λj+1 of vj∗ T (λ)vj = 0.
vj+1 = vj − T (λj )−1 T (λj+1 )vj
Normalize vj+1 .
Step 3: Replace T (λj )−1 by T (σ)−1 .
Residual inverse iteration
Choose fixed σ sufficiently close to eigenvalue of interest.
for j = 0, 1, 2, . . .
Compute solution λj+1 of vj∗ T (λ)vj = 0.
vj+1 = vj − T (σ)−1 T (λj+1 )vj
Normalize vj+1 .
end for
Finally...
... this is RESINVIT (residual inverse iteration) by Neumaier [1985].
Residual inverse iteration
Choose fixed σ sufficiently close to eigenvalue of interest.
for j = 0, 1, 2, . . .
Compute solution λj+1 of vj∗ T (λ)vj = 0.
vj+1 = vj − T (σ)−1 T (λj+1 )vj
Normalize vj+1 .
end for
I
For linear T (λ) = λI − A: Odd way of writing inverse iteration.
When using preconditioner P ≈ −T (σ)
PINVIT.
Preconditioner for nonlinear eigenvalue problems?
Model problem
Vibrating string with elastically attached mass [Solov’ëv’2006]:
−u 00 (x) = λu(x),
u(0) = 0,
u 0 (1) + φ(λ)u(1) = 0,
φ(λ) =
ληm
.
λ−η
Discretization with pw linear FEs
K (λ) − λM v = 0
with K (λ) = K0 + φ(λ)en enT and


2 −1


..
..

.
.
1 −1
,
K0 = 


..
h
.
2 −1
−1 1

4

h 1
M= 
6

1
..
.
..
.

..
.
4
1


.

1
2
Model problem
I
I
I
Aim to compute smallest eigenvalue λ1 = 4.482024295 . . ..
Apply RESINVIT with preconditioner for T (σ) = one W-cycle of
multigrid with Jacobi smoother.
Stop when residual ≤ 10−6 .
h
2-5
2-6
2-7
2-8
2-9
2-10
2-11
2-12
2-13
2-14
2-15
2-16
2-17
2-18
σ=0
12
12
12
12
12
12
11
11
11
10
10
10
9
9
#iterations
σ=2 σ=4
8
5
8
5
8
5
7
4
7
4
7
4
7
4
7
4
6
4
6
4
6
4
6
4
6
3
5
3
Goal of this talk
Derive convergence result that allows to conclude
mesh-independent asymptotic convergence rates
when using a spectrally equivalent preconditioner for T (σ).
[Solov’ëv’2006] has established such a result for “monotonic”
nonlinear eigenvalue problems
T (λ) = A(λ) − λB(λ)
with spd A(λ), B(λ).
Convergence result by Neumaier
I
Consider simple eigenvalue λ1 with eigenvector x1 normalized
s.t. w ∗ x1 = 1.
I
Assume that σ is sufficiently close to λ1 .
I
Exact preconditioner T (σ)−1 .
Then RESINVIT converges with
kvj+1 − x1 k2
= O(|σ − λ1 |),
kvj − x1 k2
|λj+1 − λ1 | = O(kvj+1 − x1 k2 ).
Convergence result by Neumaier
I
Consider simple eigenvalue λ1 with eigenvector x1 normalized
s.t. w ∗ x1 = 1.
I
Assume that σ is sufficiently close to λ1 .
I
Assume that T is Hermitian.
I
Exact preconditioner T (σ)−1 .
Then RESINVIT converges with
kvj+1 − x1 k2
= O(|σ − λ1 |),
kvj − x1 k2
|λj+1 − λ1 | = O(kvj+1 − x1 k22 ).
Convergence result by Neumaier
I
Consider simple eigenvalue λ1 with eigenvector x1 normalized
s.t. w ∗ x1 = 1.
I
Assume that σ is sufficiently close to λ1 .
I
Assume that T is Hermitian.
I
Exact preconditioner T (σ)−1 .
Then RESINVIT converges with
kvj+1 − x1 k2
= O(|σ − λ1 |),
kvj − x1 k2
|λj+1 − λ1 | = O(kvj+1 − x1 k22 ).
But: Not helpful if constant in O(|σ − λ1 |) deteriorates as h → 0.
Convergence rate
Detailed analysis of convergence rate by [Jarlebring/Michiels’2011]:
asympt convergence rate = |σ − λ1 |ρ(B1 ) + o(|σ − λ1 |)
with
B0
=
B1
=
T 0 (λ1 )x1 x1∗
x1∗ T 0 (λ1 )x1
−1
T 0 (λ1 )x1 x1∗
T 00 (λ1 )x1 x1∗
I+ ∗ 0
B0 −T 0 (λ1 )T (λ1 )+ B0 − ∗ 0
x1 T (λ1 )x1
x1 T (λ1 )x1
I−
Inexact variant of RESINVIT analyzed by [Szyld/Xue’2013].
PINVIT (Preconditioned inverse iteration)
for j = 0, 1, 2, . . .
vj+1 = vj + P −1 vj∗ Avj I − A)vj
Normalize vj+1 .
end for
Elegant convergence analyses by [D’yakonov/Orekhov’1980],
[Knyazev’1998], [Neymeyr’2001], [Knyazev/Neymeyr’2003], . . .:
I
Setting: symmetric positive definite A and aim at smallest
eigenvalue λ1 .
I
Neymeyr’s analysis relies on mini-dimensional analysis for
Rayleigh quotient.
I
Allow to conclude mesh-independent convergence rates.
PINVIT (Preconditioned inverse iteration)
for j = 0, 1, 2, . . .
vj+1 = vj + P −1 vj∗ Avj I − A)vj
Normalize vj+1 .
end for
Elegant convergence analyses by [D’yakonov/Orekhov’1980],
[Knyazev’1998], [Neymeyr’2001], [Knyazev/Neymeyr’2003], . . .:
I
Setting: symmetric positive definite A and aim at smallest
eigenvalue λ1 .
I
Neymeyr’s analysis relies on mini-dimensional analysis for
Rayleigh quotient.
I
Allow to conclude mesh-independent convergence rates.
Difficult to extend this analysis to
general nonlinear eigenvalue problems.
Instead: Extend poor men’s approach by interpreting PINVIT as
perturbed inverse iteration.
Rayleigh functionals
Back to nonlinear Hermitian T (λ).
Assume that scalar nonlinear equation
x ∗ T (λ)x = 0
admits unique solution λ ∈ Ω for every x in open set D ∈ Cn .
Defines Rayleigh functional
ρ : D → Ω,
x 7→ λ.
Additional assumption:
x ∗ T 0 (ρ(x))x > 0
I
I
∀x ∈ D.
Allows to carry over variational principles [Hadeler’1968],
[Voss/Werner’1982], . . ..
For example:
λ1 = inf ρ(x).
x∈D
−T (σ) is spd (on D) for any σ < λ1 .
Rayleigh functional for model problem
T (λ) = λM − K0 − φ(λ)en enT .
with spd M, K0 and φ(λ) =
I
ληm
λ−η .
x ∗ T (λ)x = 0 has exactly one solution in ]η, ∞] for every
x ∈ Cn \ {0}.
∃ Rayleigh functional for Ω =]η, ∞] and D = Cn \ {0}.
One step of residual inverse iteration
RESINVIT with Rayleigh functional ρ:
v + = v + P −1 T ρ(v ) v
for some spd preconditioner P.
Ignore scaling: will work with angles.
What is the right geometry?
Geometry induced by P
RESINVIT with Rayleigh functional ρ:
v + = v + P −1 T ρ(v ) v
for some spd preconditioner P.
I
Geometry induced by P:
I
hv , wiP := v ∗ Pw,
√
and induced norm kv kP := v ∗ Pv .
Angle φP (v , x1 ) between v and x1 :
cos φP (v , x1 ) :=
I
Rehv , x1 iP
kv kP kwkP
Given P-orthogonal decomposition
v = v1 + v⊥ ,
v1 ∈ span{x1 },
hv1 , v⊥ iP = 0,
tan φP (v , x1 ) = kv⊥ kP /kv1 kP .
Convergence result
Theorem [Effenberger/K.’2013].
tan φP (v + , x1 ) ≤ γ · tan φP (v , x1 ) + O(ε2 )
with ε = tan φP (v , x1 ) and convergence factor
z ∗ T (λ1 )z γ = max 1 +
.
z6=0
z ∗ Pz hz,x i =0
1 P
Proof based on expansion T (ρ(v )) = T (λ1 ) + O(ε2 ).
I Linear case T (λ) = λI − A with exact preconditioner P = A
γ =1+
I
λ1 − λ2
λ1
=
λ2
λ2
Sufficient condition for convergence: ∃0 ≤ γ̃ < 1 such that
(1 − γ̃)z ∗ Pz ≤ −z ∗ T (λ1 )z ≤ (1 + γ̃)z ∗ Pz
Satisfied, e.g., by multigrid for −T (σ) with
σ sufficiently close to λ1 .
∀z ⊥P x1 .
Summary
RESINVIT for Hermitian T (λ) with preconditioner P ≈ −T (σ) has
asymptotic linear convergence rate
z ∗ T (λ1 )z γ = max 1 +
.
z6=0
z ∗ Pz hz,x i =0
1 P
I
For P = −T (σ): γ < 1 if λ1 simple eigenvalue and σ ≈ λ1
I
Aim:
Show that γ stays away from 1 as h → 0 for discretized PDE.
Function space setting
I
I
Hilbert spaces V and H with dense and compact embedding
V ,→ H (think of H 1 and L2 )
Variational formulation of NLEVP:
Find (u, λ) ∈ V × Ω such that
a(u, v , λ) = 0 ∀v ∈ V .
I
with bounded bilinear form a(·, ·, λ) depending holomorphically
on λ.
Fix λ and consider linear eigenvalue problem associated with
ā(·, ·, λ) := −a(·, ·, λ):
Find (u, µ) ∈ V × D such that
ā(u, v , λ) = µhu, v iH ∀v ∈ V .
Assume
∃c, s
I
such that ā(v , v , λ) ≥ ckv k2V − skv k2H
∀v ∈ V .
Modification of setting considered by [Voss’2013]:
Existence of Rayleigh functional
variational principles for
eigenvalues.
Function space setting for model problem
For model problem V = H 1 (]0, 1[), H = L2 (]0, 1[), and
Z
Z
a(u, v , λ) =
∇u · ∇v dx + ϕ(λ)u(1)v (1) − λ uv dx.
Ω
all assumptions are satisfied.
Ω
Further assumptions
I
0 = µ1 (λ1 ) < µ2 (λ1 ) ≤ µ3 (λ1 ) ≤ · · ·
I
a bounded on V × V
∂
∂λ a
I ∂ a
∂λ
I
bounded on H × H
Lipschitz continuous wrt λ in neighbourhood of λ1
Attaining mesh-independent convergence rate
Outline of proof:
1. For eigenpair (λ1 , u1 ) choose σ < λ1 sufficiently close to λ1
(shifted) ā(·, ·, σ) is elliptic on V .
2. Assumptions imply existence of 0 ≤ δ < 1 such that
(1 − δ)ā(v , v , σ) ≤ ā(v , v , λ1 ) ≤ (1 + δ)ā(v , v , σ),
(1)
holds for all v 6= 0 with a(u1 , v , σ) = 0.
3. Show that (1) holds asymptotically (h → 0) for
(1 − δ)āh (v , v , σ) ≤ āh (v , v , λ1,h ) ≤ (1 + δ)āh (v , v , σ),
(2)
with ah (σ), ah (λ1,h ) obtained by Galerkin projection on Vh ⊂ V .
Main problem: λ1,h is the eigenvalue of the discrete problem!
4. Show that (1) remains true when ah (σ) is replaced by spectrally
equivalent preconditioner P.
Establishes h-independent convergence rate δ.
Conclusions
I
Proposed and analyzed preconditioned variant of RESINVIT for
nonlinear eigenvalue problems admitting a Rayleigh functional.
I
Surrogate for understanding convergence of more advanced
algorithms, e.g., nonlinear Arnoldi.
I
Analysis requires preconditioner of T (σ) with σ sufficiently close
to λ1 . In practice: Not a severe limitation.
I
Ongoing work (with A. Miedlar): Adaptive FEM.
References:
C. Effenberger, D. K. On the convergence of the residual inverse iteration for
nonlinear eigenvalue problems admitting a Rayleigh functional. Preprint
available from http://anchp.epfl.ch.
C. Effenberger. Robust solution methods for nonlinear eigenvalue problems.
PhD thesis, EPFL, 2013.
MHG70 @ EPFL, 03.10.2014
c
Alain
Herzog
http://anchp.epfl.ch/mhg70
Speakers: P. Arbenz, J.-P. Berrut, M. Hochbruck, V. Mehrmann, M.
Rozloznik, Th. Schmelzer, Z. Strakoš, P. Van Dooren, J.-P. Zemke.