On the Eigenvalues of a Class
of Saddle Point Matrices
V. Simoncini
Dipartimento di Matematica
Università di Bologna
[email protected]
. – p.1/22
Thanks to
Mario Arioli, RAL, UK
Michele Benzi, Emory University (GA)
Ilaria Perugia, Università di Pavia
Miro Rozloznik, Institute of Computer Science, Academy of
Science, Prague
. – p.2/22
Application problems
• Computational Fluid Dynamics
• Elasticity problems
• Mixed (FE) formulations of II and IV order elliptic PDEs
• Linearly Constrained Programs
• Linear Regression in Statistics
• Weighted Least Squares (Image restoration)
• ... Survey: Benzi, Golub and Liesen, Acta Num 2005


A BT
B −C


u
v


=
f
g


. – p.3/22
Motivational Application
Constrained Quadratic minimization problem
1
minimize J(u) = hAu, ui − hf, ui
2
subject to Bu = g
A
n×n
spd, B m × n, m ≤ n full rank
⇓
Lagrange multipliers approach
Karush-Kuhn-Tucker (KKT) system
. – p.4/22
Spectral properties

• M=
A
BT
B
O


0 < λn ≤ · · · ≤ λ 1
eigs of A
0 < σm ≤ · · · ≤ σ1 sing. vals of B
. – p.5/22
Spectral properties

• M=
A
BT
B
O


0 < λn ≤ · · · ≤ λ 1
eigs of A
0 < σm ≤ · · · ≤ σ1 sing. vals of B
• (Rusten & Winther 1992) Λ(M) subset of
»
1
(λn −
2
–
q
q
1
2 )
λ2n + 4σ12 ), (λ1 − λ21 + 4σm
2
∪
»
1
λn , (λ1 +
2
–
q
2 )
λ21 + 4σm
. – p.5/22
Spectral properties

• M=
A
B
BT
−C


0 < λn ≤ · · · ≤ λ 1
eigs of A
0 < σm ≤ · · · ≤ σ1 sing. vals of B
• (Rusten & Winther 1992) Λ(M) subset of
»
1
(λn −
2
–
q
q
1
2 )
λ2n + 4σ12 ), (λ1 − λ21 + 4σm
2
∪
»
1
λn , (λ1 +
2
–
q
2 )
λ21 + 4σm
• (Silvester & Wathen 1994), 0 ≤ σm ≤ · · · ≤ σ1
q
λn − λmax (C) − (λn + λmax (C))2 + 4σ12
. – p.5/22
Spectral properties

• M=
A
B
BT
−C


0 < λn ≤ · · · ≤ λ 1
eigs of A
0 < σm ≤ · · · ≤ σ1 sing. vals of B
• (Rusten & Winther 1992) Λ(M) subset of
»
1
(λn −
2
–
q
q
1
2 )
λ2n + 4σ12 ), (λ1 − λ21 + 4σm
2
∪
»
1
λn , (λ1 +
2
–
q
2 )
λ21 + 4σm
• (Silvester & Wathen 1994), 0 ≤ σm ≤ · · · ≤ σ1
q
λn − λmax (C) − (λn + λmax (C))2 + 4σ12
More results for special cases (e.g., Perugia & S. 2000)
. – p.5/22
Block diagonal Preconditioner


e 0
A

P=
e
0 C
e≈A
A
sym. pos. def.
e ≈ BA−1 B T + C
C
Rusten Winther (1992), Silvester Wathen (1993-1994), Klawonn (1998)
Fischer Ramage Silvester Wathen (1998...), . . .
. – p.6/22
Block diagonal Preconditioner


e 0
A

P=
e
0 C
e≈A
A
sym. pos. def.
e ≈ BA−1 B T + C
C
Rusten Winther (1992), Silvester Wathen (1993-1994), Klawonn (1998)
Fischer Ramage Silvester Wathen (1998...), . . .
λ 6= 0 eigs of P
− 21
MP
− 21
,
λ ∈ [−a, −b] ∪ [c, d],
a, b, c, d > 0
. – p.6/22
Triangular preconditioner
A spd,

P=
e BT
A
0
e
−C


Bramble & Pasciak, Elman, Klawonn, Axelsson & Neytcheva, S. 2004
. – p.7/22
Triangular preconditioner
A spd,

P=
e BT
A
0
e
−C


Bramble & Pasciak, Elman, Klawonn, Axelsson & Neytcheva, S. 2004
Spectrum of MP −1 :
small complex cluster around 1 + real interval
. – p.7/22
Triangular preconditioner
A spd,

P=
e BT
A
0
e
−C


Bramble & Pasciak, Elman, Klawonn, Axelsson & Neytcheva, S. 2004
Spectrum of MP −1 :
small complex cluster around 1 + real interval
θ ∈ Λ(MP −1 )
q
e−1 )
=(θ) =
6 0 ⇒ |θ − 1| ≤ 1 − λmin (AA
More precisely:
e−1 ) ≥ 0)
(if 1 − λmin (AA
=(θ) = 0 ⇒ θ ∈ [χ1 , χ2 ] 3 1
. – p.7/22
Constraint Preconditioner
Q=
e B
A
B −C
T
Axelsson (1979), Ewing Lazarov Lu Vassilevski (1990), ... many more papers 1997 -
. – p.8/22
Constraint Preconditioner
Q=
e B
A
B −C
T
Axelsson (1979), Ewing Lazarov Lu Vassilevski (1990), ... many more papers 1997 -
λ 6= 0 eigs of MQ−1 ,
λ ∈ R+ ,
λ ∈ {1} ∪ [α0 , α1 ]
. – p.8/22
Constraint Preconditioner
Q=
e B
A
B −C
T
Axelsson (1979), Ewing Lazarov Lu Vassilevski (1990), ... many more papers 1997 -
λ 6= 0 eigs of MQ−1 ,
λ ∈ R+ ,
λ ∈ {1} ∪ [α0 , α1 ]
Remark: Bxk = 0 for all iterates xk (C = 0)
. – p.8/22
Constraint Preconditioner
Q=
e B
A
B −C
T
Axelsson (1979), Ewing Lazarov Lu Vassilevski (1990), ... many more papers 1997 -
λ 6= 0 eigs of MQ−1 ,
λ ∈ R+ ,
λ ∈ {1} ∪ [α0 , α1 ]
Remark: Bxk = 0 for all iterates xk (C = 0)
Q
−1
=
I −B
O
I
T
I
O
O −(BBT + C)−1
I O
−B I
e = I if prescaling used)
(A
. – p.8/22
Computational Considerations.
3D Magnetostatic problem
with H: Incomplete Cholesky fact.
H ≈ BB T + C
(ICT package, Saad & Chow)
. – p.9/22
Computational Considerations.
3D Magnetostatic problem
with H: Incomplete Cholesky fact.
H ≈ BB T + C
(ICT package, Saad & Chow)
Elapsed Time
size
1119
2208
4371
8622
22675
MA 27
QMR
b
Q Q(2)(it)
0.6
3.0
1.7(18)
2.3
11.7
3.1(18)
10.2
64.6
8.4(20)
83.4
466.0 18.3(29)
753.5 3745.5 63.2(45)
QMR
ILDLT(10)
0.7
1.5
5.2
31.0
246.0
. – p.9/22
Computational Considerations.
3D Magnetostatic problem
with H: Incomplete Cholesky fact.
H ≈ BB T + C
(ICT package, Saad & Chow)
Elapsed Time
size
1119
2208
4371
8622
22675
MA 27
QMR
b
Q Q(2)(it)
0.6
3.0
1.7(18)
2.3
11.7
3.1(18)
10.2
64.6
8.4(20)
83.4
466.0 18.3(29)
753.5 3745.5 63.2(45)
QMR
ILDLT(10)
0.7
QMR
P
4.0
b
P(2)(it)
2.0
1.5
15.2
4.9
5.2
73.9
11.0
31.0
510.1
24.3
246.0
4161.4
128.2
. – p.9/22
Spectrum of perturbed problem
3D Magnetostatic problem
(a)
0.5
0.4
0.3
0.2
0.1
0
−0.1
−0.2
−0.3
−0.4
−0.5
0
0.5
1
1.5
2
2.5
3
3.5
. – p.10/22
Spectrum of perturbed problem
3D Magnetostatic problem
(a)
(d)
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0
0
−0.1
−0.1
−0.2
−0.2
−0.3
−0.3
−0.4
−0.4
−0.5
0
0.5
1
1.5
2
2.5
3
3.5
−0.5
0
0.5
1
1.5
k(BB T + C) − Hk∞ ≈ 2 · 10−1 kBB T + Ck∞
2
2.5
3
3.5
. – p.10/22
Eigenvalue problem
2
b=4
Q
I
O
B
I
32
54
2
4
I
O
O
−H
A
BT
B
O
32
54
32
54
I
BT
O
I
x
y
3
3
5,
2
b4
5 = λQ
H ≈ BB T ,
x
y
C=O
3
5
. – p.11/22
Eigenvalue problem
2
b=4
Q
I
O
B
I
32
54
2
4
I
O
O
−H
A
BT
B
O
32
54
32
54
I
BT
O
I
x
y
3
5,
H ≈ BB T ,
2
3
x
b4
5 = λQ
y
C=O
3
5
can be written as
2
4
I
O
−B
I
2
4
32
54
A
BT
B
O
32
54
I
−B T
O
I
A
(I − A)B T
B(I − A)
−B(2I − A)B T
32
54
32
54
u
v
3
u
v
2
3
I
O
O
−H
I
O
O
−H
32
5 = λ4
2
5 = λ4
54
32
54
u
v
u
v
3
5
3
5
. – p.11/22
A “different” perspective


A
BT
−B
C


u
v


=
f
−g


M− x = d
Polyak 1970, ..., Wathen & Fischer & Silvester 1995, Fischer & Ramage & Silvester & Wathen 1998,
Fischer & Peherstorfer 2001, Bai & Golub & Ng 2003, Sidi 2003, Benzi & Gander & Golub 2003,
Benzi & Golub 2004, S. & Benzi 2004, ...
. – p.12/22
A “different” perspective


A
BT
−B
C


u
v


=
f
−g


M− x = d
Polyak 1970, ..., Wathen & Fischer & Silvester 1995, Fischer & Ramage & Silvester & Wathen 1998,
Fischer & Peherstorfer 2001, Bai & Golub & Ng 2003, Sidi 2003, Benzi & Gander & Golub 2003,
Benzi & Golub 2004, S. & Benzi 2004, ...
• M positive real
⇒
Λ(M− ) in
C+
. – p.12/22
A “different” perspective


A
BT
−B
C


u
v


=
f
−g


M− x = d
Polyak 1970, ..., Wathen & Fischer & Silvester 1995, Fischer & Ramage & Silvester & Wathen 1998,
Fischer & Peherstorfer 2001, Bai & Golub & Ng 2003, Sidi 2003, Benzi & Gander & Golub 2003,
Benzi & Golub 2004, S. & Benzi 2004, ...
• M positive real
⇒
Λ(M− ) in
C+
• More refined spectral information possible
. – p.12/22
A “different” perspective


A
BT
−B
C


u
v


=
f
−g


M− x = d
Polyak 1970, ..., Wathen & Fischer & Silvester 1995, Fischer & Ramage & Silvester & Wathen 1998,
Fischer & Peherstorfer 2001, Bai & Golub & Ng 2003, Sidi 2003, Benzi & Gander & Golub 2003,
Benzi & Golub 2004, S. & Benzi 2004, ...
• M positive real
⇒
Λ(M− ) in
C+
• More refined spectral information possible
• New classes of preconditioners
. – p.12/22
A “different” perspective


A
BT
−B
C


u
v


=
f
−g


M− x = d
Polyak 1970, ..., Wathen & Fischer & Silvester 1995, Fischer & Ramage & Silvester & Wathen 1998,
Fischer & Peherstorfer 2001, Bai & Golub & Ng 2003, Sidi 2003, Benzi & Gander & Golub 2003,
Benzi & Golub 2004, S. & Benzi 2004, ...
• M positive real
⇒
Λ(M− ) in
C+
• More refined spectral information possible
• New classes of preconditioners
• General framework for spectral analysis of some
indefinite preconditioners
Benzi & Simoncini (tr 2005)
. – p.12/22
Spectral properties of M−

M− = 
A
A
BT
−B
C
n × n sym. semidef. matrix,


B
m×n,m≤n
M− has at least n − m real eigenvalues
Detailed analysis for A = ηI, C = O in:
? Fischer & Ramage & Silvester & Wathen 1998
?
Fischer & Peherstorfer 2001
. – p.13/22
Reality condition
Let A be spd and C = βI. If
λmin (A + βI) ≥ 4 λmax (B T A−1 B + βI),
then all eigenvalues of M− are real (and positive)
. – p.14/22
Reality condition
Let A be spd and C = βI. If
λmin (A + βI) ≥ 4 λmax (B T A−1 B + βI),
then all eigenvalues of M− are real (and positive)
——————————–
The condition is sufficient, but not necessary:


1
0 0

 2
λmin (A) = 12


M− =  0 3 1  ,


λmax (BA−1 B) =
0 −1 0
1
3
has real spectrum, but does not satisfy the condition
. – p.14/22
The (steady-state) Stokes problem



 −∆u + ∇ p = f in Ω
∇·u=0
in Ω



Bu = g
on Γ .
⇒
M±
. – p.15/22
The (steady-state) Stokes problem



 −∆u + ∇ p = f in Ω
∇·u=0
in Ω



Bu = g
on Γ .
⇒
M±
Numerical evidence: eigs of M− are all real and positive
(for several discretizations and b.c.)
. – p.15/22
The (steady-state) Stokes problem



 −∆u + ∇ p = f in Ω
∇·u=0
in Ω



Bu = g
on Γ .
⇒
M±
Numerical evidence: eigs of M− are all real and positive
(for several discretizations and b.c.)
Explanation:
Ω = [0, 1] × [0, 1], Dirichlet b.c. div-stable discr.
λmin (A) ≈ 2π 2 ≈ 19,
λmax (BA−1 B T ) ≈ 1
⇒ λmin (A) > 4λmax (BA−1 B T )
. – p.15/22
(In)definite inner product

M− = 
A
BT
−B
C



J =
I
O
O −I


J indefinite
M− is J-sym.
. – p.16/22
(In)definite inner product

M− = 
A
BT
−B
C



J =
I
O
O −I


J indefinite
M− is J-sym.
If Λ(M− ) ⊂ R+ , there exists a nonstandard inner product
with which M− is spd (and diag.ble)
. – p.16/22
(In)definite inner product

M− = 
A
BT
−B
C



J =
I
O
O −I


J indefinite
M− is J-sym.
If Λ(M− ) ⊂ R+ , there exists a nonstandard inner product
with which M− is spd (and diag.ble)

G=
A − γI B T
B
γI

,
M− G = GMT−
. – p.16/22
(In)definite inner product

M− = 
A
BT
−B
C



J =
I
O
O −I


J indefinite
M− is J-sym.
If Λ(M− ) ⊂ R+ , there exists a nonstandard inner product
with which M− is spd (and diag.ble)

G=
A − γI B T
B
γI

,
M− G = GMT−
If γ = 12 λmin (A), λmin (A) > 4λmax (BA−1 B T ) (with C = O)
⇒
G is spd
⇒
G defines an inner product for M−
. – p.16/22
Hermitian Skew-Hermitian
Preconditioning

M− = 
A
BT
−B
C


 = 
=
A O
O C
H


+
+
O
BT
−B
O
S


. – p.17/22
Hermitian Skew-Hermitian
Preconditioning

M− = 
A
BT
−B
C


 = 
=
A O
O C
H


+
+
O
BT
−B
O
S


Use the preconditioner
1
(H + αI)(S + αI)
Rα =
2α
α ∈ R, α > 0
(Bai & Golub & Ng 2003), Benzi & Gander & Golub 2003, Benzi & Golub 2004, S. & Benzi 2004,
Benzi & Ng, 2004
Spectral bounds: Simoncini & Benzi 2004
. – p.17/22
Reality condition
Assume A is spd and C = 0. If
1
α ≤ λmin (A)
2
then all eigenvalues of R−1
α M− are real
(Simoncini & Benzi ’04)
. – p.18/22
Reality condition
Assume A is spd and C = 0. If
1
α ≤ λmin (A)
2
then all eigenvalues of R−1
α M− are real
(Simoncini & Benzi ’04)
But even more general:
If λmin (A) > 4λmax (BA−1 B T ) then
all eigenvalues are real for any α
(cf. Stokes problem)
. – p.18/22
Location of M− ’s spectrum
Let λ ∈ Λ(M− ), x = [u; v] e’vect., A spd
(cf. Sidi ’03, C = O)
? If =(λ) 6= 0 then
1
(λmin (A) + λmin (C)) ≤
2
1
<(λ) ≤ (λmax (A) + λmax (C))
2
|=(λ)| ≤ σmax (B).
. – p.19/22
Location of M− ’s spectrum
Let λ ∈ Λ(M− ), x = [u; v] e’vect., A spd
(cf. Sidi ’03, C = O)
? If =(λ) 6= 0 then
1
(λmin (A) + λmin (C)) ≤
2
1
<(λ) ≤ (λmax (A) + λmax (C))
2
|=(λ)| ≤ σmax (B).
? If =(λ) = 0 then for v 6= 0
2 min{λmin (A), λmin (C)} ≤ λ ≤ max{λmax (A), λmax (C)}
and for v = 0, λmin (A) ≤ λ ≤ λmax (A)
. – p.19/22
Location of M− ’s spectrum
Let λ ∈ Λ(M− ), x = [u; v] e’vect., A spd
(cf. Sidi ’03, C = O)
? If =(λ) 6= 0 then
1
(λmin (A) + λmin (C)) ≤
2
1
<(λ) ≤ (λmax (A) + λmax (C))
2
|=(λ)| ≤ σmax (B).
? If =(λ) = 0 then for v 6= 0
2 min{λmin (A), λmin (C)} ≤ λ ≤ max{λmax (A), λmax (C)}
and for v = 0, λmin (A) ≤ λ ≤ λmax (A)


2 0 1




λ1 = 1,
M− =  0 1 0 


−1 0 1
λ2,3
√
3
3
= ±i
.
2
2
. – p.19/22
Inexact Indefinite Precond.


A
(I − A)B T
B(I − A) −B(2I − A)B T


u
v


 = λ
I
O
O −H


u
v


. – p.20/22
Inexact Indefinite Precond.


A
(I − A)B T
B(I − A) −B(2I − A)B T
Eigenvalue bounds:
Let
? If =(λ) 6= 0 then
1
b ≤
(λmin (A) + λmin (C))
2


u
v


 = λ
I
O
O −H



u

v
b = B(2I − A)B T H −1
C
<(λ)
1
b
≤ (λmax (A) + λmax (C))
2
|=(λ)| ≤ σmax ((I − A)BH
− 12
).
? If =(λ) = 0 then
b ≤ λ ≤ max{λmax (A), λmax (C)}
b
2 min{λmin (A), λmin (C)}
. – p.20/22
Spectral bounds
2
bounding box of nonreal eigs
1.5
imaginary part of eigenvalues
1
0.5
0
a
a2
1
−0.5
−1
−1.5
−2
0
0.5
1
1.5
real part of eigenvalues
2
2.5
. – p.21/22
Final Considerations
• New framework increases understanding
. – p.22/22
Final Considerations
• New framework increases understanding
• But: in general eigenvector analysis may be
challenging
. – p.22/22
Final Considerations
• New framework increases understanding
• But: in general eigenvector analysis may be
challenging
• General non-Hermitian problem not fully understood
. – p.22/22
Final Considerations
• New framework increases understanding
• But: in general eigenvector analysis may be
challenging
• General non-Hermitian problem not fully understood
• Visit
http://www.dm.unibo.it/˜simoncin
. – p.22/22