TIES594 PDE-solvers
Lecture 8, 16.4.2015
Olli Mali
Saddle point problems
We make few remarks on the structure and origins of saddle point problems.
They are defined by functionals of two arguments (let’s say L(w, q)) and the
goal is to find a pair of arguments (let the solution be (u, p)) such that the
value is minimized with respect to the first one and maximized with respect
to the second one. Mathematically, this means that
L(u, q) ≤ L(u, p) ≤ L(w, p),
∀w, q.
(1)
Other way to write the inequality (which has certain requirements for L), is
to identify
L(u, p) = min max L(v, q) = max min L(v, q).
v
q
q
v
This min-max condition is of utmost importance in the related mathematical
analysis. Moreover, this defines a primal problem
J(u) = min J(v) = min max L(v, q)
v
v
q
and the corresponding dual problem
I ∗ (p) = max I ∗ (q) = max min L(v, q).
q
q
v
Problems of this type occur in energy minimization problems (J is the primal
energy and I is the dual energy), where the energy to minimize is convex
(there are no local minima).
Finite dimensional example (Quadratic programming)
Let us consider a minimization problem
min 1 xT Ax
x∈Rn 2
− bT x,
where A ∈ Rn×n and b ∈ Rn subject to a constraint
Bx = g,
(2)
where B ∈ Rm×n (m < n). Alternative way to formulate it as an unconstrained problem is to introduce Lagrange multiplier y and to define a
problem
min max L(x, y) := 21 xT Ax − bT x + yT (Bx − g).
x∈Rn y∈Rn
(3)
Clearly the minimum is the same and the condition (2) is satisfied. Necessary
conditions for the solution (pair (u, p)) are
∇x L(u, p) = Au + BT p − b = 0,
∇y L(u, p) = Bu − g = 0,
i.e.,
A BT
B 0
u
p
=
b
g
.
(4)
The structure of the linear system is the same as in mixed formulations.
Stokes problem
Stokes problem describes a (here stationary to begin with) flow of a fluid with
high viscosity, i.e., “stiff flow”. For example, ice flows (the slow ones, not
huge chunks falling to the ocean) in Antarctica can be modelled using Stokes
(search also for Laminar flow in YouTube). It consists of a momentum and
mass conservation law.
−∆u + ∇p = f ,
divu = 0,
where u ∈ Rd is the velocity field and


−∆u1


..
−∆u := 
.
.
−∆ud
Writing component wise yields
∂p
= f1 ,
∂x1
..
.
∂p
−∆ud +
= fd ,
∂xd
divu = 0,
−∆u1 +
Again, multiplying by test function and applying Gauss-Ostrogradski formula yields
Z
Z
Z
∂v1
∇u1 · ∇v1 dx − p
dx = f1 · v1 dx, ∀v1
∂x1
Ω
Ω
Ω
..
.
Z
Z
∇ud · ∇vd dx −
Ω
∂vd
p
dx =
∂xd
ZΩ
−
Z
fd · v1 dx,
∀vd
Ω
divuw dx = 0.
Ω
Remark. This is indeed a saddle point problem, it can be written as (after
summing the first d equations)
a(u, v) + b(v, p) = `(v),
b(u, q) = 0,
∀v
∀q,
where
a(u, v) :=
d Z
X
Z
∇uj · ∇vj dx,
b(v, p) := −
j=1 Ω
divvp dx,
Ω
and
Z
f · v dx.
`(v) :=
Ω
Galerkin approximation
Since we are dealing with a saddle point problem, in which the infsup condition has to be satisfied in order to obtain a stable method, elements of
u and p have to be approximated in different basis. For notational simplicity,
we select d = 2. We define
N
X
u1 =
c1k ψk
u2 =
p=
k=1
N
X
k=1
N
X
k=1
c2k ψk
c3k φk .
The coefficients are solved from a system
N
X
k=1
N
X
Z
Ω
Z
N
X
Z
∇ψk · ∇ψi dx −
Z
c1k
k=1
∂ψk
φi dx −
∂x1
Ω
∂ψi
dx =
φj
∂x1
c3k
Ω
Ω
N
X
Z
k=1
c2k
k=1
−
∇ψk · ∇ψi dx −
c1k
N
X
Ω
N
X
Z
∂ψi
dx =
φj
∂x2
∀j ∈ {1, . . . , N }
f2 · ψj dx,
∀j ∈ {1, . . . , N }
Z
Ω
∂ψk
φi dx = 0,
∂x2
c2k
k=1
f1 · ψj dx,
Ω
c3k
k=1
Z
∀i ∈ {1, . . . , M }.
Ω
Written in a matrix form it is

S
0
 0
S
B1 T B2 T
   
B1
c1
b1




B2
c2 = b2  ,
c3
0
0
where
Z
Z
∇ψk · ∇ψi dx,
Sik =
B1ik =
ZΩ
Ω
∂ψi
dx,
∂x1
Z
B2ik =
Ω
φk
∂ψi
dx,
∂x2
Ω
Z
f1 · ψi dx,
b1i =
φk
f2 · ψi dx.
b2i =
Ω
Uzawa algorithm “schematic derivation”
Uzawa algorithm is the classical algorithm for solving saddle point problems.
It has numerous variants, which typically try to improve it’s slow convergence. The Algorithm is based on the saddle point structure of he problem,
and we illustrate its derivation from a model problem (or any discrete saddle point problem) (4). Iterative method is a rule to obtain (uk+1 , pk+1 )
from (uk , pk ). The method is generated by two inequalities of the saddle
point problem (1). The necessary condition related to the minimization of
u provides
Auk+1 = b − BT pk .
(5)
The so-called “projection step” to obtain pk+1 is less intuitive to see. Consider the inequality for maximization of p, i.e., L(u, q) ≤ L(u, p), where L
is defined in (3). It yields
(q − p)T (Bu − g) ≤ 0,
∀q ∈ Rm
We can multiply by arbitrary positive parameter α,
(q − p)T (α(Bu − g)) ≤ 0,
∀q ∈ Rm
and add and subtract p to obtain
(q − p)T (p + α(Bu − g) − p) ≤ 0,
∀q ∈ Rm .
The motivation of these manipulations is to use following Lemma. It holds
also in abstract Hilbert spaces, but here we consider only the suitable finitedimensional analog. The proof is not at all complicated, few lines only, but
omitted here.
Lemma. Let Q be a convex, closed, nonempty subset of Rn , x ∈ Rn and
u ∈ Q. Then the following conditions are equivalent
(v − u)T (x − u) ≤ 0,
∀v ∈ Rn
and
u = ΠQ x,
where ΠQ is a projection from Rn to Q.
Applying the Lemma to our inequality (where Q is the whole space and
thus ΠQ is just identity) yields
p = p − α(Bu − g)
and we rewrite it as an iteration
pk+1 = pk − α(Buk+1 − g).
(6)
The Uzawa algorithm consists of repeating steps (5) and (6). It can be
proved that for suitable values of α (depending on eigenvalues of the Schur
complement of coefficient matrix in (4)) the Uzawa iteration converges.
Inexact Uzawa
In some papers one finds the “first step” of Uzawa written as (add and
subtract Auk )
uk+1 = uk + A−1 b − Auk − BT pk . .
This form seems to be most popular in inexact Uzawa approach, where A−1
is not computed exactly, but it approximated by some preconditioner.
Summary and remarks
• Saddle point problems generate finite dimensional problems of a particular form.
• The selection of basis in Galerkin approximation is less free, since the
applied basis must satisfy the infsup-condition.
• Solving the linear system emerging from a saddle point problem is an
active area of research. Different ways to accelerate Uzawa by augmentation, preconditioners, inexact solvers are vividly studied in the
literature. Essentially all topics of numerical linear algebra combined
with different variants of the main iteration are of interest.