Approximation Method for a Non-convex
Variational Model
Tieyong Zeng
Department of Mathematics,
Hong Kong Baptist University
Email: [email protected]
Joint work with: Yumei Huang, Michael Ng
SIAM IS10, Chicago
April 12-14, 2010
First
Previous
Next
Last
1
Outline
1. Introduction
2. Approximation
3. Iterative algorithm
4. Convergence analysis
5. Numerical simulation
6. Conclusions
SIAM IS10, Chicago
April 12-14, 2010
First
Previous
Next
Last
2
Background
Image denoising
• Gaussian noise
– Total variation model, wavelet shrinkage
– NL-means, K-SVD, BM3D
• Non-Gaussian noise
– Impulse noise
– Poisson noise
– Multiplicative noise
SIAM IS10, Chicago
April 12-14, 2010
First
Previous
Next
Last
3
Multiplicative noise
Noisy model:
f = un,
where f is noisy image, u is clean image and n is multiplicative noise.
• Variational approaches
–
–
–
–
Rudin-Lions-Osher
Aubert-Aujol
Shi-Osher
Huang-Wen-Ng
• Hybrid approach
– Durand-Fadili-Nikolova
SIAM IS10, Chicago
April 12-14, 2010
First
Previous
Next
Last
4
Aubert-Aujol model
Assumptions: Ω-image region, and n is of Gamma distribution.
The AA model reads:
Z inf
u∈S(Ω)
log u +
Ω
f
u
Z
|Du|,
+λ
(1)
Ω
where S(Ω) := {u ∈ BV (Ω), u > 0}, f > 0 in L∞(Ω) and λ > 0.
• Discussion
– Non-convex
– Important
– How to solve?
SIAM IS10, Chicago
April 12-14, 2010
First
Previous
Next
Last
5
How to handle non-convex?
Huang-Ng-Wen (SIIMS, 2009) consider:
Z inf
log u +
u∈{u>0, log u∈BV (Ω)}
Ω
f
u
Z
|D log u|.
+λ
(2)
Ω
Using logarithm transformation, it can be transferred to a convex problem:
Z
inf
w∈BV (Ω)
SIAM IS10, Chicago
−w
w + fe
+λ
Ω
April 12-14, 2010
Z
|Dw|.
(3)
Ω
First
Previous
Next
Last
6
Our approach
Restricting u in:
B(Ω) := {u > 0, u ∈ W 1,1(Ω)},
where W 1,1(Ω) is Sobolev space.
For u ∈ B(Ω), consider v = log u, then
Du = ∇u = ∇ev = ev ∇v.
Replacing AA by:
Z
inf
v∈{v, ev ∈W 1,1 (Ω)}
−v
v + fe
+λ
Ω
Z
ev |∇v|,
(4)
Ω
and let u = ev as result.
SIAM IS10, Chicago
April 12-14, 2010
First
Previous
Next
Last
7
Discrete setting
The discrete version can be given as follows:
2
min E(v) ≡
v
n X
[v]i + [f ]ie−[v]i + λT Vv (v),
(5)
i=1
where
T Vv (u) :=
X
e
vj,k
|(∇u)j,k |2 =
1≤j,k≤n
SIAM IS10, Chicago
X
e
vj,k
q
|(∇u)xj,k |2 + |(∇u)yj,k |2.
1≤j,k≤n
April 12-14, 2010
First
Previous
Next
Last
8
Approximation
We study the following problem:
2
min J (v, w) ≡
v,w
n X
[v]i + [f ]ie−[v]i + α||v − w||22 + λT Vv (w),
(6)
i=1
where α > 0.
• Advantage
– Easier to handle
– Let α → +∞, solution converges to global minimizer
SIAM IS10, Chicago
April 12-14, 2010
First
Previous
Next
Last
9
Convergence
Proposition 1. The function J (v, w) is coercive.
Denote v∗-global minimizer of E(v), (vα, wα)-minimizer of J (v, w).
Proposition 2. Let α tends to +∞, then in the sense of objective function, the
minimizer of J (v, w) converges to global minimizer of E(v). Moreover, we have:
∗
∗
E(v ) ≤ E(vα) ≤ E(v ) + λ sup e
vα
√
· min{ 8nkvα − wαk2, 4kvα − wαk1}. (7)
In practice, α small is ok.
SIAM IS10, Chicago
April 12-14, 2010
First
Previous
Next
Last
10
The iterative algorithm
Object:
2
min J (v, w) ≡
v,w
n X
[v]i + [f ]ie−[v]i + α||v − w||22 + λT Vv (w).
i=1
• Alternating minimization algorithm
– fix v, update w
– fix w, update v
We thus obtain a sequences:
v(0), w(0), v(1), w(1), v(2), w(2), . . . , v(m), w(m), . . . .
SIAM IS10, Chicago
April 12-14, 2010
First
Previous
Next
Last
11
Details
It is just:






R(w(m−1)) :=









2
v
(m)
= argminv
n
X
[v]i + [f ]ie
−[v]i
+
i=1
(8)
α||v − w(m−1)||22
S(v(m)) := w(m) = argminw αkv(m) − wk22 + λT Vv(m) (w)
• first step, Newton method
• second step, weighted Chambolle algorithm
SIAM IS10, Chicago
April 12-14, 2010
First
Previous
Next
Last
12
The convergence analysis
Lemma 1.
The operator R on a bounded region {w : kwk∞ ≤ A} is a
contraction map, i.e., we have:
kR (w1) − R (w2) k ≤ q kw1 − w2k2 ,
−A
where q = 1 + e2α · inf i[f ]i
−1
∈ (0, 1).
(0)
N2
Theorem 1. For any initial guess w ∈ R , suppose w(m) is generated by the
alternating minimization algorithm, then w(m) converges q-linearly to the
minimizer of J .
SIAM IS10, Chicago
April 12-14, 2010
First
Previous
Next
Last
13
Outline
1. Introduction
2. Approximation
3. Iterative algorithm
4. Convergence analysis
5. Numerical simulation
6. Conclusions
SIAM IS10, Chicago
April 12-14, 2010
First
Previous
Next
Last
14
Gamma noise
Consider:
f = un,
where the Gamma noise n is of mean one with probability density function
 L L−1
 L n
e−Ln, n > 0,
p(n; L) =
Γ(L)

0,
n ≤ 0.
The level of noise is determined by L and the variance equals 1/L.
SIAM IS10, Chicago
April 12-14, 2010
First
Previous
Next
Last
15
Experiment I
1-D case:
Clear u:
u = [0.2 0.2 0.2 0.2 0.2 0.6 0.6 0.6 0.6 0.6].
(Quasi)-global solution of AA model (by fmin extensively)
ũ = [0.2120 0.2120 0.2120 0.2120 0.2120 0.5876 0.5876 0.5876 0.5876 0.5876].
Our model:
e 800 = [0.2141 0.2141 0.2141 0.2141 0.2141 0.5887 0.5887 0.5887 0.5887 0.5887].
u
SIAM IS10, Chicago
April 12-14, 2010
First
Previous
Next
Last
16
Experiment II
Figure 1: Barbara: middle-noisy image with L = 10; right-restored image.
SIAM IS10, Chicago
April 12-14, 2010
First
Previous
Next Last
17
Experiment III
Figure 2: Boat: middle-noisy image with L = 10; right-restored image.
SIAM IS10, Chicago
April 12-14, 2010
First
Previous
Next Last
18
Experiment-IV
Images
(initial)
“Babara”
L = 33
“Babara”
L = 10
“Boat”
L = 10
“Cameraman”
L = 10
Proposed
recorded mean
26.0
26.1
(24.2)
(30.2)
23.0
23.1
(55.6)
(51.3)
25.3
25.4
(53.4)
(49.9)
26.5
26.5
(67.5)
(56.9)
“HNW”
recorded
mean
25.9
25.9
(31.0)
(51.5)
23.3
23.3
(75.6)
(94.8)
25.4
25.2
(72.3)
(97.5)
26.5
26.3
(73.6)
(119.6)
“AA”
recorded
mean
25.1
25.9
(588.2) (602.2)
22.2
22.9
(620.6) (594.3)
25.3
21.1
(624.2) (568.3)
25.6
18.8
(587.2) (692.6)
Table 1: The recovered PSNRs and computing time for various images, the upper
value is PSNR, the lower one is computing time (seconds).
SIAM IS10, Chicago
April 12-14, 2010
First
Previous
Next
Last
19
Conclusion
Multiplicative noise removal
• AA model
–
–
–
–
approximation
alternating minimization algorithm
linear convergence
some comparison
• Further works
– extended to other non-convex models
– extended to deblurring
SIAM IS10, Chicago
April 12-14, 2010
First
Previous
Next
Last
20