Chinese Journal of Electronics
Vol.22, No.4, Oct. 2013
Iterative Regularization and Nonlinear Inverse
Scale Space in Curvelet-type Decomposition
Spaces∗
LI Min, XU Chen and SUN Xiaoli
(College of Mathematics and Computational Science, Shenzhen University, Shenzhen 518060, China)
Abstract — In this paper we generalize the iterative
regularization method and the inverse scale space method,
recently developed for wavelet-based image restoration, to
curvelet-type decomposition spaces setting. We obtain the
result that minimzer of the new model can be derived as
curvelet firm shrinkage with curvelet-type weight, which
is dynamically changing in the iteration(CDS-IRM). And
we obtain a new class of nonlinear inverse scale spaces flow
which is dependent on Curvelet-type decomposition scale
and smooth order(CDS-ISS). Numerical experiments indicate that the proposed methods are very efficient for denoising.
Key words — Denoising, Iterative regularization, Inverse scale space, Curvelet-type decomposition, Shrinkage.
I. Introduction
Variational regularization has been extremely successful
in a wide variety of image restoration problems[1−5,16,18] . Recently, iterative regularization method based on the Bregman
distances is a new task introduced in variational-based image restoration. It has been first formulated in theory by Osher et al.[10] . Later the discrete iterative regulariztion method
was generalized successfully to a time-continuous inverse scale
space formulation in Refs.[11, 12].
But one drawback is that the iterative regularization
method and inverse scale space lead to numerically intensive schemes. And then Xu and Osher obtained the discrete Wavelet-based Iterative regularization method and Inverse scale space method (abbreviated as W-IRM and W-ISS).
However, the discrete wavelet often creates artifacts in edge.
Thus this problem might be circumvented by high redundancy.
One way to achieve redundancy is given by translation invariant representations. The Ref.[6] gives the Iterative regularization method and Inverse scale space based on Translation
invariant wavelet (abbreviated as TIW-IRM and TIW-ISS).
It is well known that wavelet fails to efficiently represent
objects with edges for not fully taking advantage of the geometry of the underlying edge curve in 2D images. Hence the
Ref.[7] proposes a curvelet based treatment. Curvelets appear
to be quite similar to the curvelet-type decomposition frames,
which are defined by Borup and Nielsen in Ref.[8]. Then a nat2
ural family of sparseness spaces Gα
p,q (R )(0 < p ≤ ∞, 0 < q <
∞, α ∈ R) are obtained, which are associated to the curvelettype frames. And the curvelet-type frames are quite similar
to the second generation of curvelets. Comparing two types
of frames on a quantitative basis, the sparseness spaces associated with the two types of frames are the same. And both
are given by specific decomposition smoothness spaces. Thus,
whenever a function has a sparse curvelet expansion, the function will have an equally sparse curvelet-type decomposition
frame expansion and vice versa, which offers an effective guarantee for characterizing the curvelet-type decomposition with
curvelets.
We are encouraged by the fact that the curvelets-type decomposition spaces of interest can be characterized by means
of curvelet coefficients. Moreover, it is well-known that
the curvelet-type decomposition spaces are relative to Besov
α
(R2 ) (0 < p ≤ ∞, 0 < q < ∞, α > 0)[8] . That is,
spaces Bp,q
α+β
2
(R2 ) → Gα
Bp,q
p,q (R ),
2
α−β
Gα
(R2 )
p,q (R ) → Bp,q
(1)
where β = K/q, K = 1/2, β = K(max(1, 1/p) − min(1, 1/q)).
Here we are interested in the case p = q. Moreover, we limit
ourselves to the case p = 1, i.e. we aim at incorporating
2
the curvelet-type decomposition spaces Gα
1,1 (R ) (α > 0). Altogether this leads the somewhat different iterative regularization method and nonlinear inverse scale space based on
2
curvelet-type decomposition spaces Gα
1,1 (R ) which are given
in the Section II and the Section III (abbreviated as CDS-IRM
and CDS-ISS). The Section IV is devoted to the comparison of
denoising performance using the proposed methods with some
method available. Finally, the conclusion is reported in the
Section V.
II. CDS-IRM: Iterative Regularization
Method in Curvelet-type Decomposition
2
Spaces Gα
1,1 (R )
∗ Manuscript Received Nov. 2012; Accepted Apr. 2013. This work is supported by the National Natural Science Foundation of China
(No.11101292, No.61070087, No.61001183).
Iterative Regularization and Nonlinear Inverse Scale Space in Curvelet-type Decomposition Spaces
1. Preliminary
It is well-known that the classical variational denoising
model is the method developed by Rudin-Osher-Fatemi[9]
(ROF model), i.e.
λ
(2)
J(u) + H(f, u)
u = arg min
u∈BV (Ω)
2
where J(u) = |u|BV = Ω |∇u|dx is referred to as Total variation (TV) semi-norm of a Bounded variation function space
(BV), H(f, u) = f −u22 is a fidelity term in L2 (Ω), and λ > 0
is a balancing parameter.
Despite the advantage of ROF model for well preserving
edges and contours, it still suffers from some inherent drawbacks that are loss of small scale features and staircasing
effect[13] . Thus Osher incorporated the Bregman distance into
ROF model[10] . This leads to the TV-based iterative regularization method, i.e.
λ
(3)
u = arg min D(u, v) + H(f, u)
u
2
where the Bregman distance D(u, v) = J(u)−J(v)−u−v, p
,
p ∈ ∂J(v), ·
is the usual L2 inner product.
Instead of numerically intensive scheme of the minimization of Eq.(3), Xu poposed a wavelet based treatment, i.e.
replacing the TV constraint |u|BV by a Besov semi-norm
|u|B1 (Ω) [14,15] . Then W-IRM is the following model
1,1
λ
u = arg min |u|B1 + H(f, u)
1,1
u
2
(4)
By Eq.(1), the embedding relation between Besov space
and curvelets-type decomposition space is G11,1 (Ω) →
1/2
1
B1,1
(Ω) → G1,1 (Ω). Since Gα
1,1 (α > 0) can be describe in
terms of curvelet coefficients, we naturally propose to replace
1
α
(Ω) by Gα
B1,1
1,1 (Ω). Moreover, G1,1 (Ω) cover a wide rang of
decomposition spaces that are optimally sparse in the class
of cartoonlike images. Altogether this leads to the interested
variational problem
λ
H(f,
u)
(5)
+
u = arg min |u|Gα
1,1
u
2
2.
Iterative regularization applied to curvelet
shrinkage
Using the equivalence relation between curvelets-type decomposition spaces and curvelet coefficients[8] , we have
3 1 1
−
jp α+
2 2 p
≈
2
|u|Gα
|uμ |p , 0 < p < ∞ (6)
p,p
μ=(j,l,k)∈Λ
where ũ = {uμ } = {u, φμ } denotes the curvelets coefficients and Λ is all indices of the curvelet coefficients. Since
0
(Ω), one has L2 (Ω) = G02,2 (Ω). Applying the
L2 (Ω) = B2,2
characterization of curvelets Eq.(6), we have
3
j α−
4
|u|Gα
≈
2
|ũμ |,
1,1
μ=(j,l,k)∈Λ
H(f, u) =f − u2L2 = f˜ − ũ2L2 ≈
μ=(j,l,k)∈Λ
|f˜μ − ũμ |
(7)
703
We then approximate Eq.(5) by using the following curveletbased mtheod:
3
j α−
λ
4
ũ = arg min
2
|f˜μ −ũμ |2
|ũμ |+
ũ
2
μ=(j,l,k)∈Λ
μ=(j,l,k)∈Λ
(8)
Following the idea of the iterative regularization scheme,
the generalized Bregman distance associated with
3
j α−
4
2
|ũμ |
J(ũ) =
μ=(j,l,k)∈Λ
in Eq.(8) and p̃ ∈ ∂J(ṽ) can be denoted as
DJp̃ (ũ, ṽ) = J(ũ) − J(ṽ) − ũ − ṽ, p̃
And iterative regularization method based on curvelet-type
decomposition can be denoted as
(k−1)
λ
(ũ, ũ(k−1) ) + f˜ − ũ2L2
ũ(k) =arg min DJp̃
(9)
ũ
2
p̃(k) =p̃(k−1) + λ(f˜ − ũ(k) )
(0)
(0)
(10)
(k)
(k)
with k ≥ 1, ũ = 0, p̃ = 0, p̃ ∈ ∂J(ũ ). We call this as
CDS-IRM.
If we denote ṽ (k) = p̃(k) /λ, then ṽ (0) = 0, by plugging Bregman distance into Eq.(9) and dropping the constant
terms from the minimization, after some simplification we can
rewrite Eq.(9) as the sequence based functional
λ
ũ(k) =arg min J(ũ) + (f˜ + ṽ (k−1) ) − ũ2L2
ũ
2
3
2j(α− 4 ) |ũμ |
=arg min
ũ
+
λ
2
μ=(j,l,k)∈Λ
|(f˜μ + ṽμ(k−1) ) − ũμ |2
(11)
μ=(j,l,k)∈Λ
Using the separable property for all μ ∈ Λ, the minimizer of
Eq.(11) is the famous soft shrinkage function
ũ(k)
μ = S
j(α−
2
˜ + ṽμ(k−1) )
(fμ
3
) 1
4
λ
(12)
(0)
where k ≥ 1, ṽμ = 0 and
ṽμ(k) = f˜μ + ṽμ(k−1) − ũ(k)
μ
Plugging Eq.(12) into Eq.(13), we can rewrite
as
ũ(k)
μ =
(13)
(k)
ũμ
⎧
3
2j(α− 4 )
⎪
⎪
˜
˜
⎪
fμ ,
if |fμ | ≥
⎪
⎪
(k − 1)λ
⎪
⎪
⎪
⎪
3 1
⎪
j(α− )
⎪
˜
4
sign(f˜μ ),
⎪
⎨ kfμ − 2
λ
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎩
3
if
3
2j(α− 4 )
2j(α− 4 )
≤ |f˜μ | <
kλ
(k − 1)λ
(k)
(14)
3
2j(α− 4 )
0,
if |f˜μ | <
kλ
(k)
(k)
˜
and sign(ũμ ) = sign(fμ ), if ũμ = 0;
⎧
3
3
⎨ 2j(α− 4 ) sign(f˜μ ), if |f˜μ | > 2j(α− 4 ) /(kλ)
(k)
ṽμ =
3
⎩ ˜
kfμ ,
if|f˜μ | ≤ 2j(α− 4 ) /(kλ)
and sign(ṽμ ) = sign(f˜μ ).
(k)
and ṽμ
(15)
Chinese Journal of Electronics
704
III. CDS-ISS: Inverse Scale Space Applied
To Curvelet-type Decomposition
Due to the singularity of ∂|ũμ | at the point ũμ = 0 in
2013
This is a nonlinear equation for ũμ which can be solved numerically. But a slight extra computational cost comes with
it as compared to Eq.(14). Thus, for each μ we can rewrite
Eq.(10) as
3
Eq.(11), we approximate F (ũμ ) = 2j(α− 4 ) |ũμ | as
3 Fe (ũμ ) = 2j(α− 4 ) ũ2μ + e
(k)
(16)
Thus, one obtains
ũ(0)
μ
(17)
Replacing F (ũμ ) by Fe (ũμ ) and taking the second term as
traditional L2 -norm in Eq.(11), we have
3 λ
(18)
ũμ = arg min 2j(α− 4 ) ũ2μ + e + (f˜μ − ũμ )2
ũμ
2
Then Euler-Lagrange equation of Eq.(18) is
=p̃(0)
μ
dp̃μ /dt = f˜μ − ũμ ,
k≥1
(20)
=0
ũμ (0) = 0
(21)
From Eq.(19), we obtain
3
3
dp̃μ /dũμ = 2j(α− 4 ) e/(ũ2μ + e) 2
(22)
Hence, one has the inverse scale space flow based on curvelettype decomposition (CDS-ISS) for each ũμ as follows
3
3
2j(α− 4 ) ũμ
+ λ(ũμ − f˜μ ) = 0
ũ2μ + e
=f˜μ − ũ(k)
μ ,
Set λ = Δt, kΔt → t, Eq.(20) becomes
3
2j(α− 4 ) ũμ
p̃μ = ∂Fe (ũμ ) = ũ2μ + e
(k−1)
p̃μ − p̃μ
λ
(19)
(ũ2μ + e) 2 ˜
dũμ
=
(fμ − ũμ ),
3
dt
2j(α− 4 ) e
ũμ (0) = 0
(23)
IV. Numerical Examples
Fig. 1. Original image and nosiy image. Left: MRI; Right: Boat
Fig. 2. Dneoised results from W IRM, W ISS, TIW IRM and TIW ISS
Fig. 3. Some dneoised results from CDS IRM and CDS ISS
In this section we give two numerical examples of denoising using Wavelet-based Iterative regularization method and Inverse scale
space flow (W IRM and W ISS), Iterative regularization method and Inverse scale space
flow based on Translation invariant wavelet
(TIW IRM and TIW ISS) and iterative regularization method and inverse scale space
flow in Curvelets-type decomposition spaces
(CDS IRM and CDS ISS) we introduce above.
We add Gaussian white noise with standard deviation σ = 30 to a clean image (see
Fig.1). We choose the signal-to-noise-ratio
(SN R(g, ω) = 10 log10 (g − ḡ2 /η − η̄2 ),
where η = ω − g) to define “optimal” results
numerically. And f − ũ2 ≈ σ is the stopping
criterion for iteration of all numerical schemes
(see Fig.4). In this section, we choose db3 basis(MRI image), coif4 basis(Boat image) and
level 3 for wavelet decomposition.
Fig.2 gives denoising results on MRI image from W IRM, W ISS, TIW IRM and
TIW ISS. Apparently, there are some artifacts
in numerical experiments. This is a common
defect of wavelet imaging. And SNRs and
numbers of iteration for the above old methods are denoted in Table 1 and Fig.4. Fig.3
gives denoising results depending on α from
CDS IRM and CDS ISS. Numerical experiments show that CDS IRM and CDS ISS can
decrease the artifacts. And the most of corresponding SNR is higher than that of Fig.2.
Iterative Regularization and Nonlinear Inverse Scale Space in Curvelet-type Decomposition Spaces
Images
σ = 30
Brain
6.2389
Boat
5.7305
Table 1. SNR results from different denoising methods
W IRM
W ISS
TIW IRM
TIW ISS
α
CDS IRM
0.8
13.8558
1
14.1868
1.2
14.1236
11.2862
11.4359
13.8705
13.4787
1.5
14.1059
1.8
14.0010
2
13.9421
2.5
13.7849
0.8
13.6748
1
14.1081
1.2
14.0421
11.7625
12.3166
14.1100
13.4327
1.5
13.9626
1.8
13.7758
2
13.6493
(see Table 1). And we find that the SNRs and visibility quality of ‘MRI’ image are improved gradually with the increase
of α (see Fig.3). But the computational cost is much more
expensive due to the increase of α (see Fig.4). Furthermore, if
we choose α = 1, CDS IRM and CDS ISS reduce to the result
introduced in Ref.[7].
Fig. 4. Stopping criterion (MRI: results of f − ũ2 ≈ σ at
number of stopping iterations)
V. Conclusions
In this paper, we investigated non-linear variational denoising methods. We focused on denoising in curvelet-type
decomposition spaces setting and used the characterization of
curvelet-type decomposition spaces by curvelet coefficients to
minimize the variation functional. And minimzer can be derived as curvelet shrinkage with curvelet-type weight, which
is dynamically changing in the iteration(CDS-IRM). Furthermore, we generalize the above iterative regularization method
to a new nonlinear inverse scale space flow with curvelettype weight(CDS-ISS). And it appears that the new methods
CDS IRM and CDS ISS perform better than wavelet-based
method from the SNR point view and result in reducing of
artifacts. The main goal of this paper is to generalize the relation between TV-based methods and wavelet-based models.
Our future work will involve the choice of the parameter in the
regularization, and using our method on Shearlet[17] .
References
705
CDS ISS
13.1399
14.1853
14.3056
14.1586
14.0193
13.9372
13.7809
14.1000
14.1678
14.1006
13.9192
13.7798
13.6759
[1] G. Aubert and L. Vese, “A variational method in image recovery”, SIAM J. Numer. Anal., Vol.34, pp.1948–1979, 1997.
[2] T. Chan, S. Esedoglu, F. Park and A. Yip, “Recent developments in total variation image restoration”, UCLA-CAMReport 05-01, Los Angeles, CA, USA, 2005.
[3] D. Donoho, “Denoising by soft thresholding”, IEEE. Trans. Inform. Theory, Vol.41, No.3, pp.613–627, 1995.
[4] Patrickl. Combettes and Jean-Christophe Pesquet, “Waveletconstrained image restoration”, International Journal of
Wavelets, Multiresolution and Information Processing, Vol.2,
No.4, pp.371–389, 2004.
[5] Ashish Khare and Uma Shanker Tiwary, “Soft-thresholding for
denoising of medical images-A Multiresolution Approach”, International Journal of Wavelets, Multiresolution and Information Processing, Vol.3, No.4, 2005.
[6] M. Li, X.C. Feng, “Iterative regularization and nonlinear inverse scale space based on translation invariant wavelet shrinkage”, International Journal of Wavelets, Multiresolution and
Information Processing, Vol.6, No.1, pp.83–95, 2008.
[7] Liu Guojun, Feng Xiangchu, “Curvelet-based iterative regularization and inverse scale space methods”, Chinese Journal of
Electronics, Vol.19, No.3, pp.548–552, 2010.
[8] Lasse Borup and Morten Nielsen, “Frame decomposition of decomposition spaces”, Journal of Fourier Analysis and Applications, Vol.13, No.1, pp.39–70, 2007.
[9] L.Rudin, S. Osher and E. Fatemi, “Nolinear total variation
based noise removal algorithms”, Phys. D, Vol.60, PP.259–268,
1992.
[10] S. Osher, M. Burger, D. Goldfarb, J. Xu and W. Yin, “An iterative regularization method for total variation based image
restoration”, Multiscale Model. and Simul., Vol.4, pp.460–489,
2005.
[11] M. Burger, S. Osher, J. Xu, and G. Gilboa, “Nonlinear inverse
scale space methods for image restoration”, Lecture Notes in
Computer Science, Vol.3752, pp.25–36, 2005.
[12] M. Burger, G. Gilboa, S. Osher and J. Xu, “Nonlinear inverse
scale space methods”, Comm. Math. Sci., Vol.4, No.1, pp.175–
208, 2006.
[13] M. Nikolova, “Local strong homogeneity of a regularized estimator”, SIAM Journal on Applied Mathematics, Vol.61, No.2,
pp.633–658, 2000.
[14] Jinjun Xu and Stanley Osher, “Iterative regularization and nonlinear inverse scale space applied to wavelet based denoising”,
IEEE Trans. Image Proc., Vol.16, No.2, pp.534–544, 2007.
[15] Jinjun Xu, “Iterative regularization and nonlinear inverse scale
space methods in image restoration”, Ph.D. Thesis, UCLACAM-Report 06-33, Los Angeles, CA, USA, 2006.
706
Chinese Journal of Electronics
[16] Youwei Wen, Raymond H. Chan, Andy M. Yip, “A primal-dual
method of total-variation based wavelet domain inpainting”,
IEEE Trans. Image Proc., Vol.21, No.1, pp.106–113, 2012.
[17] Pooran Singh Negi and Demetrio Labate, “3-D discrete shearlet transform and video processing”, IEEE Trans. Image Proc.,
Vol.21, No.6, pp.2944–2954, 2012.
[18] M. Li, X.C. Feng, “Wavelet shrinkage and a new class of variational models based on Besov spaces and negative HilbertSobolev spaces”, Chinese Journal of Electronics, Vol.16, No.2,
pp.276–280, 2007.
LI Min
received the Ph.D. degree
in applied mathematics from Xidian University in 2008. She is currently a lecturer
at College of Mathematics and Computational Science, Shenzhen University, Shenzhen. Her research interests are in the
field of partial differential equations and
wavelet, with applications to image analysis. (Email: [email protected])
2013
XU Chen (corresponding author)
received the B.S. and M.S. degrees in mathematics from Xidian University in 1986
and 1989, the Ph.D. degree in mathematics from Xi’an Jiaotong University in 1992
respectively. He is currently a Professor of Mathematics and advisor for doctor students at Shenzhen University, Shenzhen. His research fields are information
and computational science, analysis and
application of wavelet. (Email: xuchen [email protected])
SUN Xiaoli received the Ph.D. degree in applied mathematics from Xidian
University in 2007. She is currently a lecturer at College of Mathematics and Computational Science, Shenzhen University,
Shenzhen. Her research interests are in the
field of partial differential equations and
wavelet, with applications to image analysis. (Email: [email protected])