Introduction to Wavelets
Ling Zhu
March 2014
Ling Zhu
Introduction to Wavelets
March 2014
1 / 21
Outline
1
Overview
2
Brief Review of Fourier Analysis
3
Wavelets
The Haar System
Smoother Wavelet Bases
Ling Zhu
Introduction to Wavelets
March 2014
2 / 21
Overview
Why Wavelets?
Wavelets:
. First developed by French scientists.
ondelette
Wavelet analysis is a renement of Fourier analysis, which describes an input
signal in terms of its frequency components.
Advantages:
Spatial adaptivity
Multi-resolution analysis
Used in:
Signal processing: denoising
Image analysis
Data compression
Ling Zhu
Introduction to Wavelets
March 2014
3 / 21
Brief Review of Fourier Analysis
The Discrete Fourier Transform
´b
Let f ∈ L2 [−π, π], where L2 [a, b] = f : a f 2 (x)dx < ∞ is the
square-integrable function space. Then Fourier analysis states that f can be
expressed as an innite sum of dilated cosine and sine functions:
f (x) =
∞
X
1
aj cos(jx) + bj sin(jx) ,
a0 +
2
j=1
(1)
where the coecients can be computed using:
1
1
aj = hf, cos(j·)i =
π
π
1
1
bj = hf, sin(j·)i =
π
π
Ling Zhu
ˆ
π
f (x) cos(jx)dx,
j = 0, 1, . . . ,
(2)
f (x) sin(jx)dx,
j = 1, 2, . . . ,
(3)
−π
ˆ
π
−π
Introduction to Wavelets
March 2014
4 / 21
Brief Review of Fourier Analysis
Fourier Basis
Denition
A sequence of function
fj 's
{fj }
are pairwise orthogonal,
orthogonal to each
Let
and
with
fj
is said to be a complete orthonormal system if the
kfj k = 1
j,
for each
and the only function
is the zero function.
gj (x) = π −1/2 sin(jx) for
j = 1, 2, . . .
hj (x) = π −1/2 cos(jx) for
j = 1, 2, . . .
√
h0 (x) = 1/ 2π
on
x ∈ [−π, π].
Then, the set {h0 , gj , hj : j = 1, 2, . . .} is a complete orthonormal system for
L2 [−π, π].
Ling Zhu
Introduction to Wavelets
March 2014
5 / 21
Brief Review of Fourier Analysis
A Simple Example
In reality, the summation in (1) can be well-approximated (in the L2 sense) by a
nite sum with upper summation limit index J :
SJ (x) =
J
X
1
aj cos(jx) + bj sin(jx)
a0 +
2
j=1
(4)
Consider the following example function:

 x + π,
f (x) =
−π/2,

π − x,
−π ≤ x ≤ −π/2
−π/2 < x ≤ π/2
π/2 < x ≤ π.
(5)
Figure 1 shows the truncated Fourier series representations (4) for J = 1, 2, and 3.
Ling Zhu
Introduction to Wavelets
March 2014
6 / 21
Brief Review of Fourier Analysis
A Simple Example
1.0
0.5
0.0
0.0
0.5
1.0
a0/2 + a1 * cos1
1.5
Reconstruction with J=1
1.5
Example function
−3
−2
−1
0
1
2
3
−3
−1
0
1
2
3
2
3
0.5
1.0
1.5
x
0.0
0.0
0.5
1.0
1.5
x
−2
Reconstruction with J=3
a0/2 + a1 * cos1 + a2 * cos2 + a3 * cos3
Reconstruction with J=2
−3
−2
−1
0
1
2
3
−3
−2
−1
0
1
Figure 1: An example function and its Fourier sum representation
Ling Zhu
Introduction to Wavelets
March 2014
7 / 21
Wavelets
The Haar System
The Haar Function (mother wavelet)
0 ≤ x < 12
1
2 ≤x<1
otherwise.
(6)
−1.0
−0.5
0.0
0.5
1.0

 1,
ψ(x) =
−1,

0,
0.0
0.2
0.4
0.6
0.8
1.0
Figure 2: The Haar function
Ling Zhu
Introduction to Wavelets
March 2014
8 / 21
Wavelets
The Haar System
Haar Wavelets
Each wavelet born of the mother wavelet can be written as:
(7)
ψj,k (x) = 2j/2 ψ(2j x − k),
j=4, k=13
j=3, k=2
j=0, k=0
−4
−2
0
2
4
where j is the dilation index and k is the translation index.
0.0
0.2
0.4
0.6
0.8
1.0
x
Figure 3: Haar wavelet examples
Ling Zhu
Introduction to Wavelets
March 2014
9 / 21
Wavelets
The Haar System
Haar Wavelets
Theorem
The set
{ψj,k , j, k ∈ Z}
constitutes a complete orthonormal system for
L2 (R).
See reference for the proof.
Basically, from the proof we know that for each level j , one can construct f j
using Haar functions to approximate the original function. This approximation can
be written as the sum of the next coarser approximation f j−1 and a detail
function g j−1 . As the index j increases, the approximations become ner. Each
detail function g j can be written as a linear combination of the ψj,k functions.
Ling Zhu
Introduction to Wavelets
March 2014
10 / 21
Wavelets
The Haar System
Multi-resolution Analysis
Dene the function space Vj , j ∈ Z to be:
n
o
Vj = f ∈ L2 (R) : f is piecewise constant on [k2−j , (k + 1)2−j ), k ∈ Z
(8)
Denote P j f to be the projection of a function f onto the space Vj , then
(9)
P j f = P j−1 f + g j−1
X
= P j−1 f +
hf, ψj−1,k iψj−1,k
(10)
k∈Z
= ···
j0
=P f+
j−1 X
X
(11)
hf, ψ`,k iψ`,k .
`=j0 k∈Z
Ling Zhu
Introduction to Wavelets
March 2014
11 / 21
Wavelets
The Haar System
Haar Scaling Functions
Haar Scaling function (father wavelet): φ(x) = I[0,1) (x).
Its dilates and translates: φj,k (x) = 2j/2 φ(2j x − k).
Then, the set {φj,k , k ∈ Z} is an orthonormal basis for Vj .
Hence,
P jf =
X
(12)
cj,k φj,k ,
k
where the scaling function coecients can be computed:
ˆ
∞
cj,k = hf, φj,k i =
(13)
f (x)φj,k (x)dx.
−∞
Ling Zhu
Introduction to Wavelets
March 2014
12 / 21
Wavelets
The Haar System
Decomposition of the Space Vj
Dene the space for the detail functions:
(14)
Wj = span{ψj,k , k ∈ Z}.
It's easily seen that hφj,k , ψj 0 ,k0 i = 0, when j ≤ j 0 . Hence, from (9) we have:
(15)
Vj = Vj−1 ⊕ Wj−1
(16)
= Vj−2 ⊕ Wj−2 ⊕ Wj−1
= ···
= Vj0 ⊕
j−1
M
(17)
W`
`=j0
=
j−1
M
(18)
W` .
`=−∞
Ling Zhu
Introduction to Wavelets
March 2014
13 / 21
Wavelets
The Haar System
Relationship Between Coecients
For the Haar scaling function coecients:
√
cj,k = (cj+1,2k + cj+1,2k+1 )/ 2
(19)
4
For the Haar wavelet coecients dj,k = hf, ψj,k i,
since
Ling Zhu
√
dj,k = (cj+1,2k − cj+1,2k+1 )/ 2,
(20)
√
ψj,k (x) = φj+1,2k (x) − φj+1,2k+1 (x) / 2.
(21)
Introduction to Wavelets
March 2014
14 / 21
Wavelets
The Haar System
Coecients for the Example Function
Haar scaling function coecients:
k
-7
-6
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
j=-1
0
0
0
0
0
0.4567
2.1548
2.1548
0.4567
0
0
0
0
0
Ling Zhu
j=0
0
0
0
0.0093
0.6366
1.4765
1.5708
1.5708
1.4765
0.6366
0.0093
0
0
0
Haar wavelet coecients:
k
-4
-3
-2
-1
0
1
2
3
4
j=2
0.0132
0.2734
0.6269
0.9774
1.1107
1.1107
1.1107
1.1107
1.1107
1.1107
0.9774
0.6269
0.2734
0.0132
Introduction to Wavelets
j=-1
0
0
-0.4435
-0.0667
0.0636
0.4496
0
0
0
j=0
-0.0093
-0.2500
-0.0943
0
0
0.0900
0.2500
0.0107
0
March 2014
15 / 21
Wavelets
The Haar System
Approximation of the Example Function Using Haar Wavelets
1.0
0.5
0.0
0.0
0.5
1.0
approx1
1.5
Projection for j= −1
1.5
Example function
−2
0
2
4
−4
−2
0
2
Projection for j= 1
x2
x2
4
1.0
0.5
0.0
0.0
0.5
1.0
approx3
1.5
Projection for j= 0
1.5
−4
−4
−2
0
2
4
−4
−2
0
2
4
Figure 4: An example function and its piecewise constant approximation
Ling Zhu
Introduction to Wavelets
March 2014
16 / 21
Wavelets
The Haar System
Detail Functions of the Example Function
Detail signal for j= 0
−0.3
−0.2
−0.2
−0.1
0.0
detail2
0.0
−0.1
detail1
0.1
0.1
0.2
0.2
0.3
Detail signal for j= −1
−4
−2
0
2
4
−4
x2
−2
0
2
4
x2
Figure 5: The piecewise constant detail functions between successive approximations for
the example function
Ling Zhu
Introduction to Wavelets
March 2014
17 / 21
Wavelets
Smoother Wavelet Bases
Daubechies Family of Wavelets:
The wavelets in the Daubechies family form an orthonormal basis for L2 (R), and
also have compact support. Members of this family are indexed by an integer N ,
with the smoothness of the functions increases as N increases.
Wavelet, N=2
1.0
psi
−1.0
0.0
0.5
1.0
1.5
2.0
2.5
x
Daub cmpct on ext. phase N=2
Ling Zhu
0.0
0.5
0.0
phi
1.0
Scaling function, N=2
3.0
−1
0
1
2
3
x
Daub cmpct on ext. phase N=2
Introduction to Wavelets
March 2014
18 / 21
Wavelets
Smoother Wavelet Bases
Daubechies Family of Wavelets:
Wavelet, N=3
0.5
1.0
0.0
−1.0 −0.5
0.0
psi
0.5
phi
1.0
1.5
Scaling function, N=3
2
3
4
5
−2
0
2
4
x
Daub cmpct on ext. phase N=3
Scaling function, N=5
Wavelet, N=5
1.0
0.5
psi
0.0
−1.0
−0.2
−0.5
phi
1
x
Daub cmpct on ext. phase N=3
0.2 0.4 0.6 0.8 1.0
0
0
2
4
6
8
x
Daub cmpct on ext. phase N=5
−5
0
5
x
Daub cmpct on ext. phase N=5
Figure 6: Three examples of scaling function/wavelet sets from the Daubechies compactly supported
family
Ling Zhu
Introduction to Wavelets
March 2014
19 / 21
Wavelets
Smoother Wavelet Bases
Approximation of the Example Function Using Daubechies Wavelets
1.0
0.5
0.0
0.0
0.5
1.0
approx4
1.5
Reconstruction with j= −2
1.5
Example function
−2
0
2
4
−4
−2
0
2
Reconstruction with j= 0
x2
x2
4
1.0
0.5
0.0
0.0
0.5
1.0
approx6
1.5
Reconstruction with j= −1
1.5
−4
−4
−2
0
2
4
−4
−2
0
2
4
Figure 7: An example function and representation using Daubechies wavelets with N=5
Ling Zhu
Introduction to Wavelets
March 2014
20 / 21
Wavelets
Smoother Wavelet Bases
Reference:
1
Essential Wavelets for Statistical Applications and Data Analysis. Todd
Ogden, Birkhauser, 1997.
2
Wavelet methods in statistics with R. Guy Nason, New York: Springer, 2008.
Thank you!
Ling Zhu
Introduction to Wavelets
March 2014
21 / 21