SCHOOL of COMPUTING
UNIVERSITY OF EASTERn FINLAND
JOENSUU, FINLAND
Advanced Digital Signal Processing
Lecture 12
Wavelet Transform
Alexander Kolesnikov
23.04.2010
Short-Term Fourier Transform
again…
Spectrums of the test signals
Signals are different,
spectrums are similar
Why?
What is wrong with the Fourier Transform?
Two basis functions: sin(t) and (t)
Support region:
In time
In frequency
sin(t)

0
(t)
0

The basis function sin(t) is not localized in time,
The (t) (sample) is not localized in frequency.
Fourier Transform is good for stationary signals,
But it can not locate drift, trends, abrupt changes,
beginning and ends of events, etc.
Short-Term Fourier Transform (STFT)
signal x(t)
window h(t-)
signal in window
~
X ( , ) 

it


x
(
t
)
h
(
t


)
e
dt


Introduce basis functions which are compact in time and frequency
domains.
Let us divide the input signal into time sub-intervals, and perform
DFT for every time sub-interval.
Gabor transfortm is a special case of STFT
Window is a Gaussian function:
~
X ( , ) 

 x(t )e

 (t  ) 2 / 2 2  it
e
dt
STFT: Time-frequency plane
Equidistant frequencies:
2
k 
k
t

t
Problems with STFT
Uncertainty Principle:
  t 
1
4

t
We cannot localize events in time and frequency
simultaneously!
Improved time resolutionDegraded frequency resolution
Improved frequency resolution Degraded time resolution
Problem: the same  and t through the entire plane!
Continuous Wavelet Transform
The main idea
  t 


1
4
 Const  t ~
1

Use wider time window for lower frequencies!
Time-frequency plane for wavelets
Heisenberg cell
Our first wavelet transform:
Note that as alpha increases: the
frequency decreases, and the
window function expands.
Time shift now also in
exponential. Can write:
Scale: Illustration
• Coarse scale:
Flying in a jet at 5 km
• Medium scale:
Bird flight at 100 m
• Fine scale:
Beach walk
Coast line
Wavelet mother & Baby function
Let we have a wavelet mother function (t).
How to obtain a set of wavelet functions?
1  t  
s , (t ) 


s  s 
Scaling (s) and Translation ()

f (t )    ( s, )
 ( s, ) 
f (t ) s , (t )dt
s ,
(t )dsd
Scaling (stretching or compressing), dilation
s=1
s=2
s=4
Translation (shift), 
Translation () and scaling (s)
s , (t ) 
1  t  


s  s 
Problems with Continuous Wavelet Transform
 f (t )  (t )dt
f (t )    ( s, )  (t )dsd
 ( s, ) 
s,
s,
1) Redundancy of Continuous Wavelet Transform (CWT) .
The WT is calculated by continuously shifting of continuously
scalable function over a signal.
2) We still have an infinite number of wavelets in the wavelet
transform and we would like to reduce the number to a more
manageable count.
3) For the most functions the wavelet transform have no analytical
solution and they can be calculated only numerically. Fast algorithms
needed.
Discrete Wavelet Transform, DWT
Solution to the problems
To overcome the problems, Discrete Wavelets have been introduced.
Discrete Wavelets can be scaled and translated in discrete steps:
 j ,k (t ) 
 t  k  0 s0j

j

j
s
s0 
0
1




Here j and k are integer numbers, s0 is a fixed dilation step.
The translation factor depends on the dilation step.
The time-scale is sampled at discrete intervals.
We choose s0=1/2. This is natural choice for human ear, musics
and computers.
For the translation factor we choose 0=1.
Discrete Wavelets
Ψ j , k (t )  2 j 2Ψ (2 j t  k )
...
...
k=0,1,...,
Discrete Wavelets
Ψ j , k (t )  2 j 2Ψ (2 j t  k )
23
Discrete Wavelets
Ψ j ,k (t )  2 j 2Ψ (2 j t  k )

Ψ
Ψ j ,k (t )Ψ *j ,k (t )dt  1
*
(
t
)
Ψ
j1 , k1
j , k (t )dt  0; j1  j , k1  k .
f (t ) 
  ( j, k )
j , k (t )
Normalization
Orthogonality
Expansion
j ,k
 ( j, k ) 

f (t ) j ,k (t )dt
Wavelet coefficients
Examples of wavelets
Haar function is the first wavelet (1909)
( j, k )  2 j / 2 (2t  k )
Scaling function
Approximation with Haar functions
Disadvantages of Haar wavelet: discontinuous and does not
approximate continuous signals very well.
k
Approximation with Haar functions
x(n)  0 (n) 
J 2 j 1
 
j , k j , k ( n)
j 1 k  0
0 
N 1
 x(n) (n)
n 0
N 1
 j ,k 
 x(n)
j , k ( n)
n 0
k
Direct approach: Example
Input data: x(n)=(2,5,8,9,7,4,-1,1)
0 :
0,0:
1,0,1,1:
0  2 3 / 2 2  5  8  9  7  4  1  1 
2 2
 1,0  2 3 / 2 (2  5  8  9)  (7  4  1  1)  
 1,0  2 2 / 2 (2  5)  (8  9) 
 2,0  2 1/ 2 2  5 
2,0, 2,1, 2,2 2,3 :
35
 2,2  2 1/ 2 7  4 
3
2
3
2
13
2 2
10
11
;  1,1  2  2 / 2 (7  4)  (1  1)  
2
2
;  2,1  2 1/ 2 8  9 
1
;  2,3  2 1/ 2  1  1 
2
2
2
Forward Haar Transform: Analysis
Input data: X={2,5,8,9,7,4,-1,1}.
Haar wavelet transform: (a,b)(s,d) where:
s is for smooth component (LPF)
d is for details (HPF)
s ( n) 
d ( n) 
x(2n)  x(2n  1)
2
x(2n)  x(2n  1)
2
1st step:
  2  5 8  9 7  4  1  1  2  5 8  9 7  4  1  1 
W {x}   
,
,
,
,
,
,
  
, 
2
2
2  2
2
2
2 
 2
  7 17 11 0    3  1 3  2  
  
,
,
,
,
,
,
 
, 
  2 2 2 2   2 2 2 2 
Forward Haar Transform: Analysis
2nd step:
  7  17 11  0   7  17 11  0    3  1 3  2  
W {x}   
,
,
,
,
,
, 
  
, 
  2 2 2 2   2 2 2 2   2 2 2 2 
  24 11    10 11    3  1 3  2  
   , , 
, , 
,
,
,
 
  2 2   2 2   2 2 2 2 
3rd step:
  24  11   24  11   10 11    3  1 3  2  
W {x}   
, , 
,
,
,

  
, 
  2 2   2 2  2 2   2 2 2 2  
  35   13    10 11    3  1 3  2  
  
, , 
,
,
,
, 
 
, 
  2 2   2 2   2 2   2 2 2 2 
Forward Haar Transform: Analysis
  35   13    10 11    3  1 3  2  
W {x}   
, , 
,
,
,
, 
 
, 
  2 2   2 2   2 2   2 2 2 2 
x ( n) 
35
2 2
0 
13
2 2
 0, 0 
10
11
3
1
3
2
 1,0   1,1 
 2, 0 
 2,1 
 2, 2 
 2, 3
2
2
2
2
2
2
Backward Haar Transform: Synthesis
x ( 2n) 
s ( n)  d ( n)
x(2n  1) 
2
s ( n)  d ( n)
2
1st step:
  35   13    10 11    3  1 3  2  
x   
, , 
,
,
,
, 
  
, 
  2 2   2 2   2 2   2 2 2 2 
  35  13   35  13    10 11    3  1 3  2  
  
, , 
,
,
,
, 
  
, 
  2 2 2   2 2 2   2 2   2 2 2 2 
  24 11    10 11    3  1 3  2  
   , , 
, , 
,
,
,
  
  2 2   2 2   2 2 2 2 
Backward Haar Transform: Synthes
2nd step:
  24 11    10 11    3  1 3  2  
x    , , 
, , 
,
,
,
  
  2 2   2 2   2 2 2 2 
  24  10 24  10 11  11 11  11    3  1 3  2  
  
,
;
,
,
,
,
, 
  
2 2
2 2 2 2   2 2 2 2 
 2 2
  7 17 11 0    3  1 3  2  
  
,
;
,
,
,
,
, 
 
  2 2 2 2   2 2 2 2 
Backward Haar Transform: Synthes
3rd step:
  7 17 11 0    3  1 3  2  
x   
,
;
,
,
,
,
, 
  
  2 2 2 2   2 2 2 2 
  7  3 7  3 17  1 17  1 11  3 11  3 0  2 0  2  
  
,
;
,
,
,
;
,
  
  2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 
 3,5;8,9,7,4;1,1
Haar Transform as subband filtering
y 0 ( n) 
y1 (n) 
x(2n)  x(2n  1)
2
x(2n)  x(2n  1)
2
 1 1 

h0 (n)  
,
 2 2
 1 1 

h1 (n)  
,
 2 2
y 0 ( n)  y1 ( n)
LPF
x (2n) 
HPF
x ( 2n  1) 
2
y 0 ( n)  y1 ( n)
2
 1 1 

g 0 (n)  
,
 2 2
 1 1 

g1 (n)  
,
 2 2
Wavelet Transform and Filter Banks
Wavelet Transform and Filter Banks
Decimation in time means stretching in frequency domain
1  
F g (at ) 
G 
|a|  a 
Wavelet Transform and Filter Banks
h0(n) is scaling function, low pass filter (LPF)
h1(n) is wavelet function, high pass filter (HPF)
is subsampling (decimation)
System of filters
Effectively, the DWT is nothing but a system of filters. There are two filters involved, one is
the “wavelet filter”, and the other is the “scaling filter”. The wavelet filter, is a high pass
filter, while the scaling filter is a low pass filter.
h0(n)
Level 1
h0(n)
h1(n)
h0(n)
h1(n)
Level 2
h1(n)
Level 3
Scaling Filter ~ Averaging Filter
Wavelet Filter ~ Details Filter
Pic from wikipedia.org
Inverse wavelet transform
is up-sampling (zeroes inserting)
In order to achieve perfect reconstruction the synthesis filters
should satisfy
g0(n) = h0(-n)
g1(n) = h1(-n)
Wavelet transform as Subband filtering
Discrete Wavelet Transform can be
Constructed via iterated
(octave-band) filter bank
Subspaces of DWT
Complexity of Discrete Wavelet Transform
N N N
N
N   2  3  ... k  2 N
2 2 2
2
 C ( N )  O( N )
Dilation equations or Two-scale relation

Scaling function:
 (t )  2  h0 (k ) (2t  k )
k  

Wavelet function:
 (t )  2  h1 (k ) (2t  k )
k  
The functions (t) and (t) are orthogonal:
h1 (k )  (1) k h0 ( N  1  k )
Scaling (father) function and Wavelets
f (t )    j 1 (k ) (2 j 1 t  k )    j 1 (k ) (2 j 1 t  k )
k
k
The most important property of the wavelets:
To obtain WT coefficients for level j we need to process
only WT coefficients for level j+1.
 j (k ) 
 j (k ) 

 h ( m  2k )
0
j 1 ( m)
m

 h ( m  2 k )
1
m
j 1 ( m)
Haar: Scaling function and Wavelets
 (t )   (2t )   (2t  1)
 (t )   (2t )   (2t  1)
h1 (k )  (1) k h0 ( N  1  k ) (t )
h0 (0)  h0 (1)  2
French Hat (piecewise linear spline) wavelet
 (t ) 
 (t ) 
1
2
1
2
 (2t )  2 (2t  1)   (2t  2) 
  (2t )  2 (2t  1)   (2t  2) 
h1 (k )  (1) k h0 ( N  1  k ) (t )
h0 (0)  h0 (1)  h0 (2)  2
French Hat wavelets
 (t ) 
 (t ) 
1
2
1
2
 (2t )  2 (2t  1)   (2t  2)
  (2t )  2 (2t  1)   (2t  2)
h0 (0)  h0 (1)  h0 (2)  2
Daubechies wavelets D4
 (t )
 (t )
 (t )  2
 (t )  2

 h (k ) (2t  k )
0
k  

 h (k ) (2t  k )
1
k  
Haar=D1
h0 (0) 
h0 (0)  h0 (1)  h0 (2)  h0 (3)  2
h0 (1) 
h0 (2) 
h0 (3) 
1 3
4 2
3 3
4 2
3 3
4 2
1 3
4 2
; h1 (0)  h0 (3)
; h1 (1)  h0 (2)
; h1 (2)  h0 (1)
; h1 (3)  h0 (0)
Daubechies wavelets
2-D Wavelet transform in JPEG2k
a00
H0
2
a0
H0
2
x
H1
a01
2
a1
H1
2
a10
H0
H1
Horizontal filtering
2
2
a11
Vertical filtering
2-D wavelet transform
Original
128, 129, 125, 64, 65, …
Transform Coeff.
4123, -12.4, -96.7, 4.5, …
2-D wavelet transform
LL3
HH4
HH3
LH2
LH1
HL2
HH2
HL1
HH1
JPEG 2000 vs JPEG
DCT
WT
JPEG 2000 vs JPEG: Quantization
JPEG
JPEG 2000
JPEG 2000 vs JPEG: 0.3 bpp
JPEG
JPEG 2000
JPEG 2000 vs JPEG: Bitrate=0.3 bpp
MSE=150
PSNR=26.2 db
MSE=73
PSNR=29.5 db
JPEG 2000 vs JPEG: Bitrate=0.2 bpp
MSE=320
PSNR=23.1 db
MSE=113
PSNR=27.6 db
Examples of Wavelets
Examples of Wavelets
Denoising with Wavelets
Denoising with Wavelets
Wavelets with Matlab
Denoising with wavelets
See in Matlab: wavemenu ->SWT Denoising 1-D
Denoising with wavelets
Denoising with wavelets
Denoising with wavelets
Denoising with wavelets
History
History