DFT Domain Image Filtering
Yao Wang
Polytechnic Institute of NYU, Brooklyn, NY 11201
With contribution from Zhu Liu, Onur Guleryuz, and
Gonzalez/Woods, Digital Image Processing, 2ed
Lecture Outline
•
•
•
•
•
•
1D discrete Fourier transform (DFT)
2D discrete Fo
Fourier
rier transform (DFT)
Fast Fourier transform (FFT)
DFT domain filtering
1D unitary transform
2D unitary transform
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
2
Discrete Fourier Transform (DFT):
DTFT for Finite Duration Signals
If the signal is only defined for n  0,1,..., N  1 :
Fourier transform becomes :
N 1
F ' (f)   f (n) exp( j 2fn),
f  (0,1)
n 0
Sampling F ' (f) at f  k/N, k  0 ,1,...,N-1, and rescaling yields :
Forward transform (DFT) :
1 N 1
k
k
F (k )  F ' (
)
f
(
n
)
exp(

j
2

n), k  0,1,..., N  1

N
N
N n 0
Inverse transform
f
(
(IDFT)
):
1
f(n) 
N
Yao Wang, NYU-Poly
N 1
k
F (k ) exp( j 2 n), n  0,1,..., N  1

N
k 0
EL5123: DFT and unitary transform
3
Property of DFT (1)
• Periodicity F (k )  F (((k )) N ), k  0 or k  N .
where ((k )) N
represents
p
modulo N .
N 1
(k  mN )
Proof F (k  mN )  1
f (n) exp( j 2
n)

N
N n 0
1 N 1
k

f
(
n
)
exp(

j
2

n  j 2mn)  F (k )

N
N n 0
F(k)
A
A
A
G
G
G
B
B
B
C
F
C
F
C
F
D
D
D
E
E
E
0
low
N-1 N
high
low
Note: Highest frequency is at k=[N/2]. k=0,1, N-1 represent low frequency.
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
4
Property of DFT (2)
• Translation
f (((n  no )) N )  F (k ) exp{{ j 2 (kn
k o / N )}
f (n) exp{ j 2k0 n / N }  F (((k  k0 )) N ).
– Special
p
case
• N is even, k0 = N/2.
f (n) exp{ jn}  f (n)(1)

n
N
F (((k  )) N ).
)
2
Shifting
g the frequency
q
y up
p by
y N/2
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
5
Property of DFT (3)
• Conjugate symmetry for real sequences
F (k )  F * (k )  F * ( N  k )
| F (k ) || F ( N  k ) | or
N
N
| F (  k ) || F (  k ) |, when N is even.
2
2
0 1 2 3 4 5 6 7
0 1 2 3 4 5 6 7 8
N=8
Yao Wang, NYU-Poly
N=9
EL5123: DFT and unitary transform
6
2D Discrete Fourier Transform
• Definition
– Assuming f(m,
f(m n)
n), m = 0
0, 1
1, …, M
M-1,
1 n=0
0, 1
1,
…, N-1, is a finite length 2D sequence
F (k , l ) 
f (m, n) 
M 1 N 1
1
MN
 f (m, n)e
 j 2 (
km ln
 )
M N
, k  0,1,..., M  1, l  0,1,..., N  1;
m 0 n 0
M 1 N 1
1
MN
 F (k , l )e
j 2 (
km ln
 )
M N
, m  0,1,..., M  1, n  0,1,..., N  1.
k 0 l 0
– Comparing to DTFT
F (u , v) 


  f (m, n)e
 j 2 ( mu  nv )
m   n  
1/ 2
f (m, n)  
1/ 2

1 / 2 1 / 2
Yao Wang, NYU-Poly
F (u , v)e
k
l
, u
,v 
M
N
j 2 ( mu  nv )
EL5123: DFT and unitary transform
dudv
7
Property of 2D DFT (1)
• Periodicity
F (k , l )  F (((k )) M , ((l )) N ),
) k  0 or k  M , l  0 or l  N .
-M
A
B
A
B
C
D
C
D
Low
Frequency
-M/2
0
(0,0)
(0,N-1)
A
B
A
B
C
D
C
D
High
Frequency
M/2
M
k
-N
Yao Wang, NYU-Poly
-N/2
(M-1,0)
0
(M-1,N-1)
N/2
N
l
EL5123: DFT and unitary transform
Displayed
area using
fftshift
8
Property of 2D DFT (2)
• Translation
F (k , l ) exp{ j 2 (km0 / M  ln0 / N )},
f (((m  m0 )) M , ((n  n0 )) N ) 
f (m, n) exp{ j 2 (k0 m / M  l0 n / N )} 
F (((k  k0 )) M , ((l  l0 )) N ).
– Special Case: M,N=even, k0=M/2, l0=N/2
f (m, n) exp{ j (m  n)}  f (m, n)(1) ( m  n )
-M
f(m,n)
-M/2
A
C
0
F(k,l) M/2
M
A
C
-N
Yao Wang, NYU-Poly
B
A
D
C
B
A
D
-N/2
C
0
-M
B
0
B
N
M
F (((k 
C
D
M
N
)) M , ((l  )) N ).
2
2
C
B
A
B
A
D
C
D
C
B
A
B
A
M/2
D
N/2
D
-M/2
D

-N
-N/2
EL5123: DFT and unitary transform
0
N/2
f(m,n)(-1)(m+n)
F(k-M/2,l-N/2)
N
9
Property of 2D DFT (3)
• Conjugate symmetry for real sequences
F (k , l )  F * (k ,l )  F * ( M  k , N  l ),
)
| F (k , l ) | F ( M  k , N  l ) |,
M
N
M
N
| F (  k ,  l ) || F (  k ,  l ) |,
2
2
2
2
for M , N  even
0
1
2
3
4
1
0
1
2
3
4
5
6
7
8
9 33 32 31
5
6
7
8
9 40 39 38 37
3
2
4
3
2
1
10 11 12 13 14 30 29 28
10 11 12 13 14 36 35 34 33
15 16 17 18 19 27 26 25
15 16 17 18 19 32 31 30 29
20 21 22 23 24 23 22 21
20 21 22 23 24 28 27 26 25
15 25 26 27 19 18 17 16
20 25 26 27 28 24 23 22 21
10 28 29 30 14 13 12 11
15 29 30 31 32 19 18 17 16
5 31 32 33 9
10 33 34 35 36 14 13 12 11
8
7
6
5 37 38 39 40 9
M N 8
M=N=8=even,
Yao Wang, NYU-Poly
real
x
x
C j
Conjugate
t pairs
i
EL5123: DFT and unitary transform
8
7
6
M=N=9=odd
10
Property of 2D DFT (5)
• Separabability
– 2D DFT can be accomplished by N-point
N point 1D DFT of
each row, followed by M-point 1D DFT of each
column
• How many 1D DFT’s?
–
–
–
–
M rows: M N-pt DFT’s
N columns:
l
NM
M-ptt DFT’s
DFT’
M=N: 2N N-pt DFT’s
Each N-pt DFT requires N2 operations,
operations total requiring
2N3 operations
• Direct calculation
– Requiring M2N2 (N4 if M=N) operations
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
11
Property of 2D DFT (6)
• 2D DFT of Separable Images
f ( m, n )  f x ( m ) f y ( n )  F ( k , l )  Fx ( k ) Fy (l )
In matrix form :
f  fx f yT  F  Fx FyT
 f (0,0)
 f (1,0)
f 
 

 f ( M 1,0)
f x   f x ( 0)
f y   f y ( 0)
f (0,1)
f (1,1)




f ( M 1,1) 
f x (1) 
f y (1) 
f (0, N 1) 
f (1, N 1) 



f ( M 1, N 1)
M  pt 1D DFT
f x ( M  1) 
  Fx
T
N  pt 1D DFT
f y ( N 1) 

 Fy
T
Real images are seldom separable. But many 2D filters are separable.
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
12
Computer Implementation of 2D DFT
• Complex array structure
• Implement comple
complex m
multiplication
ltiplication and
addition
j2 k /M for all
• Pre-compute the constants: e-j2πkm/M
k,m=0, 1, …, M-1, and e-j2πln/N for all l,n=0,
1, …, N-1.
• Use separable processing to speed up
• For real square image, only need to
calculate half of the points
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
13
Display of the Magnitude of 2D DFT
• Shifting the low frequency components
into the center
g (m, n)  f (m, n)(1) ( m  n )
A
B
 G (k , l )  F (((k 
A
B
M
N
)) M , ((l  )) N ).
2
2
D
C
F(k,l)
G(k,l)
C
D
C
D
Low
Frequency
B
A
High
Frequency
Matlab command: fftshift
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
14
Display of the Magnitude of 2D DFT
• Amplitude rescaling G (k , l )  log(1  F (k , l ))
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
15
Sample Fourier pairs
Which Fourier transform in the second row is for each image in the first row?
Yao Wang, NYU-Poly
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
EL5123: DFT and unitary transform
16
Fast Fourier Transform (FFT)
• Direct computation of N-point DFT takes N2
operations
• FFT is a fast algorithm for computing DFT,
reducing
g the computation
p
from N2 to N log
g2((N))
– Complex conjugate symmetry of e  j 2kn / N
e
 j 2k ( N  n ) / N
e
j 2kn / N
e
 j 2k (  n ) / N

 e

 j 2kn / N *
– Periodicity in n and k of e  j 2kn / N
k /N
e  j 2kn
 e  j 2k ( n  N ) / N  e  j 2 ( k  N ) n / N
• Matlab function for N-point DFT: fft(A,N),
fft2(A M N)
fft2(A,M,N)
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
17
Computation of 2D DFT
• 2D (MxN) point DFT can be computed in a
separable manner:
– First compute N-point FFT for each row (M N
log2 (N) )
– Then compute M-point FFT for each column (N M
log2 (M) )
– Total computation if M=N: 2N2log2 (N)
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
18
Convolution Theorem (1D)
• Circular convolution (N-pt)
N 1
f (n)  h(n)   f (((n  k )) N )h(k )
k 0
f(n)
f(((n-k))N)
h(n)
0 1 2 3 4
n
0 1 2 3 4
n
• Convolution theorem
f ( n)  h( n) 
Yao Wang, NYU-Poly
0
n-1
n
n+2 n+4
n+1 n+3
k
F (k ) H (k )
EL5123: DFT and unitary transform
19
Linear Convolution vs. Circular
Convolution
• f(n): N1 pt, h(n): N2 pt, assume N1>=N2
• f(n)*h(n): N=N1+N2-1 pt
• Equivalent to circular convolution of M-pt, if
M>=N
• If we do N1 pt circular convolution, which parts
of the resulting output is equal to that of linear
convolution (assume N2 is much smaller than
N1)?
– Illustrate on board
– If f(n) is from 0 to N1-1, h(n) is from 0 to N2-1
– Circular conv. = linear conv. Over N2-1 to N1-1
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
20
Calculate Linear Convolution Using DFT
• 1D case
– f(n) is length N1, h(n) is length N2
– g(n) = f(n)*h(h) is length N = N1+N2-1.
– To
T use DFT,
DFT need
d to
t extend
t d f(n)
f( ) and
d h(n)
h( ) tto
length N by zero padding.
f(n)
N-point DFT
DFT
F(k)
Yao Wang, NYU-Poly
h(n)
*
Convolution
x
H(k)
g(n)
DFT
Multiplication
EL5123: DFT and unitary transform
G(k)
21
Calculate Linear Convolution Using DFT
f(n), N1=5
Fe(k)
fe(n), N=8
n
h(n), N2=4
n
He(k)
he(n), N=8
n
n
f (n) * h(n)  f e (n)  he (n)
n
Yao Wang, NYU-Poly
k
EL5123: DFT and unitary transform
k
Fe(k) He(k)
k
22
Comparison of Complexity
• Direct calculation
– Each point,
point N2 multiplications
– Overall, N2 * N ≈ O(N2)
• Via
Vi FFT
– N-point FFT for f(n) and h(n)
– N multiplications in the DFT domain
– N-point inverse FFT for F(k)*H(k)
– Overall, 3NlogN + N ≈ O(NlogN)
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
23
Circular Convolution (2D)
• Circular Convolution
M 1 N 1
f (m, n)  h(m, n)   f (((m  k )) M , ((n  l )) N )h(k , l )
k 0 l 0
• Convolution Theorem
f (m, n)  h(m, n) 
Yao Wang, NYU-Poly
F (k , l ) H (k , l )
EL5123: DFT and unitary transform
24
Circular vs. Linear Convolution
– If f(m,n) is N1xM1 pt, h(m,n) is N2xm2 pt
– f(m,n)
f(m n)*h(m
h(m,n)
n) is N’xM’
N xM pt,
pt N
N’=N1+N2=N1+N21,M’=M1+M2-1
– Equivalent to NxM pt circular convolution if
N>=N’,M>-M‘
– Can be implemented by using NxM pt 2D
DFT!
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
25
Separable filter
• If the filter is separable,
– h(m,n)=hx(m)
h(m n)=hx(m) hy(n)
– Row-by-row circular convolution with hy(n),
then column
column-by-column
by column circulua convolution
with hx(m)
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
26
Filtering in DFT Domain
Relation between spatial and frequency domain operation:
g ( x, y )  h( x, y )  f ( x, y )  G (u, v)  H (u, v) F (u, v)
h( x, y )  IDFT ( H (u, v)),
)) H (u , v)  DFT (h( x, y )).
))
Typically DFT size=image size. This corresponds to circular convolution.
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
27
Low-Pass Filter Using DFT Window
• Filtering in DFT domain:
– G(k,l)=H(k,l) F(k,l)
• Ideal Low-Pass Filter
– H(k,l)= 1 in low frequency range (the four corners!)
= 0 in high frequency range
– Can think of H(k,l) as a mask in the DFT domain
• The (k,l)th coefficient is unchanged if H(k,l)=1, and is set to zero if
H(k,l)=0.
• More generally, H(k,l) can take on higher values in lowfrequency range, and lower values in high frequency
range
• Note that equivalent spatial filter is truncated sinc. With
Circular convolution, it leads to ringing artifacts.
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
28
Low-Pass Filter in DFT Domain
The window size where
H(k l)=1 is
H(k,l)=1
(2 W1+1)x(2 W2+1)
•
The window can be implemented in matlab via
»
»
»
»
»
•
R1=zeros(M,1);
R1(1:W1+1,M-W1+1:M)=1;
R2=zeros(N,1);
( , );
R2(1:W2+1,N-W2+1:N)=1;
H=R1*R2’;
Before
shifting: note
low frequency
are at four
corners!
Given G(k,l)=F(k,l)H(k,l), matlab may make small numerical errors in
the inverse transform leading to complex valued images. g(m,n) is
best reconstructed via:
» g=real(ifft2(F.*H));
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
29
Example
Courtesy of Onur Guleyuz
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
30
Spatial Filter Corresponding
to Ideal LPF
10
20
30
40
10
50
20
60
30
10
20
30
40
50
60
40
IDFT
50
60
10
20
30
40
50
60
Ideal LPF with
W1=W2=10
To suppress the ringing artifacts, the window function should be smoothed
by an appropriately designed smoothing function.
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
31
High Pass Filtering
• The high-pass filtered image can be
thought of as the original image minus the
low pass filtered image.
– High-pass
High pass filtering by DFT windows:
• If w(k,l) (W1 W2) is a low-pass DFT window,
simply
p y define a high-pass
g p
window h(k,
( , l)) byy h(k,
( , l))
= 1 – w(k, l).
– High-pass filtering in spatial domain:
• If L is a low-pass filter of size W, simply define a
high pass filter H via H(m,n) = δ(m,n) - L(m,n).
Courtesy of Onur Guleyuz
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
32
High Pass Filtering in DFT Domain
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
33
DFT as Unitary Transform
• We have so far explained DFT as sampled
version of DTFT for finite duration discrete time
signals
• Finite duration discrete time signal = vector (1D)
or image (2D)
• Transform = decomposition of vector or image
into a weighted sum of some basis vectors or
images.
• Unitary transform : basis functions are
orthonormal to each other
• DFT uses a special type of basis functions
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
34
One Dimensional Linear Transform
• Let CN represent the N dimensional
complex space
space.
• Let h0, h1, …, hN-1 represent N linearly
independent vectors in CN.
• For any vector f є CN,
f   t (k )h  Bt ,
N 1
k 0
k
 t (0) 
 t (1) 
.
where B  [h 0 , h1 ,..., h N 1 ], t  
  


t
(
N

1
)


Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
t  B 1f  Af
f and t form a transform pair
35
Inner Product
• Definition of inner product
N 1
 f1 , f 2  f1 f 2   f1 (n) f 2 (n)
H
*
n 0
• Orthogonal
 f1 , f 2  0
• Norm of a vector
f
2
N 1
 f , f  f f   | f (n) |2
H
n 0
1
• Normalized: unit norm
• Orthonomal = orthogonal + normalized
f
Yao Wang, NYU-Poly
2
EL5123: DFT and unitary transform
36
Orthonormal Basis Vectors (OBV)
• {hk, k=0,…N-1} are OBV if
1 k  l
 h k , h l   k ,l  
0 k  l
N 1
N 1
k 0
k 0
 h l , f  h l ,  t (k )h k   t (k )  h l , h k  t (l )  h l f
H
 h0 H 
 H 
h1 

t
f  B H f  Af
  

H
h N 1 
B 1  B H , or B H B  BB H  I.
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
B is unitary
y
37
Definition of Unitary Transform
• Basis vectors are orthonormal
t (k )  ard
h , f 
• Forward
For
transform
 h (n) f (n),
N 1
*
k
n 0
k
 h 0H 
 H 
h1 

f  B H f  Af
t
  
 H 
h N 1 
N 1
n)   t (k )htransform
•f (Inverse
k ( n),
k 0
N 1
f   t (k )h k h 0
k 0
Yao Wang, NYU-Poly
h1  h N 1 t  Bt  A H t
EL5123: DFT and unitary transform
38
Example of 1D Unitary Transform
1 / 2
 1/ 2 
 1/ 2 
 1/ 2 
1 / 2
 1/ 2 
  1 / 2
  1 / 2
, h 2  
, h 3  
,
h 0   , h1  
1 / 2
  1 / 2
  1 / 2
 1/ 2 
 






1 / 2
  1 / 2
 1/ 2 
  1 / 2
 t0  5
1 
t  2
 2
1


f
 
 3
 t2  0
 
t3  1
 4
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
39
Property of Unitary Transform
• Energy preservation: ||f||=||t||.
Proof:
f  f H f  t H AA H t  t H t  t
• Mean vector relation:
μ t  Aμ
A f , μ f  A H μ t , where
h
μ f  E{f }, and μ t  E{t}
• Covariance relation:
Ct  AC f A H , C f  A H Ct A, where
C f  E{(f  μ f )(f  μ f ) H }, Ct  E{(t  μ t )(t  μ t ) H }
Proof:
t  μ t  A(f  μ f )  Ct  E{A(f  μ f )(f  μ f ) H A H }  AC f A H .
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
40
1D DFT as a Unitary Transform
N 1
1
F (k ) 
N
 f ( n )e
 j 2
1
f ( n) 
N
N 1
j 2
kn
N
, k  0, 1, ..., N  1;
n 0
 F ( k )e
kn
N
, n  0, 1, ..., N  1.
k 0
kn
1 j 2 N
hk (n) 
e
, or
N
1


k
 j 2 N 
1  e

hk 
, k  0,1,..., N  1.



N
 j 2 ( N 1) k 
N
e

Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
41
Example: 1D DFT
N  2 case : there are only two basis vectors :
k 

exp(
j
2

0) 
1 1
1 1
1 
2
hk 
 : h0 

1, h1 
 
k
2
2  1
2  exp( j 2 1) 
2 

1 
if f   , determine t0 , t1
 2
Using t k  h k , f , we obtain
1  1 1   1
3
1   1  1   1
1








,
1
*
1
1
*
2
,
,
1
*
1
1
*
2



t0 
t




1
   
  
2  1 2 
2
2
2   1 2 
2
2
Verify : t0h 0  t1h1 
Yao Wang, NYU-Poly
3 1 1  1  1 
   f


2 1 2  1 2
EL5123: DFT and unitary transform
42
Another Example: 1D DFT
k 

j
exp(
2

0) 

4
1
1
1
1

k 

 
 
 
1  exp( j 2 4 1) 
1 1
1 j 
1  1
1  j 
N  4 case : using h k 
yields : h 0 
; h1 
; h2 
; h3 
2   1
2   1
2 exp( j 2 k 2)
2 1
21 

 
 
 

4 
1
1


j






 j 

k 
 exp( j 2 3) 
4 

 2
1
1
1
 4
t0  2  4  5  3  7; t1  2  4 j  5  3 j    (3  j );
2
2
2
f   
1
1
1
5




3  j .
t





t


j


j


2
4
5
3
0
;
2
4
5
3
2
3
 
2
2
2
 3
 14  (3  j )  (3  j ) 
8


 
1 14  (3  j ) j  (3  j ) j  1 16 
Verify : t0h 0  t1h1  t 2h 2  t3h 3 
f

4  14  (3  j )  (3  j )  4 20

 

14  (3  j ) j  (3  j ) j 
12 
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
43
Two Dimensional Decomposition
• Decompose an MxN 2D matrix F=[F(m,n)]
into a linear combination of some basic
images, Hk,l=[Hk,l(m,n)], so that:
M 1 N 1
F   T (k , l )H k ,l ,
k 0 l 0
M 1 N 1
F (m, n)   T (k , l ) H k ,l (m, n)
k 0 l 0
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
44
Graphical Interpretation
t1
+
t2
t3
t4
Inverse transform: Represent a vector (e.g. a block of image samples) as the
superposition of some basis vectors (block patterns)
Forward transform: Determine the coefficients associated with each basis vector
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
45
Two Dimensional Inner Product
• Inner Product
M 1 N 1
 F1 , F2   F1* (m, n) F2 (m, n)
m 0 n 0
• Norm of a Matrix
M 1 N 1
F  F, F   F (m, n)
2
m 0 n 0
• A set of basis images {Hk,l, k=0,1,…,M-1,
l=0 1
l=0,1,…,N-1}
N-1} is orthonormal if
 H k ,l , H i , j   k ,i l , j
Yao Wang, NYU-Poly
1, if k  i, l  j

 0, otherwise
EL5123: DFT and unitary transform
46
Two Dimensional Unitary Transform
• {Hk,l} is an orthonormal set of basis images
• Forward
For ard transform
M 1 N 1
T (k , l )  H k ,l , F   H k*,l (m, n) F (m, n)
m 0 n 0
• Inverse transform
M 1 N 1
F (m, n)   T (k , l ) H k ,l (m, n), or
k 0 l 0
M 1 N 1
F   T (k , l )H k ,l
k 0 l 0
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
47
Example of 2D Unitary Transform
1/ 2 
 1 / 2  1 / 2
1 / 2  1 / 2
 1/ 2
1 / 2 1 / 2
H 00  
H
H
,
H
,
,
,



01
10
11







 1 / 2 1 / 2 
1 / 2  1 / 2
  1 / 2  1 / 2
1 / 2 1 / 2
 T (0,0)  5
T (0,1)  2
1 2

F



3 4
T (1,0)  1
 T (1,1)  0
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
48
2D DFT as a Unitary Transform
f (m, n) 
1
MN
M 1 N 1
 F (k , l )e
j 2 (
km ln
 )
M N
, m  0,1,..., M  1, n  0,1,..., N  1.
k 0 l 0
 km ln 
H k ,l (m, n) 
j 2 
 
1
M N
e
MN
Basis Images for 8x8 DFT (Real Part)
Basis Images for 8x8 DFT (Imaginary Part)
Notice how the frequency content of the basis images change !
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
49
Separable Unitary Transform
• Let hk, k=0, 1, …, M-1 represent
orthonormal basis vectors in CM,
• Let gl, l=0, 1, …, N-1 represent
orthonormal basis vectors in CN,
• Hk,l=hkglT, or Hk,l(m,n)=hk(m)gl(n).
• Then Hk,l will form an orthonormal basis
set in CMxN.
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
50
Example of Separable Unitary Transform
• Example 1
1 / 2 
 1/ 2 
h0  
,
h

.
 1 
1 / 2 
 1 / 2 
1 / 2 1 / 2
1 / 2  1 / 2
T
H 00  h 0hT0  
H

h
h

01
0 1

1 / 2  1 / 2
1 / 2 1 / 2


1/ 2 
 1/ 2
 1 / 2  1 / 2
T
H10  h1hT0  
H
h
h


11
1 1
 1 / 2 1 / 2 


1
/
2

1
/
2




• 2D DFT
 km ln 
H k ,l (m, n) 
 
j 2 
1
M N
,
e
MN
ln
1 j 2 km
1 j 2 N
M
, g l ( n) 
hk (m) 
e
e
M
N
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
51
Example: 4x4 DFT
1
1
1
1
1
 j 
 1
 
1 
1 
1 
1  j 
Recall the 1D DFT basis are : h 0 
; h1 
;h2 
; h3 
2 1
2   1
21 
2   1

 
 
 
1
 j 
 1
 j 
using H k,l  h k h l  yields :
T
H 0, 0
1

1 1

4 1

1
1
1
1
1
1
1
1
1
1
1
 j
j
1
H1,0  
4  1 1

 j  j
1 1

1  1  1
 2,0 
41 1

 1  1
H 3, 0
1
1 j  1  j 
1  1 1




1
1 1 j  1  j 
1 1  1 1
,
H 0,1 
,
H 0, 2 
1
4 1 j  1  j 
4 1  1 1




1
1 j  1  j 
1  1 1
1
1
j 1  j
1
 1 1




j
j
1  j 1  j 1 
1 j  j
, H1,1 
, H1, 2 
j 
 1  1
4  1  j 1
4  1 1




j  1
j
j j
 j 1
 j
j 1  j
1 1
1
 1 1




 1  1
j 
1  1  j 1
1  1 1
, H 2,1 
, H 2, 2 
1 1
j 1  j
41
4  1 1




 1  1
j 
 1  j 1
 1 1
 1
1  j  1 j 



 1
1 1  j  1 j 
,
H 0,3 
 1
4 1  j  1 j 



 1
1  j  1 j 
1  1
j 
 1  j 1


j  j
1  j  1 
1 j
, H1,3 
j
1 1 
1  j
4  1



j 
j
j
1
 j  1
1  1
 1  j 1 j 


1 1 
1  j 
1  1 j
, H 2,3 
1  1
4  1  j 1 j 



1 1 
1  j
 1 j
1
1
1
j 1  j
j 
1
1
 1  1 1  1
 1  j 1
 j  j  j  j 
 j 1




j  1
j j
j 
j
1 
1
1
1  j
1  j  1

, H 3,1 
, H 3, 2 
, H 3, 3 

1  j
j 
j
4  1
4   1  1  1  1
4  1 1 1 1 
4  1  j 1








1  j  1
j
j
j 
j  j
 j
 j 1  j 1 
 j j
 j
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
52
Example: 4x4 DFT
1
0
For F  
0

1
2 0
3 1 
, compute
comp te Tk ,l
2 1

2  1
Using Tk ,l  H k ,l , F  yields, e.g,
2
1
1
2
T0,0
 1

1  1
 H 0,0 , F  
4  1
 1

T2,3
  1  j  1 j  * 1


1  j  0
1   1 j
 H 2,3 , F  
,
4   1  j  1 j  0
 
  1 j
1  j  1

Yao Wang, NYU-Poly
1
1
1
1
1
1
1
1
1 1
1 0
,
1 0
 
1 1
2
1
1
2
2 0 

3 1   1
18
 1  2  2  0  0  1  3  1  0  1  2  1  1  2  2  1 

2 1
4
4
 
2  1 
2
1
1
2
2 0  
3 1   1
1
  1  2 j  2  j  3  j  j  2  j  1  2 j  2  j   (1  j )
2 1  4
4

2  1

EL5123: DFT and unitary transform
53
Property of Separable Transform
• When the transform is separable, we can
perform the 2D transform separately
separately.
– First, do 1D transform for each row using
basis vectors gl,
– Second, do 1D transform for each column of
the intermediate image using basis vectors hk.
– Proof:
M 1 N 1
M 1
N 1
M 1
T (k , l )   H (m, n) F (m, n)   hk (m) g (n) F (m, n)   h*k (m)U (m, l )
m 0 n 0
Yao Wang, NYU-Poly
*
k ,l
*
m 0
n 0
EL5123: DFT and unitary transform
*
l
m 0
54
•
•
Computational Savings of
Separable Transform
A MxN transform takes about MxN
calculations for each transform value,
and the total number of calculation is
M2N2 ((N4 if M=N))
If the transform is separable
1. Calculate M N-point
p
1D transform,, each
requiring N2 calculations
2. Calculate N M-point 1D transform, each
requiring
i i M2 calculations
l l ti
3. Overall: MN2+NM2 (2N3, if M=N)
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
55
Other Transforms
• Discrete cosine transform (DCT)
– Basis functions are real
– Can be thought of as “real
real version
version” of DFT
– We will discuss DCT and its application in image compression
(JPEG) later
•
•
•
•
•
•
Discrete
sc ete sine
s e ttransform
a s o ((DST)
S )
Hadamard transform
Walsh transform
Haar transform
Slant transform
Wavelet transform
– Multi-resolution transform
– We will discuss wavelet transform and its application in image
compression (JPEG2000) later
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
56
Homework
1. Find the NxN point DFT of the following 2D image f(m, n), 0 ≤ m, n ≤ N:
a) f(m, n) = 1, n = 0; f(m, n) = 0; n ≠ 0.
b) f(m,
f( n)) = 1
1, m = 0
0; f(
f(m, n)) = 0
0, m ≠ 0.
0
c) f(m, n) = 1, m = n; f(m, n) = 0, otherwise.
d) f(m, n) = 1, m = N – 1- n; f(m, n) = 0, otherwise.
From the result, what can you say about the relation between the directionality of an
image with that of its DFT?
2. One can use the DFT algorithms to compute the linear convolution of an image F(m,
n) with a filter H(m
H(m, n)
n). Let the convolved image be denoted by Y(m
Y(m, n)
n).
i) Suppose the image size is 256x256 and the filter size is 11x11: What is the
required size of the DFT to obtain the convolution of these two? Explain the exact
steps to obtain the convolution result.
ii) Suppose we use a 256x256 point DFT algorithm for F(m, n) and H(m, n), and
obtain Z(m, n) as follows: Z = IDFT (DFT(X).*DFT(H)). The DFT and IDFT in the
above equation are both 256x256 points. For what values of (m, n) does Z(m, n)
equal Y(m, n) ?
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
57
Homework (cnt’d)
3.
Show that the following vectors form an orthonormal basis set for any value of 
cos  
 sin  
h0  
, h1  


 sin  
 cos  
Determine the corresponding matrix A and B for the forward and inverse transform described by
Forward transform : t  Af ; Inverse transform : f  Bt.
4.
Consider a zero mean random vector u with covariance matrix
1 
Cu  

 1 
From the class of unitary transforms given in Prob. 3, determine the value of θ for which the
components of transformed vector v will be uncorrelated (i.e., the covariance matrix of the
transformed vector is a diagonal matrix).
5.
The basis images of the 2D-DCT are:
H 00 
1 1 1
11 1
1 1  1
1  1  1
; H 01  
; H10  
; H11  
;




2 1 1
2  1  1
2 1  1
2  1 1 
Calculate the transform of the image:F  1 2
 2 3


Find the reconstructed image F̂ obtained with the 2 largest coefficients (in magnitude). Calculate
E  m,n [ F (m, n)  Fˆ (m, n)]2 You
the error between the original and reconstructed
images defined by
2
should see that E  k ,l:t ( k ,l ) are omitted | t (k , l ) | where t(k, l) represents the transform coefficient
associated with the basis image Hkl.
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
58
Computer Assignment
1.
2.
Perform 2D DFT (using the fft2 function in Matlab) on several test images of your choice
(sample images are available at the Matlab image toolbox directory). Display the magnitude of
the DFT image with and without shifting (using fftshift) and with and without logarithmic
mapping, to see the effect of shifting and logarithmic mapping. Include the printouts in your
submission Also
submission.
Also, examine in which frequency range the DFT coefficients have large
magnitudes and explain why. Note that you may want to work on a small portion of your image
(say 256x256 or less), to save computation time.
Note that if your image is an RGB image, you should convert it to a grayscale image using
"rgb2gray" function and apply the above operations on the grayscale image only. If the original
image is an index image, you should not apply DFT to the index image directly, rather you
should derive the grayscale image from it, using the "ind2gray" function.
Write a program (in Matlab or C) which filters an image by zeroing out certain DFT coefficients.
The program consists of 3 steps:
1) performing 2D DFT (you can use the “fft2" function in Matlab);
2)) zeroing
g out the coefficients at certain frequencies ((see below);
)
3) performing inverse DFT to get back a filtered image. Truncate or scale the image properly
such that its range is between 0 and 255.
For part 2, try the following two types of filters:
(a) let F(k, l) = 0 for TN < {k, l} < (1 - T )N, T = 1/4, 1/8 (i.e., low-pass filtering);
(b) let F(k, l) = 0 for the following regions i) 0 ≤ {k and l}≤
l} TN;ii) 0
0<= k <=TN,
TN, and (1 - T)N ≤ll
≤ N -1;iii) (1 - T)N ≤k≤ N -1 and 0 ≤ {l}≤ TN; iv)(1 - T)N ≤k and l ≤ N -1;T=1/4, 1/8
Compare the original and processed images. Comment on the function of the two types of
filters.
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
59
Reading
• R. Gonzalez, “Digital Image Processing,”
Chap 4 except Sec
Chap.4,
Sec. 4
4.5.
5
• A. Jain, “Fundamentals of Digital Image
Processing ” Chapter 5
Processing,”
5.
Yao Wang, NYU-Poly
EL5123: DFT and unitary transform
60