Image Filtering: Noise
Removal, Sharpening,
Deblurring
Yao Wang
Polytechnic University, Brooklyn, NY11201
http://eeweb.poly.edu/~yao
Outline
•
•
•
•
Noise removal by averaging filter
Noise removal by median filter
Sharpening (Edge enhancement)
Deblurring
©Yao Wang, 2006
EE3414: Image Filtering
2
Noise Removal (Image Smoothing)
• An image may be “dirty” (with dots, speckles,stains)
• Noise removal:
– To remove speckles/dots on an image
– Dots can be modeled as impulses (salt-and-pepper or
speckle) or continuously varying (Gaussian noise)
– Can be removed by taking mean or median values of
neighboring pixels (e.g. 3x3 window)
– Equivalent to low-pass filtering
• Problem with low-pass filtering
– May blur edges
– More advanced techniques: adaptive, edge preserving
©Yao Wang, 2006
EE3414: Image Filtering
3
Example
©Yao Wang, 2006
EE3414: Image Filtering
4
Averaging Filter
• Replace each pixel by the average of pixels in a
square window surrounding this pixel
• Trade-off between noise removal and detail
preserving:
– Larger window -> can remove noise more effectively, but
also blur the details/edges
©Yao Wang, 2006
EE3414: Image Filtering
5
Example: 3x3 average
100
100
100
100
100
100
200
205
203
100
100
195
200
200
100
100
200
205
195
100
100
100
100
100
100
©Yao Wang, 2006
EE3414: Image Filtering
100
100
100
100
100
100
144
167
145
100
100
167
200
168
100
100
144
166
144
100
100
100
100
100
100
6
Example
©Yao Wang, 2006
EE3414: Image Filtering
7
Weighted Averaging Filter
• Instead of averaging all the pixel values in the window, give the
closer-by pixels higher weighting, and far-away pixels lower
weighting.
g (m, n) =
L
L
∑ ∑ h( k , l ) s ( m − k , n − l )
l =− L k =− L
• This type of operation for arbitrary weighting matrices is
generally called “2-D convolution or filtering”. When all the
weights are positive, it corresponds to weighted average.
• Weighted average filter retains low frequency and suppresses
high frequency = low-pass filter
©Yao Wang, 2006
EE3414: Image Filtering
8
Graphical Illustration
©Yao Wang, 2006
EE3414: Image Filtering
9
Example Weighting Mask
1
×
9
1
1
1
1
1
1
1
1
1
1
×
16
1
2
1
2
4
2
1
2
1
All weights must sum to one
©Yao Wang, 2006
EE3414: Image Filtering
10
Example: Weighted Average
1
×
9
100
100
100
100
100
100
200
205
203
100
100
195
200
200
100
1
1
1
1
1
1
100
200
205
195
100
1
1
1
100
100
100
100
100
100
100
100
100
100
100
144
167
145
100
100
167
200
168
100
144
166
100
100
100
©Yao Wang, 2006
1
×
16
1
2
1
2
4
2
1
2
1
100
100
100
100
100
100
156
176
158
100
100
100
174
201
175
100
144
100
100
156
175
156
100
100
100
100
100
100
100
100
EE3414: Image Filtering
11
Filtering in 1-D: a Review
• Continuous-Time Signal
∞
Time Domain : g (t ) = s (t ) ∗ h(t ) = ∫ h(τ ) s (t −τ )dτ
−∞
Frequency Domain : G ( f ) = S ( f ) H ( f )
∞
Filter frequency response : H ( f ) = ∫ h(t )e − j 2πft dt
−∞
• Extension to discrete time 1D signals
Time Domain (linear convolution) : s (n) = s (n) ∗ h(n) = ∑ h(m) s (n − m)
m
Frequency Domain : G ( f ) = S ( f ) H ( f )
Filter frequency response (DTFT) : H ( f ) = ∑ h(n)e − j 2πfn
n
H ( f ) is periodic, only needs to look at the range f ∈ (-1/ 2 ,1/ 2 )
1/2 corresponds to f s / 2
©Yao Wang, 2006
EE3414: Image Filtering
12
Filtering in 2D
Weighted averaging = 2D Linear Convolution
l1
k1
g (m, n) = ∑
∑ h(k , l ) s (m − k , n − l )
l =l0
k = k0
In 2D frequency domain G ( f1 , f 2 ) = S ( f1 , f 2 ) H ( f1 , f 2 )
Frequency response of the 2D Filter
H ( f1 , f 2 ) =
l1
k1
∑ ∑ h(m, n)e
− j 2π ( f1m + f 2 n )
m =l0 n = k 0
H ( f1 , f 2 ) is periodic, only needs to look at the square region
f1 ∈ (−1 / 2,1 / 2), f 2 ∈ ( −1 / 2,1 / 2).
©Yao Wang, 2006
EE3414: Image Filtering
13
Frequency Response of Averaging
Filters
0
0
1
h(m, n) = 0
9
0
0
• Averaging over a 3x3
window
H ( f1 , f 2) =
1
9
1
+
9
1
=
9
1
=
9
+
0
1
1
1
0
0
1
1
1
0
(
0
0
0

0
0
1 − j 2π ( f1 ⋅( −1) + f 2 ⋅( −1)) − j 2π ( f1 ⋅( 0 )+ f 2 ⋅( −1)) − j 2π ( f1 ⋅(1) + f 2 ⋅( −1))
e
+e
+e
9
(e
− j 2π ( f1 ⋅( −1) + f 2 ⋅( 0 ))
+ e − j 2π ( f1 ⋅( 0 ) + f 2 ⋅( 0 )) + e − j 2π ( f1 ⋅(1) + f 2 ⋅( 0 ))
(e
− j 2π ( f1 ⋅( −1) + f 2 ⋅(1))
+ e − j 2π ( f1 ⋅( 0 ) + f 2 ⋅(1)) + e − j 2π ( f1 ⋅(1)+ f 2 ⋅(1))
(e
j 2πf1
+ 1 + e − j 2πf1 e j 2πf 2 +
(e
j 2πf1
+ 1 + e − j 2πf1 e j 2πf 2
©Yao Wang, 2006
0
1
1
1
0
)
)(
(
)
)
)
)
(
)
1 j 2πf1
1
e
+ 1 + e − j 2πf1 ⋅1 + e j 2πf1 + 1 + e − j 2πf1 e − j 2πf 2
9
9
1
+ 1 + e − j 2πf 2 = (1 + 2 cos 2πf1 )(1 + 2 cos 2πf 2 )
9
)
EE3414: Image Filtering
14
H ( f1 , f 2 ) = H ( f1 ) H ( f 2 )
H ( f1 ) =
1
(1 + 2 cos 2πf1 ), H ( f 2 ) = 1 (1 + 2 cos 2πf 2 )
3
3
Sketch H(f1)
1
0.5
0
-0.5
30
20
10
0
©Yao Wang, 2006
EE3414: Image Filtering
0
5
10
15
20
25
15
Frequency Response of Weighted
Averaging Filters
1
1 
1  
b  =
b [1 b 1];
2  
(1 + b)
1
1
H (u , v) = (b + 2 cos(2πu ) )(b + 2 cos(2πv) ) /(1 + b) 2
1 b
1 
2
b
b
H=
(1 + b) 2 
1 b
1
0.8
0.6
0.4
0.2
0
30
20
10
0
©Yao Wang, 2006
0
5
10
15
20
25
EE3414: Image Filtering
16
Averaging vs. Weighted Averaging
1 1 1 1
1
H = 1 1 1 = 1[1 1 1];
9
1 1 1 1
H (u , v) = (1 + 2 cos(2πu ) )(1 + 2 cos(2πv) ) / 9
1 b
1 
H=
b b2
2 
(1 + b)
1 b
1
1
1  
b [1 b 1];
b  =
(1 + b) 2  
1
1
H (u , v) = (b + 2 cos(2πu ) )(b + 2 cos(2πv) ) /(1 + b) 2
1
1
0.8
0.5
0.6
0
0.4
0.2
-0.5
30
20
10
0
0
5
10
15
20
25
0
30
20
10
0
0
5
10
15
20
25
b=2
©Yao Wang, 2006
EE3414: Image Filtering
17
Interpretation in Freq Domain
Filter response
Low-passed image spectrum
Original image spectrum
Noise spectrum
0
f
Noise typically spans entire frequency range, where as natural images have
predominantly lower frequency components
©Yao Wang, 2006
EE3414: Image Filtering
18
Median Filter
• Problem with Averaging Filter
– Blur edges and details in an image
– Not effective for impulse noise (Salt-and-pepper)
• Median filter:
– Taking the median value instead of the average or weighted
average of pixels in the window
• Median: sort all the pixels in an increasing order, take the middle one
– The window shape does not need to be a square
– Special shapes can preserve line structures
• Order-statistics filter
– Instead of taking the mean, rank all pixel values in the window, take
the n-th order value.
– E.g. max or min
©Yao Wang, 2006
EE3414: Image Filtering
19
Example: 3x3 Median
100
100
100
100
100
100
200
205
203
100
100
195
200
200
100
100
200
205
195
100
100
100
100
100
100
100
100
100
100
100
100
100
200
100
100
100
200
200
200
100
100
100
195
100
100
100
100
100
100
100
Matlab command: medfilt2(A,[3 3])
©Yao Wang, 2006
EE3414: Image Filtering
20
Example
©Yao Wang, 2006
EE3414: Image Filtering
21
Matlab Demo: nrfiltdemo
Original Image
Corrupted Image
Filtered Image
Can choose between mean, median and adaptive (Wiener) filter with different window size
©Yao Wang, 2006
EE3414: Image Filtering
22
Noise Removal by Averaging Multiple
Images
©Yao Wang, 2006
EE3414: Image Filtering
23
Image Sharpening
• Sharpening : to enhance line structures or other
details in an image
• Enhanced image = original image + scaled version of
the line structures and edges in the image
• Line structures and edges can be obtained by
applying a difference operator (=high pass filter) on
the image
• Combined operation is still a weighted averaging
operation, but some weights can be negative, and the
sum=1.
• In frequency domain, the filter has the “highemphasis” character
©Yao Wang, 2006
EE3414: Image Filtering
24
Frequency Domain Interpretation
Filter response (high emphasis)
sharpened image spectrum
Original image spectrum
0
©Yao Wang, 2006
EE3414: Image Filtering
f
25
Highpass Filters
• Spatial operation: taking difference between current and averaging
(weighted averaging) of nearby pixels
– Can be interpreted as weighted averaging = linear convolution
– Can be used for edge detection
• Example filters
0 1 0   0 − 1 0 
1 − 4 1; − 1 4 − 1;

 

0 1 0  0 − 1 0 
1 1 1 − 1 − 1 − 1
1 − 8 1; − 1 8 − 1;

 

1 1 1 − 1 − 1 − 1
– All coefficients sum to 0!
©Yao Wang, 2006
EE3414: Image Filtering
26
Example Highpass Filters
 0 −1 0 
− 1 4 − 1


 0 − 1 0 
− 1 − 1 − 1
− 1 8 − 1


− 1 − 1 − 1
8
12
6
10
4
8
6
2
4
0
30
2
0
30
20
10
0
0
©Yao Wang, 2006
5
10
15
20
20
25
10
0
EE3414: Image Filtering
0
5
10
15
20
25
27
Example of Highpass Filtering
Original image
©Yao Wang, 2006
Isotropic edge detection
EE3414: Image Filtering
Binary image
28
Designing Sharpening Filter Using High
Pass Filters
• The desired image is the original plus an appropriately
scaled high-passed image
• Sharpening filter
f s = f + λf h
hs (m, n) = δ (m, n) + λhh (m, n)
x
x
x
©Yao Wang, 2006
fs(x)=f(x)+ag(x)
g(x)=f(x)*hh(x)
f(x)
EE3414: Image Filtering
29
Example Sharpening Filters
fs(x)=f(x)+ag(x)
g(x)=f(x)*hh(x)
f(x)
x
x
x
 0 −1 0 
 0 −1 0 
1
1
H h = − 1 4 − 1 ⇒ H s = − 1 8 − 1 with λ = 1.
4
4
 0 − 1 0 
 0 − 1 0 
− 1 − 1 − 1
− 1 − 1 − 1
1
1
H h = − 1 8 − 1 ⇒ H s = − 1 16 − 1 with λ = 1.
8
8
− 1 − 1 − 1
− 1 − 1 − 1
©Yao Wang, 2006
EE3414: Image Filtering
30
Example of Sharpening
− 1 − 1 − 1
1
H h = − 1 8 − 1
4
− 1 − 1 − 1
 − 1 − 1 − 1
1
H s =  − 1 16 − 1
8
 − 1 − 1 − 1
©Yao Wang, 2006
EE3414: Image Filtering
31
Example of Sharpening
λ=4
λ =8
©Yao Wang, 2006
EE3414: Image Filtering
32
Challenges of Noise Removal and
Image Sharpening
• How to smooth the noise without blurring the details
too much?
• How to enhance edges without amplifying noise?
• Still a active research area
©Yao Wang, 2006
EE3414: Image Filtering
33
Wavelet-Domain Filtering
Courtesy of Ivan Selesnick
©Yao Wang, 2006
EE3414: Image Filtering
34
Feature Enhancement by
Subtraction
Taking an image without injecting a contrast agent first. Then take the image again after
the organ is injected some special contrast agent (which go into the bloodstreams only).
Then subtract the two images --- A popular technique in medical imaging
©Yao Wang, 2006
EE3414: Image Filtering
35
Image Deblurring
• Noise removal considered thus far assumes the
image is corrupted by additive noise
– Each pixel is corrupted by a noise value, independent of
neighboring pixels
• Image blurring
– When the camera moves while taking a picture
– Or when the object moves
– Each pixel value is the sum of surrounding pixels The
blurred image is a filtered version of the original
• Deblurring methods:
• Inverse filter: can adversely amplify noise
• Wiener filter = generalized inverse filter
• Many advanced adaptive techniques
©Yao Wang, 2006
EE3414: Image Filtering
36
Example of Motion Blur
©Yao Wang, 2006
EE3414: Image Filtering
37
Inverse Filtering vs. Wiener Filtering
©Yao Wang, 2006
EE3414: Image Filtering
38
Constrained Least Squares Filtering
©Yao Wang, 2006
EE3414: Image Filtering
39
What Should You Know
•
•
•
•
•
•
How does averaging and weighted averaging filter works?
How does median filter works?
What method is better for additive Gaussian noise?
What method is better for salt-and-pepper noise?
How does high-pass filtering and sharpening work?
For smoothing and sharpening:
– Can perform spatial filtering using given filters
– Deriving frequency response is not required, but should know the
desired shape for the frequency responses in different applications
• What is the challenge in noise removal and sharpening?
• What causes blurring?
• Principle of deblurring: technical details not required
©Yao Wang, 2006
EE3414: Image Filtering
40
References
Gonzalez and Woods, Digital image processing, 2nd edition,
Prentice Hall, 2002. Chap 4 Sec 4.3, 4.4; Chap 5 Sec 5.1 – 5.3
(pages 167-184 and 220-243)
©Yao Wang, 2006
EE3414: Image Filtering
41