Adaptive Digital Zoom
Techniques Based on
Hypothesized Boundary
Author: 藍寅峻 (Y. C. Lan)
Written by: Ting-Hsuan Chang
Mobile Phone: 09633-31533
E-Mail: [email protected]
Outline
z
Introduction
z Weighting
Weighting-Based
Based Digital Zoom
Algorithms
z Area-Based
A
B
dR
Restoration
t ti and
dR
Resample
l
z Experimental
pe e ta Results
esu ts
z Conclusion
Outline
z
Introduction
z Weighting
Weighting-Based
Based Digital Zoom
Algorithms
z Area-Based
A
B
dR
Restoration
t ti and
dR
Resample
l
z Experimental
pe e ta Results
esu ts
z Conclusion
Introduction
z
Digital zoom is the process to scale up a
g
image
g to another higher-resolution
g
digital
image by using a computer.
z We can observe details in an image by
applying digital zoom algorithms.
Introduction (cont.)
z
The major problem of digital zoom
q is that we only
y have little
technique
information to generate a high-resolution
image from a low-resolution
low resolution one
one.
z Many researches have focused on
super-resolution algorithms using
p images.
g
multiple
Introduction (cont.)
z
The super-resolution
Th
l ti ttechnique
h i
d
deals
l
with images containing stationary scene
with objects and captured by a moving
camera.
z However, we do not always have many
ages of
o stationary
stat o a y scene
sce e with
t objects
objects.
images
z The digital zoom approaches using a
single image are developed to solve this
difficulty.
Introduction (cont.)
z
Because of the lack of information the
y value of interpolated
p
p
pixel is
intensity
guessed or interpolated by its
neighboring pixels
pixels.
z We can handle the problems in
frequency domain or in spatial domain.
Introduction (cont.)
Introduction (cont.)
Introduction (cont.)
z
Left: Original image (Digital zooming in
g )
red rectangle)
z Top right: nearest neighbor pixel copy
z Bottom
B tt
right:
i ht bilinear
bili
iinterpolation
t
l ti
Introduction (cont.)
z
Because the blurry and blocky effects
pp
on the edges
g when applying
pp y g
appear
bilinear interpolation.
z We propose “adaptive
adaptive digital zoom
techniques based on hypothesized
boundary”” to deal with the effects
ff
in this
thesis.
Outline
z
Introduction
z Weighting
Weighting-Based
Based Digital Zoom
Algorithms
z Area-Based
A
B
dR
Restoration
t ti and
dR
Resample
l
z Experimental
pe e ta Results
esu ts
z Conclusion
Introduction
z
Our motivation is to keep the object
g sharp
p and to have better results.
edges
Terminologies and Algorithm
O
Overview
i
z
Object: An image is composed of many
j
An object
j
in an image
g is defined
objects.
as a region where the pixels in it have
similar property (such as intensity).
intensity)
z Run: A run is defined as a segment of
pixels in an image scan line and with
p p y
similar property.
Terminologies and Algorithm
O
Overview
i
(cont.)
(
t)
z
Run boundary: The boundary of two
g scan
different runs in the same image
line is called run boundary.
z Hypothesized boundary (HB): The
hypothesized boundaries are obviously
located from
f
the run boundaries in the
nearest scan line to the nearest ones in
the third scan line.
Terminologies and Algorithm
O
Overview
i
(cont.)
(
t)
z
HB pixel set: A pixel set that hypothesized
boundary passes through.
Algorithm Overview
z
We divide the scale up process into two subprocesses, one for vertical and the other one
for horizontal process
Algorithm Overview (cont.)
z
In the beginning of the adaptive
g
, we copy
py the intensity
y values of
algorithm,
pixels in the original image to the
corresponding pixels in the scale-up
scale up
image.
z How to generate a pixel to be
p
in our algorithm
g
depends
p
on
interpolated
whether the pixel is in HB pixel set or not.
Algorithm Overview (cont.)
z
If itits gradient
di t values
l
off its
it interpolating
i t
l ti
pixels is larger than a user-defined
threshold, the pixel falls in an HB pixel
set.
z We use our weighting-based algorithm to
deal with
dea
t these
t ese kinds
ds o
of p
pixels
e s because
the hypothesized boundary passes
through it.
z If not, we use the linear interpolation.
Algorithm Overview (cont.)
Linear Weighted Sum Algorithm
NRB: Nearest Run-Boundary
LRB: Left Run-Boundary
RRB: Right Run
Run-Boundary
Boundary
D: Distance
Linear Weighted Sum Algorithm
(
(cont.)
t)
Linear Weighted Sum Algorithm
(
(cont.)
t)
Nearest neighborhood
pixel
i l copy
1x
4x
2x
Bilinear interpolation
1x
4x
2x
Linear weighted sum
1x
4x
2x
Sigmoid Weighted Sum Algorithm
Sigmoid Weighted Sum Algorithm
(
(cont.)
t)
I ( P1 .5 ) = Sig ( D NRB ( P1 ) − D NRB ( P2 )) × I ( P1 ) + (1 − Sig ( D NRB ( P1 ) − D NRB ( P2 ))) × I ( P2 ),
Sigmoid Weighted Sum Algorithm
(
(cont.)
t)
C = 0.01
1x
4x
2x
C = 1.0
1x
4x
2x
Realization for a Binary Image:
L
Larger-NRBD
NRBD S
Selection
l ti
NRBD: Nearest Run Boundary Distance
(a) Nearest-neighbor pixel copy
(c) Linear weighted sum
(b) Bilinear interpolation with binarization (d) Linear weighted sum with binarization
Realization for a Binary Image:
L
Larger-NRBD
NRBD S
Selection
l ti ((cont.)
t)
z
We propose a one-pass algorithm to
p
the two-pass
p
algorithm
g
linear
replace
weighted sum followed by binarization.
z We may call the process as larger
largerNRBD selection algorithm.
Realization for a Binary Image:
L
Larger-NRBD
NRBD S
Selection
l ti ((cont.)
t)
Realization for a Binary Image:
L
Larger-NRBD
NRBD S
Selection
l ti ((cont.)
t)
I ( P1 .5 ) = I (arg max( D NRBD ( P1 ), D NRBD ( P2 ))),
Take a Break
Outline
z
Introduction
z Weighting
Weighting-Based
Based Digital Zoom
Algorithms
z Area-Based
A
B
dR
Restoration
t ti and
dR
Resample
l
z Experimental
pe e ta Results
esu ts
z Conclusion
Introduction
z
We assume an intensity value of a pixel
g
of the light
g energy
gy in a
is the integration
Charge-Coupled Device (CCD) grid.
z Based on the CCD grid mode and the
hypothesized boundary concept we can
restore the infinite-high-resolution
f
or
g
locally.
y
continuous signal
Introduction (cont.)
CCD Grid Mode and the Hypothesized
B
Boundary
d
L
Localization
li ti
z
We want to scale up the image two-byy divide a pixel
p
in a lowtwo. We may
resolution image into four pixels to get a
high-resolution
high
resolution image
image.
z The pixel center in the low-resolution
image locates on one pixel center in the
g
image.
g
high-resolution
CCD Grid Mode and the Hypothesized
B
Boundary
d
L
Localization
li ti
CCD Grid Mode and the
H
Hypothesized
th i d B
Boundary
d
L
Localization
li ti
(a) Area-based algorithm.
((b)) Linear weighted
g
sum
algorithm.
Local Restoration
(d), (e), (f) : mirrors of (a), (b), (c)
Local Restoration (cont.)
z
Use (a) for an example. To count the
area of A1
I ( P1 ) = (1 − A1 ) × I R 1 + A1 × I R 2
I ( P2 ) = I R 2
z
IR1: Original intensity of P1
IR2: Original
g
intensity
y of P2
Let
et tthe
e area
a ea o
of eac
each p
pixel
e be 1
I R1 =
I ( P1 ) − A1 × I R 2
(1 − A1 )
I R 2 = I ( P2 )
Local Resampling for Scaling up by
Two
z
Using the high-resolution area divided by
yp
boundary
y to calculate
the hypothesized
the intensity
z Each high
high-resolution
resolution pixel’s
pixel s area
becomes 0.5
z Take (a) for example
Local Resampling for Scaling up by
T
Two
(cont.)
(
t)
I ( P1 ) = (0.5 − A1 ) × I R1 + A1 × I R 2
'
'
I ( P1.5 ) = I R 2
'
I ( P2 ) = I R 2
'
'
Area-based
algorithm
g
1x
4x
2x
Outline
z
Introduction
z Weighting
Weighting-Based
Based Digital Zoom
Algorithms
z Area-Based
A
B
dR
Restoration
t ti and
dR
Resample
l
z Experimental
pe e ta Results
esu ts
z Conclusion
Experimental Results
z
z
z
Visualization results
Digital
g
zoom on red
rectangle
Five methods will be
compared.
Experimental Results (cont.)
(a) Nearest-neighbor pixel copy
(b) Bili
Bilinear iinterpolation
t
l ti
(c) Linear weighted sum
(d) Sigmoid weighted sum
(e) Area-based algorithm
Experimental Results (cont.)
((a)) Original
g
image
g
(b) Nearest-neighbor
pixel copy
(c) Bilinear interpolation
(d) Larger-NRBD
selection
Experimental Results (cont.)
((a)) Original
g
image
g
(b) Nearest-neighbor
pixel copy
(c) Bilinear interpolation
(d) Larger-NRBD
selection
Experimental Results (cont.)
((a)) Original
g
image
g
(b) Nearest-neighbor
pixel copy
(c) Bilinear interpolation
(d) Larger-NRBD
selection
SNR and Sharpness Comparison
Original Image
Sharpness: 153,037,221
153 037 221
SNR: Signal-to-Noise Ratio
4x4 nearest neighbor pixel copy
SNR: 9
9.551811
551811
Sharpness: 99,597,786
SNR and Sharpness Comparison
(
(cont.)
t)
4x4 bilinear interpolation
4x4 linear weighted sum
SNR: 12
12.807445
807445
SNR: 12
12.730128
730128
Sharpness: 23,033,292
Sharpness: 26,264,322
SNR and Sharpness Comparison
(
(cont.)
t)
4x4 sigmoid weighted sum
4x4 area-based
area based algorithm
SNR: 12
12.655672
655672
SNR: 13.046390
13 046390
Sharpness: 23,194,836
Sharpness: 75,389,265
Outline
z
Introduction
z Weighting
Weighting-Based
Based Digital Zoom
Algorithms
z Area-Based
A
B
dR
Restoration
t ti and
dR
Resample
l
z Experimental
pe e ta Results
esu ts
z Conclusion
Conclusion
z
Nearest-neighbor pixel copy
Advantage:
g fast,, simple
p
z Disadvantage: blocky, useless in scaling up
process
z
z
Bilinear interpolation
Advantage: fast, simple
z Disadvantage:
g blocky
y and blurry
y effects on
edges
z
Conclusion (cont.)
z
Linear weighted sum
z
z
z
Advantage: efficient, fast, no blocky effects on
edges, good visualization
Disadvantage: not sharp enough on edges
Sigmoid weighted sum
z
z
Advantage: a user-defined parameter for tuning,
sharpness larger than bilinear interpolation even
linear weighted sum
Disadvantage: SNR smaller than bilinear
interpolation
Conclusion (cont.)
z
Larger-NRBD selection
Advantage:
g suitable for binary
y images
g
z Disadvantage: only for binary images
z
z
A
Area-based
b
d algorithm
l ith
Advantage: good SNR and sharpness
values
z Disadvantage
g over-enhancement on edges,
g ,
worse visualization
z