I. He et al.: Modified Fast Climbing Search Auto-focus Algorithm with Adaptive Step Size Searching Technique far Digital Camera
257
Modified Fast Climbing Search Auto-focus Algorithm with
Adaptive Step Size Searching Technique for Digital Camera
Jie He, Rongzhen Zhou, and Zhiliang Hong
Abstract - A practical real-time auto-focus algorithm for
digital camera ispresented. and it improves the reliability and
speed of auto-focus process, especially suitable for mega-pixel
high definition camera. The proposed algorithm adopts
threshold gradient and edge point count technique besides
focus value function, instead of traditional two-stage climbing
search algorithm that uses focus value function only.
Additionally, a relative difference ratio circuit is also
proposed, which can implement adaptive step size searching
to increase the searching speed. By adopting the modified
algorithm on the protoiype of our mega-pixel digital camera,
real-time auto-focus function is verified. The proposed
algorithm is implemented in test camera chip that has been
manufactured in 0.25um CMOS digital process.
Index Terms - modified fast climbing search, focus
value, threshold gradient, edge point, adaptive step size
searching, relative difference ratio.
I.
INTRODUCTION^
I
n the past years, digital camera has been playing a more and
more important role in the consumer electronics market.
During the recent years, the market of digital camera has been
obtaining an average growth of 130% per year in China. The
low cost, high definition and compact digital camera will be
the focus in consumer market. Auto-focus is a basic function
in mega-pixel digital camera.
There have been several auto-focus techniques reported in
[I]-[3]. The most basic method is to calculate the focus value
and derive the hest-focused lens position by climbing search
method. Because auto-focus algorithm must be real-time, the
traditional auto-focus algorithm will have some problems due
to computation expanding with pixel number increasing. An
obvious problem is that convergence speed of auto-focus is
slowed down due to more computation. Additionally, the
probability of defocus may increase because of the tradeoff
between the computation and the reliability. Sub-sampling
method seems to be usually employed to reduce the
''
This work was supported in pan by the 863 Plan of China under Grant
No. 2002AAIZ1450.,
Jie He is with the Microelectronics Department, hformation School of
Fudan University, Shanghai China (e-mail: hjee@ 263.net).
Rongzhen Zhou is with the Microelectronics Department, Information
(e-mail: mho"@
School of Fudan University, Shanghai China
fudan.edu.cn).
Zhiliang Hong is with the Microelectronics Department, Information
(e-mail: zlhong@
School of Fudan University, Shanghai China
fudan.edu.cn).
Contributed Paper
Manuscript received A p d 3,2003
computation especially in high definition camera. However, in
the sub-sampling method, some detailed information is lost
and noise floor increases. As a result, the lens deviates from
the best-focused position. To handle this problem, sub-window
method [2] was proposed which can keep more detailed
information. However, when sub-window becomes small, the
focus reliability decreases due to losing information outside
the windows. Based on above problems, we develop modified
fast- climbing search (MFCS) method to reduce the
computation and improve the reliability.
Furthermore,
adaptive step size technique for searching process has faster
convergence speed than constant step size. A circuit named
relative drfference ratio (RDR) 'is also introduced to realize
adaptive step size searching and save the focusing time.
Firstly, some fundamentals in auto-focus are introduced in
part 11. In part I11 our modified fast climbing search method
will be described in detail and in part IV adaptive step size
searching with a relative dfference ralm circuit will be
discussed. The experiment results will be given in part V.
Conclusion is in part VI.
11. FOCUS VALUE AND CLIMBING SEARCH ALGORITHM
A . Focus value
Focus value is a performance parameter to measure the
focus degree of an image. Focus value is generally based on
illuminant gradient of an image. It often refers to the sum of
the absolute gradient or the gradient energy of pixels in an
image as following
where FV is focus value and g (x, y) is the illuminant gradient
at point (x. y), The sum of the gradient energy is more
preferred, because it can make more difference between the
focused image and the defocused image.
There are various methods for gradient computation. In
general, gradient can be defined as
r
0098 3063100 $10.00 0 2003 IEEE
Authorized licensed use limited to: Guangdong Univ of Tech. Downloaded on December 25, 2008 at 00:25 from IEEE Xplore. Restrictions apply.
iEEE Transactions on Consumer Electronics, Vol. 49, No. 2, MAY 2003
258
focused
Lnns Posltlm
Fie. 2 Focus value versus lens oarition
Fig.1 Illuminant window centered st point (x, y)
in whichf(x. yjlx3is the 3 by 3 illuminant matrix with point (x,
+ + +
y) centered as shown in Fig.1, and (x, i&, i&,are weighted
is defined as: for two M X N matrixes
vectors, &d operator
M
A and B, A O B = z
N
z a , b , . According
i
to Equation
j
(Z), various gradient methods can be realized by choosing
+
corresponding vectors
+
+
a, i&2ie3,
+
+
=(-I, -2, - I ) ~ ,a2 =(a, a,
a3=(1,
2, ilT,
Sobel gradient [3] is realized, which is a specific type of
Tenengrad gradient[4 j .
+
+
+
(2) If a, =(a, -1, O)T, a, =(O, 1, O)T, a3 =(O, 0 ,
Robert gradient is realized, and it can also be used to calculate
Laplacian gradient [ 5 ] if f(x, y) is replaced by the first
derivative of the original illuminant matrix.
algorithm is split to two different searching stages in order to
obtain fast convergence speed. Generally, in the first searching
stage, a large step size is used for lens moving, which is always
a constant. When the mountain peak is found, it enters into the
second searching stage, and the smallest step size is used for
lens moving toward the hest focus position. Here the first stage
searching can be defined as out focused region searching
(OFRS), as well as the second stage searching can be defined
as focused region searching (FRS). The operations in these
two stages with modification will be discussed in detail at next
section.
--*
(1) if
a,
oy,
+
+
(31 If a, =(O, a, , 017, a, =(O, a,, 017, a3=(O,a3,
O)l, FSWM gradient reported in [2] can be realized only if the
transform with median filter is applied to the original
illuminant matrix firstly.
The gradient operator for auto-focus must be able to extract
the high frequency components out of the image and suppress
the white and impulse noise. Other effective gradient operators
[6]-[8] are introduced on the topic of image edge detection. In
fact, the idea used in edge detection is very similar to the idea
for auto-focus, so the real-time edge detection techniques can
be modified for auto-focus, which is also adopted in the
proposed modijkd fast climbing search algorithm. Fig.2 is
known as focus cume, which indicates the relationship
between focus value and lens position. The peak of this curve
is referred as the hest focused position. The region near the
peak can be referred to focused region, and the region far from
peak can be referred to out focused region.
B. Climbing Search Algorithm
Another topic in this section is on searching.~
algorithm.
Climbing search algorithm has been developed for fast
searching, which is described as MCS (mountain Climb servo)
in [I] or HCS (hill-climbing search) in [21. Climbing search
111. PROPOSED MODIFIED
FAST CLIMBING SEARCH
ALGORITHM
Modified fast climbing search (MFCS) is developed to
improve the focus reliability and speed up the searching
process, which adopts different methods for out focused region
searching and focused region searching. The center area of
an image is the most useful area, which is always used to
reduce the computation and the complexity of an auto-focus
system by the sub-window method. As shown in Fig.3, the
center area consists of AREAl and AREA2, where AREAl
involves AREA2.
,
<
.
W
F i g 3 Image area planning
As mentioned afore, sub-window method will encounter
difficulties in dealing with trade-off between computation area
and focus reliability. Traditional climbing search algorithm
Authorized licensed use limited to: Guangdong Univ of Tech. Downloaded on December 25, 2008 at 00:25 from IEEE Xplore. Restrictions apply.
J. He et al.: Modified Fast Climbing Search Auto-focus Alganthm with Adaptive Step Size Searching Technique for Digital Camera
...!.............................
AREA1
gradient
(f1X.Y))
threshold
detect
i
!
edge points
1
COULll
j
j
........................
:
~...L
image
oat3 inpvt
259
i
,._
..........................
i
j
(FV)cnerge
....................AREA2
............
algorithm
j
1
Fig. 4 Flow diagram of modified fast climbing search
with sub-window method and constant moving step size is not
very suitable for fast auto-focus control in mega-pixel camera.
Compared with traditional climbing.,.:search algorithm,
MFCS has the following modifications: . :
( I ) AREAl is selected for out focused region searching
(OFRS), and AREA2 is for focuked region searching (FRS).
However, in traditional methods,.AREAZ is for OFRS in order
to reduce the computation and AREA1 is for FRS in order to
hold the accuracy.
(2) Traditionally, the focus value method is used for
computaiion in both OFRS and FRS only. In MFCS, edge
detection method is used in OFRS and focus value method is
used in FRS.
( 3 ) Threshold detect in proposed MFCS algorithm can he
used to suppress the background noise.
(4) Adaptive step size searching technique is developed to
accelerate the searching in OFRS. A smart circuit named
relative difference ratio (RDR)is designed to realize it.
<ill'
_ _ _ _>
0
in OFRS it enables the edge point count module and disable
the FV module, in FRS it enables the FV and disables the edge
point count.
As shown in Fig.5, the algorithm begins with OFRS state,
and at first it decides the moving direction of lens. ARer the
direction is decided, focus decision module compares the edge
point number (EPN) between current image and last image. If
edge point number of current image is larger than that of last,
direction is kept and OFRS goes on, otherwise direction is
reversed and OFRS changes to FRS, which means that the
peak is found. In FRS, the initial operation is the same as in
OFRS, and the difference is that focus decision module
compares the focus value of two consecutive images instead of
edge point number. In FRS, When the condition of direction
reverse is met the same as in OFRS, the focused position is
found.
OFRS
FRS
I"lll2l
po'illon
LZ", p o l l l l o n
Fig. 5 FOCUS
curve of MFCS algorithm
MFCS is a real-time algorithm and the main flow is shown
in Fig.4. The first two lines of computed area are stored in
memories, so it realizes the real-time processing. The 3 X 3
pixel window is for gradient calculation of pixel (x?y). The
output gradient passes through the threshold detect module. If
gradient is larger than threshold, the gradient value is output to
the next module with a valuable flag, otherwise zero is output
with a valueless flag. Edge point count module receives the
valuable flag and increases the edge point counter by one. FV
module receives both valuable flag and gradient, and
calculates the focus value of the computation area. Focus
decision module controls the operation of OFRS and FRS, and
Because only the edge point count method is used in out
focus area searching, the hardware requirement is much
smaller than that using focus value method. Because the most
time of the focus searching is in out focus area searching
stage, the heavily reduced computation results in the power
saving. It is ease to increase the size of sub-window AREA1
without significant length increasing of EPN counter. With the
increased size, more image information can be kept. It is
possible to increase size of sub-window AREAl to include
more detailed image information. By the edge point counter
method, noise performance is also improved. If impulse noise
appears, its effect on the edge point counter is attenuated to
Authorized licensed use limited to: Guangdong Univ of Tech. Downloaded on December 25, 2008 at 00:25 from IEEE Xplore. Restrictions apply.
IEEE Transactions on Consumer Electronics, Vol. 49, No. 2, MAY 2003
260
one. Additionally, threshold gradient detector reduces the
effect of background noise. Thus, the computation area
increases in MFCS without increasing hardware consumption
and at the same time the system becomes more robust and
more power efficient.
IV. ADAPTIVE STEP SIZE SEARCHING WITH RELATIVE
DIFFERENCE
RATIO CIRCUIT
Adaptive step size searching technique can be applied in
OFRS of proposed modifiedfasf climbing search algorithm.
Adaptive step size searching is based on the following
principles:
(1) If the edge point numbers of two consecutive images
are in similar scale, it indicates that the last change of lens
position could not improve the focus quality much, which
means the position of lens is far away from the focused area,
and the next moving step size should he,increased.
(2) If the edge point numbers of two consecutive images
are in large relative difference, it indicates that the last change
of lens position has improved the focus quality significantly,
which means that the lens has been stepping into the focused
area, and the next moving step size should be decreased.
Because the edge point number is a variable, the concept
relative difference ratio (RDR) is used to determine how
similar edge point numbers are. The definition is: for A and E,
if A>B, then (A-B)/A is defined as relative diffeerence ratio
between A and B. Let’s define output C= ClCO as shown in
Table 1.
If C2CI=O0, relative difference ratio is within lis.
TABLE11
LSL AND COMPARATOR BASED ENCODING FOR 118 RDR
LSLby 1 bit
>=A
<A
LSLhy2bits
-
>=A
<A
LSL b y 3 bits
-
-
>=A
LSLby4 bits
-
ClCO
11
10
<A
-
>=A
01
or <A
00
Adaptive step size searching is shown with line OFRS in
Fig.5. At the beginning, where focus curve is flat and RDR is
small, larger step size is used for focus search. It is faster to
reach the peak region.of focus curve than the traditional
constant step size method. When approaching the peak region,
where focus curve is steep and RDR is large, small step size is
used.
State flow diagram for MFCS algorithm is shown in Fig.6,
which is split into two stages: OFRS and FRS. The above
mentioned adaptive step size searching is included in OFRS
stage
I
I
I
I
I
I
I
TABLE I
LOGSCALEENCODING TABLE FOR 118 RDR
ClCO
(A-B)/A, if A>B
00
[0, 1/81
01
10
11
[lis, 1/41
[1/4, 1/21
[1/2, 11
Here a simple method is given to implement the RDR
circuit. For example, A=AlA2AI& and B= BIB2BlBo, A>B,
A-B=B’= B’,B’,B’,B’o, to realize a 1/8 RDR, at first expand
A and B’ by adding four zeros before the MSB, then
A=0000A3A2Al& and B’=0000B’3B’2B’IE’o. Logical shift
left (LSL) B’ by one bit and B’=OOOB’lB’~B’IB’oO,
if B ’ a A ,
output CICo=ll;if B’<A, repeat LSL and comparison
operations, the results are shown in Table 11. It is easy to
extend the method from 4-bits to N-hits by the larger
comparator, from U8 RDR to 1/2” RDR by more left-shift
operations.
I
I
I
I
I
I
I
I
-
Ng.6 State flow diagram for MFCS algorithm
Authorized licensed use limited to: Guangdong Univ of Tech. Downloaded on December 25, 2008 at 00:25 from IEEE Xplore. Restrictions apply.
261
I. He et al.: Modified F a t Climbing Search Auto-focus Algorithm with Adaptive Step Size Searching Technique for Digital Camera
Fig.7 Prototype test camera
V. EXPERIMENT
RESULTS
In our prototype mega-pixel camera, shown in Fig.7, we
realized the interface of CCD, LCD, TV, CFA, USB, and
Step-Motor etc. In the middle of hoard is the test camera chip,
Both under median and weak exposure condition (The strong
exposure condition is similar to weak exposure condition.), the
ratio of edge point numbers in focused and out focused area
with threshold value above 32 is larger than 1.2 (i.e.
RDR=1/6), and this is the explicit reason to adopt 1/8 RDR
circuit. With the threshold increasing, the ratio increases as
well as the relative difference ratio. When the exposure time
increases, the optimal threshold gradient should he set smaller
due to the decreasing luminance gradient. An additional
average luminance detector realizes the threshold control.
Fig. 9 shows the test images in different distance with
prototype camera, 1 meter and'5 meters. In Fig.9 (a) and (c)
are images hefore auto-focus, (h) and (d) are images after
auto-focus. Many experiments results show that the auto-focus
fknction realized by proposed MFCS methods is real-time,
robust and practical. Fig.10 is the die'photo of the test camera
chip, and auto-focus module is in the white frame at the rightbottom.
a
Fig.8 Edge point number in focused and out-focus area with different threshold gradient
(a) Medium exposure (b) Weakexposure
Fig.9 Test images before autofoeus versus after autofoeus (a) Image in 1 meter before autofocus (b) Image in 1 meter after autofocus
(c) Image in 5 meters before autofoeus (d) Image in 5 meters after autofoeus
which was manufactured in 0.251 'CMOS digital process
technology. The test step motor and lens are shown at the right
bottom of the figure. Fig.8 shows the experimental results for
edge point numbers and their ratio. The horizontal axle is the
threshold gradient. The left vertical axle is edge point numbers
and the right vertical axle is their ratio r (It is not the relative
difference ratio). The lines of square points are the tested edge
point numbers in focused and out focused region versus
different threshold gradient. The line of dot points indicates
the edge point number ratio of focused region to out focused
region, and the defined relative difference ratio RDR = 1- lir.
Authorized licensed use limited to: Guangdong Univ of Tech. Downloaded on December 25, 2008 at 00:25 from IEEE Xplore. Restrictions apply.
IEEE Transactions on Consumer Electronics, Vol. 49, No. 2, MAY 2003
262
ACKNOWLEDGMENT
The authors thank for the work of the implementation of the
camera test system by Xuefeng Chen, Feng Liu, Jinghua Ye,
Yanfeng Su, Yajie Qin, Xiaofeng Yi and Tiankang Liao.
Without their diligent work, there would be nothing on this
work.
REFERENCES
[I] Kazushige Ooi etc. “An advanced autofocus system for video camera
usine ouasi
~,~condition reasonid. IEEE Trans. an Consumer Electronics,
Vo1.36, No.3, pp526-530, Aug. 1990
121 K. S. Chai, J. S. Lee, S. 1. KO, “New autofocus technique using the
freauencv selective weighted median filter for video cameras”. IEEE Trans.
On Consumer Electronics, Vo1.45, Na.3, pp.820-827, Aug. 1999.
[3] J. H.’Lee, K. S. Kim, B.D. Nam, “Implementation of a passive automatic
focusing algorithm for digital Still camera”, IEEE Trans. On Consumer
Electronics, Vo1.41, No.3, pp.449-454, Aug. 1995.
in ,computer vision, Ph.D. thesis,
[4] J. M. Tenenbaum, Acca&”dation
Stanford University, 1970.
[5] S. K. Nayar, Y. Nakagawa, “Shape from focus”, IEEE Trans. Pattem
Analysis and Machine Intelligence, Vol. 16, pp.824-831, Aug. 1994.
[6] M.B. Ahmad, T.S. Choi. “Local threshold and Boolean function based
edge detection”, IEEE Trans. On Consumef Electronics, Vol. 45, No. 3,
pp.674-679, Aug. 1999.
[7] Chang J. J., “Modified 2D median filter for impulse noise suppression in a
real-time system”, IEEE Trans. On Consumer Electronics, Vo1.41, No.1,
pp.73-80, Feb. 1995.
[SI C.T. Chen, L.G.Chen, J.H. Hsiao. “VLSI implementation of a selective
median filter”, IEEE Trans. On Consumer Electronics, Vol. 42, No.1, pp.3342. Feb. 1996.
I
.~
Fig.10 Die photo of camera chip, embedded with proposed
auto-focus module
VI. CONCLUSION
In this article, a real-time auto-focus algorithm named
modified fast climbing search (MFCS), is presented, which is
able to apply in mega-pixel digital camera. MFCS uses the
threshold gradient and edge point counter methods to suppress
the noise effectively. Also a relative dfference ratio circuit is
developed to realize the adaptive step size searching and
increase the searching speed. Totally in our prototype megapixel camera, MFCS with all presented methods is realized
and verified.
I
~
I
Authorized licensed use limited to: Guangdong Univ of Tech. Downloaded on December 25, 2008 at 00:25 from IEEE Xplore. Restrictions apply.