BULETINUL INSTITUTULUI POLITEHNIC DIN IAŞI
Publicat de
Universitatea Tehnică „Gheorghe Asachi” din Iaşi
Tomul LVII (LXI), Fasc. 4, 2011
SecŃia
AUTOMATICĂ şi CALCULATOARE
RECOGNITION METHOD FOR 2D POLYGONAL OBJECTS
BY
FLORIN ROTARU∗1, SILVIU-IOAN BEJINARIU1, MIHAI BULEA2,
CRISTINA DIANA NIłĂ1 and RAMONA LUCA1
1
Institute of Computer Science, Romanian Academy, Iaşi Branch, România
2
Synaptics, Santa Clara, CA, USA
Received: September 19, 2011
Accepted for publication: November 16, 2011
Abstract. A 2D polygons recognition method is proposed. First, a
polygonal vertex detection is applied and the two strongest guesses are retained.
Then a polygonal fitting algorithm using as input the two vertices and the object
contour provides a precise object identification and description. The algorithm is
suitable for robot vision, quality control or photogrammetry applications when
the image objects to be processed have polygonal shapes. A software
environment was designed to test and use the proposed method, and to evaluate
its speed and accuracy.
Key words: vertex detection, polygonal shape approximation, 2D recognition.
2000 Mathematics Subject Classification: 68T10, 17B69.
1. Introduction
The proposed method is a generalization of a quadrilateral recognition
and fitting algorithm proposed in (Bulea, 1996; Bulea, 2003), applied also in
(Rotaru et al., 1997). Initially the 2D polygon adaptive recognition approach was
developed as part of a much larger project, (Teodorescu et al., 2010), to operate
on low resolution ultra-sound images and identify convex polygonal shapes,
∗
Corresponding author; e-mail: [email protected]
116
Florin Rotaru et al.
(Rotaru et al., 2011a). In (Rotaru et al., 2011b) the recognition was extended to
any kind of polygonal shapes for ultrasound images. This paper is a synthesis of a
general method to recognize any kind of poligonal shape on any kind of images.
In the noisy images the retrieved polygons have quite round vertices, as
illustrated by Fig. 1 and the classical recognition methods might fail.
The paper describes only the object recognition part. The further use of
the results depends on the specific application. In robot vision applications the
recognized features can be used for pose calculation to reorient the robot head
during the execution step in order to reproduce the operation learned in a previous
training phase, (Rotaru et al., 2010). Also, the result of the proposed method can
be used to build object descriptors as in (Museros & Escrig, 2005) and then apply
it for autonomous robot navigation. In quality control the extracted features can
be used as statistical descriptors. For instance in (Bulea, 1996), based on the
recognition part, warp and weft yarn density, gap size between yarns, yarn
diameters, cover factor are computed to estimate the quality of textile sieves. In
(Rotaru et al., 1997) the Vickers hardness was computed from diagonal
measurements of the recognized quadrilateral obtained with a Vickers indenter.
2. Pre-Processing Steps
The first two steps of image processing chain are classical: binarization
and component labeling. However while the component label is quite the same
with the one used in (Bulea, 1996; Bulea, 2003; Rotaru et al., 1997; Rotaru et
al., 2011a; Rotaru et al., 2011b) the segmentation step depends on image type.
a
b
Fig. 1 − Test images; a − Original images, b − Preprocessing results.
Bul. Inst. Polit. Iaşi, t. LVII (LXI), f. 4, 2011
117
2.1. Binarization Methods for Ultrasonic and Raster Images
1. Binarization for ultrasonic images
The object detection is performed in the following steps:
− shadow elimination - a specific characteristic of the ultrasonic images
is the presence of the acoustic shadow as a compact dark area (Fig. 2). This area
is detected and replaced by gray (or white) pixels.
− edge detection - a Gabor filter based procedure is used to detect the
edge pixels.
− a sequence of dilation, erosion and fill operations are applied to
connect the contours, and fill the obtained objects.
− objects filtering. Because of noisy images, a lot of false objects are
detected. Depending on the area of the expected objects, some of these false
objects are removed, using an area based criterion.
The binarization result is illustrated by Fig. 3.
More elaborated methods for objects detection, based on edge
information fusion techniques (Bejinariu et al., 2010) can be used in the
segmentation step.
Fig. 2 − Original test image.
Fig. 3 − Preprocessing results.
2. Binarization for raster images
Using a linear digital non-recursive filter the histogram vector H(j) of
current image is filtered until it has only two local maxima and one minimum
between them. A third order filter:
y[ j ] =
1
( H [ j − 1] + H [ j ] + H [ j + 1])
3
(1)
was successively applied until the filtered histogram has only two local
maximum values.
The binarization threshold is a function of the positions of maximum
points. If m1 , m2 are the gray-levels for which the histogram vector reaches its
118
Florin Rotaru et al.
two maxima and H [m1 ],H [m 2 ] are these maximum values, then the
segmentation threshold is:
T=
m1 + m2
2
or T =
m2 H [ m2 ] − m1 H [ m1 ]
H [ m2 ] − H [ m1 ]
(2)
Please note that this method requires a distinct bimodal histogram, with
modes not very different in size, otherwise, if one mode is very small, it might
be filtered out. For this kind of situations other methods might be needed.
2.2. Component Labeling
By using a contour following/tracking algorithm, (Jain, 1989) or (Pratt,
2007), all image components (objects) are located and for each of them the
corresponding area ( Ai ) and the contour points are stored in a list. Then only
the objects of interest are retained, in this case those ones most likely to be
polygons images. The following selection rules are used:
− An object contour is not included in another contour.
− An object contour is not touching the image boundary (it might be
incomplete, so it must be ignored).
− The horizontal and vertical sides of the object surrounding rectangle
must fulfill the condition:
1 / K1 < ∆x / ∆y < K1
(3)
− The object area must be above a minimum level:
Ai > Amin
(4)
− The ratio of the object area and the area of the biggest object in the
image must be greater than a minimum level
Ai / max( A j ) > K 2
j
(5)
− The object contours must not be touching each other.
Those objects not fulfilling all previous rules are deleted from object
list. The pre-processing step results are illustrated also on test raster images in
Figs. 1 a and 1 b.
Bul. Inst. Polit. Iaşi, t. LVII (LXI), f. 4, 2011
119
3. Object Vertex Detection and Polygonal Approximation
An adaptive object vertex identification method is first applied in order
to obtain the best guess of polygon vertices. The input is the object contour
chain, so there is no need to reorder later the vertex list to compute polygon area
and to do all tests. In a first approach we were looking for vertex configuration
with the best ratio polygon area/object area. Polygon area is the area calculated
using the polygon vertices and object area is computed from object contour
points. This way the method worked only for convex polygons.
Later the method was adapted to recognize any kind of polygonal
shape. Alternative vertex identification methods can be found in (Hermann &
Klette, 2007; Klette & Rosenfeld, 2004; Liu et al., 2008; Liu et al., 2007;
Nguyen & Debled-Rennesson, 2011; Nguyen & Debled-Rennesson, 2009).
Our object vertex identification method computes for each contour
point the line determined by neighbors of current point in contour chain. The
neighborhood size is function of maximum expected number of vertices. For
instance if the application involves images including at most hexagons and no
polygons with more vertices, the ratio would be 1/6 of contour length. Then,
using the fitting point set, the fitting error is calculated and stored in an error
list. After processing all contour points, each contour point with fitting error
being a relative maximum in the error list and greater than an adaptive threshold
is classified as vertex.
We employ the same vertex identification approach as the one
presented in (Rotaru et al., 2011a) for convex shapes.
The equation of the line passing through a point ( x0 , y0 ) and having the
slope ϕ0 is:
( x − x0 )sin ϕ0 − ( y − y0 )cosϕ0 = 0
(6)
The equation of the line passing through a point ( x k , y k ) and being
perpendicular to the previous line is:
( x − xk )cos ϕ0 + ( y − yk )sin ϕ0 = 0
(7)
The distance from the point ( x k , y k ) to the line passing through the
point ( x0 , y 0 ) and having the slope ϕ 0 is given by:
2
d k2 = ( x0 − xk ) sin 2 ϕ0 − ( y0 − yk ) sin ϕ0 cosϕ0  +
+ ( y0 − yk ) cos2 ϕ0 − ( x0 − xk ) sin ϕ0 cos ϕ0 
2
(8)
120
Florin Rotaru et al.
Then, the mean square error corresponding to N contour points is:
E=
1
N
N
∑d
2
k
(9)
k =1
Relation (9) can be expressed as:
E=
1
N
∑{ ( x
N
0
k =1
2
− xk ) sin 2 ϕ0 − ( y0 − yk ) sin ϕ0 cosϕ0  +
+ ( y0 − yk ) cos ϕ0 − ( x0 − xk ) sin ϕ0 cosϕ0 
2
2
(10)
}
and if the next quantities are defined:
x( 0 )
x
2
(0)
1
=
N
1
=
N
N
∑x
k
,
y (0)
k =1
N
∑x
2
k
,
y
2
(0)
k =1
1
=
N
1
=
N
N
∑y
k
, xy ( 0 )
k =1
1
=
N
N
∑x
k
yk
k =1
N
∑y
(11)
2
(0)
k =1
the next expression for E is:
2
E = ( x02 − 2 x0 x(0) + x(0)
)sin 2 ϕ0 − 2( x0 y0 − x0 y(0) −
2
− x(0) y0 + xy(0) )sin ϕ0 cos ϕ0 + ( y02 − 2 y0 y(0) + y(0)
)cos 2 ϕ0
(12)
By minimizing this error, the unknown values of x 0 , y 0 and ϕ 0 are found:
∂E
=0⇔
∂ x0
( x0 − x(0) )sin ϕ0 − ( y0 − y(0) )cosϕ0 = 0 ⇔
(13)
x0 = x(0)
∂E
=0⇔
∂ y0
−( x0 − x(0) )sin ϕ0 + ( y0 − y(0) )cosϕ0 = 0 ⇔
y0 = y(0)
(14)
Bul. Inst. Polit. Iaşi, t. LVII (LXI), f. 4, 2011
121
∂E
=0⇔
∂ϕ0
1
2
ϕ0 = arctan
2( xy(0) − x(0) y(0) )
(15)
2
2
x(0)
− ( x(0) )2 − y(0)
+ ( y(0) )2
Considering the current image includes polygons up to a hexagon, N in
the above equations is 1/6 fraction of number of object contour points.
Having computed for each current contour point the line with x0, y0, and φ0
then the error E from (9) is calculated considering its fitting points, the N/2 previous
chain neighbors and the next N/2 chain neighbors of the current point.
Finally, the two contour points with greatest error (9) greater than an
adaptive threshold and being relative maximums in the error fitting chain are
retained.
The above procedure can be synthesized as follows:
Objective
Given the object contour points and the parameter percent compute the
best two vertices.
Algorithm The best two vertices (object contour, percent)
1. For each contour point compute xi, yi and φi, using its N/2 precedents
chain neighbors and N/2 succeeding chain neighbors. Then compute fitting error
Ei using the N fitting points and store it in a list. The fitting points are current
point neighbors in the object contour;
2. Using an adaptive threshold, th = percent * max ( Ei ) declare vertex
any contour point which is a relative maximum in the error list and having
Ei > th . Add the point to a vertex list;
3. Select the pair from the vertex list with the greatest errors Ei.
The overall vertex computation procedure is:
Objective
Given the initial two vertices compute the all vertex set.
Algorithm Vertex (object contour, vertex list)
1. Initialize the vertex list with two vertices provided by previous
procedure;
2. For each vertex pair of the vertex list compute the fitting error of the
contour points located on contour fraction between the two vertices. If the error
is significant compute the best guess of a potential vertex located on the object
contour between the two current vertices;
122
Florin Rotaru et al.
3. Update the vertex list;
4. Repeat steps 2-3 until there is no more candidates.
Obviously, if less than three vertices were detected or the final ratio
Apol/Aobj does not fit the required conditions, the object does not qualify as a
possible polygon. Otherwise, next step of the adaptive method is to validate the
vertex list. It is possible the list contains superfluous vertices, generated by
contour irregularities.
It was implemented the next validation procedure:
Objective
Given object vertex list and the object area Aobj validate each vertex and
finally obtain the minimum vertex set that satisfies Apol /Aobj conditions.
Algorithm Validation (object vertices, Aobj)
1. For each element of the vertex list do:
a) Save the current polygon area: A’pol= Apol;
b) Eliminate the current vertex and compute Apol area for the polygon
formed by the remaining vertices;
c) If thr1 < Apol/Aobj < thr2 and thr1 < Apol/A’pol < thr2.
Keep track of vertex k with min(Apol(k)/ A’pol).
2. If there is vertex k eliminate it from vertex list and resume the
process from first step.
The experiments revealed that optimal values for parameters are thr1 =
0.9 and thr2 = 1.2 for polygons with at most 6 vertices. For polygons with
greater number of vertices, the value of thr1 is automatically increased and thr2
is automatically decreased.
The last step of the proposed method is the polygonal fitting using the
obtained vertex list. It is the same procedure used in (Rotaru et al., 2011a;
Rotaru et al., 2011b). The fitting error gives a measure of the polygon
recognition accuracy. If the error is less than a specific threshold and implicit a
few fitting iterations are needed, the vertex list is a possible object description
and eventually can be used in further applications. Otherwise the object is
rejected as a possible polygon.
Using the vertex list an initial polygon is built. For each vertex pair
[vi ( xi , y i ), v j ( x j , y j )] provided by two neighbors in the vertex list the
corresponding polygon edge equation is:
( x − x n ) sin ϕ n − ( y − y n ) cos ϕ n = 0
(16)
Bul. Inst. Polit. Iaşi, t. LVII (LXI), f. 4, 2011
where : xn =
xi + x j
2
1
 y −y
 a tan  i j
 x −x
ϕn =  2
 i j

a cos(0)
; yn =
yi + y j
2
123
;

 if x ≠ x
i
j


if xi = x j
(17)
Then each object contour point is allocated to the closest
[vi ( x i , y i ), v j ( x j , y j )] edge. The fitting process continues building new
polygon edges for each edge set. The fitting process goes on until the fitting
error cannot be minimized any more: E ( t +1) − E ( t ) < ε 0 , where ε 0 is error
threshold and E (t ) is the total fitting error in fitting step t.
For a list of m vertices, the total fitting error in step t is:
 1


k =1  N k
m
E (t ) =
∑
Nk
∑d
j =1
2
j




(18)
Fig. 4 illustrates the results of the proposed method applied on the
contour detected in the ultra-sound image depicted in Fig. 2. In the first case,
the contour was approximated by a convex quadrilateral polygon. In Fig. 4 b, an
optimal approximation through a concave polygon is depicted. The recognized
polygons were overlapped on the original images (Figs. 5 and 6).
a
b
Fig. 4 − Algorithm results; a − Polygon approximation by a convex
quadrilateral, b − optimized polygon approximation.
124
Florin Rotaru et al.
4. Conclusions
The method is suited for applications where the user has to deal with
images containing unknown polygons but it works also for a specific polygon or
a restricted polygon class.
The adopted approach assures recognition invariance to scale and noise.
The method was tested on a large set of images (ultra-sound and raster images)
with large object scale variance. The test image set included the convex
polygons used to test the method presented in (Rotaru et al., 2011a) and
additional images including concave and combination of convex and concave
polygons, as depicted in Fig. 7.
The same recognition rate as for the first version of the adaptive
recognition method was obtained. Also the fitting error obtained in the case of
side-scan sonar images used for the presented application demonstrates the
practical utility of the algorithm.
The method was tested on an Intel Core2Duo, 2.33 GHz processor. The
processing time is about 20 ms for a 256x256 gray scale image and 65 ms sec
for a 512x512 gray scale image.
For an error threshold ε 0 = 0.001 , the maximum number of iterations to
complete the polygonal fit is 3. In our experiments we used the next values for
the previously described constants: K1 = 4 , Amin = 10 , and K 2 = 0.2 .
Fig. 5 − Algorithm results - object
approximation by a convex quadrilateral.
Fig. 6 − Algorithm results - optimized
polygon approximation.
Fig. 7 − Test images.
Bul. Inst. Polit. Iaşi, t. LVII (LXI), f. 4, 2011
125
REFERENCES
Bejinariu S., Rotaru F., NiŃă C.D., Luca R., Information Fusion Techniques for Image
Segmentation. ECIT 2010, ECIT2010 – 6th European Conference on Intelligent
Systems and Technologies, Iaşi, România, October 07-09, 2010.
Bulea M., Automatic Measurement of Textile Sieves by Image Analysis. Research
Report, I.I.T, 1996.
Bulea M., Prelucrarea imaginilor si recunoasterea formelor: Teorie si aplicatii. Edit.
Academiei Romane, 2003.
Hermann R., Klette R., A Comparative Study on 2D Curvature Estimators. Int.
Conference on Computing: Theory and Applications (ICCTA’07), 2007.
http://www.l-3klein.com
Jain A.K., Fundamentals of Digital Image Processing. Prentice Hall, Englewood Cliffs,
1989.
Klette R., Rosenfeld A., Digital Geometry. Geometric Methods for Digital Picture
Analysis. Elsevier Inc., 2004.
Liu H., Latecki L.J., Liu W., A Unified Curvature Definition for Regular, Polygonal
and Digital Planar Curves. Int. Journal of Computer Vision, 80, 104−124,
2008.
Liu H., Latecki L.J., Liu W., Bai X., Visual Curvature. IEEE Conf. on Computer Vision
and Pattern Recognition (CVPR), June 2007.
Museros L., Escrig M.T., A Qualitative Theory for Shape Matching Applied to
Autonoumous Robot Navigation. In “Artificial Intelligence Research and
Development. Frontiers in Artificial Intelligence and Applications”. IOS
Press., 243−250, 2005.
Nguyen T.P., Debled-Rennesson I., A Discrete Geometry Approach for Dominant Point
Detection. Pattern Recognition, Vol. 44, 1, 32−44, 2011.
Nguyen T.P., Debled-Rennesson I., Fast and Robust Dominant Point Detection on
Digital Curves. Proc. of ICIP, 953−956, 2009.
Pratt W.K., Digital Image Processing. John Wiley & Sons, Fourth Edition, 2007.
Rotaru F., Bejinariu S., Bulea M., Nita C.D., Luca R., Adaptive Recognition Method for
2D Polygonal Objects. International Symposium on Signal, Circuits &
Systems – ISSCS 2011, Iaşi, România, June 30- July 1 2011a.
Rotaru F., Bejinariu S., Bulea M., Nita C.D., Luca R., Recognition Method for 2D
Polygonal Objects in Ultrasound Images. International Conference on EHealth and Bioengineering – EHB 2011, Iaşi, România, 24-26 November
2011b.
Rotaru F., Pescaru S., Bejinariu S., Nita C.D., Luca R., Lazar C., 3D Pose Computation
in Robot Vision Applications. Proc. 4th IEEE International Workshop on Soft
Computing Applications (SOFA), Arad, România, July 2010, 181−184, 2010.
Rotaru F., Roman M., Bulea M., Bejinariu S., A Method for Evaluating Vickers
Hardness by Automatic Image Analysis. FSAI, Vol. 6, 1-3, 21−31, 1997.
Teodorescu H.N. et al., ADBIOSONAR – Adaptive Bio-Mimetic Sonar Heads for
Autonomous Vehicles. Project PNCDI-II, nr. 12079/2008, Research Report,
November 2010.
126
Florin Rotaru et al.
METODĂ PENTRU RECUNOAŞTEREA OBIECTELOR
POLIGONALE 2D
(Rezumat)
Lucrarea propune o metodă de recunoaştere a obiectelor având forme
poligonale. Pentru început este aplicată o procedură de identificare a vârfurilor formei
poligonale, fiind reŃinute cele mai puternice 2 vârfuri identificate. În continuare este
aplicat un algoritm de potrivire, care pornind de la cele 2 puncte identificate şi
descrierea conturului, determină o descriere precisă a conturului obiectului. Metoda
propusă poate fi utilizată în aplicaŃii industriale de robot vision, controlul calităŃii sau
fotogrametrie, în cazul în care obiectele de procesat au forme poligonale. Pentru testare
şi evaluarea rezultatelor, metoda propusă a fost implementată în C++.