M.S. THESIS
A Skewed Rotational Symmetry Detection
Technique Using Symmetry-Growing
대칭성분 성장을 이용한 편향 회전대칭 검출 기법
BY
HYO JIN KIM
FEBUARY 2012
DEPARTMENT OF ELECTRICAL ENGINEERING AND
COMPUTER SCIENCE
COLLEGE OF ENGINEERING
SEOUL NATIONAL UNIVERSITY
M.S. THESIS
A Skewed Rotational Symmetry Detection
Technique Using Symmetry-Growing
대칭성분 성장을 이용한 편향 회전대칭 검출 기법
BY
HYO JIN KIM
FEBUARY 2012
DEPARTMENT OF ELECTRICAL ENGINEERING AND
COMPUTER SCIENCE
COLLEGE OF ENGINEERING
SEOUL NATIONAL UNIVERSITY
A Skewed Rotational Symmetry Detection
Technique Using Symmetry-Growing
대칭성분 성장을 이용한 편향 회전대칭 검출 기법
지도교수 이 경 무
이 논문을 공학석사 학위논문으로 제출함
2012년 2월
서울대학교 대학원
전기 컴퓨터 공학부
김효진
김효진의 공학석사 학위 논문을 인준함
2012년 2월
위 원 장:
부위원장:
위
원:
Abstract
This paper introduces a robust and accurate algorithm for detecting skewed rotationally symmetric patterns from real-world images. Previous local feature-based rotational symmetry detection methods have several problems such as unreliable performance depending on the number of features and weakness in coping with deformation
and background clutters. And dealing with skew is another important issue because
perspective skew maligns the detection performance. However, no straightforward solution has been reached yet. Our method overcomes the limitations of the existing local
feature-based methods via symmetry-growing scheme that efficiently explores image
space and grows rotational symmetry from a few symmetrically matched initial feature pairs. Furthermore, a powerful method for handling skewed rotational symmetry
is presented. Unlike previous iterative methods that consider center points and orientations only, the proposed method exploits the shape of affine covariant features, thus
giving a closed form solution for the rotational axis of a feature pair on an affine plane.
The experimental evaluation demonstrates that the proposed method successfully detects and segments multiple rotational symmetric patterns from real-world images, and
clearly outperforms the state-of-the-art methods in terms of accuracy and robustness.
keywords: skewed rotational symmetry, symmetry-growing, skew-robust rotational
center computation, symmetry detection
student number: 2010-20791
i
Contents
Abstract
i
Contents
ii
List of Tables
iv
List of Figures
v
1 Introduction
1
1.1
Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1
1.2
Objective and Contribution . . . . . . . . . . . . . . . . . . . . . . .
2
1.3
Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
2 Related Work
4
2.1
Mirror Symmetry Detection . . . . . . . . . . . . . . . . . . . . . .
4
2.2
Rotational Symmetry Detection . . . . . . . . . . . . . . . . . . . .
5
2.2.1
6
Skewed Rotational Symmetry Detection . . . . . . . . . . . .
3 Skewed Rotational Symmetry Detection Technique Using Symmetry-Growing
3.1
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
3.2
Rotational Symmetry Seeds . . . . . . . . . . . . . . . . . . . . . .
10
3.3
Skew Robust Rotational Center Computation . . . . . . . . . . . . .
11
3.4
Rotational Symmetry-Growing Framework . . . . . . . . . . . . . .
17
ii
8
3.4.1
Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . .
17
3.4.2
Initialization . . . . . . . . . . . . . . . . . . . . . . . . . .
18
3.4.3
Expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19
3.4.4
Merging . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
3.4.5
Termination Criteria . . . . . . . . . . . . . . . . . . . . . .
21
4 Symmetry Analysis
23
4.1
Symmetry Center Localization . . . . . . . . . . . . . . . . . . . . .
23
4.2
Ellipse Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24
4.3
Number of Folds Estimation . . . . . . . . . . . . . . . . . . . . . .
26
4.3.1
Local Match-Based Histogram Approach . . . . . . . . . . .
26
4.3.2
Region-Based Frequency Analysis Approach . . . . . . . . .
27
Outlier Elimination . . . . . . . . . . . . . . . . . . . . . . . . . . .
32
4.4
5 Experimental Results
34
5.1
Quantitative Evaluation . . . . . . . . . . . . . . . . . . . . . . . . .
34
5.2
With and Without the Skew Robust Rotational Center Computation . .
43
5.3
Skew Robustness of the Algorithm . . . . . . . . . . . . . . . . . . .
44
5.4
Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
45
6 Conclusion
49
Abstract (In Korean)
55
Acknowlegement
57
iii
List of Tables
4.1
The evaluation of two number of folds estimation method. The selected 100 images from the dataset of [27] with 140 ground truth is
used.
5.1
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
32
Quantitative evaluation of the proposed algorithm compared with four
other state-of-the-art methods. The dataset used for performance evaluation in the work of Lee and Liu [9] is used. . . . . . . . . . . . . .
5.2
35
The proposed approach compared with a competitor and the baseline
algorithm in the ”Symmetry Detection from Real World Images - A
Competition,” the CVPR 2011 Workshop [37]. The same dataset used
5.3
in the competition was used. . . . . . . . . . . . . . . . . . . . . . .
41
The major-to-minor radius ratio according to the skew intensity level.
45
iv
List of Figures
3.1
Overview of the proposed approach. (a) Original image. (b) Feature
extraction. (c) Rotationally symmetric pairs obtained by matching the
extracted features. (d) Expansion of an initially detected symmetry
pair done through investigation of nearby regions inside the expansion
layer, which is a set of overlapping circularly gridded regions. (e)-(f)
Rotational symmetries grown by expansion and merging of clusters
that contain rotationally symmetric feature pairs. (g) Estimation of the
final center of rotation through a voting scheme when a symmetry is
all grown up. (h) Final result of the algorithm. The centers of rotational
symmetry plotted in circles. . . . . . . . . . . . . . . . . . . . . . . .
3.2
9
Matching rotationally symmetric feature pairs. The region normalizing
matrices are denoted as Hi and Hj . A symmetric feature pair (Zi , Zj )
is matched since the descriptors ki and kj are similar enough. . . . . .
3.3
10
(a) Conventional approach [11] to find the center of rotation using a
perpendicular line and the orientation of matched two features. (b) The
conventional approach fails to locate the center in the correct place.
(c) The proposed method uses the shape of matched affine covariant
regions to locate the center robust to affine skew. . . . . . . . . . . .
v
12
3.4
(a) A rotationally symmetric feature pair Zi and Zj (located at pi and
pj , respectively), and its center c for a skewed rotationally symmetric
object. (b) The transformations within the matched features. (c) The
corresponding undistorted model of (a). . . . . . . . . . . . . . . . .
3.5
13
Estimating the center of rotation from a single pair of rotationally symmetric features for both the undistorted case and the skewed case. (a)
The result of the proposed method. (b) The result of the conventional
approach [11]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.6
15
Some examples of estimation of the ellipse when given a match. The
estimated ellipses are shown in yellow lines. The black dots are the
estimated points, pk ’s; the red and the blue dots represent the features
of the given match, pi ’s and pj ’s, and the greens for the centers c’s. .
3.7
16
Sampling image regions for refining initial seeds. (a) An initial seed
with rotationally symmetric regions Ri (blue) and Rj (red). A new region Rk (black) associated with the same rotational symmetry is generated by the pair. (b) All sampling regions computed iteratively. (c)
Original feature regions (upper row) and the sampled regions (lower
row) show appreciably high similarity; thus, the seed is reliable. . . .
3.8
18
Propagating symmetry. (a) A supporter match Ma = (Zi , Zj ) and
a neighboring region Zp picked near Zi , where TA is the transform
within the supporter match Mi . (b) The region Zp is mapped to the region Zq via TA , creating a propagated match Mb = (Zp , Zq ). Because
the displacement between the centers of Zi and Zp is known to be ti ,
the displacement tj between Zj and Zq is calculated as tj = TA ti .
Furthermore, using the shape of Zp , Σp , the shape of Zq is calculated
−1
as: Σq = T−T
A Σ p TA . . . . . . . . . . . . . . . . . . . . . . . . .
vi
19
4.1
Symmetry center localization for grown-up clusters. (a) Matched symmetric features in each cluster depicted in different colors. (b) Voting
for the rotational center of each cluster. (c) Voting result for cluster 1
(red). (d) Voting result for cluster 2 (green). (e) Final centers of rotation. 24
4.2
Ellipse shape determination. (a) A grown cluster. (b) Ellipses of various size and shape lie inside the grown cluster. (c) The ellipses are then
normalized into the same scale. (d) The normalized ellipses are then
voted in a parameter space to acquire the most popular elliptic shape
that exists in the cluster. . . . . . . . . . . . . . . . . . . . . . . . . .
4.3
26
Determining the scale of ellipses. (a) Grown clusters with the minimum distance from the center to its convex hull specified in colored
lines. (b) These distances are initially taken as the minor radii of ellipses. (c) A three-step search algorithm is used to precisely estimate
the scale of the ellipse. (d) The regions that lie outside the ellipses are
eliminated from their clusters. . . . . . . . . . . . . . . . . . . . . .
4.4
27
Removing a global symmetry (across objects) accumulated with a local symmetry (within an object). (a) Original image. (b) Initially detected result. (d) The clusters and their covering region. (e) Ellipse
estimation result. (c) Final detection result. The unwanted global symmetries are removed from local symmetry clusters. . . . . . . . . . .
4.5
Number of folds estimation results via the local match-based histogram
approach. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.6
28
29
Local match-based histogram approach for the number of folds estimation. (a) The number of fold wrongly estimated as 12 directly using
features’ orientation difference [11]. (b) The number of fold correctly
estimated as 5 using angles recovered from Eq.(3.18). . . . . . . . . .
vii
30
4.7
(a)Rotational symmetry detection result. (b) Enlarged view of (a) showing a rotational symmetry and its estimated ellipse. (c) The symmetry
exhibiting region being rectified via the estimated ellipse. . . . . . . .
4.8
30
Frequency analysis scheme for the number of folds estimation. (a) The
rotational symmetry of Fig.4.7 depicted in polar coordinate as ρ(r, φ).
(b)Sn s, the strength of nth order component. A clear peak at n = 9 is
shown.
5.1
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
31
Example results of the proposed algorithm on the real images with a
single symmetry in the data set of [9]. LE06, and LL10 denote the
methods of [11] and [9], respectively.
5.2
. . . . . . . . . . . . . . . . .
36
Example results of the proposed algorithm on the real images with
multiple symmetries of natural objects in the data set of [9]. LE06,
and LL10 denote the methods of [11] and [9], respectively. . . . . . .
5.3
37
Example results of the proposed algorithm on the real images with
multiple symmetries of artifacts in the data set of [9]. LE06, and LL10
denote the methods of [11] and [9], respectively. . . . . . . . . . . . .
5.4
38
Quantitative comparison of the proposed algorithm (Red on the right),
the competitor algorithm (Green on the left) the baseline algorithm
(Blue in the middle) on the dataset of [37] in the five main categories
[37]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.5
42
Results showing the effectiveness of the skew robust rotational center computation. (a) The result of the previous approach [11]. (b) The
result of [11] with affine covariant features. (c) The skew robust rotational center computation applied on (b). (d) The results of the symmetry growing without the skew robust rotational center computation.
5.6
(e) The result of the overall algorithm . . . . . . . . . . . . . . . . .
46
Examples showing different levels of skew (from level 0 to level 10) .
47
viii
5.7
Accuracy according to the intensity of skew. We tested our algorithm
on 275 images consisting of 11 levels of skew synthesized from 25
original images (13 synthetic images and 12 real images). . . . . . . .
5.8
47
Results showing limitations of the algorithm. (a) Featureless image.
(b) Irregular continuous symmetry. LE06, LL10, and KP11 denote the
methods of [11], [9], and [38], respectively. . . . . . . . . . . . . . .
ix
48
Chapter 1
Introduction
1.1 Motivation
Symmetry is a highly salient visual phenomenon and plays an important role in perceptual grouping [1], [2]. From natural structures to manufactured artifacts, countless
objects around our lives exhibit symmetric patterns. Accordingly, symmetry detection
and analysis are useful for many computer vision applications, such as object recognition, matching, segmentation [3], [4], [5]. Despite active research for almost four
decades [6], most previous methods for symmetry detection have been demonstrated
only on cleanly segmented artificial shapes or images with dominant global symmetries [7]. Only recently have symmetry detection methods been proposed to face variety
of obstacles in the images taken from the real world. Moreover, the recent performance
evaluation result shows that robust and widely applicable symmetry detectors for realworld images are still lacking [8].
Detecting symmetries in real-world images is not an easy task; there exists occlusion, illumination changes, deformation, background clutters, and perspective skew.
As a consequence, a reliable and consistent cue for inferring symmetry under such
condition is needed. Thanks to the recent development of local invariant features, the
methods that use local features have been thought to be promising. These methods
1
group the sets of local features to detect symmetry; thus, they are robust to occlusion
and even illumination changes when used with invariant features. Yet, there are some
problems left. For local feature-based methods, there is the problem of limited number
of features. Feature detectors often cannot detect enough features for grouping to detect symmetric patterns. In addition, real-world images taken from an arbitrary angle
often contain perspective distortion, due to projection of a 3D object into a 2D planar
space [9], which is referred to as perspective skew. The perspective skew introduces
a number of additional challenges when detecting symmetry from real-world images.
Moreover, the problems regarding deformation and background clutters still need to
be solved.
1.2 Objective and Contribution
The presented work introduces a novel and effective method for skewed rotational
symmetry detection based on a symmetry-growing approach [10], that is robust to realworld images. The proposed algorithm grows symmetric region by exploring the image space beyond initially detected symmetric features considering both photometric
similarity and geometric constraints for rotational symmetry. Different from previous
local feature-based methods, it can handle low number of seeds and deformations. The
method also detects multiple rotational symmetry and symmetric objects in complex
backgrounds.
Furthermore, in this paper, an efficient and effective method to deal with skewed
rotational symmetry using shape of affine covariant features is proposed. The method
computes affine transformation function that maps one feature region to the other
within a matched feature pair given their shapes, obtaining a closed form solution
for the center of rotation.
Therefore, the algorithm obtains an accurate detection result for a symmetric pattern regardless of any affine transformation that was applied. Also, the approach is
2
robust to outliers not only because it considers skew, but also the symmetry growing
framework provides the environment where only the inliers can grow well.
This work makes the following specific contributions: (1) A novel symmetrygrowing algorithm for rotational symmetry detection is proposed; (2) An effective approach that figures out the center of rotation, the ellipse shape, and the rotation angle
of a symmetric feature pair in a skewed rotationally symmetric pattern is introduced.
Together with stated contribution points above, the proposed method effectively and
accurately localizes, segments and analyzes multiple and skewed rotational symmetry
in real-world images, thus avoiding the adverse effect of outliers in symmetry detection.
1.3 Outline
In this paper, the related work is summarized in chapter 2 and the proposed rotational
symmetry detecting algorithm is introduced in chapter 3. The analysis scheme for
detected rotational symmetry and the post-processing step for the outlier elimination
is described in chapter 4. In chapter 5, quantitative and qualitative experimental results
are demonstrated. The paper concludes in chapter 6.
3
Chapter 2
Related Work
2.1 Mirror Symmetry Detection
Previous symmetry detection algorithms can be roughly classified into two different
types, namely, global methods and local methods.
Global methods treat the entire image as a signal from which symmetric patterns
are inferred [11]. Derrode and Ghorbel [12] introduce the analytical Fourier-Mellin
transform to detect symmetry. Keller and Shkolnisky [13] present a global geometric
structure method, that uses pseudo-polar Fourier transform to find repetition over angular direction. Sun et al. [14] show that the orientation of the dominant bilateral symmetry axis is computed from the histogram of gradient orientations. However, these
global methods are usually limited to detecting a single incidence of symmetry, and
are greatly influenced by background clutters.
Local methods, on the other hand, use local features, such as edges, contours, or
regions to detect symmetry by grouping. Masuda et al. [15] adopt an image similarity measure based on the directional correlation of edge features and proceeded to
detect rotational and mirror symmetry. Zabrodsky et al. [16] introduce a continuous
measure of the amount of symmetry present in a shape by computing the minimum
mean squared distance between the contour points. Nevertheless, a reliable and con-
4
sistent feature is needed for real-world images where there are many obstacles such as
occlusion, illumination changes, and deformations.
Owing to the recent advancement of local invariant features [17] ,[18] ,[19], a significant progress has been achieved in this approach. Tuytelaars et al. use cascaded
Hough transform to detect regular repetitions of planar patterns. Lazebnik et al. use
affine invariant clusters of features to detect symmetries, but the use of rotationally
invariant descriptors restricted its performance. Loy and Eklundh [11] propose an efficient method to exploit the properties of local invariant features for grouping symmetric constellations of features and detecting symmetry. Cornelius et al. [20] extend this
approach by constructing local affine frames. Although all these methods show robust
performance on problems with occlusion and background clutters, they are largely
influenced by feature detection step, and cannot exploit further information beyond
the initially detected features. To solve the limitation of the local method, Cho and
Lee [10] propose a region growing method for mirror symmetry detection inspired by
match-growing approaches [21], [22], [23], [24] used in object recognition and image registration. The algorithm grows mirror symmetric patterns by investigating the
image space beyond the detected reflectional symmetric features.
2.2 Rotational Symmetry Detection
Although primary interest has been on the detection of mirror symmetry [25], rotational symmetry detection became an active research area as well [9], [7], [16], [26],
[11], [13], [27]. For local methods, Zabrodsky et al. [16] propose a method using the
minimum mean squared distance between the contour points for rotational symmetry
as well as mirror symmetry. Prasad and Davis [26] proposed a method using Gradient
Vector Flow (GVF) to create a confidence map for detecting the candidate rotation
symmetry centers. However, it requires the number of folds to be given as a prior, and
has a high computational complexity. The state-of-the-art local method for rotational
5
symmetry detection is the work of Loy and Eklundh [11], where both mirror and rotational symmetries can be detected, that groups the matched feature pairs according to
their center of rotation. However, it still has the problem of limited number of features
as in the mirror symmetry case. Also, it has difficulties dealing with skewed rotational
symmetries since it considers only the point locations and the orientations of a feature
pair to estimate the rotational center, that do not work for the skewed case.
For global methods, the work of Derrode and Gorbel, and the work of Keller et
al. also cover rotational symmetry as they deal with mirror symmetry, but they are
greatly affected by background clutters. The work of Lee and Liu [27] is the most
prominent one. They use Frieze-expansion and frequency analysis, calculating a rotation symmetry strength map for all pixels in the image. Recently, Lee and Liu propose
a new rotational symmetry detection scheme [9] that hybrids the Frieze-expansion approach [27] and the local feature-based method [11]. The method is quite powerful
since it combines the two state-of-the-art algorithms. Although it shows a remarkable
true positive rate, however, it often suffers from high false positive rate resulting from
deformation and background clutters.
2.2.1 Skewed Rotational Symmetry Detection
Dealing with skewed rotational symmetry is another important issue. Although there
were numerous works on affine or skewed mirror symmetry detection [28], [29], [30],
[31]. there is relatively little literature on effective skewed rotational symmetry detection for real images. Cornelius and Loy [7] extend the work of [11] by computing
centers of rotation with respect to all discretized orientations by tilting angles, and
then finding the most likely center of rotation by voting. Lee and Liu [9] propose an
approach using phase analysis of Frieze expansion plane followed by rectification and
re-calculation of RSS value for all possible aspect ratio until the maxima is found.
In this work, the symmetry-growing approach [10] for overcoming the limitation
of the local feature-based method is extended to introduce a novel symmetry growing
6
framework for rotational symmetry detection. The robustness to skew is achieved by
exploiting the shape of affine covariant features. Unlike previous methods that deal
with skew iteratively, we provide an exact closed form solution for the center of skewed
rotational symmetry; therefore, the algorithm obtains an accurate center of rotation for
a symmetric pattern regardless of any affine transformation that was applied.
7
Chapter 3
Skewed Rotational Symmetry Detection Technique Using Symmetry-Growing
3.1 Overview
The proposed method aims to localize and segment all the rotationally symmetric patterns in an image considering perspective skew and analyze them to find out the center
(cx , cy ) and the number of folds n. Fig.3.1 illustrates a brief overview of the approach.
First, affine covariant features are extracted from the given image as in Fig.3.1(b). Second, using the appearance around the detected features, rotationally symmetric feature
pairs, which are used as seeds in the symmetry-growing framework, are constructed,
as shown in Fig.3.1(c). Third, starting from singleton symmetry clusters each containing a seed match, symmetry clusters are simultaneously expanded and merged by
exploring the image space as illustrated in Fig.3.1(e)-(f), where the same color of dots
represent the features in the same cluster. In the expansion step, a set of element regions consisting of overlapping, circularly gridded local regions, which is referred to
as an “expansion layer,” is used as shown in Fig.3.1(d). Using expansion layers, the
proposed algorithm generates new feature pairs, gradually growing symmetry clusters. In the merging step, the clusters that share the same center of rotation are merged.
8
The rotational center of a cluster is computed through a voting scheme as shown in
Fig.3.1(g). Finally, the reliable symmetric patterns grown well enough are chosen, and
the final center for the symmetry is computed as depicted in Fig.3.1(h). In addition, the
analysis of the symmetry is performed to estimate the number of folds of the detected
symmetry.
D
E
F
G
HIJK
Figure 3.1: Overview of the proposed approach. (a) Original image. (b) Feature extraction. (c) Rotationally symmetric pairs obtained by matching the extracted features. (d)
Expansion of an initially detected symmetry pair done through investigation of nearby
regions inside the expansion layer, which is a set of overlapping circularly gridded regions. (e)-(f) Rotational symmetries grown by expansion and merging of clusters that
contain rotationally symmetric feature pairs. (g) Estimation of the final center of rotation through a voting scheme when a symmetry is all grown up. (h) Final result of the
algorithm. The centers of rotational symmetry plotted in circles.
9
3.2
Rotational Symmetry Seeds
First, local candidate matches for rotational symmetry are constructed from the given
image. These matches are used as seeds of the proposed symmetry-growing algorithm
(Sec.3.3). Many of modern local feature detectors provide robust means for generating
and matching dense features [17], [18], [19].
In this paper, affine invariant region detectors and their matrix parameterizations
[18] are used. They are the most widely used ones: Maximally Stable Extremal Region (MSER) [19], Harris-Affine [18], and Hessian-Affine [18]. A detected region
Zi = (pi , Σi , oi ) consists of a point vector pi = (xi , yi ) which denotes the location
of the center, a covariance matrix Σi , and a scalar oi that represents orientation. An
interior pixel x of the feature region Zi satisfies (x − xi )T Σi (x − xi ) ≤ 1. In addition,
SIFT keypoint detector is also used [32] for more features. After feature extraction, a
feature descriptor ki is generated for each feature region, encoding the local appearance of the feature after its normalization with respect to the orientation and affine
distortion.
5HJLRQ
1
1RUPDOL]DWLRQ
OL WL
0DWFKLQJ
6,)7
GHVFULSWRUV
6,)7
GHVFULSWRUV
Figure 3.2: Matching rotationally symmetric feature pairs. The region normalizing
matrices are denoted as Hi and Hj . A symmetric feature pair (Zi , Zj ) is matched
since the descriptors ki and kj are similar enough.
10
Potential rotationally symmetric matches are obtained by constructing a set of feature descriptors k and matching them against each other. As shown in Fig.3.2, a feature
pair (Zi , Zj ) is matched since the descriptor ki and kj are similar enough. The similarities of descriptor pairs for all the possible symmetric feature pairs are calculated,
and the best 2400 matches (600 matches per a type of feature) are collected allowing
multiple correspondences for each feature.
3.3 Skew Robust Rotational Center Computation
In the proposed symmetry-growing framework, the rotational center of each rotationally symmetric feature pair serves as the key criteria for the whole process. However,
the radius and the rotation angle of an symmetric object are distorted under perspective
projection. Thus, computing the true center of rotation from such distortion is crucial
for accuracy of the detection.
For the accurate rotational center computation from a skewed symmetric object,
choosing a cue that captures the skew information of the object is important. The
conventional method [11] uses only the center points and the orientation difference
of features to locate the center, assuming that the center is on the perpendicular line
drawn from the midpoint of a matched feature pair. Therefore, when it comes to a
skewed rotational symmetry, the approach fails because the center is no longer on the
perpendicular line as shown in Fig.3.3(b).
The proposed algorithm utilizes affine covariant regions, not just the points and
orientations, to capture skew information of the given symmetric object. Given a pair
of affine covariant regions Zi = (pi , Σi , oi ) and Zj = (pj , Σj , oj ) of a rotationally symmetric match Ma = (Zi , Zj ), the transformation function TA that maps one
region Zi to the other Zj is computed using the following formula:
zj − pj = TA (zi − pi ),
(3.1)
where the points zi , zj are defined as zi ∈ Zi and zj ∈ Zj . This computed transfor-
11
…αȋ…Ȍ
(a)
(b)
(c)
Figure 3.3: (a) Conventional approach [11] to find the center of rotation using a perpendicular line and the orientation of matched two features. (b) The conventional approach
fails to locate the center in the correct place. (c) The proposed method uses the shape
of matched affine covariant regions to locate the center robust to affine skew.
mation function TA is the same as the rotation matrix in the skewed symmetric object
Rθskewed . The proof is as follows.
Proof 1. (TA as a rotation matrix on a skewed rotationally symmetric object, Rθskewed ):
Let a model be assumed, that is, a model that is an un-skewed version of a given rotationally symmetric object that contains regions Yi and Yj that corresponds to Zi and
Zj of the skewed object as shown in Fig.3.4(c). Then, let TS be a skewing matrix
that skews the model to the skewed object, and θ be an angle between Yi and Yj . The
following holds for all TS :
yi = TS −1 zi ,
(3.2)
yj = TS −1 zj ,
(3.3)
yj − TS −1 pj = Rθ (yi − TS −1 pi ),
(3.4)
where the points yi and yj are defined as yi ∈ Yi , and yj ∈ Yj , and Rθ is a rotation matrix of the angle θ. The last equation Eq.(4) comes from the fact that on a
not skewed, perfectly circular rotational symmetry, a pair of rotationally symmetric
12
SM
SL
=L
=M
<M
=L
T VNHZHG
=M
+L
F
+M
7$
(a)
<L
76 S M
T
76 S L
76 F
(b)
(c)
Figure 3.4: (a) A rotationally symmetric feature pair Zi and Zj (located at pi and
pj , respectively), and its center c for a skewed rotationally symmetric object. (b) The
transformations within the matched features. (c) The corresponding undistorted model
of (a).
regions with an angle θ are alike when either of them is rotated by θ.
Therefore, based on Eq.(3.2)-(3.4),
zj − pj = TS Rθ TS −1 (zi − pi ).
(3.5)
Moreover, based on eq(3.1),
TA = TS Rθ TS −1 .
(3.6)
Now, when letting the rotation matrix of Zi and Zj on the skewed object as Rθskewed ,
so that it corresponds to Rθ in the model, the following holds for a point xi and its
rotated pair xj on the skewed rotationally symmetric object:
(xj − c) = Rθ skewed(xi − c),
(3.7)
where c denotes the rotational center of the skewed object as in Fig.3.4(a). Also, the
following holds for TS −1 xi and TS −1 xj on the model that corresponds to xi and xj
on the skewed plane:
TS −1 xj − TS −1 c = Rθ (TS −1 xi − TS −1 c).
13
(3.8)
Thus, from Eq.(3.7)-(3.8),
Rθ skewed = TS Rθ TS −1 = TA .
(3.9)
This completes the proof.
With the transformation TA being a rotational matrix on the skewed rotationally
symmetric object, the center of rotation is the point that is invariant to the transformation TA as in Fig.3.3(c). Hence, the center computed from a match Ma = (Zi , Zj ),
where Zi = (pi , Σi , oi ) and Zj = (pj , Σj , oj ), is as follows:
c = pj − (I − TA )−1 (pj − pi ).
(3.10)
This is a closed form solution for the center of rotation that is invariant to affine skew.
The transformation TA is calculated using region normalizing matrices of the
rotationally symmetric feature pair. For affine covariant regions Zi = (pi , Σi , oi )
and Zj = (pj , Σj , oj ) of a rotationally symmetric match Ma = (Zi , Zj ), regionnormalizing matrices Hi and Hj are calculated from covariance matrices as follows:
1
Hi = Roi −1 Σi 2 ,
1
Hj = Roj −1 Σj 2 ,
(3.11)
(3.12)
where Roi and Roj represent rotation matrix with angle oi and oj , respectively. These
normalization matrices map the points in the elliptical regions onto normalized unit
circles with a constant orientation as in Fig.3.4(b). Hence, the transformation TA that
maps region Zi to its corresponding region Zj is defined as:
TA = H−1
j Hi .
(3.13)
The effectiveness of this approach is demonstrated in Fig.3.5 which shows the
rotational center estimation result of the conventional method [11] and the proposed
approach from a matched feature pair. Both the conventional method and the proposed
scheme works well on un-skewed case as in the upper row of Fig.3.5. However, when
14
DE
Figure 3.5: Estimating the center of rotation from a single pair of rotationally symmetric features for both the undistorted case and the skewed case. (a) The result of the
proposed method. (b) The result of the conventional approach [11].
the symmetric pattern is affinely skewed, conventional method fails to locate the center
of rotation while the proposed approach finds an accurate location for the center.
Having found the center of rotation, ellipse that fits the given pair of feature points
is also estimated. Although at least three points are required to define an ellipse, only a
pair of features is given. However, this problem is solved through a simple application
of what has been found just before (Eq.(3.9)). Using Rθskewed = TA , a point pk that
potentially is in a rotational symmetry with the existing pair of feature points, pi and
pj , is generated, such that:
(pk − c) = TA (pj − c),
(3.14)
where (pj − c) = TA (pi − c).
Some examples of computed pk ’s can be seen in Fig.3.6. Now, let the skewed
rotational symmetry be centered at origin, as the center is known from Eq.(3.10). An
15
DEF G
Figure 3.6: Some examples of estimation of the ellipse when given a match. The estimated ellipses are shown in yellow lines. The black dots are the estimated points, pk ’s;
the red and the blue dots represent the features of the given match, pi ’s and pj ’s, and
the greens for the centers c’s.
ellipse centered at the origin is represented as follows:
xT Ax = 1,
⎡
where A = ⎣
(3.15)
⎤
a b
⎦ and x = (x, y). In this case, x1 = pi − c, x2 = pj − c, and
b c
x3 = pk − c satisfy the equation.
The ellipse parameter matrix A is obtained by solving the following equation for
a, b, and c:
⎡
⎤ ⎡ ⎤ ⎡ ⎤
a
1
⎥ ⎢ ⎥ ⎢ ⎥
⎥ ⎢ ⎥ ⎢ ⎥
y2 2 ⎥ · ⎢2b⎥ = ⎢1⎥ .
⎦ ⎣ ⎦ ⎣ ⎦
2
c
1
y3
x1 2 x1 y 1 y 1 2
⎢
⎢ 2
⎢ x2 x2 y 2
⎣
x3 2 x3 y 3
(3.16)
Using these ellipse parameters, the rotation angle is recovered by rectification. The
matrix A is decomposed into DT D since A is symmetric and positive definite. The
matrix D is the normalizing matrix that converts the ellipse to a unit circle. Thus,
normalized vectors Dx are obtained for each x’s, such that:
xT Ax = xT DT Dx = (Dx)T (Dx).
(3.17)
Therefore, the angle of rotation recovered from the deformation due to skew transfor-
16
mation becomes:
θ = cos−1 (
D(pi − c) · D(pj − c)
).
||D(pi − c)||||D(pj − c)||
(3.18)
These recovered angles are used to estimate the order of symmetry as in Sec.4.3 (Fig.
4.6) for the symmetry analysis.
The case of rotational symmetry that is undistorted by perspective skew can be
perceived as a special case of skewed rotational symmetry with Rθ skewed = TA =
Rθ .
3.4 Rotational Symmetry-Growing Framework
The proposed symmetry-growing framework efficiently grows rotational symmetric
patterns from real-world images. The symmetry-growing framework consists of two
iterative steps. One is the expansion step and the other is the merging step. The expansion process propagates the rotational symmetry to neighboring regions using initially
matched feature pairs as seeds, thus generating new rotationally symmetric feature
pairs. The merging process groups those rotationally symmetric feature pairs with the
same center of rotation to specify dominant symmetries within the image. The algorithm simultaneously repeats these two steps, searching the image space to grow symmetry. Through these cooperative processes, the algorithm robustly finds rotational
symmetric patterns despite unsatisfactory number of initial seed matches.
3.4.1 Preprocessing
During the preprocessing stage, we refine the initial seed matches to further reduce
outliers and to reduce the computation time. The unreliable matches are filtered out,
and only the remaining matches are used as seeds. Along with Eq.(3.14) which generates a point pk that potentially makes rotational symmetry with existing feature points
pi and pj , other sample points that are in the same rotational symmetry are generated
17
4M
4M
4K
4K
4L
4N
D
4O
4
4L
K
4L
4M
E
4N
F
4O
Figure 3.7: Sampling image regions for refining initial seeds. (a) An initial seed with
rotationally symmetric regions Ri (blue) and Rj (red). A new region Rk (black) associated with the same rotational symmetry is generated by the pair. (b) All sampling
regions computed iteratively. (c) Original feature regions (upper row) and the sampled
regions (lower row) show appreciably high similarity; thus, the seed is reliable.
as well. The following holds for a rotational symmetry on any affine planes:
(pk+1 − c) = TA (pk − c),
(3.19)
Σk+1 = TA −T Σk TA −1 ,
(3.20)
where pk and Σk represents the location and the covariant matrix of elliptical feature region of k-th sample, respectively. Using Eq.(3.19)-(3.20), image patches that
are potentially rotationally symmetric with original feature regions are taken out iteratively as illustrated in Fig. 3.7. Then, the similarity with the original feature regions
is computed to check the reliability of each match. Finally, the seed matches that have
high similarity among sampled patches remain, and the rest of the seed matches are
discarded.
3.4.2 Initialization
When the process starts, each initial seed match of Sec.3.2 forms a singleton cluster
that contains themselves. Moreover, the matches are listed on the “supporter list” Q
for propagating symmetry. Each match expands according to the order of the list. The
18
=L
=T
WM
WL
7$
=M
=S
=S
(a)
7$
(b)
Figure 3.8: Propagating symmetry. (a) A supporter match Ma = (Zi , Zj ) and a neighboring region Zp picked near Zi , where TA is the transform within the supporter
match Mi . (b) The region Zp is mapped to the region Zq via TA , creating a propagated match Mb = (Zp , Zq ). Because the displacement between the centers of Zi
and Zp is known to be ti , the displacement tj between Zj and Zq is calculated as
tj = TA ti . Furthermore, using the shape of Zp , Σp , the shape of Zq is calculated as:
−1
Σq = T−T
A Σ p TA .
order is determined by the photometric similarity of matched features, so that strong
matches have the priority. Once a match performs expansion, it is then removed from
the list Q.
3.4.3 Expansion
From Eq.(3.13) of Sec.3.3, we have defined the transformation function from region
Zi to Zj as TA for a given rotationally matched feature pair Ma = (Zi , Zj ) as in
Eq.(3.1). Suppose that an arbitrarily region Zp = (pp , Σp , op ) that is close enough to
Zi is given. Then, another candidate rotationally symmetric match Mb = (Zp , Zq ) is
generated using TA and Zp . Letting the center point pp of the region Zp to be ti away
from the center point pi of region Zi so that pp = pi + ti , then the generated region
19
Zq is located at pq , where
pq = pj + TA ti .
(3.21)
Also, the covariance matrix Σq of the elliptical region Zq is determined as:
−1
Σq = T−T
A Σ p TA .
(3.22)
This process is illustrated in Fig.3.8. Next, whether the regions Zp and Zq actually
make a rotationally symmetric match photometrically should be checked. The Zeromean Normalized Cross Correlation (ZNCC) is used for evaluating the photometric
similarity. If the similarity is high enough, then the propagated match Mb = (Zp , Zq )
is accepted as a valid rotationally symmetric match. The seed match Ma = (Zi , Zj ) is
called as a “supporting match,” and the generated match Mb = (Zp , Zq ) is called as a
“propagated match.” The accepted propagated match Mb is stored in the same cluster
Ca of its supporter match Ma . Also, it is added to the list Q for further expansion.
An “expansion layer” that is the sets of element regions consisting of an overlapping circularly gridded regions is used, as shown in Fig.3.1(c), to select candidate for
the neighborhood region Zp . A candidate neighborhood region is selected from the
“current expansion layer” Gi of the currently expanding cluster Ci , which consists of
regular regions in the expansion layer that are close to the larger region of the matched
pair.
3.4.4 Merging
In the merging process, clusters with the same center of rotation are grouped together. The rotational centers of each match in a cluster are computed and accumulated
through a voting scheme introduced in Sec.4.1 to determine the rotational center of the
cluster as a whole. The rotational centers of each cluster are updated at every iteration. A tolerance threshold δc is set proportional to the image size is used to determine
the same center of rotation. If the clusters that share the same center of rotation with
the current cluster Ca are detected, the merging begins. All the matches of detected
20
clusters are added to the current cluster Ca , and their expansion layers are combined
with the current expansion layer Ga . Finally, the detected clusters are erased from the
memory.
3.4.5 Termination Criteria
The process of expanding and merging is repeated until the list of the seed matches Q
is empty. The proposed symmetry-growing algorithm is summarized in Algorithm 1
and is depicted in Fig.3.1(e)-(f).
21
Algorithm 1 Pseudo code of the proposed Symmetry-Growing framework
1: Construct seed matches from the extracted features. M = {M1 , M2 , ..., Mn }.
2:
For each seed, make an initial cluster that contains the seed itself. Ci = {Mi }
(i = 1, 2, ..., n).
3:
Generate an expansion layer Gi for each cluster Ci by given grids.
4:
Register all the seeds into the supporter list Q.
5:
repeat
6:
The supporter Ma with the highest similarity is removed from the supporter list
Q.
7:
Identify the cluster Ca which contains the supporter match Ma .
8:
Propose expansion to the neighborhood region via the supporter Ma , generating
a potential match Mb .
9:
The accepted match is stored in the cluster Ca , and added to the supporter list
Q.
10:
The propagated regions are eliminated from the expansion layer Ga .
11:
Update the rotational center of each cluster Ci .
12:
if Clusters that share the same rotational center with Ca are detected then
13:
Merge the clusters with Ca .
14:
Combine the expansion layers of the merged clusters.
15:
end if
16:
until The supporter list Q is empty
17:
Eliminate unreliable symmetry clusters.
22
Chapter 4
Symmetry Analysis
To analyze a rotational symmetric pattern, the center of rotational symmetry and the
number of folds need to be computed. The proposed algorithm first finds the center
of rotation for the cluster, then its elliptical shape and size to localize the symmetric
pattern in the image. Next, the algorithm figures out the number of folds in two ways:
the local-match-based histogram approach and the region-based frequency analysis
approach. Lastly, the algorithm discards unrelaible clusters in the outlier elimination
process.
4.1 Symmetry Center Localization
After the symmetry growing process, a cluster Ci contains symmetric matching pairs
that support a rotational symmetry. Every match Mij ’s in the cluster Ci has its own
rotational center cij ’s, although they do not vary greatly from each other because they
all support the same rotational symmetry with a constant center of rotation ci . The
rotational center of each match is used to localize the rotational center of the cluster as
a whole through a voting scheme. The centers are accumulated in a voting plane that
is the same size as the input image. The voting plane is then Gaussian-blurred, and the
maxima peak of the plane is identified as the center of rotational symmetry ci as in
23
D
F
G
E
H
Figure 4.1: Symmetry center localization for grown-up clusters. (a) Matched symmetric features in each cluster depicted in different colors. (b) Voting for the rotational
center of each cluster. (c) Voting result for cluster 1 (red). (d) Voting result for cluster
2 (green). (e) Final centers of rotation.
Eq.(4.1).
ci = max
x
j
1
1
exp(− 2 (x − cij )T (x − cij )).
2
2πσv
2σv
(4.1)
Fig.4.1 shows the process of the symmetry center localization.
4.2 Ellipse Estimation
Each rotational symmetric pattern has its own elliptical shape. Some are near circle
and some are highly elliptic. The fact that the ellipse parameter that fits a symmetric
feature pair can be estimated is shown in the last part of Sec.3.3 (Eq.(3.14)-(3.16)).
Based on these ellipse parameters determined by each symmetric feature pair in a
cluster (Fig.4.2(b)), the ellipse which fits the symmetric pattern that lies in the cluster
is computed.
An ellipse is assumed to be composed of its shape and its size. Hence, the elliptical
24
shape of the symmetric pattern that lies in the cluster must first be voted for. To do so,
the ellipses are scaled in a same size as in Fig.4.2(c). Then, the resulting normalized
parameters from each symmetric feature pairs should be accumulated in a voting plane,
and the maxima would be identified as the dominant elliptical shape of the pattern
that lies within the cluster, as in the localizing symmetry center problem of Eq.(4.1).
However, different from localizing symmetry center problem, it is hard to scale the
voting plane because the parameters are very sensitive to scales. Therefore, the random
Tukey depth [33] is used instead to find the dominant elliptical shape of the rotational
symmetry as in Fig.4.2(d). The random Tukey depth has been verified to be accurate
and instantaneous for finding the median for high dimensional data [33].
Next, the scale of the ellipse is estimated. Given the shape of the ellipse determined
from the previous step, the shortest distance to the cluster’s convex hull (Fig.4.3(a)) is
taken as the minor radius of the ellipse for a rough estimation as in Fig4.3(b). Then, a
simple three-step search [34] is performed on the length of the minor radius based on
the number of folds which is discussed in the next subsection (Sec.4.3). The search
moves to the direction where the number of folds and its strength increases as in
Fig4.3(c). After the estimation, the regions that lie outside the ellipse are eliminated
from the cluster as shown in Fig4.3(d). The approach also solves the problem of a local
symmetry (within an object) and a global symmetry (across objects) being merged together. Since the algorithm clusters the matches that share the same centers of rotation,
sometimes a local symmetry and a global symmetry get accumulated in the same cluster just because their centers of rotation are nearby as shown in Fig.4.4(b),(d). This
is problematic because not only does it obstruct clean segmentation of a symmetric
object, but it also prevent accurate symmetry analysis in the following steps. However,
using the approach, unwanted global symmetries are removed from local symmetry
clusters as shown in Fig.4.4(c),(e).
25
Ĕ
D
Ĕ
E F
G
Figure 4.2: Ellipse shape determination. (a) A grown cluster. (b) Ellipses of various
size and shape lie inside the grown cluster. (c) The ellipses are then normalized into
the same scale. (d) The normalized ellipses are then voted in a parameter space to
acquire the most popular elliptic shape that exists in the cluster.
4.3 Number of Folds Estimation
A pattern is called an N -fold rotationally symmetric pattern, with N being an integer
number, if it is invariant to rotations of 2π/N and its integer multiples about the pattern’s barycenter [35]. There are two types of approach for estimating number of folds.
One is the local match-based histogram approach and the other is the region-based
frequency analysis approach. Both of two approaches were tested in the proposed algorithm, and compared their results to figure out which approach performs the best
combined with the algorithm.
4.3.1 Local Match-Based Histogram Approach
The local match-based histogram approach uses angles of each rotationally symmetric
pairs to generate an angular histogram, and then estimates the number of folds from the
angles that frequently occur in the histogram. While it is robust to occlusion and noise,
it greatly depends on the existing local matches. In [11], Loy and Eklundh propose the
26
D EFG
Figure 4.3: Determining the scale of ellipses. (a) Grown clusters with the minimum distance from the center to its convex hull specified in colored lines. (b) These distances
are initially taken as the minor radii of ellipses. (c) A three-step search algorithm is
used to precisely estimate the scale of the ellipse. (d) The regions that lie outside the
ellipses are eliminated from their clusters.
following formula for the number of fold estimation function O(n):
1 2πk
2π(k − 1)
(h(
+ q) − h(
+ q)).
n−1
n
n
−q
n−1 q
O(n) =
(4.2)
k=1
Each number of fold n defines a set of rotation angles A =
2πk
n
= 1, 2, ..., n that should
frequently occur in the angular histogram as it exists. The formula prevails different
number of folds by calculating the mean number of rotations in some vicinity q of
the angles in A and subtracting the mean number of rotations that are
2π(k−1)
n
out of
phase with these angles. The order n of highest O(n) is selected to be the number
of the fold of the region. The angles recovered from Eq.(3.18) are used to achieve
robustness to skew in computing number of folds, while the original method plainly
uses the orientation difference of features and fails to estimate the correct number of
folds in skewed objects as in Fig.4.6.
4.3.2 Region-Based Frequency Analysis Approach
The region-based frequency analysis approach, on other hand, utilizes all image region
that contains the rotationally symmetric pattern. It is the most widely used approach
27
D
E
F
G
H
Figure 4.4: Removing a global symmetry (across objects) accumulated with a local
symmetry (within an object). (a) Original image. (b) Initially detected result. (d) The
clusters and their covering region. (e) Ellipse estimation result. (c) Final detection
result. The unwanted global symmetries are removed from local symmetry clusters.
for estimating the number of folds [27], [36]. However, it is vulnerable to occlusion
and noise. From the estimated symmetry center and the estimated ellipse, the elliptical
image region that emits rotational symmetry is derived as shown in Fig.4.7(b). Given
the ellipse parameters, the elliptical image region is then converted into a circular
image region using a simple perspective projection scheme as depicted in Fig.4.7(c).
After obtaining the circular region, the region is converted into polar coordinates (r, φ)
for convenience as in Fig.4.8(a). Crowther and Amos [36] expand the optical density
ρ(r, φ) within the image in cylindrical waves as:
ρ(r, φ) =
∞
gn (r)exp(inφ),
n=−∞
28
(4.3)
2ULJLQDO,PDJH&HQWHU(VWLPDWLRQ)LQDO5HVXOWRI)ROGV(VWLPDWLRQ
0D[SHDN
DW
D
0D[SHDN
DW
E
0D[SHDN
DW
F
Figure 4.5: Number of folds estimation results via the local match-based histogram
approach.
where gn (r) represents the weight of the n-fold azimuthal component of the image at
a radius r. Since the image density ρ is real, gn must satisfy the condition g−n (r) =
gn ∗ (r). By integrating each gn (r) over the radius a of the circular region, a measure
of the total n-fold rotational component of the image region is obtained. The strength
Sn of the n-fold component is computed as:
a
|gn (r)|dr.
S n = n
(4.4)
0
Note that only AC components are considered because DC component is always stronger
than any other AC component. Fig.4.8(b) shows the strength of each n-fold component,
Sn , plotted along n. The order n of the highest strength Sn is chosen to be the number
of the fold of the selected region.
The methods are tested on 100 selected images from the dataset of [27]. The
evaluation result is illustrated in Table 4.3.2. The result shows that the region-based
frequency analysis approach performs better than the local match-based histogram
method. This indicates that although local match-based angular histogram is robust
29
(a)
(b)
Figure 4.6: Local match-based histogram approach for the number of folds estimation.
(a) The number of fold wrongly estimated as 12 directly using features’ orientation
difference [11]. (b) The number of fold correctly estimated as 5 using angles recovered
from Eq.(3.18).
(OOLSVH(VWLPDWHG
D
E
5HFWLILHG
F
Figure 4.7: (a)Rotational symmetry detection result. (b) Enlarged view of (a) showing
a rotational symmetry and its estimated ellipse. (c) The symmetry exhibiting region
being rectified via the estimated ellipse.
30
(a)
(b)
Figure 4.8: Frequency analysis scheme for the number of folds estimation. (a) The
rotational symmetry of Fig.4.7 depicted in polar coordinate as ρ(r, φ). (b)Sn s, the
strength of nth order component. A clear peak at n = 9 is shown.
31
Table 4.1: The evaluation of two number of folds estimation method. The selected 100
images from the dataset of [27] with 140 ground truth is used.
Method Local-match-based
Region-based Frequency
Histogram Approach Analysis Approach
(extended from [11]) (extended from [27], [36])
Fold
88/140 = 63%
98/140 = 70%
to outliers and noise, its limited information about the image region bounds the performance, whereas the frequency analysis approach takes the whole image region into
consideration.
4.4 Outlier Elimination
The proposed algorithm definitely has a competitive edge regarding outliers. First is
because matches are grouped well based on their rotational centers due to skew robust
rotational center computation, and second is because the symmetry-growing framework provides the environment where only inliers can grow well. Thus, outliers are
easily distinguishable by size.
However, although the size of a cluster conveys large information about its reliability, there are a few more things to consider to derive a more accurate result. A
post-processing step is performed for elimination of outliers following the rotational
symmetry growing framework. The process is composed of five major steps.
Step 1. Since reliable symmetry clusters are likely to grow large, the algorithm
eliminates clusters whose size of the convex hull is small. A symmetry cluster is determined to be an inlier if the size of convex hull is larger than δa |I| where |I| denotes
the area of the given image I.
Step 2. As the inlier clusters have a tendency to have a dense distribution of rotational centers within the symmetric pattern as shown in Fig.4.1(c)-(d), for each cluster,
32
how dense the center distribution is compared to the size of the pattern is measured.
The algorithm calculates σc /r̄e , where σc is the standard deviation of the centers, and
r̄e is the geometric mean of the major and the minor radius of the estimated ellipse that
encloses the pattern. The smaller the value of σc /r̄e is, the more likely the cluster is an
inlier. A threshold δb is set, so that only clusters that satisfy σc /r̄e < δb are regarded
as inliers.
Step 3. The algorithm eliminates clusters whose estimated ellipse have a high ratio
of the major radius to the minor radius with a threshold of δd because a cluster whose
estimated ellipse is too elliptic is likely to be an outlier.
Step 4. The coverage ratio, which is defined as the area actually covered by a
cluster over the area of the estimated ellipse, is used. This value determines how partial
the symmetry pattern is due to occlusion or missing parts. Partial symmetries are likely
to be outliers if their coverage is too small compared with that of the whole. Therefore,
clusters with coverage ratio of less than a threshold δe is discarded.
Step 5. Finally, the algorithm discards clusters Ci , whose center of rotation ci has
a low peak value (Sec.4.1) compared with that of the largest cluster in the image. A
threshold δf is set, so that δf · peakvalue(cmax ) > peakvalue(ci ) for outlier clusters
Ci ’s.
Through these outlier elimination steps, most of the common outliers are discarded.
33
Chapter 5
Experimental Results
In all experiments, the parameter values for the algorithm are fixed as follows: The
ZNCC similarity threshold δs = 0.7 is used for rejecting photometrically unreliable
symmetry matches in expansion. In generating expansion layers, the radius of the overlapping circular regions re = l/25, where l denotes the shorter length of the axes of the
given image, is set. The cluster size threshold δa = 0.3, the center distribution threshold δb = 0.26, the elliptical cluster threshold δd = 2, the coverage ratio threshold
δe = 0.3, and the center peak value threshold δf = 0.2 are used.
The experiments conducted includes: the comparable quantitative evaluation on
the test dataset of the state-of-the-art algorithm [9] and on the dataset of recently held
symmetry detection competition [37], the demonstration of the skew robust rotational
center computation, and the skew robustness of the overall algorithm.
5.1 Quantitative Evaluation
The quantitative evaluation of our approach compared to four other state-of-the-art
methods ([11], [27], and the two proposed algorithms of [9]) is shown in Table 5.1.
The dataset used in the work of Lee and Liu [9] was used. In the sense of rotational
symmetry detection itself, the proposed method performs clearly better than four other
34
Table 5.1: Quantitative evaluation of the proposed algorithm compared with four other
state-of-the-art methods. The dataset used for performance evaluation in the work of
Lee and Liu [9] is used.
Algorithm
Data Set
TP Center Rate FP Center Rate # of Folds
Synthetic(29 images/48 GT)
31/48 = 65 %
4/48 = 8%
22/49 = 45%
Loy and Eklundh
Real-Single(58 images/58 GT) 50/58 = 86%
41/58 = 71%
16/64 = 25%
ECCV 2006 [11]
Real-Multi(21 images/78 GT) 32/78 = 41%
6/78 = 8%
12/42 = 29%
Overall(108 images/184 GT)
113/184 = 61% 51/184 = 28%
50/155 = 32%
Synthetic(29 images/48 GT)
36/48 = 75 %
0/48 = 0%
42/54 = 78%
Lee and Liu
Real-Single(58 images/58 GT) 25/58 = 43%
33/58 = 57%
22/32 = 69%
CVPR 2008 [27]
Real-Multi (21 images/78 GT) 19/78 = 24%
21/78 = 27%
18/25 = 72%
Overall(108 images/184 GT)
80/184 = 43%
54/184 = 29%
82/111 = 74%
Synthetic(29 images/48 GT)
43/48 = 90 %
12/48 = 25%
44/62 = 71%
Real-Single(58 images/58 GT) 47/58 = 81%
37/58 = 64 %
30/59 = 51%
PAMI 2010 #1 [9] Real-Multi(21 images/78 GT) 52/78 = 67%
22/78 = 28%
37/67 = 55%
Lee and Liu
Overall(108 images/184 GT)
142/184 = 77% 71/184 = 39%
111/188 = 59%
Synthetic(29 images/48 GT)
43/48 = 90 %
12/48 = 25%
44/62 = 71%
Real-Single(58 images/58 GT) 54/58 = 93%
31/58 = 53 %
35/66 = 53%
PAMI 2010 #2 [9] Real-Multi(21 images/78 GT) 55/78 = 71%
22/78 = 28%
40/70 = 57%
Lee and Liu
Overall(108 images/184 GT)
152/184 = 83% 65/184 = 35%
119/198 = 60%
Synthetic(29 images/48 GT)
42/48 = 88 %
1/48 = 2%
32/48 = 67%
Real-Single(58 images/58 GT) 55/58 = 95%
26/58 = 45%
34/58 = 59%
Real-Multi(21 images/78 GT) 57/78 = 73%
18/78 = 23%
44/78 = 56%
Ours
Overall(108 images/184 GT)
154/184 = 84% 45/184 = 24%
35
110/184 = 60%
2ULJLQDO,PDJH/(//2XUV
D
E
F
G
H
Figure 5.1: Example results of the proposed algorithm on the real images with a single
symmetry in the data set of [9]. LE06, and LL10 denote the methods of [11] and [9],
respectively.
36
2ULJLQDO,PDJH/(//2XUV
D
E
F
G
Figure 5.2: Example results of the proposed algorithm on the real images with multiple symmetries of natural objects in the data set of [9]. LE06, and LL10 denote the
methods of [11] and [9], respectively.
37
2ULJLQDO,PDJH/( //2XUV
D
(CKNGF
E
F
G
Figure 5.3: Example results of the proposed algorithm on the real images with multiple
symmetries of artifacts in the data set of [9]. LE06, and LL10 denote the methods of
[11] and [9], respectively.
38
methods. For comparison, TP (true positive) rates are adjusted by arranging thresholds so that they are similar to that of the highest performing state-of-the-art method
(the second algorithm of [9]). It turns out that with the same or higher TP rates, the
proposed algorithm shows lower FP (false positive) rates compared to the second algorithm of [9] for all categories in the dataset including synthetic, real images with
single symmetry, and the real images with multiple symmetries. In particular, for the
overall result, the FP rate is 10% lower than that of the second algorithm of [9]. The
performance of the proposed algorithm in estimating the number of folds is the second
best, it is tied with that of the second algorithm of [9]. Here, the result of region-based
frequency analysis approach is displayed because its performance turned out to be better than the local match-based angular histogram approach (Table 1). Some example
results of the proposed algorithm are illustrated in Fig.5.1, Fig.5.2, and Fig.5.3, where
detected symmetries are shown in different colors that represent their clusters. As can
be seen, the proposed method is highly robust and accurate in detecting rotational
symmetry in real-world images. The algorithm is not disturbed by background clutters
(Fig.5.1(a)-(d)), or occlusion (Fig.5.1(e), Fig.5.3(d)). Also, it detects highly skewed rotational symmetries (Fig.5.3(a)-(c)), and successfully deals with multiple symmetries
(Fig.5.2, Fig.5.3).
The proposed algorithm also is examined on the dataset of “Symmetry Detection
from Real World Images - A Competition,” the CVPR 2011 Workshop [37]. The code
of the initial form of the algorithm was submitted for the competition, but the result
was not so significant due to poor post-processing. However, the post-processing is
now improved, and as a result, the proposed algorithm clearly outperforms the competitors [38] and the baseline algorithm [11] as can be seen in Table 5.1 and in Fig.5.4.
The algorithm shows higher recall rate compared to its competitors while showing similar, or higher precision rate in most of the categories, including images with discrete
single and multiple symmetries, continuous multiple symmetries, and deformed single
symmetries as in Table 5.1. While showing better performance in most of the cate-
39
gories, the algorithm shows relatively poor precision ratio in the continuous single and
the deformed multiple case, although it shows similar recall rate. This is because the
algorithm has difficulties in dealing with featureless objects. Typically, for continuous
symmetries, they have comparably smaller number of initial features than other types
of symmetries. However, as can be seen in Fig.5.4, in terms of the five main categories
[37], namely, the images with single symmetries, multiple symmetries, discrete symmetries, continuous symmetries, and deformed symmetries, the proposed algorithm
performs the best in all categories.
The experiments demonstrate that the proposed method significantly outperforms
the state-of-the-art methods in terms of accuracy and robustness, and is widely applicable to the real-world complex images. The reason that the gap between the proposed
algorithm and [11] in the dataset of [37] is not so large as in the dataset of [9] is because
of the bias between the datasets. Whereas the dataset of [9] mainly consists of images
with discrete symmetries that the proposed algorithm is proficient at, the dataset of
[37] contains many images with continuous symmetries that outnumber those with
discrete symmetries.
40
Table 5.2: The proposed approach compared with a competitor and the baseline algorithm in the ”Symmetry Detection from Real World Images - A Competition,” the
CVPR 2011 Workshop [37]. The same dataset used in the competition was used.
Algorithm
Data Set
Recall
Precision
F1 score
Discrete-Single (11 images/11 GT)
4/11 = 36%
4/22 = 18%
24%
Discrete-Multi (3 images/16 GT)
3/16 = 19%
3/6 = 50%
27%
Continuous-Single(10 images/10 GT) 3/10 = 30%
3/20 = 15%
20%
Kondra and
Continuous-Multi(5 images/25 GT)
3/25 = 12%
3/7 = 43%
19%
Petrosino [38]
Deformed-Single(8 images/8 GT)
2/8 = 25%
2/13 = 15%
19%
Deformed-Multi(4 images/11 GT)
2/11 = 18%
2/10 = 20%
19%
Overall(41 images/81 GT)
17/81 = 21% 17/78 = 22% 21%
Discrete-Single (11 images/11 GT)
10/11 = 91% 10/17 = 59% 71%
Discrete-Multi (3 images/16 GT)
8/16 = 50%
8/11 = 73%
59%
Continuous-Single(10 images/10 GT) 7/10 = 70%
7/10 = 70%
70%
Loy and Eklundh [11] Continuous-Multi(5 images/25 GT)
5/25 = 20%
5/15 = 33%
25%
(Baseline)
Deformed-Single(8 images/8 GT)
5/8 = 63%
5/9 = 56%
59%
Deformed-Multi(4 images/11 GT)
6/11 = 55%
6/7 = 86%
67%
Overall(41 images/81 GT)
41/81 = 51% 41/69 = 59% 55%
Discrete-Single (11 images/11 GT)
10/11 = 91% 10/17 = 59% 71%
Discrete-Multi (3 images/16 GT)
11/16 = 69% 11/14 = 79% 73%
Ours
Continuous-Single(10 images/10 GT) 8/10 = 80%
8/16 = 50%
62%
Continuous-Multi(5 images/25 GT)
8/25 = 32%
8/15 = 53%
40%
Deformed-Single(8 images/8 GT)
7/8 = 88%
7/7 = 100%
93%
Deformed-Multi(4 images/11 GT)
5/11 = 46%
5/7 = 71%
55%
Overall(41 images/81 GT)
46/81 = 60% 46/71 = 64% 62%
41
<ŽŶĚƌĂ
>ŽLJ
KƵƌƐ
ϭ
Ϭ͘ϵ
Ϭ͘ϴ
Ϭ͘ϳ
Ϭ͘ϲ
Ϭ͘ϱ
Ϭ͘ϰ
Ϭ͘ϯ
Ϭ͘Ϯ
Ϭ͘ϭ
Ϭ
^ŝŶŐůĞ
DƵůƚŝƉůĞ
ŝƐĐƌĞƚĞ
ŽŶƚŝŶƵŽƵƐ
ĞĨŽƌŵĞĚ
ůů
(a) Recall on the dataset of [37] in five main categories.
<ŽŶĚƌĂ
>ŽLJ
KƵƌƐ
Ϭ͘ϵ
Ϭ͘ϴ
Ϭ͘ϳ
Ϭ͘ϲ
Ϭ͘ϱ
Ϭ͘ϰ
Ϭ͘ϯ
Ϭ͘Ϯ
Ϭ͘ϭ
Ϭ
^ŝŶŐůĞ
DƵůƚŝƉůĞ
ŝƐĐƌĞƚĞ
ŽŶƚŝŶƵŽƵƐ
ĞĨŽƌŵĞĚ
ůů
(b) Precison on the dataset of [37] in five main categories.
<ŽŶĚƌĂ
>ŽLJ
KƵƌƐ
Ϭ͘ϴ
Ϭ͘ϳ
Ϭ͘ϲ
Ϭ͘ϱ
Ϭ͘ϰ
Ϭ͘ϯ
Ϭ͘Ϯ
Ϭ͘ϭ
Ϭ
^ŝŶŐůĞ
DƵůƚŝƉůĞ
ŝƐĐƌĞƚĞ
ŽŶƚŝŶƵŽƵƐ
ĞĨŽƌŵĞĚ
ůů
(c) F1-score on the dataset of [37] in five main categories.
Figure 5.4: Quantitative comparison of the proposed algorithm (Red on the right), the
competitor algorithm (Green on the left) the baseline algorithm (Blue in the middle)
on the dataset of [37] in the five main categories [37].
42
5.2 With and Without the Skew Robust Rotational Center
Computation
To demonstrate the effectiveness of the skew robust rotational center computation, the
approach was applied on the previous method [11] to see how the proposed idea boosts
the conventional method. The examples are shown in Fig.5.5(c), where the result of
the original method [11] is shown in Fig.5.5(a). As can be seen, the one with our
approach clearly outperforms the original method. However, since [11] uses only SIFT
features, whereas our algorithm also uses affine covariant features, the result of the
modified version of [11] that uses the same covariant features as ours is also subjected
for comparison as shown in Fig.5.5(b). The one with our skew robust rotational center
computation shows better results in this case also. Observe that the estimation result of
the rotational center is also better with the proposed scheme as shown in Fig.5.5(c) of
Img.2. Furthermore, although the methods started with the same features, the features
are grouped better using the skew robust rotational center computation as shown in
Fig.5.5(b)-(c) of Img.2.
Next, the symmetry growing algorithms with and without the proposed skew robust
rotational center computation are compared to further investigate the performance of
the skew robust rotational center computation. In Fig.5.5(d), the result of the symmetry growing algorithm without skew robust rotational center computation is displayed.
When the symmetry growing framework is used by itself, the performance is insignificant compared with the one without it (Fig.5.5(b)). However, when the skew robust
rotational center computation is used together, the algorithm performs much better
than other listed methods in terms of accuracy and robustness as shown in Fig.5.5(e).
Because it performs better than the case when only the skew robust rotational center
computation is used, as shown in Fig.5.5(c), the skew robust rotational center computation and the symmetry growing framework can be said to create a mutually supportive
synergy effect. This can be explained as follows: As demonstrated in Fig.5.5(c), the
43
skew robust rotational center computation helps to detect more inliers. As a result,
these inliers form large inlier clusters in the symmetry growing framework and leave
few regions in the expansion layer, thereby strongly suppressing outliers from growing.
Thus, the inliers are discriminated even well from outliers by their size.
In summary, the skew robust rotational center computation improves the results of
both the previous method [11] and the symmetry growing framework used by itself.
5.3 Skew Robustness of the Algorithm
The proposed algorithm’s robustness to skew is evaluated by testing it on images varying the intensity of the skew deformation. An affine transform was applied multiple
times to get differently skewed images. The intensity level of skew is defined to be the
times of the affine transform [1 0 0; .25 1 0; 0 0 1] is applied on the original image.
The affine transform is chosen so that an increase of the intensity level corresponds
to a gradual increase of the major-to-minor radius ratio as in the Table 5.3. Examples
showing different levels of skew are shown in Fig.5.6. Experiments were performed
on 275 images consisting of 11 levels of skew synthesized from 25 original images
(13 synthetic images and 12 real images). Fig.5.7 shows the accuracy of the algorithm
according to the 11 intensity levels of skew. From the graph of Fig.5.7, it is observed
that the accuracy remains 100% until level 3, and it declines almost linearly with the
slope of −6.3%/level. from level 4. Considering that most of the rotational symmetry
that is taken in real-world fall into the category with the major-to-minor radius ratio
ranging from 1 to 3, it can be said that the algorithm is sufficiently robust to skew since
the algorithm performs well with the accuracy over 88% at level 0 to 5 while it also
persists the good performance at level 6 to 10 that corresponds to extreme radius ratios
of 4 to 8 with the lowest accuracy 64% at the very last level. The accuracy should not
decline according to the intensity of the skew as shown in the Sec.3.3, if the matching
result is the same. However, the matching result becomes degenerated as the intensity
44
of the skew becomes stronger, resulting only few number of inlier matches to remain.
Therefore, the performence declines as the number of inlier matches decreases.
Table 5.3: The major-to-minor radius ratio according to the skew intensity level.
Skew Intensity Level
0
1
2
3
4
5
6
7
8
9
10
major-to-minor radius ratio 1.00 1.28 1.64 2.08 2.62 3.26 4.00 4.86 5.83 6.92 8.13
5.4 Limitations
Although the overall success rates of the proposed method are significantly better than
those of existing rotational symmetry detection algorithms, the method has some limitations. Since the algorithm is a feature-based one, it fails to detect symmetry for
featureless object as shown in Fig.5.8(a). Also, the proposed algorithm has difficulties in irregularly shaped continuous symmetric objects. Although the SIFT features
are used to capture the features on edges, it sometimes cannot create enough initial
matches. Moreover, because the shape is irregular, the symmetry growing framework
can hardly help the symmetry grow as shown in Fig.5.8(b).
45
,PDJH,PDJH,PDJH,PDJH
D
( KN F
(CKNGF
E
F
G
H
Figure 5.5: Results showing the effectiveness of the skew robust rotational center computation. (a) The result of the previous approach [11]. (b) The result of [11] with affine
covariant features. (c) The skew robust rotational center computation applied on (b).
(d) The results of the symmetry growing without the skew robust rotational center
computation. (e) The result of the overall algorithm
46
/HYHO/HYHO
/HYHO/HYHO/HYHO/HYHO
/HYHO/HYHO/HYHO/HYHO/HYHO
Figure 5.6: Examples showing different levels of skew (from level 0 to level 10)
ĐĐƵƌĂĐLJ;йͿ
ϭϮϬ
ϭϬϬ
ϴϬ
ϲϬ
ϰϬ
ϮϬ
Ϭ
Ϭ
ϭ
Ϯ
ϯ
ϰ
ϱ
ϲ
ϳ
ϴ
ϵ
ϭϬ
Figure 5.7: Accuracy according to the intensity of skew. We tested our algorithm on
275 images consisting of 11 levels of skew synthesized from 25 original images (13
synthetic images and 12 real images).
47
2ULJLQDO,PDJH
/(//
2XUV
(CKNGF
(CKNGF
(a)
2ULJLQDO,PDJH
J
J
/(.32XUV
(CKNGF
(CKNGF
(CKNGF
(b)
Figure 5.8: Results showing limitations of the algorithm. (a) Featureless image. (b)
Irregular continuous symmetry. LE06, LL10, and KP11 denote the methods of [11],
[9], and [38], respectively.
48
Chapter 6
Conclusion
In this paper, a novel approach for detecting and localizing rotationally symmetric patterns in real-world images is presented. It efficiently deals with skewed rotationally
symmetric objects by utilizing the shape of affine covariant features, giving a closed
form solution for the center of rotation for a rotationally symmetric feature pair on any
affine plane. Based on this, the elliptical shape, and the rotation angle of a symmetric
feature pair also are discovered. On top of that, the proposed symmetry-growing framework explores the image space to grow rotational symmetry beyond the symmetrically
matched initial feature pairs, thereby solving the problem of limited number of features
of the conventional local-feature-based methods. The experimental results demonstrate
that the proposed method clearly outperforms state-of-the-art methods and is widely
applicable to real-world images with complex backgrounds. The proposed approach
can be extended to a variety of pattern analysis tasks such as vehicle detection, visual
attention, completion of occluded shapes, segmentation, object detection, and object
classification. For the remaining research problems, the authors would like to further
investigate methodologies to deal with featureless objects and severe deformations.
49
Bibliography
[1] F. Labonte, Y. Shapira, and P. Cohen, “A perceptually plausible model for global
symmetry detection,” in Proc. IEEE Int’l Conf. Computer Vision, 1993, pp. 258–
263.
[2] H. B. Barlow and B. C. Reeves, “The versatility and absolute efficiency of detecting mirror symmetry in random dot displays,” Vision Res., vol. 19, pp. 783–793,
1979.
[3] P. Flynn, “3-d object recognition with symmetrical models: symmetry extraction
and encoding,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16,
no. 8, pp. 814–818, 1994.
[4] T. Riklin-Raviv, N. Kiryati, and N. Sochen, “Segmentation by level sets and symmetry,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1,
2006, pp. 1015–1022.
[5] W. C. Lin, “A lattice-based mrf model for dynamic near-regular texture tracking and manipulation,” Ph.D. dissertation, Robotics Institute, Carnegie Mellon
University, Pittsburgh, PA, 2005.
[6] Y. Liu, “Computational symmetry,” Wenner Gren Int’l Series, pp. 231–246, 2000.
[7] H. Cornelius and G. Loy, “Detecting rotational symmetry under affine projection,” in Proc. Int’l Conf. Pattern Recognition, vol. 2, 2006, pp. 292–295.
50
[8] M. Park, S. Lee, P. Chen, S. Kashyap, A. A. Butt, and Y. Liu, “Performance
evaluation of state-of-the-art discrete symmetry detection algorithms,” in Proc.
IEEE Conf. Computer Vision and Pattern Recognition, 2008, pp. 1–8.
[9] S. Lee and Y. Liu, “Skewed rotation symmetry group detection.” IEEE Trans.
Pattern Analysis and Machine Intelligence, vol. 32, no. 9, pp. 1659–1672, 2010.
[10] M. Cho and K. M. Lee, “Bilateral symmetry detection via symmetry-growing,”
in Proc. British Machine Vision Conf., 2009, pp. 1–8.
[11] G. Loy and J. Eklundh, “Detecting symmetry and symmetric constellations of
features,” in Proc. European Conf. Computer Vision, 2006, pp. 508–521.
[12] S. Derrode and F. Ghorbel, “Robust and efficient fourier-mellin transform approximations for gray-level image reconstruction and complete invariant description,” Computer Vision and Image Understanding, vol. 83, no. 1, pp. 57–78,
2001.
[13] Y. Keller and Y. Shkolnisky, “An algebraic approach to symmetry detection,” in
Proc. Int’l Conf. Pattern Recognition, vol. 3, 2004, pp. 186–189.
[14] C. Sun and D. Si, “Fast reflectional symmetry detection using orientation histograms,” J. Real-Time Image Processing, vol. 5, no. 1, pp. 63–74, 1999.
[15] T. Masuda, K. Yamamoto, and H. Yamada, “Detection of partial symmetry using
correlation with rotated-reflected images,” Pattern Recognition, vol. 26, no. 8,
pp. 1245–1253, 1993.
[16] H. Zabrodsky, S. Peleg, and D. Avnir, “Symmetry as a continuous feature,” IEEE
Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 12, pp. 1154–1166,
1995.
[17] D. Lowe, “Distinctive image features from scale-invariant keypoints,” Int’l J.
Computer Vision, vol. 60, no. 2, pp. 91–110, 2004.
51
[18] K. Mikolajczyk and C. Schmid, “Scale and affine invariant interest point detectors,” Int’l J. Computer Vision, vol. 60, no. 1, pp. 63–86, 2004.
[19] J. Matas, O. Chum, M. Urban, and T. Pajdla, “Robust wide-baseline stereo
from maximally stable extremal regions,” Image and Vision Computing, vol. 22,
no. 10, pp. 761–767, 2004.
[20] H. Cornelius and G. Loy, “Efficient symmetry detection using local affine
frames,” in Proc. Scandinavian Conf. on Image Analysis, 2007, pp. 152–161.
[21] M. Cho, Y. Shin, and K. Lee, “Co-recognition of image pairs by data-driven
monte carlo image exploration,” in Proc. IEEE Int’l Conf. Computer Vision,
2008, pp. 144–157.
[22] V. Ferrari, T. Tuytelaars, and L. Van Gool, “Simultaneous object recognition and
segmentation from single or multiple model views,” Int’l J. Computer Vision,
vol. 67, no. 2, pp. 159–188, 2006.
[23] J. Kannala, E. Rahtu, S. Brandt, and J. Heikkila, “Object recognition and segmentation by non-rigid quasi-dense matching,” in Proc. IEEE Conf. Computer
Vision and Pattern Recognition, 2008, pp. 1–8.
[24] M. Lhuillier and L. Quan, “Match propagation for image-based modeling and
rendering,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24,
no. 8, pp. 1140–1146, 2002.
[25] T. Tuytelaars, A. Turina, and L. V. Gool, “Noncombinatorial detection of regular
repetitions under perspective skew,” IEEE Trans. Pattern Analysis and Machine
Intelligence, vol. 25, no. 4, pp. 418–432, 2003.
[26] S. Prasad and L. Davis, “Detecting rotational symmetries,” in Proc. IEEE Int’l
Conf. Computer Vision, vol. 2, pp. 954–961.
52
[27] S. Lee, R. T. Collins, and Y. Liu, “Rotation symmetry group detection via frequency analysis of frieze-expansions,” in Proc. IEEE Conf. Computer Vision and
Pattern Recognition, 2008, pp. 1–8.
[28] L. V. Gool, T. Moons, and M. Proesmans, “Mirror and point symmetry under
perspective skewing,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1996, pp. 285–292.
[29] R. K. K. Yip, W. C. Y. Lam, P. K. S. Tam, and D. N. K. Leung, “A hough transform technique for the detection of rotational symmetry,” Pattern Recogn. Lett.,
vol. 15, no. 9, pp. 919–928, 1994.
[30] D. Shen, H. Ip, and E. Teoh, “Robust detection of skewed symmetries,” in Proc.
Int’l Conf. Pattern Recognition, vol. 3, 2000, pp. 1010–1013.
[31] S. Lee and Y. Liu, “Curved glide-reflection symmetry detection,” in Proc. IEEE
Conf. Computer Vision and Pattern Recognition, 2009, pp. 1046–1053.
[32] D. Lowe, “Object recognition from local scale-invariant features,” in Proc. IEEE
Int’l Conf. Computer Vision, vol. 2, 1999, pp. 1150–1157.
[33] J. Cuesta-Albertos and A. Nieto-Reyes, “The random tukey depth,” Computational Statistics & Data Analysis, vol. 52, no. 11, pp. 4979–4988, 2008.
[34] T. Koga, K. Iinuma, A. Hirano, Y. Iijima, and T. Ishiguro, “Motion compensated
interframe coding for video conferencing,” in Proc. Nat. Telecommun. Conf,
vol. 5, no. 3, 1981, pp. 1–5.
[35] L. Lucchese, “Frequency domain classification of cyclic and dihedral symmetries
of finite 2-d patterns,” Pattern recognition, vol. 37, no. 12, pp. 2263–2280, 2004.
[36] R. Crowther and L. Amos, “Harmonic analysis of electron microscope images
with rotational symmetry,” J. Molecular Biology, vol. 60, no. 1, pp. 123–126,
1971.
53
[37] I. Rauschert and Y. Liu, “Evaluation details,” presented at the CVPR Workshop
on Symmetry Detection from Real World Images - A Competition, 2011.
[Online]. Available: http://vision.cse.psu.edu/research/symmComp/index.shtml
[38] A. Ferone, S. Kondra, and A. Petrosino, “Multi-scale kernel operators for
reflection and rotation symmetry,” presented at the CVPR Workshop on
Symmetry Detection from Real World Images - A Competition, 2011. [Online].
Available: http://vision.cse.psu.edu/research/symmComp/workshop/index.shtml
54
초록
대칭은 매우 두드러진 시각 정보이며, 이미지로부터 검출될 경우 물체 인식
및 분류 등 많은 영상이해 분야에서 활용도가 높은 현상이다. 본 논문에서는 그
중에서도 회전 대칭을 다루고자 하며, 실제 영상에서 빈번히 발생하는 원근에
의한 편향(perspective skew)에 강인한 회전 대칭 검출 기법을 제안하고자 한다.
기존의 회전대칭 검출 기법들의 경우, 완벽한 원을 가정하고 검출을 하였기 때문
에 임의의 각도에서 3D 물체를 카메라로 찍었을 때에 생기는 원근 변환에 의한
왜곡, 원이 즉 타원에 가까운 형태가 되어버리는 문제에 취약하여 합성이미지가
아닌, 실제 사진으로부터 원형대칭을 검출하는 데에 어려움이 있었다. 본 논문에
서는, 기존의 방법들이 매칭된 특징점의 위치와 방향만을 고려하여 편향에 대해
반복 연산으로 접근했던 것과 달리, 특징점 주변의 특징 부분(feature region)의 형
태 정보를 이용하여 어떠한 편향에도 강인한 닫힌 해(closed form solution)로써의
회전 중심을 구해낸다. 또한, 기존의 특징 부분을 이용한 검출 방법들의 경우, 이
미지로부터 추출할 수 있는 특징 부분이 제한되어있기 때문에 충분한 특징 부분
매치가 존재하지 않아 대칭인 부분을 구하기 힘든 고질적인 문제가 있었으나, 본
알고리즘에서는 대칭 증식(Symmetry Growing) 기법을 회전 대칭으로 확장하여
제한된 특징 부분 개수로 인한 한계를 극복한다. 제안된 대칭 증식 기법은 회전
대칭을 이루는 특징점들의 인접 영역을 탐색하여 이미지 내에 존재하는 회전
대칭 영역을 찾아 확장시켜 나감으로써 실제 영상에 강인한 회전대칭 검출이
이루어 질 수 있도록 한다. 실험결과를 통해 제안된 기법의 성능이 실제 영상에
매우 효과적이며 기존 방법들에 비해 우수한 성능을 보임을 알 수 있다.
55
주요어: 왜곡된 회전 대칭, 회전 대칭 검출, 대칭 증식 기법, 대칭 중심
학번: 2010-20792
56