Finding Symmetry Plane of 3D Face Shape∗
Gang Pan
Yueming Wang
Yipeng Qi
Zhaohui Wu
Department of Computer Science, Zhejiang University, Hangzhou, 310027, China
{gpan,ymingwang}@zju.edu.cn
Abstract
The symmetry plane detection from 3D face shape is
much helpful for pose estimation and feature extraction for
a wide range of applications, e.g. 3D face recognition. This
paper proposes a robust symmetry plane detection method
for 3D face models, which is an extension of EGI-based
approach. The experiments using two data sets, USF HumanID 3D face database and 3DFED database covering
significant variation in facial expression, demonstrate the
effectiveness.
1
Introduction
Numerous objects in the world have a property of symmetry, such as the human face and body, animal’s body,
chair, cup, etc. Symmetry is one of the most important geometry properties for objects.
Symmetry could be defined in terms of three transformations in n-d Euclidean space E n : reflection, rotation and
translation. Formally, a subset S of E n is symmetric with
respect to a transformation T if T (S) = S. Reflectional
symmetry is a kind of symmetry without rotation, for which
one half part is a mirror of the other half part. The reflectionally symmetrical object has a symmetry plane.
The 3D human face shape could be considered as a
nearly reflectionally symmetrical object. It comes up with
the robust and fast detection of symmetry plane for 3D face,
since the symmetry plane is much helpful for pose determination and 3D face feature extraction for wide applications,
e.g. 3D face recognition [11, 12]. This paper addresses
finding the reflectional symmetry plane of 3D face shapes.
Generally, there are two kinds of symmetry detection
methods. The first one uses geometry information of the
object to construct an intermediate dense representation to
conduct symmetry detection [1, 2, 3, 4, 5]. This kind of
methods is suitable for objects with a lot of detailed information.
∗ The authors are grateful for the grants from NSF of China (60503019,
60525202) and Program for New Century Excellent Talents in University
(NCET-04-0545).
The other kind of methods emphasizes particularly on
extraction of the structural information. It regards the object
as a discrete and sparse structure. After simply abstracting,
the analysis based on graph theory, topology theory etc. is
performed [6, 7, 8]. This kind of methods is appropriate to
deal with objects with prominent structure characteristics.
There is few method dealing with symmetry plane detection particularly for 3D face shapes. The most relative
work is the reference [5], obtaining the symmetry information of the 3D model based on extended Gaussian image
(EGI) [10]. However, this approach is hard to handle the
models with noise data. This paper gives a two-step approach to realize the robust symmetry plane detection from
3D face shapes.
2
The Method
A plane in 3D Euclidean space could be formalized as
−
→
→
n ·−
x +k =0
(1)
→
where −
n is the unit normal vector of the plane, and k is a
constant. Thus, the task of seeking the reflectional symme→
try plane of 3D face shape is to solve two parameters −
n and
k.
We propose a two-step method to sequently solve the two
→
parameters. 1) Firstly, the normal −
n is calculated based on
EGI. 2) Secondly, a novel procedure is conducted to determine the parameter k, the offset position of the symmetry
plane. The method diagram is shown in Fig. 1.
2.1
Finding normal of symmetry plane
Calculation of the normal of the symmetry plane is
mainly based on EGI, similar to [5] but more robust. EGI
[10] is a 3D shape representation, which is a function defined on the unit sphere(called Gauss sphere [17]), with the
value in the direction of the unit normal n given by K(n).
Minkowski proved that, if two convex objects have the same
EGIs, those objects are congruent [9]. EGI also has some
good properties[10]: 1)The EGI mass on the sphere is the
inverse of Gaussian curvature on the object surface. 2)The
0-7695-2521-0/06/$20.00 (c) 2006 IEEE
1. Finding the normal of the symmetry plane
Computing
Gaussian
Curvature
Map
onto
Gauss
sphere
Search the optimal
normal around the
principal axes
EGI
2. Determining the offset position of the symmetry plane
Input
Result
Sampling N point pairs from the
model via drawing N lines with
the direct of the normal
The offset position is
obtained by averaging the N
segments' midpoints
Figure 1. The method diagram
(a)
In order to get the model’s EGI, we should map the curvature’s reciprocal to the tessellation. For a 3D face shape,
some obtained curvature would be very big or very small
due to noise data. If we map the too small curvature to the
tessellation, there will be a quite big value on the tessellation, which would disturb the symmetry detection. Thus,
we filter the curvatures before mapping to remove those are
not in a given range, empirically setting to (0.001, 0.1).
(b)
Figure 2. EGI illustration. (a) face, (b) its EGI
mass center of the EGI is at the origin of the sphere. 3) As
an object rotates, its EGI also rotates in the exact same way.
Once the EGI of the 3D face shape is constructed, the
normal of the symmetry plane could be derived directly
from the EGI without need of the original 3D face model.
2.1.1
Constructing EGI
Let us assume that an object surface is evenly sampled into
patches. At each surface patch, we can define a surface normal with a single unit of mass. Each surface normal is assumed to be able to vote the mass to the corresponding point
on the Gaussian sphere.
Actually, EGI could be regarded as an orientation histogram {{pi , Mi }|i ∈ 0, 1, . . . , k} for each point pi on
Gauss sphere has a value Mi , shown in Fig. 2. The mass
distribution of the model is shown in Fig. 2(b), with voting
by the all surface patches over the object surface. According
to the characteristic of EGI that the EGI mass on the sphere
is the inverse of Gaussian curvature on the object surface,
we could use the inverse of Gaussian curvature to compute
the orientation histogram. In this paper, the Gauss sphere is
constructed by subdivide icosahedron [6], and the Gaussian
curvature is mapped onto Gauss sphere.
There are many discrete methods to estimate the Gaussian curvature on triangular meshes. We simply employ
the paraboloid fitting method recommended by [16] to discretely estimate the Gaussian curvature.
2.1.2
Finding the normal
After construction of EGI for a 3D model, the problem of
solving the normal of the symmetry plane is converted into
finding the symmetry plane of its EGI, since the normal of
symmetry plane is same for 3D model and its EGI. The following will focus on how to find the EGI’s symmetry plane
which must pass through the mass center of Gauss sphere.
Reference [5] gives a method to reduce the search space
for solving the symmetry plane, whose motivation is based
on theorems stating that: any symmetry plane a body is perpendicular to a principal axis [13]. So, it only checks the
directions of the principal axes of the models. However, it
may result in inaccurateness in existence of noise. Here, we
relax the search space to seek the EGI’s symmetry plane ΛT
in a wider range to increase robustness
The principal axis could be obtained by the inertia matrix
J of an EGI, which defined as:
J
= tr(C)I − C

µ020 + µ002
−µ110
= 
−µ101
−µ110
µ200 + µ002
−µ011
(2)

−µ101

−µ011
µ200 + µ020
where tr(·) denotes the trace of a matrix and I is the 3 × 3
identity matrix. C is the covariance matrix of an EGI, derived from the second-order central moments. The eigenvectors of J are the principal axes of EGI, and the eigenvalue varies inversely with the variance of EGI along the
corresponding axis.
0-7695-2521-0/06/$20.00 (c) 2006 IEEE
Assume that EGI is represented as {{pi , Mi }|i ∈
0, 1, . . . , k} , where pi is a point on the Gaussian sphere,
and Mi is the histogram value on point pi . Given a candidate symmetry plane Λt , for an arbitrary point pi , we can
find its symmetrical point ps(i) with respect to Λt . Then
symmetry plane could be solved by maximizing the correlation measure:
ΛT = arg max Ct ,
Λt
Ct =
k
Mi Ms(i)
(3)
i=0
Fig. 3(a). In order to improve the performance, here we give
a simple but efficient scheme.
→
Assume that the normal −
n of the symmetry plane is
known. For a 3D human face shape represented by {P, F }
where P is vertex set {p1 , p2 , . . . , pn } and F is index set of
→
the points, if we draw a line with the direction of −
n which
passes through a point p1 on the face shape, the line often intersects at another point p2 of the face shape. If the
shape is strictly reflectional symmetrical, the midpoint of
the segment between p1 and p2 will lie on symmetry plane,
shown in Fig. 4. To cope with the noise data, we sample
For 3D face shape with noise data, the direction of the
principal axis is usually not near to the normal of the symmetry plane. Thus, to solve the optimization problem of
Equ. 3, we should expand the search space but limited a reasonable range to balance accuracy and computational cost.
We define a distance Ω to quantitatively control the search
space:
(4)
Ω = (ω0 − n0 )2 + (ω1 − n1 )2 + (ω2 − n2 )2
→
where unit vector −
n = {n0 n1 n2 }T is the direction of a
→
principal axis, and unit vector −
ω = {ω0 ω1 ω2 }T is the
normal of a candidate symmetry plane.
→
−
→
−
√ Since n and ω are both unit vector, Ω must fall into [0,
2]. Thus we could set a threshold Ω0 on Ω to constrain
the search space. Smaller the threshold, less the computational cost, but less accuracy of the result. We choose the
threshold in bound [0.3, 0.8].
2.2
Figure 4. Illustration of finding the offset
N points of the face shape, and, for each point, draw a line
passing through this point with the direction of the normal
→
−
n . In this manner a set of points pair are obtained, denoted
as {(pi , qi }, the average midpoint P could be simply computed, through which the symmetry plane passes:
Determining the offset position
P =
The symmetry plane of a 3D model could be determinate together by its normal and its offset. The normal has
been solved in the previous section. This section gives an
efficient step to compute the offset of the symmetry plane.
3.1
Sun et al[5] assume that the symmetry plane passes
through the model’s mass center, but for those models with
noise data and with non-rigorous symmetry, the mass center will be not exactly on the symmetry plane, shown in
(5)
Combining with Equ. 1, the offset position k of symmetry
plane is easily solved.
3
Figure 3. Result comparison. (a) by Sun’s
method [5], (b) by our method
N
1 pi + q i
N i=1
2
Experimental Results
Data Sets
The experiments use two 3D face databases. One is USF
Human ID 3D face database [15], 136 models totally. The
other is 3DFED (3D Face Expression database) built by our
group using InSpeck 3D MEGA Capturor DF [14] with totally 360 scans, in which there are 9 scans for each subject (3 neutral scans, 2 smile scans, 2 surprise scans, 2 sad
scans). To reduce the computational costwe simplify each
scanned model to nearly 2000 vertices.
3.2
Results
For all 3D face models, there are more than 95% of the
models have good detection results by our method. For
0-7695-2521-0/06/$20.00 (c) 2006 IEEE
most face models, the computational cost is less than 1 second on Pentium IV 2.0GHz. Figure 5(a) shows detection
results in USF Human ID 3D face database, the first row
is the input models, the second row is the detection results.
Figure 5(b) illustrates some results in 3DFED database with
the various facial expressions.
4
Conclusions
A robust and effective method for detecting the symmetry plane from 3D human face has been presented, which
could deal with different facial expressions and noise data.
The method consists of two sequent steps—-normal-finding
and offset-finding. The experimental results demonstrate its
effectiveness.
References
(a)
(b)
Figure 5. Detection results
(a)
(b)
(c)
(d)
Figure 6. Results with different threshold. (a)
input model, (b) Ω0 = 0.2, (c) Ω0 = 0.3, (d)
Ω0 = 0.4
If the threshold Ω0 is small, the computational cost
would be less, but the result be more inaccurate. Otherwise,
if Ω0 is big, the result would be accurate while the computational cost will increase. Figure 6 shows the results with
different threshold Ω0 , accuracy is significantly improved
from left to right.
[1] D. Shen, H.H.S.Ip, K. Cheung, E. K. Teoh, “Symmetry Detection by Generalized Complex (GC) Moments: A Close-Form
Solution”, IEEE PAMI, 21(5):466-476, 1999.
[2] D. Shen, H.H.S. Ip and E. K. Teoh, ‘Detecting reflection axes
by energy minimization”, ICPR’00.
[3] H.Zabrodsky, S. Peleg and D. Avnir, “Symmetry as a Continuous Feature”, IEEE PAMI, 17(12):1154-1166, 1995.
[4] M.Kazhdan , B.Chazelle, D.Dobkin, et al, “A reflective symmetry descriptor”, ECCV’02.
[5] Changming Sun, J.Sherrah, “3D Symmetry Detection Using
The Extended Gaussian Image”, IEEE PAMI, 19(2), 1997.
[6] Lei Yiwu and W. K. Cheong, “A novel method for detecting
and localising of reflectional and rotational symmetry under
weak perspective projection”, ICPR’98.
[7] S.Fukushima, “Division-based analysis of symmetry and its
application,” IEEE PAMI, 19(2):144-148, 1997.
[8] H.Heijmans and A.Tuzikov, “Similarity and symmetry measures for convex shapes using Minkowski addition”, IEEE
PAMI, 20(9):980-993, 1998.
[9] L.A. Lysternik , “Convex Figures and Polyhedra”, Dover Publications, New York,1963.
[10] B. Horn, “Extended Gaussian Image,” Proc. of IEEE,
72(12):1671-1686, 1984.
[11] Gang Pan, Z.Wu, ”3D Face Recognition from Range Data”,
Int’l Journal of Image and Graphics, 5(3):573-593, 2005.
[12] Gang Pan, S.Han, Z.Wu, Y.Wang, “3D Face Recognition using Mapped Depth Images”, CVPR’05, FRGC Workshop.
[13] P.Minovic, S.Ishikawa, K.Kato, “Three Dimensional Symmetry Identification, Part I: Theory”, Technical Report 21,
Kyushu Institute of Technology, Japan, 1992.
[14] Y.Wang, G.Pan, Z.Wu, “Exploring Facial Expression Effects
in 3D Face Recognition using Partial ICP”, ACCV’06.
[15] V.Blanz, T.Vetter, “Morphable Model for the Synthesis of 3D
Faces”, SIGGRAPH’99, pp.187-194, 1999.
[16] T.Surazhsky, E.Magid, O.Soldea, et al, “A comparison of
Gaussian and mean curvatures estimation methods on triangular meshes”, IEEE ICRA’03, pp.1021-1026, 2003.
[17] K. F.Gauss, General Investigations of Curved Surfaces,
Raven Press, New York, 1965.
[18] P.Minovic, S.Ishikawa, K.Kato, “Symmetry Identification of
a 3D Object Represented by Octree”, IEEE PAMI, 15(5):507514, 1993.
0-7695-2521-0/06/$20.00 (c) 2006 IEEE