Modelling of Overlapping Circular Objects
Based on Level Set Approach
Eva Dejnozkova and Petr Dokladal
School of Mines of Paris, Centre of Mathematical Morphology
35, Rue Saint Honoré
77 300 Fontainebleau, FRANCE
[email protected]
Keywords: Shape analysis, computer vision, smoothing, segmentation, curvature, level set, part decomposition
Abstract. The paper focuses on the extraction and modelling of circular objects embedded up to any extent. The proposed method is inspired
from the continuous Level Set theory and consists of two stages. First,
by using the local curvature and the normal vector on the boundaries
are detected the shape parameters of the sought circular objects. Second, an area-based matching procedure detects such cases where several
identified circles correspond to only one, partially ocluded object.
1
Introduction
The detection of occluded objects and the shape fitting are basic computer vision
tasks. Many methods dealing with the detection and separation of overlapping
objects can be found in the litterature. The Hough transform (HT) [1] and
its extensions [2] represent a popular method for extracting analytic curves.
However, its implementation is often memory consuming.
Another group of methods is based on classical morphological tools. Meyer [3]
proposes a method based on a bisectrice function of the distance to the complement. However, this method can only be used to separate objects, embedded up
to a limited extent. A more recent algorithm by Talbot [4] computes the skeleton on the elliptical distance. This method is computationally expensive and the
author does not explain up to which extent this algorithm works.
The scale-space approach constitutes another separation technique. As an
example, one can cite Lindeberg [5] who focuses on the junction detection by
the normalized curvature in automatically selected scale-space. However, this
method can lead to poor junction localization and requires an additional correction of the localization. Zhang [6] proposes a direct part decomposition of
triangulated surfaces by thresholding the estimated gaussian curvature.
The second problem, shape fitting, usually deals with some optimization
method. A widely used approach for shape fitting is the minimization of the
squared error (with constraints or not) [7], [8]. The problem of these methods is
the robustness and its numerical stability. Another approach using a Bayesian
formulation is shown in Werman [9]. However, this approach results in optimizing
nontrivial cost functions for the fitted model.
This paper proposes a new method to detect and model embedded circular
objects by using the local curvature of the object borders and tries to show that
a local information can also be used for a global shape analysis. The parameter estimation algorithm is based on the continuous Level Set theory [10] which
introduces the sub-pixel precision and thus allows to improve the numerical accuracy of the parameter estimation. In addition, the computation cost is reduced
by the measuremements performed only on the narrow band of the objects contours. The second goal is to minimize the optimization effort of the shape fitting
task by using a clustering method on a set with a reduced number of elements.
The paper is organized as follows: the basic notions and principles are introduced first, followed by the algorithm description. Finally an application example
used for an analysis of microscope photographs of polymer crystals is presented.
2
Basic Notions
Below we use the following notations and definitions.
Let X be a discrete, binary object X: Z2 → {0, 1}, where points at 1 denote
the objets and 0 the background. Let C denote a closed curve placed in R2 such
that C is a continuous representation of the boundary of X. C is obtained from
X by some interpolation method, see e.g. Siddiqi [11] or Shu [12]. For an easy
manipulation, the curve C is defined implicitly by the signed distance function
u : Z2 → R such that
u(x) = d(x, C)
(1)
The distance u is assumed positive outside and negative inside the object X. C
is therefore the zero-level set of u.
To describe the curve C, it is useless to compute the distance u on the entire
image. In order to limit the computation cost, the distance u is calculated only
on the narrow band close to the curve. The narrow band is the set of points
NB = x x ∈ Z2 , |u(x)| < NBwidth
(2)
where NBwidth > 0. Outside NB, the values of u are limited by putting u equal
to ± NBwidth (exterior/interior).
Let X be some set, and P(X) the set of all subsets of X. The mapping
CCγ : X → P(X) denotes the set of connected components of X, with a
given neighbourhood connectivity γ (i.e. 4- or 8-neighbourhood). Note that a
γ-connected component of X is the equivalence class for γ-connected path of
points in X.
In the following, a circle is defined by a pair {c, r}, where c ∈ R2 stands for
the centre and r ∈ R+ for the radius.
2.1
Local Curvature
The curvature analysis is a basic tool in computer vision. Defined in a given
point as the inverse of the radius r of the osculating circle in this point, the
curvature κ of a curve is proportional to the angular speed of the normal vector
n0 travelling alongside the curve :
κ = div n0
(3)
An exhaustive discussion of the curvature representations can be found in Sapiro
[13]. We focus on the curvature computed on the implicit representation of the
curve. In terms of the curve implicit description by the distance function u, the
curvature κ is given by:
∇u
κ = div
(4)
|∇u|
In this paper the curvature κ is used for two objectives: 1) smoothing and 2)
radius estimation.
The smoothing makes use of the traditional discretization scheme which is
obtained by using central differences in Eq. (4).
κ=
uxx u2y − 2uxuy uxy + uyy u2x
(u2x + u2y )3/2
(5)
The main feature of this discretization scheme is that it allows to estimate the
curvature even in singular points, i.e. the points where ux = uy = 0, by solving the limit case. However this scheme is less robust for the radius estimation
because the surface deformations can influence the resulting level-set curvature.
The radius estimation requires a better accuracy of the curvature measurement.We propose to compute first the normal vector n0 with a more sophisticated scheme taking the mean of four normal vectors obtained by one-sided
differences as proposed by Sethian [10].
n00 = (n00x , n00y ) =
+
(u+
x , uy )
+ 2
2
(u+
x ) + (uy )
+
and after normalization :
1/2 +
+
(u−
x , uy )
+ 2
2
(u−
x ) + (uy )
n0 =
−
(u+
x , uy )
− 2
2
(u+
x ) + (uy )
1/2 +
n00
|n00 |
1/2 +
−
(u−
x , uy )
− 2
2
(u−
x ) + (uy )
1/2
(6)
The divergence operator Eq. (3) is only applied after having numerically obtained
n0 . In this case, the curvature estimation is based on the fine measurements of the
changes in the normal vector direction and the resulting curvature estimation is
more homogeneous. Nevertheless, this directional approach to the approximation
is to be used carefully. The divergence operator can only be applied directly on
such points where the normal vector n0 exists both in the point itself and in its
neighbourhood. Otherwise the obtained numerical value is incorrect.
3
Algorithm
Before the actual identification, the contours are decomposed in parts to separate the fused objects. The algorithm consists in several steps, described in this
section in the order they are applied.
3.1
Construction of the level-set function
The initial objects are defined as X: Z2 → {0, 1}. The boundary level-set function is the distance function u calculated by using Eq. (1). Next, the distance
u has to be smoothed to eliminate the discretization effect and the segmentation artefacts (cf. Fig. 2). Recall that u is calculated with a sub-pixel accuracy.
Unless one uses some higher-order interpolation to obtain the initial line C, the
iso-distance lines will have a staircase-like aspect, unusable for global shape estimation based on the local curvature.
(a)
(b)
Fig. 1. Iso-distance lines before (a), and after smoothing (b).
The level-set smoothing can be considered as a choice of the appropriated
scale-space used for the part decomposition. There exist many smoothing methods; see e.g. [14], [15] or [5]. We have adopted the smoothing governed by the
equation called geometric heat flow [15]:
∂u
= κ|∇u|
∂t
(7)
This choice has been made for the following reasons. It has been shown that any
closed curve will be deformed to become convex and will converge to a circular
form without developping self-intersections. The expected circular shape of the
contours is then naturally preserved. Eq. (7) applies the stronger smoothing
factor (see Fig. 1), the higher the local curvature κ is. The smoothing stops
when some norm of the difference of the successive iterations is smaller than
some arbitrary limit.
3.2
Boundary decomposition
The overlapping objects are separated by detecting the cusp points in the curve
C. After the segmentation of C in these points one obtains a set of smooth circular
arcs.
The segmentation is based on sign(κ) similarly to the methods proposed
by [6] and [16]. The contour is decomposed in convex parts (κ > 0) by splitting
it where κ changes sign and by dropping the concave parts. The concave parts
correspond to the cusp points before the smoothing.
Recall, that here the set of arcs is described implicitly by the set A of portions of the narrow band around the decomposed curve C. The set of arcs A =
{A1 , A2 , . . .}, A ⊂ Z2 , is obtained as : A0 = CCN4 {x | x ∈ NB and κ(x) > 0}.
The 4-connectivity N4 was used for this application. Optionally, the arc segments that are too short are filtered out: A = {Ai | Ai ∈ A0 , #Ai > const.},
where # stands for the cardinal of a set.
3.3
Detection of centres and radii of the circles
The centre coordinates of the circle approximating a given arc Ak ∈ A are
calculated in all points of Ak where the curvature is not biased by the border
effect (introduced by points close to the border of NB where the derivatives of
u use neighbors from outside the narrow band). Let xi ∈ Ak be any such point.
The circle approximating the iso-distance line in xi is given by: the radius ri =
1/κ(xi ) and the centre ci = xi +n0 (xi )/κ(xi ), ci ∈ R2 , where n0 (xi ) is the normal
vector (Eq. (6)) of the iso-distance line in xi , and κ(xi ) the curvature. Recall that
negative values of κ were filtered out (cf. section “Boundary decomposition”).
Let C(Ak ) = {ci } denote the set of the circle centre coordinates, obtained
for all xi ∈ Ak . The centre ĉk of the circle, approximating a given arc Ak , is
obtained by taking the median: ĉk = med(C(Ak )). Finally, the radius is r̂k =
med(de (ĉk , xi )), for all xi ∈ Ak , where de : R2 → R+ , denotes the euclidean
distance from a to b calculated de (a, b) = [(ax − bx )2 + (ay − by )2 ]1/2 .
Note that the computationaly expensive voting and following maxima searching processes of the standard HT are replaced by simple statistical measure.
3.4
Circles matching
Obviously, several arcs may form one circular object and have to be matched. We
need some arbitrary condition authorizing to couple circles even if their centres
and radii are not perfectly identical. This condition is a trade-off between the
accuracy and the capacity to model circles fused to an unlimited extent. We
use a classical clustering technique, proceeding iteratively by agglomerating the
most similar pairs.
The procedure uses the similarity matrix Ξ = {ξij } where ξij denotes the
similarity of circles i and j. The similarity criterion ξ of two circles is the area of
their intersection divided by the area of their union. The strictly positive values
of ξ are allowed only for the circles of similar radii and position (i.e. low de ).
ξij =
(
S(A)
S(Ai )+S(Aj )−S(A)
if de (ĉi , ĉj ) <
0
elsewhere
min{r̂i ,r̂j }
2
and i > j
(8)
(a)
(b)
Fig. 2. Original image: (a) A microscope photograph of polymer crystals, (b) Segmentation of the original image.
where S(Ai ) and S(Aj ) are the areas of the i-th and j-th circle and S(A) the
area of their intersection. The values ξij are calculated for i > j only to have a
triangular matrix under the main diagonal. The matching algorithm reads:
repeat while (maxi>j (ξij ) > 0)
[k, l] := arg maxi>j (ξij ) // get the most similar pair k,l of all i,j
Ak := Ak ∪ Al
// merge the circle l to k
ĉk := med(C(Ak ))
// recompute the centre and radius
r̂k := med(de (ĉk , xi )) for all xi ∈ Ak
delete Al , line and column l from Ξ and recompute Ξ
This algorithm is run on a reduced population set and is not computationally
expensive. It stops as soon as there are no more pairs of circles verifying ξij > 0.
4
Experiment Results
The motivation of the study presented above was an automatic crystal growth
analysis 1 . In the early stage of the growth, the crystals are circular. The goal is to
successively study crystal properties such as the size or the degree of incrustation
of artificial polymer crystals (Fig. 2).
4.1
Segmentation
Although the segmentation is not the objective of this paper, we briefly describe how the initial contours are obtained. Recall the basic operators from the
mathematical morphology. Let f denote an image and F the family of images.
Then εX , δX : F → F denote respectively the erosion and dilation by X. If no
structuring element is given then the unitary disk is used.
The segmentation strategy has been chosen according to the observations
made on the original image (Fig. 2). One can see that the image background
is almost flat. On the other hand, the range of intensity in the crystal interior
can be high. The gray levels on the crystal borders can reach both low and high
1
The microscope images of polymer crystals were used with kind permission of the
Centre for Material Forming (CEMEF), School of Mines of Paris, France
(a)
(b)
(c)
Fig. 3. (a) Arcs segmented by the boundary decomposition,(b) Circles detected from
the arcs. (c) Result of the circles matching superposed on the original gray-scale image.
values. In order to overcome this problem we start the segmentation procedure by
computing the morphological gradient which allows to extract the information
about the contrast changes alongside the borders. Let f denote the original
grey-scale image and fg = δ(f ) - ε(f ) the morphological gradient of f .
Since the next objective is to segment the image by a threshold we first have
to equalize the values in the crystal interior. For this purpose we use the hole filling operator HoleFill. HoleFill(fg ) fills the attraction basins of fg (corresponding
to the agglomerated crystals): f1 = HoleFill(fg ). Note, that the HoleFill operator can be applied either to binary or grey-scale images. More details can be
found in [17], for example.
Finally, the following thresholding extracts the gradient crests delimiting the
circular objects:
1, if f1 > T h
f2 =
0, otherwise
where T h > 0 (a convenient value for this application is T h = 15). In the next
stage a morphological closing filter will smooth noisy borders: f3 = εX (δX (f2 )),
where X is a disk of a four point radius. In order to eliminate the possible holes
inside these objects, a following hole-filling operator is applied to fill the interior
of the objetcs: f4 = HoleFill(f3 ). Finally, an area opening suppresses small, noisy
objects in the background: f5 = AreaOpenN (f4 ), with N = 300 (to eliminate
objects smaller than 300 points). f5 is the result of the segmentation (see Fig. 2
(b)).
The binary objects are then submitted to boundary decomposition to extract
the smooth arcs, see Fig. 3(a). For every extracted arc one osculating circle is
found (Fig. 3(b)). Finally, if several circles correspond to only one circular object
they are matched and replaced by only one circle. The parameters of the new
circle are accordingly adjusted. See the results of the matching (superposed to
the original image) at Fig. 3(c).
5
Conclusions
The paper shows the use of local curvature measure for a global shape analysis
and gives a specific application example, where the curvature is used to separate
circular objects fused theoretically up to any extent.
The proposed method uses a distance-based, implicit description of the contours for the estimation of the radii of circular objects. The measurements are
performed only on a narrow band around the contours. Obviously, the curvature
varies on different levels of the level set, even if measured in the normal direction.
Nevertheless, the osculatory circles tangent to points laying on the normal are
cocentric. Using larger narrow band gives birth to a more numerous population
of candidates, and consequently to an increase of the accuracy.
Concerning the radius estimation, it has been observed that higher curvature
offers better accuracy for the radius estimation. One improvement consists in
giving stronger weights to points belonging to level sets closer to the circle centre,
and having therefore a higher curvature. The second one consists in using an
asymmetric narrow band, larger towards the centre of the circles.
In addition, this paper presents a matching procedure based on the analysis
of the circle intersection area. The proposed matching condition allows to couple
circles representing one binary object and, at the same time, to separate crystals
embedded up to high degree of incrustation.
The computation complexity of this technique remains quite low. The only
computationally extensive step is the preprocessing where the distance function
is iteratively smoothed. The needs of smoothing could be limited by improving
the segmentation quality or by using some more sophisticated interpolation for
the construction of the distance function [11]. However, the proposed method
have a high degree of parallelism and can be efficiently implemented on a specific
parallel hardware [18] without any constraint.
References
1. Hough, P.V.C.: Methods and means for recognizing complex patterns. US Pattent
3069654 (1962)
2. Atiquzzaman, M.: Coarse-to-fine search technique to detect circles in images. International Journal of Advanced Manufacturing Technology 15 (1999) 96–102
3. Meyer, F.: Cytologie quantitative et morphologie mathématique. PhD thesis, Ecole
des Mines de Paris (1979)
4. Talbot, H., Appleton, B.: Elliptical distance transform and the object splitting
problem. In: International Symposium on Mathematical Morphology, Australia.
(2002)
5. Lindeberg, T.: Scale-Space Theory In Computer Vision. Kluwer Academic Publishers, Monograph 1994 (1994)
6. Zhang, Y., Paik, J., Koschan, A., Abidi, M.A.: A simple and efficient algorithm
for part decomposition of 3-d triangulated models based on curvature analysis.
In: International Conference on Image Processing ICIP02. Volume III., Rochester,
N.Y. USA (2002) 273–276
7. Fitzgibbon, A., Pilu, M., Fisher, R.: Direct least square fitting of ellipses. IEEE
Transactions on Pattern Analysis and Machine Intelligence 21 (1999) 476–480
8. Gander, W., Golub, G.H., Strebel, R.: Least-squares fitting of circles and ellipses.
In editorial board Bulletin Belgian Mathematical Society, ed.: Numerical analysis
(in honour of Jean Meinguet). (1996) 63–84
9. Werman, M., Keren, D.: A bayesian method for fitting parametric and nonparametric models to noisy data. IEEE Transactions on Pattern Analysis and Machine
Intelligence 23 (2001) 528–534
10. Sethian, J.: Level Set Methods. Cambridge University Press (1996)
11. Siddiqi, K., Kimia, B., Shu, C.W.: Geometric shock-capturing ENO schemes for
subpixel interpolation, computation and curve evolution. Graphical models and
image processing: GMIP 59 (1997) 278–301
12. Osher, S., Shu, C.W.: High-order Essentially Non-oscillatory schemes for HamiltonJacobi equations. SIAM Journal of Numerical Analysis 28 (1991) 907–922
13. Sapiro, G.: Geometric Partial Differential Equations and Image Analysis. Cambridge University Press (2000)
14. Leymarie, F., Levine, M.D.: Curvature morphology. Technical Report TR-CIM-891, Computer Vision and Robotics Laboratory, McGill University, Montreal, Quebec, Canada (1989)
15. Kimia, B., Siddiqi, K.: Geometric heat equation and nonlinear diffusion of shapes
and images. Computer Vision and Image Understanding: CVIU 64 (1996) 305–322
16. Siddiqi, K., Kimia, B.: Parts of visual form: Computational aspect. IEEE Transactions on Pattern Analysis and Machine Intelligence (1995)
17. Serra, J.: Image analysis and mathematical morphology. Academic Press, London
(1982) ENSMP - CMM Fontainebleau.
18. Dejnožková, E., Dokládal, P.: Asynchronous multi-core architecture for level set
methods. In: International Conference on Acoustics, Speech and Signal Processing,
Canada, IEEE (2004) Proceedings.