EM-Style Geo-Cuts Segmentation for MRI Brain Images
Jie Zhu, Ashish Raj, and Ramin Zabih

Abstract— Segmentation of MRI brain images has great
clinical and academic importance. The overlap of MR
intensities of different tissue types and the vast amount of thin
structures in brain images make segmentation of MRI brain
images difficult. In this paper, we present an EM-style geo-cutsbased segmentation method to over come these challenges. We
classify the brain images into three tissue types: white matter,
gray matter, and CSF. We iteratively classify the voxels and
calculate the intensity profile. We use region bias and automatic
seed setting combined with intensity profile induced
Riemannian metrics for the classification of voxels. We then use
this classification to re-estimate the intensity profile.
Experimentally, our method gives very good performance on
both synthetic images with ground truth segmentation and real
images with the segmentation of white matter and CSF
improved over the widely used EMS method.
use a Gaussian mixture to model the intensity probability
distributions for the different tissues. Our method iteratively
estimates the parameters for the Gaussian mixture and
classifies the voxels using geo-cuts with region bias,
automatically set seeds and intensity profile induced
Riemannian metrics.
The paper is organized as follows. We start by giving a
summary of Expectation-Maximization for segmentation and
state its limitations in section II. The geo-cuts method and
image segmentation using Riemannian metrics is described
in section III. In section IV, we present our MRI brain
segmentation method. We give our experimental results and
compare them with results using the popular EMS [1]
method in section V. In section VI, we give our conclusion
and discuss future work directions.
I. INTRODUCTION
S
of brain images obtained through
magnetic resonance imaging (MRI) into different tissue
types has great importance in the studies of many brain
disorders. One possible application is to characterize the
volumes of Alzheimer patients through the advancement of
the disease. The studies of these brain disorders usually
involve a vast amount of data and classifying white matter,
gray matter and cerebrospinal fluid (CSF) by human experts
is too time-consuming and is also likely to show inter and
intra-observer inconsistency. Automatic brain MRI
segmentation, however presents many challenges. The
overlap of MR intensities of different tissue classes creates
challenges for intensity-based algorithms. In [1], an
expectation-maximization (EM) [2] based method called
EMS is proposed. In addition to intensities, this method also
attempts to incorporate spatial constraints. The second issue
which makes automated MRI brain segmentation challenging
is the fact that the surface area of each of the tissue types is
extremely large due to large amount of thin structures. Most
image segmentation algorithms assume the contour of the
object being segmented to be relatively smooth, therefore,
tend to miss-classify these thin structures.
In this paper, we present an EM-style geo-cuts [3] based
segmentation method for MRI brain images. We look at
three tissue types: white matter, gray matter, and CSF and
EMGMENTATION
Jie Zhu is with the Department of Electrical and Computer Engineering
at Cornell University, Ithaca, NY 14850 USA (phone: 607-253-5535; email: [email protected]).
Ashish Raj is with the Department of Radiology, UCSF, San Francisco,
CA 94115 USA (e-mail: [email protected]).
Ramin Zabih is with the Computer Science Department, Cornell
University Ithaca, NY 14850 USA (e-mail: [email protected]).
II. IMAGE SEGMENTATION USING EM
A. Brain Image Segmentation as a Missing Data Problem
Expectation-Maximization [2] is an iterative method for
finding maximum likelihood in presence of missing data. In
EM, the likelihood function does not necessarily converge at
a local maximum although it is guaranteed to be nondecreasing.
With brain image segmentation, we can usually assume a
Gaussian Mixture Model (GMM) for the intensity profile
with one Gaussian characterizing the intensity distribution of
one of the tissue types. The missing data is then which
Gaussian produced a certain voxel, or the classification of
the voxel. The EM method in this case iterates between
calculating the model parameters using the current
classification of the voxels (Maximization step) and
classifying the voxels using the current GMM (Estimation
step).
B. EMS for Brain Image Segmentation
Because of the overlaps in intensities between white matter
and gray matter and gray matter and CSF in MR images,
using EM alone to segment MRI brain images is difficult. In
[1], an EM-based method called EMS is proposed. In
addition to intensities, this method also attempts to
incorporate spatial constraints.
EMS incorporates spatial constraints by assigning a
penalty for each tissue type pairs being neighbors. More
specifically, a penalty is assigned for white matter and gray
matter as neighbors, white matter and CSF as neighbors and
so forth. Intuitively, more likely neighboring pairs such as
white matter and gray matter would have a smaller penalty
than a less likely pair such as white matter and CSF. These
penalties are calculated using frequencies of the label
configurations from the previous iteration. One problem with
this set up is that it favors segmentations where classes are
spatially clustered and erases the fine structures in the brain.
To overcome this problem and preserve the edges between
white matter/gray matter and gray matter/CSF, EMS applies
a constraint where white matter/gray matter neighbors must
have the same penalty as white matter/white matter
neighbors and so on. The actual decisions now for white
matter/gray matter and gray matter/CSF are then purely
intensity based, leaving the spatial penalties to come into
play only for unlikely neighboring configurations. In other
words, no regularization is applied between white
matter/gray matter and gray matter/CSF.
Another challenge for EMS is that it tends to classify
partial volumes of white matter and gray matter as gray
matter. In most MRI brain images, gray matter has a much
larger variance than white matter. Since between white
matter and gray matter, EMS segments using intensities
alone, partial volumes are more likely to be classified as the
class with larger variance.
III. ENERGY MINIMIZATION USING GEO CUTS
A. Riemannian Metrics and Image Segmentation
Active contour models, such as the snake, have been
widely used in image segmentation. The most recent and
elegant approach is the “geodesic active contour” [4], which
minimize functional (1) through curve evolution.
|C |
E (C ) 
 g (| I (C (s)) |)ds
(1)
for anisotropic Riemannian metrics. The shortest curve
(minimal surface) in a Riemannian space then becomes the
minimum cost cut on this corresponding graph, the
approximation of which can be found using graph cuts.
In [3], the minimum cost contour (surface) that separates
pre-determined “object” and “background” seeds gives the
image segmentation.
Based on (2), Boykov defined a better Riemannian metric
for segmentation as in (3):
D( p)  g (| I ( p) |)I  (1  g (| I ( p) |))uu T (3)
where u is the unit vector in the direction of the image
gradient at pixel p. This new metric takes into account not
only the gradient magnitudes of the image but also the
directions of the image gradients. Here, the unit Riemannian
vectors at pixel p would not have small Euclidean lengths in
all directions when the gradient magnitude at p is large as
they would be in (2) but would only do in the directions
close to the image gradient at p.
B. Geo-cuts for Brain Image Segmentation
The Riemannian metric given in (3) is especially effective
in the case of brain image segmentation. Brain images have a
vast amount of fine structures, which are easily missclassified using conventional graph cuts set up.
For instance, in the case of the thin structure of white
matter in the circled area in Fig.1, conventional graph cuts
method with edge weights defined by intensity differences is
likely to eliminate this structure. Whereas the geo-cuts
method based on (3) tends to preserve it.
0
Here parameter s is the arc length on the contour,
| C | is
the Euclidean length of the contour, I represents intensity
and g is a strictly decreasing function with the range of 0 to
1. In this functional, the smoothing term in the original snake
approach is replaced by the idea that minimizing the
weighted length produces a smooth contour, since curvier
contours tend to be longer.
Curve evolution, however, only produces local optima. In
[3], Boykov proposed the geo-cuts algorithm that combines
the geodesic active contour idea and the graph cuts method
which finds global optima. The above energy functional from
the geodesic active contour is equivalent to the length of
curve C in a Riemannian space with isotropic Riemannian
metric
D( p)  g (| I ( p) |)I
(2)
where I is the identity matrix and D(p) is the Riemannian
metric at voxel p. Thus, minimizing (1) is equivalent to
finding the shortest curve (minimal surface) in the
Riemannian space defined by (2). The geo-cuts method takes
cue from this and segments images by finding the shortest
curve (minimal surface) in an image-induced Riemannian
space. Boykov showed in [3] how to build a grid graph and
set its edge weights so that the cost of cuts is arbitrarily close
to the length (area) of the corresponding contours (surfaces)
Fig.1. (a) Likely segmentation by conventional graph cuts. (b) Likely
segmentation by geo-cuts based on (3).
IV. OUR SEGMENTATION METHOD
A. Voxel Classification and Intensity Profile Estimation
We propose an EM-style geo-cuts-based segmentation
method that iteratively estimates the classification of voxels
using geo-cuts, finds the intensity profile and sets the seeds
for segmentation.
In our method, we assume a Gaussian Mixture Model
(GMM) for the intensity profile with one Gaussian
characterizing the intensity distribution of one of the tissue
types. We use the segmentation result from the previous
iteration to estimate the parameters for the intensity profile.
When segmenting the images, we use region bias and
seeds set automatically based on the intensity profile in place
of the pre-determined seeds in [3]. The Riemannian space we
use is induced by the intensity profile instead of pure
intensities as in [3].
B. Energy Minimization for Image Segmentation
We denote the parameters for our GMM as θ and the
observed image intensities for all voxels (P) in an array form
as I. What we want to find is the label configuration L for
the entire image. Say we have Ns number of seeds (S), we
use Is as the observed intensities of these seeds and Fs as the
label configuration assigned to the seeds. We also have a
brain atlas, which we will use A to represent.
An energy function is defined in (4) to guide the image
segmentation so that the best segmentation is given by the
label configuration that minimizes this function.
E  CR (θ, L)    log P(L, I | θ)
  T   ( Ls  Fs )
(4)
sS
  ( N S   log( P(FS | I S , θ) P(FS | A)))
where T is a very large constant and  () is a delta
function.•  ,  ,  are all constants representing coefficients.In
(4), the first term tries to minimize the Riemannian length
(area) of the segmentation boundary and the second term
tries to maximize the likelihood of the label configuration
and the observed image intensities given the model
parameters. The third term assigns large penalties for
violating the seed assignments whereas the fourth term looks
to increase the number of seeds while keeping the seeds
accurate according to the atlas and the intensity profile.
We have three sets of variables θ , L and {S, Fs} in the
energy function shown in (4). Our method iterates between
three steps: 1) finding L using the current θ and {S, Fs}, 2)
calculating θ using the current L and {S, Fs}, and 3) finding
the seeds and their labels given the current θ and L. In each
of the above three steps, we fix two sets of variables and
minimize the function w.r.t. the remaining one set. Overall,
the energy function should be constantly decreasing through
the iterations.
The geo-cuts method is used for step 1). Since we have
three labels (white matter, gray matter and CSF), we use the
   swap [5] for finding the segmentation. During this
step, the fourth term in (4) can be ignored since the seeds do
not change labels. The third term in (4) can be added as
terminal-links (t-links) in the graph. The second term in (4)
can also be added as t-links since it can be broken down into
a sum of functions of individual voxels:
where Lp and Ip are respectively the labeling and intensity
for voxel p. We define our Riemannian space using
normalized probabilities of labels given voxel intensities and
the model parameters. For instance, when segmenting
between white matter and gray matter, we use (6) in place of
the intensity I in (3). This configuration also helps with the
over-segmentation of gray matter suffered by EMS as shown
in subsection C.
The minimum cut on the graph is found using the “maxflow” algorithm [6].
In step 2), model parameters are calculated by finding
stationary points. We take the first derivative of the energy
function in (4) w.r.t. each of the model parameters, and
assign the value that makes this derivative zero to the
parameter.
In step 3), we only have to look at the fourth term in (4)
since the rest of the terms are fixed. This term can be
•broken down into a sum of functions of individual of seeds:
  U (s)
sS
   (1   log P( Fs | I s , θ)   log P( Fs | A))
(7)
sS
•Adding all voxels where U(s)>0 as seeds minimizes the
energy function in (4) while other variables are fixed.
C. Biased vs. Unbiased Miss-Classification
Brain images usually contain partial voxels that are made
up of tissues from both white matter and gray matter. Since
these voxels do not belong solely to either of one these tissue
types, what we want is for these voxels to be unbiasly
classified sometimes as one tissue type and sometimes as the
other. We achieve this by using a region bias that favors gray
matter and spatial constraints that favors white matter.
Our region bias is based on the joint probabilities of voxel
intensities and labelings as shown in (5). Since in most brain
images, gray matter tends to have a much larger variance
than white matter, this joint probability tends to be larger for
partial voxels when classified as gray matter.
Our spatial constraint is based on the normalized
probability shown in (6). Consider a 2D case where a partial
voxel has a gray matter neighbor and a white matter
neighbor. Since white matter has a much smaller variance
than gray matter, this normalized probability for the gray
matter neighbor tends to be very close to 0 while that for the
white matter one tends to be considerable smaller than 1.
Hence, the normalized probabilities of the partial voxel and
its white matter neighbor are closer producing a stronger nlink in the graph.
V. EXPERIMENTAL RESULTS
We experimented with synthetic images with ground truth
segmentation from the Simulated Brain Database and real
volumetric T1-weighted brain data (1x1x1mm3 resolution)
acquired using a Magnetization-Prepared RApid Gradient
Echo (MPRAGE) sequence with TI/TR=950/2300ms timing
and a small flip angle of around 8-10°sequence. Our
segmentation is done in 3 dimensions and is fully automated.
In EMS, we used a 4th order 3D polynomial to model the
bias field as in [1].
Table 1 shows the percentages of miss-classifications
using our method and the EMS method in the synthetic
images. We investigated three cases: no noise and no bias
field (bf), no noise and a strong bias field (bf=20) and noisy
image (noise=7) with no bias field. Our method gives
comparable performance with EMS with the classifications
of white matter and CSF improved.
All
WM
GM
CSF
classes
noise=0
bf=0
noise=0
bf=20
noise=7
bf=0
our method
3.5683%
0.16717% 7.00301% 1.40074%
%
EMS
8.2742%
21.0636% 0.40414% 4.17696%
our method
6.0719%
6.21011% 7.72402% 1.67149%
EMS
8.7639%
22.6044% 0.76464% 3.07330%
our method
11.9716% 9.94290% 16.2416% 5.06710%
EMS
9.5884%
[3] Y. Boykov and V. Kolmogorov, "Computing geodesics and minimal
surfaces via graph cuts," in Proceedings: Ninth IEEE International
Conference on Computer Vision, Oct 13-16 2003, 2003, pp. 26-33.
[4] N. Paragios and R. Deriche, "Geodesic active contours for supervised
texture segmentation," Proc. IEEE Comput. Soc. Conf. Comput. Vision
Pattern Recognit., vol. 2, pp. 422-427, 1999.
[5] Y. Boykov, O. Veksler and R. Zabih, "Fast approximate energy
minimization via graph cuts," IEEE Trans. Pattern Anal. Mach. Intell.,
vol. 23, pp. 1222-1239, 2001.
[6] Y. Boykov and V. Kolmogorov, "An experimental comparison of mincut/max-flow algorithms for energy minimization in vision," IEEE Trans.
Pattern Anal. Mach. Intell., vol. 26, pp. 1124-1137, 2004.
13.4606% 8.25075% 5.81081%
2D slices of our segmentation results and the EMS results
for the real images are shown in Fig.2. Compared to the
EMS results, our method does a better job capturing finer
structures of white matter and CSF and white matter voxels
with intensities similar to that of gray matter.
(a) original un-bias corrected MRI brain images
VI. CONCLUSION
In this paper, we present an EM-style geo-cuts [3] based
method that classifies MRI brain images into three tissue
types: white matter, gray matter, and CSF. Our method
iteratively estimates the parameters for the intensity profile
and classifies the voxels using geo-cuts with region bias,
automatically set seeds and intensity profile induced
Riemannian metrics. The geo-cuts method provides a good
framework for preserving thin structures of white matter and
CSF in the brain. Whereas, the intensity profile estimated
using EM provide the bases for region bias, seeds and
Riemannian metrics necessary for the geo-cuts segmentation.
We also attempted unbiased partial voxel segmentation in
our set up.
We tested our method on both synthetic brain images
with ground truth segmentation and real 3D brain MRIs.
Our method and EMS has comparable accuracy in the case
of synthetic images. In real images, compared to the
results produced by the popular EMS method, our method
does a better job with fine structures and white matter
voxels with intensities similar to that of gray matter.
(b) segmentation of (a) using EMS
References
[1] K. Van Leemput, F. Maes, D. Vandermeulen and P. Suetens,
"Automated model-based tissue classification of MR images of the brain,"
IEEE Trans. Med. Imaging, vol. 18, pp. 897-908, 1999.
[2] R. A. Redner and H. F. Walker, "MIXTURE DENSITIES, MAXIMUM
LIKELIHOOD AND THE EM ALGORITHM," SIAM Rev, vol. 26, pp.
195-239, 1984.
(c) segmentation of (a) using the proposed method
Fig.2. 2D slices of segmentation results using EMS and
using our proposed method for real MRI brain images