Variational Approach Based Image PreProcessing Techniques for Virtual
Colonoscopy
DONGQING CHEN, ALY A. FARAG
Computer Vision & Image Processing (CVIP) Laboratory
Department of Electrical & Computer Engineering
University of Louisville
Louisville, Kentucky, 40292
U.S.A
ROBERT L. FALK
Department of Medical Imaging
Jewish Hospital & St. Mary’s Healthcare
Louisville, Kentucky, 40202
U.S.A
GERALD W. DRYDEN
Division of Gastroenterology/Hepatology
Department of Medicine
University of Louisville
Louisville, Kentucky, 40202
U.S.A
Abstract
Colorectal cancer includes cancer of the colon, rectum, anus and appendix. Since it is largely
preventable, it is extremely important to detect and treat the colorectal cancer in the earliest stage.
Virtual colonoscopy is an emerging screening technique for colon cancer. One component of
virtual colonoscopy, image pre-processing, is important for colonic polyp detection/diagnosis,
feature extraction and classification. This chapter aims at an accurate and fast colon
segmentation algorithm and a general variational-approach based framework for image preprocessing techniques, which include 3D colon isosurface generation and 3D centerline
extraction for navigation. The proposed framework has been validated on 20 real CT
Colonography (CTC) datasets. The average segmentation accuracy has achieved 96.06%, and it
just takes about 5 minutes for a single CT scan of 512*512*440. All the 12 colonic polyps with
sizes of 6 mm and above in the 20 clinical CTC datasets are found by this work.
Keywords: Colorectal Cancer, Virtual Colonoscopy, Variational approach, Adaptive
Level Sets, Image Segmentation, Skeleton Extraction
1
1. INTRODUCTION
Colorectal cancer includes cancer of the colon, rectum, anus, and appendix. Colorectal cancer can develop
from the mucosa located throughout the colon or rectum. Most colorectal cancers progress through a
series of mutations which confer a growth advantage, leading to polyp formation and eventually an
invasive tumor.
In most cases, colorectal cancers develop slowly and normally take a period of several years to grow
from the earliest lesion to an advanced cancer. Adenocarcinomas account for about 95 percent of
colorectal cancers, arising from the intestinal epithelial cells that line the colon and rectum. It is the
second leading cause of cancer-related death and the third most common form of cancer in the United
States (Abbruzzese, 2004).
Colorectal cancer is largely preventable. Several screening tests, including the digital rectal exam, fecal
occult blood test (via guaiac or fecal immunochemical test), flexible sigmoidoscopy, double-contrast
barium enema, and colonoscopy are recommended for all people age 50 and over. Currently, optical
colonoscopy is considered the gold standard for colorectal cancer screening. During optical colonoscopy,
a thin flexible video endoscope is inserted into the patient's rectum and advanced to the cecum.
Inspection of the colon for polyps generally occurs during the withdrawal phase of the colonoscopy.
Although a colonoscopy can detect more than 90% of colorectal cancers, it is invasive, sometimes
uncomfortable, the preparation is inconvenient, and inability to the cecum results in an incomplete exam
(Macari, 1999).
On the other hand, computed tomographic colonography (CTC), also known as virtual colonoscopy
(VC), is a computer-based alternative to optical colonoscopy. It has evolved rapidly over the past decade
due to advances in manufacturing of high-resolution helical spiral computed tomography scanners.
Ferrucci (2001) comprehensively analyzed the promise, polyp detection and politics of colon cancer
screening with VC. The accuracy (mainly the sensitivity and specificity of polyp detection) is comparable
to the conventional colonoscopy for the significant size greater than 10 mm with few false-positive.
VC has many advantages, including relative lack of invasiveness, lower incidence of complications and
side effects, patient tolerability and preference (Juchems, 2005). VC is not intended to replace traditional
optical colonoscopy (Summers, 2002; Zalis, 2005; Chen, 2008), but rather to complement it by providing
an additional mechanism for providing CRC screening. Benefits of VC include visualization of
neighboring structures outside the colon, visualization of difficult anatomical locations (i.e. behind
flexures), the ability to bypass high grade stenoses, and providing an alternative to colonoscopy in those
patients who either refuse optical colonoscopy or cannot tolerate it due to severe illness.
The process for computer aided diagnosis (CAD) assistance for colorectal cancer screening involves: 1)
colon segmentation, 2) colon isosurface generation and rendering, 3) 3D centerline extraction for
navigation, 4) colonic polyp detection, 5) color coding for polyp candidate visualization, 6) features
extraction, and 7) benign/ malignant polyp classification.
Accurate and reliable colon segmentation are important, since any incorrect segmentation, for example,
missing colonic segments, containing non-colon tissue (e.g. small bowel) or reconstructing colon wall of
poor quality, impairs interpretation of 3D visualization. Moreover, inaccurate 3D colon segmentation and
visualization diminish the perception of polyp detection, classification and the whole performance of
CAD system. The 3D colon segmentation normally suffers from the following difficulties. First, CTC
sometimes contains disconnected regions of the colon because collapsed segments result from different
2
reasons such as colon spasm, insufficient distension, etc. Second, in clinical practice, oral contrast agent
may be given to patients for CTC, however, oral contrast in colon can cause leakage into small bowel.
Third, opacified liquid is generated by ingesting iodine and barium of the oral contrast, and then some
polyps may submerge under the liquid, which causes hard colonic polyp detection. Fourth, the
segmentation accuracy and speed are greatly affected by the complicated colon structure and topology.
The literature has introduced several sophisticated image segmentation procedures. Most are based on two
concepts: thresholding and connectivity (Franazek, 2006). Tagged CT colonography uses thresholding
for digital stool subtraction. Opacified liquid is generated by ingesting iodine and barium. When contrast
enters the colon and tags residual fluid, electronic cleaning can occur. Electronic cleaning (Zalis, 2004;
Chen, 2001) served as a very useful technique to balance CT values in fluid-filled lumen parts and others
filled with air, and to remove the contrast fluid. However, some artifacts were generated which affected
image interpretation. Yoshida and Nappi (2001) designed a CAD framework to detect and classify the
colonic polyps by using shape index and curvedness. The first stage of their CAD system involved colon
segmentation, consisting of two major steps: 1) anatomy based extraction and 2) colon based analysis.
Recently, Franazek et al. (2006) proposed an entire framework for hybrid segmentation of colon tissue in
CT colonoscopy. It achieved good accuracy for the segmentation results; however it consisted of eight
different steps, so the average processing time was very long. For a single CT scan 512*512*400, it took
18 minutes on 1.8 GHz PC (even without I/O operations).
This chapter aims at an accurate and fast colon segmentation algorithm and a general variational-approach
based framework for image pre-processing techniques, which include 3D colon isosurface generation and
3D centerline extraction for navigation. A statistical model of regions is explicitly embedded into partial
differential equations, which describes the evolution of the level sets. The probability density function for
each region (bi-model in this work, colon tissue and non-colon tissue) is modeled by a Gaussian function
with adaptive parameters. These parameters include the mean values and variances, which are estimated
by the maximum likelihood estimation (MLE) method. The prior probabilities of two regions are
automatically re-estimated during each iteration. Finally, the pixel candidate is classified into colon tissue
or non-colon tissue by using Bayesian decision theory (Duda, 2001). The designed level set model
depends on these density functions. The region information over the image is also taken into account.
The proposed framework has been validated on 20 real CTC datasets. The average segmentation accuracy
has achieved 96.06%, and it just takes about 5 minutes for a single CT scan of 512*512*440. The rest of
the chapter is organized in the following manner: Section 2 presents the mathematical and theoretical
background of curvature formulation, curvature based curve evolution, and variational calculus, Section 3
introduces variational methods for colon tissue segmentation, Section 4 discusses colon isosurface
generation and 3D colon centerline extraction for navigation. The proposed framework has been validated
in Section 5. Two future directions are introduced in Section 6. Section 7 concludes the book chapter.
2. MATHEMATICAL BACKGROUND
Three Dimension Differential Geometry (Manfredo, 1976)
Let p(x, y, z) denote a voxel on a 3D surface S, and (u, v) is an orthogonal basis on the plane tangent to S
at p shown in Figure 1.
An isosurface S with intensity level I in a 3D space R3 is given as S  { p( x, y, z )  R 3 | h( p)  I }
3
Figure 1. 3D surface patch parameterized by (u , v)  [0,1]  [0,1]
In a small neighborhood around each point p, z can be expressed as a function of (x, y), say r(x, y). Then,
the iso-surface S can be represented by denoting (x, y) as (u, v).
p(u, v)  {(u, v)  R 2 | h(u, v, r (u, v))  I }
Now, consider the distance ds between p: r(u, v) and q: r(u + du, v + dv) inside the neighborhood of p :
r(u, v), and it can be expressed as




 
 
 
(ds) 2  dr  dr  (ru du  rv dv)  (ru du  rv dv)  ru  ru (du) 2  2ru  rv dudv  rv  rv (dv) 2 (1)
If Equation (1) is rewritten as
(ds ) 2  Eu (du ) 2  Fdudv  G (dv) 2




(2)

Let us define ru , rv the first order partial derivatives, ruu , ruv , rvv the second order partial derivatives of




S at point p, the first fundamental forms E, F and G can be expressed as: E  ru  ru , F  ru  rv and
 
G  rv  rv , where  is the dot product.
The first fundamental forms denote the coefficients of the differential quadratic norm that gives
the length of a differential arc on the surface.
If we continue to consider the perpendicular distance ± between q: r (u + du, v +dv) and the
tangential plane to the 3D surface S at p: r(u, v).



  (r (u  du, v  dv)  r (u, v))  n
(3)

where, n is the normal vector of surface S at p.

Taking the Taylor expansion to r (u  du, v  dv) , then








1 
r (u  du, v  dv)  r (u, v)  ru (u, v)du  rv (u, v)dv  ruu (du ) 2  2ruv dudv  rvv (dv) 2   (du, dv)
2
4

Substituting r (u  du, v  dv) in Equation (3), we can get





1 
  (ru (u, v)du  rv (u, v)dv  ruu (du ) 2  2ruv dudv  rvv (dv) 2    (du, dv))  n
2
(4)
 
Since (ru , rv ) is the tangential vector of surface S, and if the residual  (du , dv ) is ignored, Equation
(4) is represented by






1 
ruu (du ) 2  2ruv dudv  rvv (dv) 2  n
2
Similarly, the equation above can take the form
1
  L(du ) 2  2Mdudv  N (dv) 2 
2
 
 
The second fundamental forms L, M, and N are expressed as: L  ruu  n , M  ruv  n , and
 
N  rvv  n .
(5)
(6)
The second fundamental forms give the coefficients of a local tangent quadratic approximation
to the surface.
Curvature Formulation
Given the first fundamental forms E, F and G, and the second fundamental forms L, M and N of surface S,
the maximum principal curvature 1 and the minimum principal curvature  2 are defined as the two
roots 1 and  2 , which satisfy the following identity:
E  L
F  M
F  M
G  N
0
(7)
As a result, Gaussian curvature K and mean curvature H can be defined as:
LN  M 2
K   1 2  12 
EG  F 2
H
1
1
EN  2 FM  GL
( 1   2 )  (1  2 ) 
2
2
2( EG  F 2 )
(8)
(9)
Then the two principal curvatures can be computed as:
1  H  (H 2  K )
 2  H  (H 2  K )
(10)
(11)
5
The principal curvatures 1 and  2 represent the inverses of the maximum and minimum radii
of curvature of all the curves passing through a given point of the surface. The mean curvature is
just the average of 1 and  2 . The Gaussian curvature is the product of 1 and  2 which is thus
independent of the orientation of the surface.
In general, two approaches are used to compute the curvatures for isosurfaces from a volume
dataset. The first category works directly on gray value information by exploiting the local
differential structure of the image, while the second approach estimates curvatures from
isosurfaces created by the Marching Cube algorithm (Lorensen, 1987).
The geometry of the deformation (evolution) of C(X, t) is only dependent of the normal component of the
velocity field, where C(X, t) denote a family of 2D closed curves, where t parameterizes the family and X
= (x, y) parameterizes the curve.
Following Lemma 1, the general geometric deformation for curve C is given by (Sapiro, 2001; Chen,
2008).

C ( X , t )
  ( X , t)N ( X , t)
t
(12)
where,  ( X , t ) is the geometric velocity on the direction of the 2D normal.
A well-known problem with the parametric representation of curves is that during evolution the points on
the curve bunch close together at certain regions and they space out elsewhere. It increases error in
numerical approximation of curves measures like tangent and curvature. This may lead to formation of
discontinuities even explosion of in the curve.
Variational Calculus and Euler-Lagrange Formula
Variational calculus is a field of mathematics that deals with functionals, as opposed to ordinary
calculus which deals with functions. Such functionals can be formed as integrals involving an unknown
function and its derivatives. The functional could attain a maximum or minimum value. The Gâteaux
differential is often used to formalize the functional derivative, which is commonly adopted in the
calculus of variations. Unlike other forms of derivatives, the Gâteaux differential of a function may be
nonlinear. If the Gâteaux differential is linear and continuous, then the resulting linear operator is called
the Gâteaux derivative (wiki web, 2008).
Given a one dimensional function u ( x) : [0,1]  R , the basic problem is to minimize a given energy E,
which satisfies the following equation:
1
E (u( x))   F (u, u )dx
0
with the given boundary conditions u(0) = a and u(1) = b, and F : R 2  R .
The solution to the above equation is the first order Euler-Lagrange formula (Sapiro, 2001).
6
F d  F 
 
0
u dx  u  
(13)
For the 2D problem, give a 2D function u(x, y) and an energy E, where
2
2


E (u ( x, y ))   F  u, u u  u 2  u 2 dxdy

x
,

y
,

x
,

y


The Euler equation is written as (Sapiro, 2001)
F d  F
 
u dx  u x
 d  F
  

 dy  u y
 d2

 dx 2

 F

 u xx
 d2
  2
 dy
 F

 u
 yy

0


(14)
3. VARATIONAL METHOD BASED IMAGE SEGMENTATION
Traditional Active Contour Method for Image Segmentation
Variational method based deformable modeling is a breakthrough of image processing, especially in
image segmentation. A famous application of curve or surface evolution is the deformable model, which
is supported by the explicit deformable models (e.g. snake (Kass, 1987), balloon force model (Cohen,
1991) and gradient vector flow (GVF) model (Xu, 1998)) and the implicit deformable models (e.g.
geodesic active contour (Caselles, 1997), level sets (Osher & Sethian, 1988), and prior shapes based
model (Abd El Munim, 2007). This section will briefly go over the snakes and GVF deformable models.
The governing equations of deformable models are dependent on Lagrangian continuum mechanics
formulations, in a volume dedicated to the Eulerian formulations, which are associated to level set
methods introduced in Section 3.2. The deformable models mimic different generic behaviors of natural
non-rigid materials in response to forces, e.g. continuity, smoothness, and elasticity (Osher, 2003).
Snake
Snake (Kass, 1987) as a deformable model, is defined as a parametric contour embedded in the image
plane (x, y). The contour is required to cover the desired object in the image I(x, y) and minimize a certain
energy function defined as follows:
E (C )  Eint ernal (C )  Eexternal (C )
(15)
where, C is the 2D contour (closed curve).
The first term is defined as the internal energy which characterizes the deformation of the contour:
1


1
Eint ernal (C )   1 ( p) | C p | 2  2 ( p) | C pp | 2 dp
20
(16)
where, Cp and Cpp are the first and second order partial derivatives of closed curve C with
parameterization point p, and p  [0,1] .
7
The non negative values ω1 and ω2 have the important rule in controlling the stretching and bending of
the contour. The length of the contour is reduced by increasing ω1 which removes the ripples and loops.
Big value of ω2 increases the bending and makes the contour smoother.
The external energy represented by the second term in Equation 17. It is derived from the input image I
(x, y), where the typical external energy functions are listed as follows:
Eexternal ( x, y )   I ( x, y )
2
(17)
and, ▽ is the gradient operator. Or
Eexternal ( x, y )   ( I ( x, y ) * G ( x, y ))
2
(18)
where, G(x, y) is a 2D Gaussian function with the standard deviation σ.
Using the variational calculus, the contour evolves in term of the following vector-based PDE.
Ct 
1 



(t1 | C p | 2 )   2 (t 2 | C pp | 2 )  E external

2  P
P

(19)
where, t1 and t2 are two constant, and 0< t1, t2<1, t1+t2=1. Ct is the partial derivative of C with respective to
t, governing the contour evolution, and  is the gradient operator.
Gradient Vector Flow
GVF is a bi-directional external force field that moves active contours in highly concave regions (Xu,
1998). The GVF is the vector field V(X)= [u(X), v(X)]T, which minimizes the following energy function,
E (V )    (| u ( X ) | 2  | v( X ) | 2 ) | f ( X ) | 2 | V ( X )  f ( X ) | 2 dX
(20)
where X=(x, y),  is a regularization parameter, and f(X) is an edge map derived from the image I(X).
For a binary volume, I(X) = -f(X).
The interpretation of Equation 20 is that if | f (x) | is small, E (V ) is dominated by the sum of squares
of the partial derivatives of the vector field, yielding a slowly varying field. On the other hand, if
| f (x) | is large, E (V ) becomes dominated by the second term, and can be minimized when
V (x)  f (x) . This produces a vector field V(X) that is nearly equal to the gradient of the edge map
f (x) when it is large and slowly varying in homogeneous regions. V(X) can be computed iteratively by
solving the following decoupled PDE in u and v (Xu, 1998).
u ( X , t  1)  u ( X , t )  t (  2 u ( X , t )  (u ( X )  f x ( X ))) | f ( X ) | 2
(21)
v( X , t  1)  v( X , t )  t ( 2 v( X , t )  (v( X )  f y ( X ))) | f ( X ) |2
(22)
8
To set up the iterative solution, let indices i, j and n correspond to x, y, and t, respectively, the numerical
solutions to Equations 21 and 22 are given as follows:
uin, j 1  (1  bi , j t )uin, j  r (uin1, j  uin, j 1  uin1, j  uin, j 1  4uin, j )  ci1, j t
(23)
vin, j 1  (1  bi , j t )vin, j  r (vin1, j  vin, j 1  vin1, j  vin, j 1  4vin, j )  ci2, j t
(24)
where, b( x, y)  f x ( x, y) 2  f y ( x, y) 2 , c1 ( x, y )  b( x, y ) f x ( x, y ) , c 2 ( x, y)  b( x, y) f y ( x, y) ,
and r 
t
xy
.
The iterative process is guaranteed to converge when the Courant–Friedrichs–Lewy (CFL) stepsize r  1/ 4 , given that b, c1, and c2 are bounded. So, the time-step t on 2D case is:
t 
xy
4
(25)
where x and  y are the data spacing of a given dataset.
Figure 2 shows one result of real colonic polyp segmentation using gradient vector flow active contour.
The initial contour, the contour after five iterations, the contour after ten iterations and the final result are
shown in Figure 2 (a), (b), (c) and (d). The evolving speed is fast, and it averagely takes 15 iterations to
obtain the results on a computer with 4GB RAM and Intel Pentium 4 CPUs at 2.6 GHz.
Problems of Classical Deformable Models
The deformable models efficiently apply to many applications. However they have some disadvantages:
1). It is almost impossible to handle the topology changes, for example, merging and splitting;
2). It is difficult to avoid self-intersection of the parametric curve or surface;
3). It is impractical to choose a very small value of time interval; and
4). Initialization needs to be near the steady state solution which represents a big problem in 3D.
Figure 2(e) to 2(h) shows two examples of self-intersection of the traditional active contour in (e) and (g),
respectively, when GVF is used to segment the two colonic polyps. The final segmentation results are
given in (f) and (h), respectively. From the results, the self-intersections are indicated by the red contours.
Level Sets Method
Level sets representation was proposed (Osher and Sethian 1988; Sethian 1989) to overcome the
problems of the classical deformable models. As mentioned before, the topology change problem, for
example, merging or splitting, is almost impossible to be handled by the conventional explicit deformable
models. However, it is done naturally by implicit level sets, and the demo is as shown in Figure 3. The
surface function evolves with time and then the evolution front is always represented as the zero level, the
following equation can be written as a general description:
( X , t )  0
(26)
9
Taking derivative on both sides of the above equation, it leads to:
(a)
(c)
(b)
(d)
(e)
(f)
(g)
(h)
Figure 2. GVF based active contour for colonic polyp segmentation. (a) initial contour, (b) after
5 iterations, (c) after 10 iterations, and (d) final result. Two colonic polyps in (e) and (g), while
the segmentation results using GVF active model are shown in (f) and (h).
10
 ( X , t )
0
t
In terms of chain rule, the above equation can be written as follows.
( X , t ) ( X , t ) X


0
t
X
t
 t  ( X , t )  V  0
where,  and V represent the gradient of © and the velocity field, respectively.
The velocity vector which can be set in terms of the tangent and normal vectors as


V  VT T  VN N resulting in:


 t  ( X , t )  (VT T  VN N )  0

Then substituting for  |  | N gives the following:
 t  | ( X , t ) | VN  0
Letting VN  F , the basic level sets function is given as follows.
 t  F | ( X , t ) | 0
(27)
Equation 27 can be interpreted as follows.
If  t is written as  t 
( X , t ) ( X , t  t )  ( X , t )

t
t
then it follows
( X , t  t )  ( X , t )
 F | ( X , t ) | 0
t
Multiplying t on both sides, we can get
( X , t  t )  ( X , t )  t  F | ( X , t ) | 0
By switching the second and third terms to the right hand side, the above equation becomes as follows.
( X , t  t )  ( X , t )  F  t | ( X , t ) |
(28)
11
(a)
(b)
(c)
Figure 3. Topology changes, for example, merging and splitting, are dealt naturally with the
implicit representation. This figure shows two curves during the merge procedure: the initial (a),
intermediate (b) and final (c) positions. 3D models are represented on the first row and 2D
curves on the second row.
If F  0 , the original curve shrinks, while it expands when F  0 . And the curve remains unchanged,
if F  0 . Normally, F  1   , where  is the curvature. ε is the parameter controlling the bending of
the curve. If the curve is very sharp (high degree bending), ε normally takes a small value, for example,
0.01, 0.001, etc. Otherwise, ε needs to be assigned a relatively big value to the curve with low degree
bending in order to keep the smooth evolution.
The numerical algorithm implementation for Level Sets is summarized as follows.
1). Initialize using signed distance function (SDF)
(1) Detect the edge points of the original figure using Canny detector (Canny, 1986);
(2) Compute the Euclidean distances between a given point and all the edge points, to find the
minimum distance dmin, if dmin > 0, the given point is located inside the region, otherwise
(3) Belong to the outside region.
2). Control the curve or surface evolution using Equation 26;
3). Find and mark all the points with Ф > 0;
4). Mark all the contour points by applying Canny detection;
5). During each iteration, re-initialization; and go back to Step 2, until iterations stops.
Adaptive Level Sets Technique
A segmented image consists of homogeneous regions characterized by statistical properties. For the bimodel system, we use Gaussian distributions for the colon (foreground) and other tissue (background).
12
If we only know the general knowledge of the bi-model system, maximum-likelihood estimation (MLE) is
classical in statistics to estimate the unknown probabilities and probability densities (Duda, 2001).
For the Gaussian distribution, the unknown mean μ and variance σ are estimated by MLE as:


1 n
1 n


x k and  2   ( x k   ) 2

n k 1
n k 1
where, x is the sample, and n is the number of samples.
For each class i (i=1, 2) in this work, in accord to the estimation method in (Vese & Chan, 2002), the
mean  i , variance  i , and the priori probability  i are updated during each iteration as follows.
i 
 H  ( ) I ( x)dx
 H  ( )dx
i
(29)
i

2
i
 H  ( )(   I ( x))

 H  ( )dx
i
i
2
dx
(30)
i
i 
 H  ( )dx
  H  ( )dx
i
2
(31)
i
i 1
where, H  () is the Heaviside step function as a smoothed differentiable version of the unit step
function. I(x) is the input image.
The Bayesian decision theory (Duda, 2001) is a fundamental statistical approach to quantify the
tradeoffs between classification decision using probability and the cost.
We let ω denote the state of nature in this work, with ω= ω1 for colon tissue and ω= ω2 for non-colon
tissue. Since it is unpredictable, ω is considered to be a variable, which must be probabilistically
described. We assume the priori probability π1 that next pixel candidate is colon, and some prior
2
probability π2 which is non-colon tissue.

i 1
i
1
In seems logical to use the decision rule: decide ω1 if π1 >π2; Otherwise, decide ω2.
Finally, the classification decision at pixel x is based on the Bayesian criteria as follows.
i * ( x)  arg( max ( i pi ( I ( x))))
i 1, 2
(32)
For further details, please refer to our work in (Abd El Munim 2004; Chen, 2008).
An example of simple level sets function is the signed distance function (SDF) to the curve. However, its
initialization needs not to be close to the desired solution, and one level sets function could only represent
two phases or segments in the image. Compared with the simple level sets function, the adaptive level-
13
sets function represents boundaries with more complicated topologies, for example, triple junction. In the
piecewise constant case, it only needs log2(n) level sets functions for n phases. Moreover, under the
piecewise smooth case, only two level sets functions are sufficient to represent any partition. In this work,
only one level-sets function has been used for colon tissue segmentation. This function was successfully
applied for multi-modal (3 or more tissues) images, which could be found in (Abd El Munim, 2004).
COLON ISOSURFACE GENERATION & VARATIONAL SKELEON EXTRACTION
3D Colon Isosurface Generation
Due to the existence of oral contrast, which generates opacified liquid, then three layers: air-liquid, airmucosa (colon inner wall), and liquid-mucosa, after colon segmentation by using adaptive level sets
method, sometimes there are very few liquid dots inside the colon tissue, we apply 3D median filter to
remove these tiny sparkles and 3D Gaussian filter for smoothness.
The 3D colon object is reconstructed by searching its isosurface using Marching Cubes, and it is
visualized by surface rendering. The Marching Cubes algorithm (Lorensen, 1987) was proposed to create
a constant density surface from a 3D array of data, especially the medical datasets. The main idea of the
Marching Cubes is summarized as follows: 1) creation of a triangular mesh which will approximate the
isosurface, and 2) calculation of the normals to the surface at each vertex of the triangle.
The Marching Cubes algorithm implementation is summarized as follows.
1). Read four slices into memory;
2). Create a cube from four neighbors on one slice and four neighbors on the next slice;
3). Calculate an index for the cube;
4). Look up the lists of edges from a pre-created table;
5). Find the surface intersection visa linear interpolation;
6). Calculate a unit normal at each cube vertex and interpolate a normal to each triangle vertex;
7). Output the triangle vertices and vertex normals.
This procedure is accomplished by Visualization ToolKit (VTK) 5.0 under Windows XP. The VTK is an
open source and freely available software system for 3D computer graphics and medical image
visualization (http://www.vtk.org).
3D Colon Centerline Extraction
The 3D centerline, also known as curve skeleton CS, is considered as the optimal flying path for
navigation inside the colon lumen. All the 3D centerlines in this work are generated by using the previous
work in (Hassouna 2007, 2008) based on the gradient vector flow (GVF) (Xu, 1998) algorithm.
This book chapter assumes that there only exists a single 3D centerline/flying path, which connects
between the cecum and rectum inside the colonic lumen.
Consider the minimum-cost path problem that finds the path C ( s) : [0, )  R n that minimizes the
cumulative travel cost from a starting point A to some destination X. If the cost U is only a function of the
14
location X in the image domain, the cost function is called isotropic, and the minimum cumulative cost at
X is defined as
T ( X )  min

X
A
U (C (s))ds
(33)
The path that gives the minimum integral is the minimum cost path. The solution of Equation 33
satisfies the solution of a nonlinear partial differential equation known as the Eikonal equation 34,
where F ( X )  1 / U ( X ) , and T(X) is the time at which the front crosses X.
| T ( X ) | F ( X )  1.0
(34)
Let A and B be medial voxels. Assume that A is a point source Ps that transmits a high speed front as
given by Equation 9, where  ( X ) is a medialness function that distinguishes medial voxels from others
and α controls the curvature of the front at medial voxels.
F ( X )  e ( X )
(35)
Because x is intrinsic to the image, only X can not change the speed of the front of the propagated wave
from the point source Ps. By multiplying by α, the speed varies and hence the curvature. Since the GVF
does not form medial surfaces in 3D, we propose the following medialness function as given by Equation
36, where the magnitude of the GVF goes to zero at medial voxels.
(36)
 ( X )  max(| V ( X ) | q ) | V ( X ) | q
where 0<q<1.
The propagating front is monotonically increasing with time; e.g., there is only one global minimum over
the cumulative travel time T, which is Ps, which has zero travel time. Then, the path between B and A can
be found by backtracking from B along the gradient of T until A is reached. The extraction process can be
described by the following ordinary differential equation 37, where C(t) traces out the CS and is found by
solving Equation 37 using Runge-Kutta of order 2.
dC
T ( X )

dt
| T ( X ) |
(37)
The proposed curve skeleton extraction framework can be summarized as follows:
1) Construct the distance transform D(x);
2) Compute the point source Ps and Identify extreme nodes;
3) Construct the new GVF based medial function;
4) Propagate an α-front from Ps and solve for a new distance field D(x);
5) Extract those curve skeletons that originate from extreme nodes and end at Ps.
VALIDATION, RESULT AND DISCUSSION
This book chapter validates the proposed framework on 20 CTC datasets. One has been provided by the
3DR Inc., Louisville, KY, and the remaining 19 CTC data were received from the Virtual Colonoscopy
Center at the Walter Reed Army Medical Center, Washington, DC, U.S.A. The patients underwent
standard 24-hour colonic preparation by oral administration of 90 ml of sodium phosphate and 10 mg of
15
bisacodyl; then consumed 500 ml of barium (2.1 percent by weight) for solid-stool tagging and 120 ml of
Gastrogra_n to opacify luminal fluid (Pickhardt, 2003). A four-channel or eight-channel CT scanner was
either GE LightSpeed or LightSpeed Ultra. The CT protocol included 1.25 mm to 2.5 mm collimation, 15
mm/second table speed, and 100 mAs and 120 kVp scanner settings. Each dataset contains 400~500
slices. The spatial resolution for each dataset is 1.0  1.0  1.0mm3 .
All the experiments have been carried out on a computer with 4GB RAM and Intel Pentium 4
CPUs at 2.6 GHz. The software programs are developed by using Visual C++ under Microsoft
Windows XP, and Visualization Toolkit (VTK) 5.0 as Visualization libraries.
(a)
(b)
(c)
(e)
(f)
(d)
(g)
Figure 4. Segmentation results of colon in tagged CTC data: (a) A CTC slice containing
opacified liquid-filled part and air-filled part, (b) original CTC slice with oral contrast agent, (c)
16
manual seed initialization inside colon after thresholding, intermediate results after (d) 10, (e)
20, (f) 30 iterations, and (g) final result.
Colon Tissue Segmentation
An original CTC slice as shown in Figure 4(a) typically consists of colon lumen including opacified
liquid-filled part and air-filled part, small intestine, and some other tissues, for example: bones having the
similar image intensity with liquid part.
Firstly, the opacified liquid is removed by using a simple thresholding method to equalize image
intensity inside the colon as shown in Figure 4(c). During this process, most opacified liquid is roughly
removed, and iso-dense tissues such as boney structures on the same slice are removed as well. However,
threshold subtraction does not affect colon segmentation, since we manually implant the initial seeds
totally inside the colon region, which guarantees the segmentation results.
Results following 10, 20 and 30 iterations of level sets evolutions after seed implantation are shown in
Figure 7 (c), (d), (e), and (f), respectively. The final result of zero level sets convergence to the lumen-air
boundaries is shown in (g). During the iterations, every candidate pixel is automatically classified as
either colon associated or background based on its probability density function (PDF) by using Bayesian
decision criteria. After each iteration, the average values and variances for both foreground and
background are estimated by using the MLE. The iteration procedure stops when the level sets curve
converges with the lumen-air boundaries, which means that the total number of newly classified colon
tissue pixels does not change.
In order to evaluate the segmentation accuracy, one CTC data set containing 461 slices was manually
segmented under expert guidance. Then, the accuracy  is calculated by computing the overlap between
the results by manual and algorithm segmentations using Equation (38) (Yao, 2004).

Sa  Sm
Sa  Sm
(38)
where, and denote the results by manual and algorithm segmentation, respectively.
Due to the space limitation, only 3 examples are listed in Figure 5(a) to (f). The manual segmentation
results are shown on the first row, while automated segmentation by the proposed algorithm is listed on
the second row. The comparisons below demonstrate that the proposed method works well on these CTC
slices.
This is verified by the accuracy curve (only represents the first 30 slices) in Figure 5(g), which shows
the overlap ranging from 94.5% up to almost 97%. The maximum and minimum overlaps are given in
Table 1. The average accuracy of total 461 slices dataset has achieved 96.06%  0.56% .
Table 1: AVERAGE OVERLAP AND ITS STANDARD DEVIATION (STD)
Overlap
Maximum
Minimum
Average
STD
96.99%
94.48%
96.06%
±0.56%
17
(a)
(d)
(b)
(e)
(c)
(f)
(g)
Figure 5. Comparison results by manual (a), (b), and (c), and algorithm segmentation (d), (e),
and (f). Accuracy curve (g) of calculating overlaps of the segmentation results with the
average at 96.06%±0.56%.
18
(a)
(c)
(b)
(d)
Figure 6. An Example of 3D colon segmentation by the proposed algorithm after (a) 20, (b) 40,
(c) 60 iterations, and (d) final result
Figure 6 shows how the 3D segmentation algorithm works. Figure 6 (a) and (b) show the early stage
results just after 20 and 40 iterations, respectively. From the segmented results, it is clearly observed that
most of the parts have not reached the desired lumen air-tissue boundaries, especially in (a), some of the
colon parts are not connected due to the insufficient iterations. Compared with those in (a) and (b), the
result in (c) shows that most of the colon parts have achieved the boundaries. Figure 6 (d) shows nice
final results with smooth colon surface and clear haustral folders.
3D Colon Isosurface Generation and Centerline Extraction
After colon segmentation, all the isosurfaces are generated by using the Marching Cubes algorithm. All
the isosurfaces are triangulated to generate mesh surfaces in order to calculate the 3D centerlines for
navigation. Table 2 shows the volume size, vertices number and centerline execution time for each
dataset. It is found that all the vertices numbers range from 2.5 Million to about 5.5 Million. All the
centerlines are generated and superimposed for demonstration as shown in Figure 11. For each dataset, it
takes from 6 to 12 minutes to load in the segmented data, generate and render the 3D colon object, and
extract 3D centerline, depending on the presence of complicated shapes and larger numbers of vertices.
We have compared the results by the proposed 3D colon segmentation with those by 3D region growing
(Gonzalez, 2003). From three locally zoomed-in results in Figure 8(a), the colon surface generated by the
proposed method has been improved greatly.
19
(1)
(5)
(3)
(2)
(6)
(7)
(4)
(8)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
(9)
(19)
(20)
Figure 7. 20 colon objects and their centerline.
20
(a)
(c)
(b)
(d)
(e)
(f)
Figure 8. Comparison results by 3D region growing (a) and the proposed method on the right
(b). All the three local surface regions have been improved. (c) polyp 1: size-9 mm in
descending segment, (d) found during 3D navigation, (e) polyp 2: size-6 mm submerged under
the opacified liquid in the rectum segment, (f) found during 3D navigation
A significant colonic polyp is defined as one larger than 5 mm in diameter (Kreeger 2002). The polyps
no less than 6 mm are only focused. For the 12 colonic polyps with sizes of 6 mm and above in the 20
clinical CTC datasets, they are all found by the proposed framework, and compared with the reported
ground truth.
Two examples in Figure 8 show that this work can find and identify colonic polyps. The first one is
located in the descending segment (c), and is successfully found during both 3D supine and prone
navigation (d). Moreover, even the second polyp submerged under the opacified liquid in the rectum
segment (e), it is identified by the proposed method and visualized during 3D virtual navigation (f).
21
Table 2. VOLUME SIZE, VERTICES NUMBER AND EXECUTION TIME
(* PROVIDED BY 3DR INC. LOUISVILLE, KY, U.S.A).
Volume Size
Number of Vertices
Execution Time
1*
394*305*541
5,048,836
9 min 57 sec
2
512*512*432
3,262,143
7 min 6 sec
3
512*512*497
4,568,371
8 min 16 sec
4
512*512*431
5,395,522
12 min 35 sec
5
512*512*496
3,438,940
9 min 10 sec
6
512*512*399
5,438,468
12 min 50 sec
7
512*512*437
3,272,356
7 min 10 sec
8
512*512*413
4,551,929
9 min 49 sec
9
512*512*428
2,928,778
7 min 14 sec
10
512*512*438
2,502,288
6 min 29 sec
11
512*512*468
6,319,156
13 min 9 sec
12
512 *512*387
4,893,086
9 min 37 sec
13
512*512*432
2,988,630
7 min 37 sec
14
512*512*405
3,694,700
8 min 08 sec
15
512*512*430
2,877,740
6 min 56 sec
16
512*512*380
3,902,101
7 min 30 sec
17
512*512*417
4,893,901
11 min 31 sec
18
512*512*461
5,724,714
11 min 38 sec
19
512*512*470
3,060,831
8 min 2 sec
20
512*512*484
3,618,593
7 min 26 sec
22
FUTURE RESEARCH DIRECTIONS
Given the fact that colorectal cancer is a largely preventable disease through routine detection and
removal of adenomatous polyps, colon cancer prevention has now moved to the forefront. Most colorectal
cancers begin as polyps. As polyps enlarge, they are more likely to develop into a cancer, which has the
ability to disseminate through the body. It is clearly demonstrated that the percentage of polyps containing
cancer increases from 1% to 30%~50%, when comparing polyps ranging from 5mm to 22mm. Therefore,
polyp size is considered to be one of the most important factors in distinguishing benign polyps from
cancerous ones. After detection and classification, accurate polyp segmentation could provide an easy
way to measure polyp size, enhancing and improving the detection of significant lesions.
CAD systems for colorectal neoplasiadepend on image preprocessing of clinical colon datasets, colonic
polyp detection and visualization, and polyp features extraction. The next step will involve potential
classification of benign or malignant polyps. We are currently investigating whether polyp architectural
features could be analyzed to generate reliable classification into benign or malignant lesion. We must
conduct prospective trials to validate the clinical applicability of these methods.
The corresponding evaluation of the images will occur within a two hour period, and the results will be
placed into five envelopes, corresponding with five recognized segments of the colon. An additional
reading by a second study radiologist (blinded to the results of the initial reading) will be obtained prior to
proceeding with the follow up colonoscopy, and will be combined with the initial radiologist’s reading, to
prevent possible missing of colonic lesions. After the scanning and reading process is complete, the
subject will be transported to the endoscopy area in University of Louisville Hospital for the colonoscopy.
After reaching the cecum or small intestine, the scope will be slowly withdrawn in the usual manner. At
this point, each of the segmental results from the CT colonoscopy will be revealed after the endoscopist
has completed the pertinent segment. If a polyp was discovered on the CTC, a thorough search of the
segment will be initiated until the lesion has been found. Characteristics of each polyp will be recorded
(size, as determined by measuring catheter), segmental location, growth characteristic (sessile vs
pedunculated), and histology will be matched up for each polyp.
CONCLUSION
This book chapter describes a general variational framework of image pre-processing techniques which
have been proposed as the basis for a colorectal neoplasia CAD system. It relies upon adaptive level sets
based colon segmentation, colon isosurface generation, and 3D centerline extraction. The proposed
framework has been successfully validated on fifteen real CTC datasets. Future work will involve
combining the proposed framework with other colonic polyp detection and visualization techniques
recently developed by our group.
REFERENCES
Abbruzzese, J.L., Pollock, R.E., Ajani, J.A., Curley, S.A., Janjan N.A. & Lynch, P.M. (2004).
Gastrointestinal Cancer. Verlag, Berlin, Heidelberg: Springer.
Abd El Munim, H.H. & A. A. Farag (2004). Adaptive segmentation of multimodal 3d data using robust
level set techniques. In Proceedings of the 8th International Conference on Medical Image Computing
and Computer Assisted Intervention (MICCAI'04) (pp.143-150), MICCAI Press.
23
Abd El Munim, H.H, & Farag, A. A. (2007) Curve/surface representation and evolution using vector level
sets with application to the shape-based segmentation problem. IEEE Transactions on Pattern Analysis
and Machine Intelligence, 29(6), 945-958.
Canny, J.F. (1986). A computational approach to edge detection. IEEE Transaction Pattern Analysis and
Machine Intelligence, 8(6): 679-698.
Caselles, V., Kimmel, R. & Sapiro, G., (1997). Geodesic active contours. International Journal of
Computer Vision. 22(1), 61-79.
Chen, D., Farag, A.A., Hassouna, M. S., Falk, R.L., & Dryden, G.W. (2008). Curvature flow based 3D
surface evolution model for polyp detection and visualization in CT colonography. In T. Smolinski (Ed.),
Computational Intelligence in Biomedicine and Bioinformatics (pp. 201-222), Verlag, Berlin: Springer
Publishing.
Chen, D., Abd El Munim, H.H., Farag, A.A., Falk, R.L., & Dryden, G.W. (2008) Adaptive Level Sets
Based Framework for Segmentation of Colon Tissue in CT Colonography. In H.Yoshida (Ed.), Workshop
on Computational &Visualization Challegences in the New Era of Virtual Colonoscopy with MICCAI'08
(pp.108-115), MICCAI Press.
Chen, D., Liang, Z., Wax, M., Li, L., Li, B., & Kaufman, A. (2000). A novel approach to extract colon
lumen from CT images for virtual colonoscopy. IEEE Transactions on Medical Imaging, 19(12), 12201226.
Cohen, L.D. (1991). On active contour models and balloons. Computer Vision, Graphics, and Image
Understanding, 53(2), 211-218.
Duda, R.O., Hart, P. E. & Stork D.G. (2001). Pattern Recognition, JohnWiley & Sons, Inc. 2nd Edition
Ferrucci, J.T. (2001). Colon Cancer Screening with Virtual Colonoscopy: Promise, Polyps, Politics.
American Journal of Roentgenology, 177(11), 975-988.
Franazek, M., Summers, R., Pickhardt, P., & Choi, J. (2006). Hybrid segmentation of colon filled with air
and opacified fluid for ct colonography. IEEE Transactions on Medical Imaging, 25(3), 358–368.
Gonzalez, R.C., Woods, R.E., & Eddins, S. L. (2003). Digital Image Processing Using MATLAB,
Prentice Hall
Hassouna, M. S. & Farag, A. A. (2007), On the extraction of curve skeletons using gradient vector flow.
In Eleventh IEEE International Conference on Computer Vision (pp. 694-699). IEEE Press.
Hassouna, M. S. & Farag, A. A. (in press). Variational Curve Skeletons Using Gradient Vector Flow.
IEEE Transactions on Pattern Analysis and Machine Intelligence.
Juchems, M. S., Ehmann, J., Brambs, H. J., & Aschoff, A . J. (2005). A retrospective evaluation of
patient acceptance of computed tomography colonography ("virtual colonoscopy") in comparison with
conventional colonoscopy in an average risk screening population. Acta radiologica 46(7):664-70.
Kass, M., Witkin, A., & Terzopolos. D. (1987). Snakes: Active contour models. InternationalJournal of
Computer Vision, 1(4), 321-331.
Kreeger, K., Dachille, F., Wax, M., & Kaufman, A. E. (2002). Covering all clinically significant areas of
the colon surface in virtual colonoscopy. In SPIE medical imaging, (pp. 198–206). SPIE Press.
24
Lorensen, W. E. & Cline, H. E. (1987). Marching cubes: A high resolution 3d surface construction
algorithm. Computer Graphics, 21(4), 163-169.
Macari, M., Berman, P., Dicker, M., Milano, A. & Megibow, A. J. (1999). Usefulness of CT
colonography in patients with incomplete colonoscopy. Journal of Roentgenol, 173, 561-564.
Malladi, R., Sethian, J. A., & Vemuri. B. C. (1995). Shape modeling with front propagation: A level set
approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(2), 158-175.
Manfredo D. C., Differential Geometry of Curves and Surfaces (1976). Prentice Hall
Osher S. & Sethian. J. A. (1988). Fronts propagating with curvature-dependent speed: Algorithms based
on hamilton-jacobi formulations. Journal of Computational Physics, 79(1), 12-49.
Osher S. & Paragios, N. (2003). Geometric Level Set Methods in Imaging, Vision and Graphics, Springer.
Pickhardt, P.J., Choi, J. R., Hwang, I., Butler, J. A., Puckett, M. L., Hildebrandt, H. A., Wong, R. K,
Nugent, P. A., Mysliwiec, P. A., & Schindler, W. R. (2003). Computed tomographic virtual colonoscopy
to screen for colorectal neoplasia in asymptomatic adults. The New England Journal of Medicine,
349(23), 2191-2200.
Sapiro, G. R. (2001). Geometric Partial Differential Equations and Image Processing. New York, NY:
Cambridge University Press.
Sethian. J. A. (1989). Level set methods and fast marching methods: evolving interfaces in computational
geometry, fluid mechanics, computer vision, and materials science. Cambridge University Press.
Summers, R. M., Jerebko, A. K., Malley, J. D., & Johnson, C. D. (2002) Colonic polyps: Complementary
role of computer-aided detection in CT colonography. Radiology, 225(2), 391-398.
Vese, L & Chan, T. (2002). A multiphase level set framework for image segmentation using the
mumford and shah model. International Journal of Computer Vision, 50(3):271-293.
Tikhomirov, V.M. (2001), "Gâteaux variation", Encyclopaedia of Mathematics, Kluwer Academic
Publishers. Retrieved April 9, 2009, from
http://en.wikipedia.org/wiki/G%C3%A2teaux_derivative
Xu C. & Prince, J. L. (1998) Snakes, shapes, and gradient vector flow. IEEE Transactions on Image
Processing, 7(3), 359-369.
Yao, J., Miller, M., Franazek, M. & Summers, R.M. (2004) Colonic polyp segmentation in CT
colonography-based on fuzzy clustering and deformable models. IEEE Transactions on Medical Imaging,
23, 1344–1352.
Yoshida, H. & Nappi, J. (2001). Three-dimensional computer-aided diagnosis scheme for detection of
colonic polyps. IEEE Transactions on Medical Imaging, 20(12), 1261-1274.
Zalis, M. E., Barish, M. A., Choi, J. R., Dachman, A. H., Fenlon, H. M., Feeucci, J. T., Glick, S. N.,
Laghi, A., Macari, M., McFarland, E. C., Morrin, M. M., Pickhardt, P. J., Soto, J., & Yee. J. (2005). CT
colonography reporting and data system: A consensus proposal. Radiology, 236(1), 3-9.
Zalis, M., Perumpillichira, J., & Hahn, P. (2004). Digital subtraction bowel cleaning for CT colonography
using morphological and linear filteration methods. IEEE Transactions on Medical Imaging, 23(2), 1335–
1443.
25