Digital Image Processing
Segmentation
Lecture 8
Course book reading: GW 10
Lucia Ballerini
Image analysis
Image
Pre-processing, image
enhancement
Data
Binary
operations
Classification and
matching
Morphological operations and
feature extraction
Segmentation
"What are the objects to be analyzed?"
Segmentation
►
Full segmentation: Individual objects are separated from the
►
Partial segmentation: The amount of data is reduced (usually by
►
Segmentation is often the most difficult problem to solve in the
process; there is no universal solution!
►
The problem can be made much easier if solved in cooperation
with the constructor of the imaging system (choice of sensors,
illumination, background etc) .
background and given individual ID numbers (labels).
separating objects from background) to speed up the further
processing.
Three types of segmentation
►
Classification – Based on some similarity measure between pixel
►
Edge-based – Search for edges in the image. They are then
►
Region-based – Region growing, merge & split
values. The simplest form is thresholding.
used as borders between regions
Common idea:
search for discontinuities or/and
similitudes in the image
Thresholding
global or local
►
global: based on some kind of histogram: grey-level, edge,
feature etc.
 Lighting conditions are extremely important, and it will only work under
very controlled circumstances.
►
Fixed thresholds: the same value is used in the whole image
►
local (or dynamic thresholding): depends on the position in the
image. The image is divided into overlapping sections which are
thresholded one by one.
Classical automatic
thresholding algorithm
1.
2.
3.
4.
5.
►
►
Select an initial estimate for T
Segment the image using T. This produces 2 groups: G1 ,
pixels with value >T and G2 , with value <T
Compute µ1 and µ2, average pixel value of G1 and G2
New threshold: T=1/2(µ1+µ2)
Repeat steps 2 to 4 until T stabilizes.
Very easy + very fast
Assumptions: normal dist. + low noise
Optimal Thresholding
►
based on the shape of the current image histogram. Search for
valleys, Gaussian distributions etc.
Foreground
Background
Optimal
threshold ?
Both
Real histogram
Histograms
To love…
…and to hate
Thresholding and illumination
► Solutions:
 Calibration of the
imaging system
 percentile filter with
very large mask
 Morphological operators
MR non-uniformity
median filtering
thresholding
More thresholding
►
Can also be used on other kinds of histogram: grey-level, edge,
feature etc.
Multivariate data ( see next lectures)
►
Problems:
 Only considers the graylevel pixel value, so it can leave “holes” in
segmented objects.
► Solution:
post-processing with morphological operators
 Requires strong assumptions to be efficient
 Local thresholding is better  see region growing techniques
Edge-based Segmentation
Based on finding discontinuities (local variations of image intensity)
Apply an edge detector
1.
ex
gradient operator (Sobel)
second derivative (Laplace)
Threshold the edge image to get a binary image
Depending on the type of edge detector:
2.
3.


Link edges together to close shapes (using edge direction for ex)
Remove spurrious edges
Gradient based procedure
Sobel
Sobel
Zero-crossing based procedure
LoG
Laplacian of Gaussian
Edge-based Segmentation: examples
Prewitt: needs edge linking
Canny: needs “cleaning”
Region based segmentation
► Work
by extending some region based on local
similarities between pixels
 region growing (bottom-up method)
 region splitting and merging (top-down method)
Bottom-up: from data to representation
 Top-down: from model to data

Region growing
(bottom-up method)
1.
Find starting points
2.
Include neighbouring pixels with similar features
(grey-level, texture, color).
3.
Continue untill all pixels have been included with one
of the starting points.
►
Problems:


Not trivial to find good strating points, difficult to automate
Need good criteria for similarity.
Watershed
(a kind of region growing)
►
Think of the grey-level image as a landscape. Let water rise
from the bottom of each valley (the water from each valley is
given its own label). As soon as the water from two valleys
meet, build a dam, or watershed. These watersheds will then
define the borders between different regions.
Example of watershed
directly on a graylevel image
Example of Watershed on a binary image
Watershed: problems and solutions
► Oversegmentation
 watershed from markers
► Computation
 new algorithm for fast watershed
► Graylevel
might not be the optimal choice as the local
similarity measure
 bigger neighborhood when growing
 other local features (statistical, edge enhanced image,
distance transformed image…)
Split & Merge
(top-down metod)
1.
Set up som criteria for what is a uniform area (ex mean,
variance, bimodality of histogram, texture, etc…)
2.
Start with the full image and split it in to 4 sub-images.
3.
Check each sub-image. If not uniform, divide into 4 new subimages.
4.
After each iteration, compare touching regions with
neighboring regions end merge if uniform.
The method is also called "quadtree" division
(and is also used for compression)
Split & Merge
The Hough transform
►
A method for finding global relationships between pixels.
Example: We want to find straight lines in an image
 1. Apply edge enhancing filter (ex Laplace)
 2. Set a threshold for what filter response is considered a true ”edge
pixel”
 3. Extract the pixels that are on a straight line using the Hough transform
original image
edge enhanced
image
thresholded
edge image
The Hough transform
Finding straight lines:
1. consider a pixel in position (xi,yi)
2. equation of a straight line yi=axi+b
3. set b=-axi+ yi and draw this (single) line in ”ab-space”
4. consider the next pixel with position (xj,yj) and draw the line
b=-axj+ yj ”ab-space” (also called parameter space). The poins
(a’,b’) where the two lines intersect represent the line y=a’x+b’
in ”xy-space” which will go through both (xi,yi) and (xj,yj).
► 5. draw the line in ab-space corresponding to each pixel in xyspace.
► 6. divide ab-space into accumulator cells and find most common
(a’, b’) which will give the line connecting the largest number of
pixels
►
►
►
►
The Hough transform
y
b
x
xy-space
a
ab- or parameter space
The Hough transform
►
In reality we have a problem with y=ax+b because a reaches
infinity for vertical lines.
Use x cos   y sin    instead.
►
It is common to use ”filters” for finding the intersection: ”butterfly
filters”
►
Different variations of the Hough transform can also be used for
finding other shapes of the form g(v,c)=0, v is a vector of
coordinates, c is a vector of coefficients.
►
Possible to find any kind of simple shape
ex. circle: ( x  c ) 2  ( y  c ) 2  c2 (3D parameter space)
1
2
3
The Hough transform
Conclusions
►
The segmentation procedure
1. Pre-processing
2. Segmentation
3. Post-processing
 Like any IP procedure
There exists NO universal segmentation method
► Evaluation of segmentation performance is important
►
Snakes
► Example:
segmentation of the brain in MRI
User interaction
Snake after initialization
Snake at equilibrium
Snakes (active contours)
►A
snake is an active contour parametrically represented
by its position v(s)=(x(s), y(s))
► Each position is associated to an energy:
1
E snake 

0
► The
1
E int[ v ( s )]ds   E ext[ v ( s )]ds
0
final position corresponds to the minimum of the
energy
Internal Energy
The internal energy of the snake is due to
bending and it is associated with a priori
constraints:
2

1
v ( s)
 2 v ( s)
E int[ v ( s )]  a ( s )
 b ( s)
2 
s
s 2
a(s) controls the tension of the contour
► b(s) controls its rigidity
►
2



External Energy
► The
external energy depends on the image and
accounts for a posteriori information
► Several energy forms have been proposed based on
features of interest in the image
► An energy commonly used to attract snakes towards
edges is:
Eext[ v ( s)]   G  I ( x , y )
2
Applications
Applications
(by Terzopoulos)
Considerations
►
►
►
►
►
The number of nodes is an important factor for the behavior
of the snake. Ability to resample the contour may be
necessary.
If we want a closed contour, we set the first and the last
point equal.
Anchor points are necessary to keep the snake in position if
the image forces are not enough.
It may be necessary to allow a snake contour to divide into
two contours, or two contours to merge into one contour.
Different applications may need different potential functions
and different settings of the control parameters (damping,
tension and rigidity).
Applications
►
Tracking of a moving object
 An initial estimate for the contour (e.g. interactively defined) is used in
the first frame.
 The contour at equilibrium is used as the starting contour for the next
frame. The snake locks on to the object.
►
Reconstruction from serial sections
 The user draws an approximate contour in the first slice.
 The contour at equilibrium is used as the starting contour in the next
slice.
 The 3D object is reconstructed from the contours using triangulation.
►
…..
More segmentation
Important in Image Processing in general:
“If you can use expert knowledge (user
interaction, modelling,…) at relatively low cost
(development, computational,…)”
JUST DO IT!!