Mean Shift Segmentation
Jan Kybic
winter semester 2007
Mean shift segmentation overview
I
I
I
No assumptions about probability distributions — rarely
known
Spatial-range domain x, y , f (x, y ) — normally f (x, y )
Find maxima in the x, y , f space — clusters close in space
and range correspond to classes.
Mean shift procedure
Goal: Find local maxima of the probability density (density modes)
given by samples.
ld
O
New window
tro
n
ce
id
ew
N
ce
id
ro
nt
Initial window
Mean shift
1. Start with a random region of interest.
2. Determine a centroid of the data.
3. Move the region to the location of the new centroid.
4. Repeat until convergence.
Kernel estimation
K (x) = ck k kxk2
(radial symmetry)
Epanechnikov kernel (other choices possible)
(
1−r
k(r ) =
0
for r ≤ 1
otherwise
(profile, r = kxk2 )
Kernel density estimator
n
X
x
−
x
1
i
e
K
f (x) =
h
n hd
i=1
Mean shift procedure
At density maxima ∇e
f =0
e
f (x) =
n
1 X
x − xi
K
h
n hd
i=1
!
n
X
x − xi 2
2
c
k
0
(x − xi ) k ∇e
f (x) =
h n h(d+2) i=1
! P
n
n
X
xi gi
2 ck
i=1
Pn
−x
gi
=
n h(d+2) i=1
i=1 gi
for g (r ) = k 0 (r ), gi = g ||(x − xi )/h||2
Mean shift procedure
At density maxima ∇e
f =0
e
f (x) =
n
1 X
x − xi
K
h
n hd
i=1
2 !
n
X
2
c
x
−
x
i k
(x − xi ) k 0 0 = ∇e
f (x) =
(d+2)
h nh
i=1
! P
n
n
X
xi gi
2 ck
i=1
Pn
gi
=
−x
n h(d+2) i=1
i=1 gi
|
{z
}
mean shift vector — must be 0 at optimum
for g (r ) = k 0 (r ), gi = g ||(x − xi )/h||2
Mean shift procedure (2)
Mean shift vector
Pn
xi gi
m(x) = Pi=1
−x
n
i=1 gi
gi = g ||(x − xi )/h||2
g (r ) = k 0 (r )
Successive locations yj of the kernel:
yj+1 =
n
X
i=1
! X
!
n
yj − xi 2
yj − xi 2
g xi g h h i=1
Mean shift procedure (2)
Mean shift vector
Pn
xi gi
−x
m(x) = Pi=1
n
i=1 gi
gi = g ||(x − xi )/h||2
g (r ) = k 0 (r )
Successive locations yj of the kernel:
! X
n
n
X
yj − xi 2
yj+1 =
xi g g
h i=1
i=1
!
yj − xi 2
h Theorem: Ifk is convex and monotonically
decreasing, the
sequence yj j=1,2,... converge and e
f (yj ) j=1,2,... increases
monotonically.
For Epanechnikov kernel → convergence in finite number of steps.
Mean shift mode detection
Points from a basin of attraction converge to the same mode.
Algorithm:
1. Using multiple initializations covering the entire feature space,
identify modes (stationary points).
2. Using small random perturbation, retain only local maxima.
100
100
80
80
60
60
u*
u*
Mean shift mode detection example
40
40
20
20
0
0
−20
20
30
40
50
60
L*
70
80
90
100
−20
20
30
40
50
60
L*
70
80
90
100
Mean shift discontinuity preserving filtering
Combine spatial and range values
c
K (x) = ([x x ]) = d p k
hs hr
s
r
s 2 !
x k
hs r 2 !
x ,
hr Algorithm:
1. For each image pixel xi , initialize yi,1 = xi .
2. Iterate the mean shift procedure until convergence.
r
3. The filtered pixel values are defined as zi = (xsi , yi,con
); the
s
value of the filtered pixel at the location xi is assigned the
r .
image value of the pixel of convergence yi,∞
Mean shift discontinuity preserving filtering
Mean shift segmentation
1. Mean shift discontinuity preserving filtering
2. Determine the clusters {Cp }p=1,...,m by grouping all zi , which
are closer than hs in the spatial domain and hr in the range
domain, i.e. merge the basins of attractions.
3. Assign class labels to clusters
4. If desired, eliminate regions smaller than P pixels.
Mean shift segmentation examples
Mean shift segmentation examples