Mean Shift
算法原理和在目标跟踪上的应用
Agenda
• Mean Shift Theory
• What is Mean Shift ?
• Density Estimation Methods
• Deriving the Mean Shift
• Mean shift properties
• Applications
• Clustering
• Discontinuity Preserving Smoothing
• Object Contour Detection
• Segmentation
• Object Tracking
Mean Shift Theory
Intuitive Description
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Distribution of identical billiard balls
Intuitive Description
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Distribution of identical billiard balls
Intuitive Description
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Distribution of identical billiard balls
Intuitive Description
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Distribution of identical billiard balls
Intuitive Description
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Distribution of identical billiard balls
Intuitive Description
Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
Distribution of identical billiard balls
Intuitive Description
Region of
interest
Center of
mass
Objective : Find the densest region
Distribution of identical billiard balls
研究现状
Mean shift算法是Fukunaga于1975年提出的，
其含义即偏移的均值向量。随着Mean shift理论的
发展，它的含义也发生了变化。现在一般是指一
个迭代的步骤，即先算出当前点的偏移均值，移
动该点到其偏移均值,然后以此为新的起始点，继
续移动，直到满足一定的条件结束。Cheng
Yizong定义了一族核函数 ，将Mean shift算法引入
到计算机视觉领域。Bradski G R对Mean shift算
法进行改进，发展建立了Camshift算法，将Mean
shift方法扩展应用到了目标跟踪中来。
Mean shift的基本形式
给定d维空间 R d 中的n个样本点,i=1,…,n,在点
x 的Mean Shift向量的基本形式定义为:
1
Mh  x 
k
 x
xi Sh
i
 x
其中，S h 是一个半径为h的高维球区域， k表
示在这n个样本点中，有k个点落入区域 S h 中.
Mean shift的扩展
核函数：
代表一个d维的欧氏空间， x 是该空间中的一
个点，用一列向量表示。 x 的模 x  x x
。
R 表示实数域。如果一个函数 K : X  R存在一个剖
面函数 k : 0,   R ，即
X
2
K ( x)  k

x
2

剖面函数的性质：
（1）k 是非负的 ；
（2）k 是非增的；

k
（3） 是分段连续的,并且 0 k (r )dr

T
Kernel Density Estimation
Various Kernels
1 n
P ( x)   K ( x - x i )
n i 1
例:
在选定的空间中，x1…xn
是有限的样本点。


 c 1 x
• Epanechnikov Kernel K E (x)  

 0
2

x 1
otherwise
• Uniform Kernel
（均匀核函数）
c
x 1
KU (x)  
 0 otherwise
• Normal Kernel
（高斯核函数）
 1 2
K N (x)  c  exp   x 
 2

核密度 估计
梯度
1 n
P(x)   K ( x - xi )
n i 1
使用核函数
的形式:
得到 :
 x - xi
K (x - xi )  ck 
 h

2



窗宽带宽
 n

xi gi

2c n
c  n  
P ( x ) 
ki    g i  g  i 1n
 x

nh i 1
n  i 1  

gi

 i 1

g(x)  k (x)
Computing The Mean Shift
 n

x
g
i i

c n
c n
 
i 1
P ( x )    ki    g i   n
 x
n i 1
n  i 1  

gi

 i 1

Yet another Kernel
density estimation !
Simple Mean Shift procedure:
• Compute mean shift vector
 n

 x - xi 2 
  xi g 




h
 i 1

  x
m ( x)  

2
n


x
x


i
g





 h 

 i 1 

•Translate the Kernel window by m(x)
g(x)  k (x)
Non-Rigid Object Tracking
…
…
Mean-Shift Object Tracking
General Framework: Target Representation
Choose a
reference
model in the
current frame
…
Current
frame
Choose a
feature space
…
Represent the
model in the
chosen feature
space
Mean-Shift Object Tracking
General Framework: Target Localization
Start from the
position of the
model in the
current frame
Search in the
model’s
neighborhood
in next frame
Find best
candidate by
maximizing a
similarity func.
Repeat the
same process
in the next pair
of frames
…
Model
Candidate
Current
frame
…
Mean-Shift Object Tracking
Target Representation
Choose a
reference
target model
Represent the
model by its
PDF in the
feature space
Choose a
feature space
0.35
Quantized
Color Space
Probability
0.3
0.25
0.2
0.15
0.1
0.05
0
1
2
3
.
color
Kernel Based Object Tracking, by Comaniniu, Ramesh, Meer
.
.
m
Mean-Shift Object Tracking
Target Model
Target Candidate
(centered at 0)
(centered at y)
0.35
0.3
0.3
0.25
0.25
Probability
Probability
PDF Representation
0.2
0.15
0.1
0.2
0.15
0.1
0.05
0.05
0
0
1
2
3
.
.
.
m
1
2
color
q  qu u 1..m
3
.
.
.
m
color
m
q
u 1
u
1
Similarity f y  f  q , p y 
    
Function:
p  y    pu  y u 1..m
m
p
u 1
u
1
Mean-Shift Object Tracking
Finding the PDF of the target model
xi i1..n
candidate
model
Target pixel locations
y
0
k ( x)
A differentiable, isotropic, convex, monotonically decreasing kernel
b( x )
The color bin index (1..m) of pixel x
• Peripheral pixels are affected by occlusion and background interference
Probability of feature u in model

b ( xi ) u
 
 y  xi
pu  y   Ch  k 
 h
b ( xi ) u 
2
k xi
0.3
0.3
0.25
0.25
0.2
Pixel weight
0.15
0.1
Normalization
factor
Probability
Normalization
factor
Probability
qu  C
Probability of feature u in candidate
Pixel weight
0.1
0
0
1
2
3
.
color
.
.
m



0.2
0.15
0.05
0.05
2
1
2
3
.
color
.
.
m
Mean-Shift Object Tracking
Similarity Function
Target model:
q   q1 ,
, qm 
Target candidate:
p  y    p1  y  ,
Similarity function:
f  y   f  p  y  , q   ?
, pm  y  
The Bhattacharyya Coefficient
q 

q1 ,
, qm

p1  y  ,
p  y  

q
1
, pm  y 

y
1
m
p  y  q 
f  y   cos  y 
  pu  y  qu
p  y   q u 1
T
p  y 
Mean-Shift Object Tracking
Target Localization Algorithm
Start from the
position of the
model in the
current frame
q
Search in the
model’s
neighborhood
in next frame
p  y
Find best
candidate by
maximizing a
similarity func.
f  p  y  , q 
Mean-Shift Object Tracking
Approximating the Similarity Function
m
f  y    pu  y  qu
u 1
Linear
approx.
(around y0)
Model location:
y0
Candidate location:
1 m
1 m
f  y    pu  y0  qu   pu  y 
2 u 1
2 u 1
y
qu
pu  y0 
 y  xi
pu  y   Ch  k 
 h
b ( xi ) u 
Independent of
y
Ch
2
 y  xi
wi k 

 h
i 1

n
2



2



Density
estimate!
(as a function
of y)
Mean-Shift Object Tracking
Maximizing the Similarity Function
The mode of
Ch
2
 y  xi
wi k 

 h
i 1

n
Important Assumption:
The target
representation
provides sufficient
discrimination
One mode in
the searched
neighborhood
2

 = sought maximum

Mean-Shift Object Tracking
Applying Mean-Shift
The mode of
Ch
2
 y  xi
wi k 

 h
i 1

2
n

 = sought maximum

 y0  xi 2 
xi g 



h
i 1


y1 
n
 y0  xi 2 
g



h
i 1


n
n
 y  xi
Original
Find mode of c k 
 h
Mean-Shift:
i 1

2



using
 y0  xi 2 
xi wi g 


2

h

i 1
y  xi


y

using

1
2
n
 y0  xi 
h

wi g 



h
i 1


n

Extended
Find mode of c wi k 
Mean-Shift:
i 1

n
Mean-Shift Object Tracking
About Kernels and Profiles
A special class of radially
symmetric kernels:
 
K  x   ck x
2
The profile of
kernel K
k  x  g  x
 y0  xi 2 
xi wi g 


2

h

i 1
y  xi


y

using

1
2
n
 y0  xi 
h

wi g 



h
i 1


n

Extended
Find mode of c wi k 
Mean-Shift:
i 1

n
Mean-Shift Object Tracking
Choosing the Kernel
A special class of radially
symmetric kernels:
Epanechnikov kernel:
1  x if x  1 
k  x  

0
otherwise


 y0  xi 2 
xi wi g 



h
i 1


y1 
n
 y0  xi 2 
wi g 



h
i 1


 
K  x   ck x
2
Uniform kernel(单位均匀核函数):
1 if x  1 
g  x   k  x   

0
otherwise


n
n
y1 
xw
i 1
n
i
i
w
i 1
i
Mean-Shift Object Tracking
Adaptive Scale
Problem:
The scale of
the target
changes in
time
The scale (h)
of the kernel
must be
adapted
Solution:
Run
localization 3
times with
different h
Choose h
that achieves
maximum
similarity
完
谢谢