Boosted Particle Filter: Multitarget
Detection and Tracking
Fayin Li
Motivation and Outline
• For a varying number of non-rigid objects, the
observation models and target distribution be highly nonlinear and non-Gaussian.
• The presence of a large, varying number of objects
creates complex interactions with overlap and
ambiguities.
• How object detection can guide the evolution of particle
filters?
• Mixture particle filter
• Boosted objection detection
• Boosted particle filter
• Observation model in this paper
Multitarget Tracking Using Mixture Approach
• Given observation and transition models, tracking can be
considered as the following Bayesian recursion:
p( yt | xt ) p( xt | y0:t 1 ) p( yt | xt )  p( xt | xt 1 ) p( xt 1 | y0:t 1 )dxt 1
p( xt | y0:t ) 

p( yt | y0:t 1 )
 p( yt | xt ) p( xt | y0:t 1 )dxt
• To deal with multiple targets, the posterior is modeled as
M-component non-parametric mixture approach
M
p( xt | y0:t )    j ,t p j ( xt | y0:t )
j 1
• Denote p j ( xt | y0:t 1 )   p( xt | xt 1 ) p j ( xt 1 | y0:t 1 )dxt 1
Mixture Approach and Particle Approximation
• Then the prediction step p( xt | y0:t 1 ) 
• And the updated mixture p( xt | y0:t ) 
• where p ( x | y ) 
j
t
0:t

j 1
M

j 1
 p (y
t
j ,t
p j ( xt | y0:t 1 )
p j ( xt | y0:t )
| xt ) p j ( xt | y0:t 1 )dxt
 j ,t 1  p j ( yt | xt ) p j ( xt | y0:t 1 )dxt
k 1  k ,t 1  pk ( yt | xt ) pk ( xt | y0:t 1 )dxt
M
j ,t 1
p j ( yt | xt ) p j ( xt | y0:t 1 )
j
• and  j ,t 
M

 j ,t 1 p j ( yt | y0:t 1 )

M
k 1
 k ,t 1 pk ( yt | y0:t 1 )
• The new filtering is again a mixture of individual component filtering.
And the filtering recursion can be performed for each component
individually. The normalized weights is only the part of the procedure
where the components interact.
Particle Approximation
• Particles filters are popular at tracking for non-linear
and/or non-Gaussian Models.
• However they are poor at consistently maintaining the
multi-modality of the target distributions that may arise
due to ambiguity or the presence of multiple objects.
• In standard particle filter, the distribution can be
i
i N
represented by N particles {xt , wt }i 1 . During recursion,
first sample particles from an proposal distribution
i
i
i
xti ~ q ( xt | x0:t 1 , y0:t ) with weight wti  wti1 p( yt | ixt ) ip( xt | xt 1 )
q( xt | x0:t 1 , y0:t )
• Resample the particles based the weights to approximate
the posterior p( xt | y0:t )
Particle Approximation
• Because each component can be considered individually
in mixture approach, the particles and weights can be
updated for each component individually.
• The posterior distribution
is approximated by
M
p( xt | y1:t )    j ,t  wti xi ( xt )
j 1
iI j
t
• And the particle weight updated rule is
wti 

~i
w
t
~i 
,
w
t
~
w
t
wti1 p j ( yt | xti ) p j ( xti | xti1 )
j
jI m
q j ( xt | xti1 , yt )
• And the mixture weights can be updated using particle
weights
~i ,  
p j ( yt | yt 1 )   w
t
j ,t
iI j
 j ,t 1w j ,t

M
n 1
 n ,t 1wn ,t
~i
, w j ,t    j ,t 1w
t
iI j
Example
• A simple example
governed by the
equations
p( xt | xt 1 )  N ( xt | xt 1 ,  x2 )
p( yt | xt )  N ( yt | xt2 ,  y2 )
Mixture Computation and Variation
• The number of modes is rarely known ahead and is
unlikely to remain fixed.
• It may fluctuate as ambiguities arise and are resolved, or
objects appear and disappear.
• It is necessary to recompute the mixture representation
• Based on the particles and weights, we can use k-means
to cluster the sample set and update the number of
modes, particles weights, and mixture weights.
• In stead of M modes, we can use M different likelihood
distributions. When one or more new objects appear,
they are detected and initialized with an observation
model. Different observation model (data association)
allow us track objects.
AdaBoost
• Given a set of weak classifiers
originally : h j (x)  {1,  1}
also h j (x) { ,  } " confidence rated"
– None much better than random
• Iteratively combine classifiers
– Form a linear combination


C ( x )     ht ( x )  b 
 t

– Training error converges to 0 quickly
– Test error is related to training margin
Adaboost Algorithm
(Freund & Shapire)
ht  min  Dt (i ) e  yi h ( xi )
h
i


Zt
1  W 
t , t   log  
2  W 
Weak
Classifier 1
Weights
Increased
Weak
Classifier 2
Dt (i )e  yi ht ( xi )
Dt 1 (i ) 
Zt
Weak
classifier 3
Final classifier is
linear combination of
weak classifiers
A variant of AdaBoost for aggressive
feature selection
 Given example images (x1,y1) , … , (xn,yn) where yi = 0, 1 for negative and positive
examples respectively.
 Initialize weights w1,i = 1/(2m), 1/(2l) for training example i, where m and l are the
number of negatives and positives respectively.
For t = 1 … T
1) Normalize weights so that wt is a distribution
2) For each feature j train a classifier hj and evaluate its error j with respect to wt.
3) Chose the classifier hj with lowest error.
4) Update weights according to:
1 i
wt 1,i  wt ,i 
t
where ei = 0 is xi is classified correctly, 1 otherwise, and

t


t
1
t
 The final strong classifier is:

1
h( x )  
0
1
  h ( x)  2  
T
t 1
t
t
otherwise
T
t 1
t
,
where

t
 log(
1

)
t
Cascading Classifiers for Object Detection
• Given a nested set of classifier
hypothesis classes
% False Pos
0
50
% Detection
100
vs false neg determined by
IMAGE
SUB-WINDOW
T
Classifier 1
F
NONNON-Object
50
• Computational Risk Minimization.
Each classifier has 100% detection
rate and the cascading reduces the
false positive rate
T
Classifier 2
F
NON-Object
T
Classifier 3
F
NON-Object
Object
Boosted Particle Filter
• Cascading Adaboost algorithm gets high detection rate but
large number of false positives, which could be reduced by
considering the motions of the objects (players).
• As with many particle filters, the algorithm simply proceeds
by sampling from the transition prior p( xt | xt 1 ) without
using the data information.
• Boosted Particle Filter uses the following mixture
distribution as the proposal distribution for sampling
qB* ( xt | xt 1 , y1:t )  qada ( xt | xt 1 , yt )  (1   ) p( xt | xt 1 )
• Here qada is a Gaussian distribution and  can be set
dynamically with affecting the convergence of the particle
filter. If there is overlap between a component of mixture
particle filters and the nearest cluster detected by
Adaboost, use the mixture proposal distribution, otherwise
set  = 0
Observation Model
• Hue-Saturation-Value (HSV) histogram is used to
represent the region containing the object. It has N =
NhNs + Nv bins.
• Then a kernel density estimation of the color distribution
at time t is given: k (n; xt )    [bt (d )  n]
• Bhattacharyya coefficient is applied to measure the
distance between two color histograms

N


n 1

 [ K * , K ( xt )]  1   k * (n; x0 )k (n; xt ) 
1
2
  2 [ K * , K ( xt )]
• And the likelihood function is p( yt | xt )  e
• If the object is represented by multiple regions, the
likelihood function will be p( yt | xt )  e  [ K , K ( x )]
*
2
j
j
j
t
Experiments and Conclusion
• Boosted particle filter works well no matter how many
objects and adapts successfully to the changes (players
come in and out).
• Adaboost detects the new players and BPF assigns the
particles to them.
• Mixture components are well maintained even Adaboost
fails.
• Object detection and dynamics are combined by forming
the proposal distribution for the particle filter: the
detections in current frame and the dynamic prediction
from the previous time step.
• It incorporates the recent observations, which improves
the robustness of the dynamics
• The detection algorithm gives a powerful tool to obtain
and maintain the mixture representation.
Tracking Results
• Video 1 and Video 2