Visual Tracking
CMPUT 615
Nilanjan Ray
What is Visual Tracking
• Following objects through image sequences or
videos
• Sometimes we need to track a single object,
sometimes a number of them
• Sometimes we just track the object centroid,
sometimes entire object boundary (shape)
Theoretical Foundation
• Visual tracking is a “state” estimation problem
• Bayesian inference is at the heart of visual
tracking; it is called sequential Bayesian
estimation
• We form the posterior probability of the state,
given all evidence or measurements up to the
current time point
• Inference is performed from the posterior
density
Setting The Stage
Some notations:
Xt: unknown state we want to estimate at time point t; e.g., object centroid
Zt: Measurement/observation made at time point t; e.g., image intensities
The sequential estimation model assumes that we know three probability
densities:
p(X0): The initial state density
p(Xt|Xt-1): State transition density or motion model
p(Zt|Xt): Measurement/observation/likelihood density
Sequential Bayesian Estimation
(AKA Sequential Bayesian Filtering)
• We want to recursively estimate the state Xt given
the observations Z1:t = {Z1, Z2, …, Zt}
Sequential Bayesian Estimation…
Filter:
Prediction
Previous posterior
Likelihood/observation density
Bayes’ Rule:
Current posterior
Bayes’ Rule Derivation
Conditional probability rule
Marginal density rule
Also, because measurement Zt is conditionally independent on the current state Xt:
So, we have the sequential Bayes’ rule:
Filter Derivation
Rule of marginal density
Rule of conditional probability
Also, note that Xt is conditionally independent on Xt-1 (Markovianity), so:
Thus we have the filter rule:
Important Assumptions
• Observation is conditionally independent on the
current state
• Current state is conditionally independent on the
immediate previous state
Computation
• Theory is all good, however we need to show people
that it works in practice…
• We will study Particle filter, the framework that can
compute the recursive state estimation, i.e.,
sequential Bayesian estimation
• We will also study Kalman filter, a popular sequential
state estimation technique with some more
assumptions
What is a Particle Filter?
Let the particles
represent the previous density
So, the filter step is now:
And the Bayes’ rule is now:
We need to generate the current particle set from p(Xt|Zt):
Particle filter
Factored Sampling
Let h(x) = f(x)g(x) is a product of two functions, where say, g(x) is a density and
f(x) is another non-negative function
Factored sampling says that to represent h(x) non-parametrically by a set of
particles, generate samples from g(x) and assign weights by f(x)
i.e., {(s1 , w1), …, (sn, wn)}, where si are generated from g(x) and wi = f(si).
This is closely related to another sampling method called importance sampling.
Conditional Density Propagation
(CONDENSATION)
This a product of two functions: (1)
and (2)
Following the principle of factored sampling, CONDENSATION generates samples from (1)
And assigns weights using (2)
Samples From a Mixture Density
Notice that
is a mixture density
To generate samples from the mixture density these two steps are followed:
CONDENSATION Algorithm
How to estimate the state?
• OK, we generated samples, what do we do with them:
estimate the current state:
h is any function of the state, for example when h(x) = x,
we are performing state estimation
Other PFs
• To date lots of particle filters have been
proposed:
– Sequential importance re-sampling (SIR)
– Auxiliary particle filter (APF)
– Likelihood particle filter
– Rao-Blackwellized particle filter
– A ton others
• A leading researcher in PF : Arnoud Doucet
Some Points to Ponder about PF
• The good point about PF is that it can handle very
general likelihood and motion models
• PF inherits a serious shortcoming from nonparametric density representation – curse of
dimensionality – when the state space x is large, for
example large multiple number of objects etc.