Tracking
We are given a contour G1 with coordinates G1={x1 , x2 , … , xN} at the initial
frame t=1, were the image is It=1 . We are interested in tracking this contour
through several image frames, say through T image frames given by
IT = IT , IT 1 ,..., I1
We will denote each of these contours by Gt and so G1 = Gt=1 . We will track
each of the coordinates of the initial contour, i.e., we will focus on the
tracking of the N initial coordinates. Thus, at any time the new contour will be
characterized by Gt = {x1 , x2 , … , xN }, and these coordinates need not be
connected. In order to create a connected contour at each frame one may fit a
spline or another form of linking of this sequence of N coordinates. In a
Bayesian framework we attempt to obtain the probability
p(Gt 1 | It 1 )
(1)
We can make measurements at each image frame, accumulate them over
time denoting by It 1 and we ask “Which state (coordinates) the contour will
be at that time given all the measurements ?”
© 2006 by Davi Geiger
Computer Vision
November 2006
L1.1
Propagating Probability Density
An interesting expansion of (1) is given by:
P (Gt 1 | It 1 ) = P (Gt 1 | It 1 , It )
=
=
1
p ( It 1 | It )
Baye' s (condition al on C ) : P( A | B, C ) =
p ( It 1 | Gt 1 , It ) p ( Gt 1 | It )
1
P( B | A, C ) P( A | C )
P( B | C )
p ( It 1 | Gt 1 , It )
p ( Gt 1 | It )
p ( It 1 | It )
Marginal Probabilit y : P( A | C ) =  P( A, B | C )
p ( It 1 | Gt 1 , It )
=
p ( Gt 1 , Gt | It ) dGt

p ( It 1 | It ) Gt
B
P( A, B | C ) = P( A | B, C ) P( B | C )
=
p ( It 1 | Gt 1 , It )
p ( Gt 1 | Gt , It ) P( Gt | It )dGt

p ( It 1 | It ) Gt
(2)
Assuming that such probabilities can be obtained, namely
p( It 1 | Gt 1 , It ), p( Gt 1 | Gt , It ), p( It 1 | It ) ,
than we have a recursive method to estimate P(Gt 1 | It 1 )
© 2006 by Davi Geiger
Computer Vision
November 2006
L1.2
Assumptions
p ( It 1 | Gt 1 , It ) Independence assumption on data generation, i.e.,
the current state completely defines the probability
or
of the measurements/data (often the previous data
p ( It 1 | Gt 1 )
is also neglected)
p ( Gt 1 | Gt , It ) First order Markov process, i.e., the current state
or
of the system is completely described by the
immediate previous state, and possibly the data
p ( Gt 1 | Gt )
(often the data is also neglected).
p ( It 1 | It )
© 2006 by Davi Geiger
This is a normalization constant that can always
be computed by normalizing the recurrence
equation (2)
Computer Vision
November 2006
L1.3
Linear Dynamics
p(Gt | Gt 1 ) = N ( Dt 1Gt 1;  dt )
p( It | Gt ) = N ( M t Gt ;  mt )
where N (  ; ) is the multivaria ble normal distributi on , i.e., a Gaussian
with mean vecto r  and covariance matrix . M t and Dt are linear matrices.
Examples
Random Walk
Points moving with constant velocity
Points moving with constant acceleration
Periodic motion
© 2006 by Davi Geiger
Computer Vision
November 2006
L1.4
Random Walk of a Point
Drift: New position is the previous one plus noise (d)
  xt 
xt =  
 yt 
© 2006 by Davi Geiger
  

 (D
p
(
x
|
x
)
=
N
x
 t t 1
xt
t 1 t 1 ;  dt )


 p (m | x ) = N ( M x ;  )
t
t
mt
t t
mt


Computer Vision
1 0 
Dt 1 =  
 0 1
1 0 
M t =  ,  mt = 0
 0 1
November 2006
L1.5
Point moving with constant velocity
dx
= v  x(t  dt ) = x(t )  v dt
dt
  
1 t

dx
 (D u

p
(
u
|
u
)
=
N
;

)
D
=
 t t 1
=v
xt
t 1 t 1
dt
t 1
dt
  xt 
 1 t  

0 1
 ut 1  
ut =    ut = 
v
0
1
1 0 


 t
 p (m | u ) = N ( M u ;  )
 
M
=
t
t
mt
t t
mt
t

0 0

 
Note : p (ut | ut 1 ) is equivalent to a second order model
Motion equation :



p ( xt | xt 1 , xt  2 ) = N ( xt 1  t ( xt 1  xt  2 );  dt )
© 2006 by Davi Geiger
Computer Vision
November 2006
L1.6
Point moving with constant acceleration
Motion wit h constant accelerati on : v(t  dt ) = v(t )  a dt

1 t 0 



  
 (D u
p
(
u
|
u
)
=
N
;

)
D
=
0
1

t


ut
t 1 t 1
dt
t 1
 t t 1
 xt 
0

0
1 
  

ut =  vt   
1 0 0 
a 





 t
 p(m | u ) = N ( M u ;  )
M
=
0
0
0


t
t
mt
t t
mt
t




0 0 0 
 
1
Note : p (ut | ut 1 ) is equivalent to a third order model, using x(t ) = x(t )  v(t )dt  a (dt ) 2 ,
2
1


p ( xt | xt 1 , xt  2 , xt 3 ) = N  xt 1  t ( xt 1  xt  2 )  (t ) 2 ( xt 1  xt 3  2 xt  2 );  dt 
2


© 2006 by Davi Geiger
Computer Vision
November 2006
L1.7
Point moving with periodic motion
d 2x
Periodic Motion :
=  x(t )  periodic solution x(t ) = A cos wt
dt 2
 dx
 dt = v(t )  x(t  dt ) = x(t )  v(t )dt
equivalent to 
 dv =  x(t )  v(t  dt ) = v(t )  x(t )dt
 dt
  xt 
ut =  
 vt 
  
 1 t 

 (D u


p
(
u
|
u
)
=
N
;

)
D
=
 t t 1
ut
t 1 t 1
dt
t 1

  t 1 

 p (m | u ) = N ( M u ;  ) M = 1 0 
t
t
mt
t t
mt
t
0 0

 

© 2006 by Davi Geiger
Computer Vision
November 2006
L1.8
Tracking one point (dynamic programming like)
Equation (2) is written as
P( xt | It ) =
p( It | xt , It 1 )
p( xt | xt 1 , It 1 ) P( xt 1 | It 1 )dxt 1
p( It | It 1 ) xt1
F ( xt | It ) = p( It | xt , It 1 ) p( xt | xt 1 , It 1 ) P( xt 1 | It 1 )
xt 1
P( xt | It ) =
F ( xt | It )
 F ( xt | It )
xt
xt 1
note that : p ( It | It 1 ) =  F ( xt | It )
xt
xt
P( It 1 | xt 1 )
P( xt 1 | It 1 )
p( It | xt )
p( xt | xt 1 )
P ( xt | It )
t1
© 2006 by Davi Geiger
t
t
Computer Vision
November 2006
L1.9
Kalman Filter
Simplify calculation of the probabilities P(Gt | It ) due to good Gaussian
properties (applied to p( It | Gt , It 1 ) and p( Gt | Gt 1 , It 1 ) )
Reminder: From equation (2) we have
P(Gt | It ) =
p( It | Gt , It 1 )
p( Gt | Gt 1 , It 1 ) P( Gt 1 | It 1 )dGt 1
p( It | It 1 ) Gt1
1D case: For one dimension point and Gaussian distributions
N yt (mt xt ; mt )
P( xt | yt ) =
 N xt ( dt xt 1; dt ) P( xt 1 | yt 1 )dxt 1
p( yt | yt 1 )
xt 1
Kalman observed the recurrence: if P( xt 1 | yt 1 ) = N xt 1 ( t 1;t 1 )

where  t =
( mt  t ) 2

( mt )  ( t )
© 2006 by Davi Geiger
2
 2

then P( xt | yt ) = N xt ( t ;t )
, t = 

2
dt
 (dt 

2
t 1
Computer Vision
)
and t =
mt yt ( t ) 2  dt t 1  m2t
(mt  t ) 2   m2t
November 2006
L1.10
Kalman Filter and Gaussian Properties
Proof: Assume
P( xt 1 | yt 1 ) = N xt 1 ( t 1;t1 )
Then P( xt | yt ) = ct N y (mt xt ;  m )  N x ( dt xt 1 ; d ) N x ( t 1 ;  t1 )dxt
t
t
t
t
t 1
xt 1
since
( xt  dt xt 1 )2
1
2
2
dt
e
2 d2t
(d
dt t
1
=
2
1
2
2
dt
e
xt  xt 1
2 d2t
)
(xt
2
=
dt1
(
2 dt  dt
1
)
2
e
1 
(
dt1 xt
2 dt1 dt
)
)
2
2
= ct dt1 N yt (mt xt ;  mt )  N xt 1 ( dt1 xt ; dt1 dt ) N xt 1 ( t 1 ;  t1 )dxt
xt 1

and since
e
x = -
( x  1 )2 ( x   2 )2
2 12
e
2 22
( 1   2 )2
dx = e 2 ( 1  2 ) , ct' = ct dt1
2
(
2
)
) )
= ct' N yt (mt xt ;  mt ) N t 1 dt1 xt ; dt1 t = (dt1 dt ) 2  ( t1 ) 2
(
= ct' N xt (mt1 yt ; mt1 mt ) N xt dt t 1 ;  t = ( dt ) 2  (dt  t1
2
One can interpret it as a product of two Gaussian models : (i) The " accumulate d prior model"
where the prediction is " t = dt t 1" with vari ance ( t ) 2 and (ii) The data measuremen t
which gives prediction mt1 yt with varia nces (mt1 mt ) 2 .
© 2006 by Davi Geiger
Computer Vision
November 2006
L1.11
Kalman Filter …(continuing the proof)
P( xt | yt ) = ...
(
= ct' N xt (mt1 yt ; mt1 mt ) N xt dt t 1 ;  t = ( dt ) 2  (dt  t1 ) 2
( x  1 )2
and since e
2 12
( 
where  =
1
( x   2 )2
2 22
e
= b( 1 ,  2 ,  12 ,  22 )
)
  2  12
 12  22
2
,  = 2
   12
 2   12
2
2
2
2
(
)
( x   )2
1
2
2
e
22
= bN x (  ;  ),
)

mt1 yt ( t ) 2  dt t 1 (mt1 mt ) 2
(mt1 mt ) 2 ( t ) 2


= ct bt N xt t =
; t =

( t ) 2  (mt1 mt ) 2
( t ) 2  (mt1 mt ) 2

'
 2
2
2
 2

m
y
(

)

d



(

)
t
t
t
t
t

1
m
m
t

t
t
= N xt  t =
;

=
t

(mt  t ) 2   m2t
(mt  t ) 2   m2t

= N xt t ;  t
(
© 2006 by Davi Geiger
)
Computer Vision








November 2006
L1.12
Using Kalman Filter
The best estimate of the point x at t - 1, given the data hystory yt 1 is
(
xt 1 = t 1 = argmax xt 1 P( xt 1 | yt 1 ) = argmax xt 1 N xt 1 t 1 ;  t1
)
Note : the mean and the max values of a Gaussian distributi ons are the same
Assuming the estimate t 1 and  t1 to be known at t - 1 we estimate x at frame τ as follows
1.
( t ) 2 = ( dt ) 2  (dt  t1 ) 2
2.
( t ) =
3. t =
 2
(mt1 mt ) 2 ( t ) 2
( t ) 2  (mt1 mt ) 2
mt1 yt (mt  t ) 2  dt t 1  m2t
(mt  t ) 2   m2t
=
mt1 yt ( t ) 2  dt t 1 (mt1 mt ) 2
( t ) 2  (mt1 mt ) 2
and that is our best estimate of xt . One can interpret it as a weighted average between
the prediction from the (prior) dynamical model applied to the previou estimation
t = dt t 1 and the " correction " from the new data mt1 yt . The weights are given by the
variances of the data, (mt1 mt ) 2 , and variance of the accumulate d prior model , ( t ) 2 , respective ly.
© 2006 by Davi Geiger
Computer Vision
November 2006
L1.13
Kalman Filter (Generalization to N-D)
The generaliza tion to many dimensions (N - D) is straightfo rward, though one
has to be careful with matrix multiplica tions ...
xt

X t = [ xt1 ,..., xtN ]
1.
( t ) 2 = ( dt ) 2  (dt  t1 ) 2

t =  dt  Dt t1 Dt
2.
( t ) =
 2
3. t =
( mt ) 2 ( t ) 2 mt2

( t ) 2 mt2  ( mt ) 2
( t ) 2 mt yt  ( mt ) 2 dt t 1
( t ) mt  ( mt )
 2
2

1

2


t1 =  mt M t t M tT M t t M tT   mt
X t = t M tT M t t M tT   mt


1

Yt   mt M t t M tT   mt
sometimes the matrix Kt = t M tT M t t M tT   mt

1

1
Dt X t 1 (3)
is called Kalman Gain and (3)
X t = Kt Yt  ( I  Kt M t ) Dt X t 1 = Dt X t 1  Kt (Yt  M t Dt X t 1 )
Note : lim Kt = M t-1
 mt 0
and
lim Kt = 0
t 0
as measuremen ts become more reliable Kt weights the data residuals more, and
as the prior covariance , t , approaches 0, measuremen ts are ignored.
© 2006 by Davi Geiger
Computer Vision
November 2006
L1.14