Rao-Blackwellised Particle Filtering for
Dynamic Bayesian Network
Arnaud Doucet
Nando de Freitas
Kevin Murphy
Stuart Russell
Introduction
 Sampling in high dimension
 Model has tractable substructure
 Analytically marginalized out, conditional on certain other nodes
being imputed
 Using Kalman filter, HMM filter, junction tree algorithm for general
DBNs
 Reduce size of the space over we need to sample
 Rao-Blackwellisation
 marginalize out some of the variables
Problem Formulation
 general state space model/DBN with hidden variables tz
and observed variables ty.
 tz is a Markov process of initial distribution) 0z (p
 Transition equation ) 1 tz | tz (p
 Observations } ty ,, 1y { 
y
t:1
 Estimate ) t:1y | t:0z (p
 recursion
) 1 tz | tz (p ) tz | ty (p
) 1 t:1y | 1 t:0z (p  ) t:1y | t:0z (p
) 1 t:1y | ty (p
 not analytically, numerical approximation scheme
Cont’d
 Divide hidden variables tz into two groups
) 1 tr | tr(p ) 1 tx , t;1 tr | tx (p  ) 1 tz | tz (p
 Conditional posterior distribution
is analytically tractable. ) 1 tr | tr( p ) 1 tx , t;1 tr |
r and tx
t
x (p  ) 1 tz | tz (p
 Focus on estimating ) t:1y | t:0r(p (reduced dimension)
 Decomposition of posterior from chain rule
) t:1y | t:0r(p ) t:0r , t:1y | t:0x (p  ) t:1y | t:0x , t:0r(p
t
 Marginal distribution
) 1 t:1y | 1 t:0r(p ) 1 tr | tr(p ) t:0r ,1 t:1y | ty (p
 ) t:1y | t:0r(p
) 1 t:1y | ty (p
Importance sampling and RAO-Blackwellisation
 Sample N i.i.d. random samples(particles)
} N ,,1  i ;) )ti:0( x , ) it:(0r ({ according to p(r0:t , x0:t | y1:t )
 Empirical estimate
1
p N (r0:t , x0:t | y1:t ) 
N
N

i 1
( r0(:ti ) , x0(:it) )
(dr0:t dx0:t )
 Expected value of any function of hidden variables ) t f ( I
xd t:0rd) t:1y | t:0x , t:0r( N p ) t:0x , t:0r( t f
t:0
)
) i(
x,
t:0
) i(
N
r( t f
t:0

1 i
) f(
1

N
t
N
I
Cont’d
 Strong law of large numbers
 ) t f ( N I converges almost surely towards ) t f ( N I as   N
 Central limit theorem
) t2f  ,0(
 N
 ]) t f ( I  ) t f ( N I [ N
Importance Sampling
 impossible to sample efficiently from target
 Importance distribution q
 Easy to sample
 p>0 implies q>0
)) t:0x , t:0r(w ) t:0x , t:0r( t f ( ) t:1y | t:0x , t:0r ( qE
 ) tf ( I
)) t:0x , t:0r(w ( ) t:1y | t:0x , t:0r ( qE
) t:1y | t:0x , t:0r(p
 ) t:0x , t:0r(w erehw ,
) t:1y | t:0x , t:0r(q
))
) i(
t:0x ,
) i(
))
t:0r (w )
) i(
t:0x ,
) i(
) i(
t:0x ,
) i(
t:0r ( t f ( 1 i
N
t:0r (w ( 1 i
N
) t f ( N1Aˆ
1ˆ


)
f
(
t
NI
1ˆ
) t f ( NB
N
~ 
) t:0x , t:0r( f w

) i(
) i(
) i(
t t:0
1 i
Cont’d
 The case where one can marginalize out t:0x analytically,
) tf ( I
propose alternative estimate for
 Alternative importance sampling estimate of ) t f ( I
)
) i(
t:0r (w )
t:0x ,
) i(
)
) i(
t:0r ( t f ( ) ) i (
E
x ( p 1 i
N
t:0r , t:1y | t:0
t:0r (w ( 1 i
N
) t:1y | t:0r(p
 ) t:0r(w
) t:1y | t:0r(q
) t f ( N2Aˆ
2ˆ


)
f
(
t
NI
2ˆ
) t f ( NB
erehw
xd) t:1y | t:0x , t:0r( q   ) t:1y | t:0r(q
t:0
 To reach a given precision, ) tf ( N2ˆI will require a reduced
number N of samples over
) tf ( N1ˆI
Rao-Blackwellised particle filters
 Sequential importance sampling step
 For i=1,…,N sample: ) t:1y , 1) it:(0r | tr( q ~ ) ) i ( tˆr(
and set: ) 1) it:(0ˆr |
) i(
ˆr(q  ) ) it:(0ˆr(
t
 For i=1,…,N evaluate importance weights up to a normalizing constant:
) t:1y | ) it:(0ˆr(p
) i(

tw
) i(
) i(
) i(
) 1 t:1y | 1 t:0ˆr(p ) t:1y , 1 t:0r | tˆr(q
 For i=1,…,N normalize importance weights:
N
~
] w [ ) i(tw  ) i(tw
1 ) j (
t
1 j
 Selection step
) i(
~
 multiply/suppress samples ) t:0ˆr ( with high/low importance weights ) i (tw
) i(
to obtain random samples ) t:0~r ( approximately distributed ) t:1y | ) it(:0~r(p
 MCMC step
 Apply a markov transition kernel with invariant distribution given by
) i(
to obtain ) t:1y | t:0r(p to obtain ) ) it:(0r (
Robot localization and MAP building
 Problem of concurrent localization and map learning
 Location } L N ,...,1{  tL
 Color of each grid cell L N ,...,1  i ,} CN ,...,1{ )i( t M
 Observation )) tL ( t M ( f t Y
 Basic idea of algorithm
 Sample t:1L with PF
 Marginalize out )i( t M since they are conditionally
independent given t:1L