Announcements
CS 188: Artificial Intelligence
ƒ Optional midterm
Fall 2006
ƒ On Tuesday 11/21 in class
ƒ Review session 11/19, 7-9pm, in 306 Soda
Lecture 20: Filtering and Tracking
11/07/2006
ƒ Projects
ƒ 3.2 due 11/9, but no coding
ƒ 3.3 is the coding part, due 11/15
ƒ 3.4 is dynamic battleship, due 11/27 (but try to do it before the
midterm)
ƒ Contest
ƒ Pacman contest details on web site this week
ƒ Entries due 12/3
Dan Klein – UC Berkeley
Hidden Markov Models
Example
ƒ Hidden Markov models (HMMs)
ƒ Underlying Markov chain over states X
ƒ You observe outputs (effects) E at each time step
ƒ As a Bayes’ net:
X1
X2
X3
X4
X5
E1
E2
E3
E4
E5
ƒ An HMM is given by
ƒ Initial distribution:
ƒ Transitions:
ƒ Emissions:
Real HMM Examples
ƒ Speech recognition HMMs:
ƒ Observations are acoustic signals (continuous valued)
ƒ States are specific positions in specific words (so, tens of
thousands)
ƒ Machine translation HMMs:
ƒ Observations are words (tens of thousands)
ƒ States are translation positions (dozens)
ƒ Robot tracking:
ƒ Observations are range readings (continuous)
ƒ States are positions on a map (continuous)
Conditional Independence
ƒ HMMs have two important independence properties:
ƒ Markov hidden process, future depends on past via the present
ƒ Current observation independent of all else given current state
X1
X2
X3
X4
X5
E1
E2
E3
E4
E5
ƒ Quiz: does this mean that observations are independent
given no evidence?
ƒ [No, correlated by the hidden state]
1
Filtering / Monitoring
Example: Robot Localization
Example from
Michael Pfeiffer
ƒ Filtering, or monitoring, is the task of tracking the
distribution B(X) (the belief state)
ƒ We start with B(X) in an initial setting, usually uniform
ƒ As time passes, or we get observations, we update B(X)
Prob
0
1
t=0
Sensor model: never more than 1 mistake
Motion model: may not execute action with small prob.
Example: Robot Localization
Prob
0
1
Example: Robot Localization
Prob
0
1
t=1
t=2
Example: Robot Localization
Example: Robot Localization
Prob
0
1
t=3
Prob
0
1
t=4
2
Passage of Time
Example: Robot Localization
ƒ Assume we have current belief state B(X) = P(X |
evidence to date)
ƒ Then, after one time step passes:
ƒ Or, compactly:
Prob
0
1
ƒ Basic idea: beliefs get “pushed” through the transitions
ƒ With the “B” notation, we have to be careful about what time step
the belief is about, and what evidence it includes
t=5
Example: Passage of Time
ƒ As time passes, uncertainty “accumulates”
Observation
ƒ Assume we have current belief state B(X) = P(X |
evidence to date):
ƒ Or:
T=1
T=2
T=5
ƒ Basic idea: beliefs reweighted by likelihood of evidence
ƒ Unlike passage of time, we have to renormalize
Transition model: ships usually go clockwise
Example: Observation
Example with HMM
ƒ As we get observations, beliefs get
reweighted, uncertainty “decreases”
Before observation
After observation
3
Example with HMM
Time Complexity
ƒ Every time step, we start with current P(X)
ƒ We must update for time:
ƒ We must update for observation:
S
S
S
E
E
The Forward Algorithm
ƒ Can do belief propagation exactly as in previous slides,
renormalizing each time step
ƒ In the standard forward algorithm, we actually calculate
P(X,e), without normalizing
ƒ So, linear in time steps, quadratic in number of states |X|
ƒ Of course, can do both at once, too
Particle Filtering
ƒ Sometimes |X| is too big to use
exact inference
ƒ |X| may be too big to even store B(X)
ƒ E.g. X is continuous
ƒ |X|2 may be too big to do updates
0.0
0.1
0.0
0.0
0.0
0.2
0.0
0.2
0.5
ƒ Solution: approximate inference
ƒ Track samples of X, not all values
ƒ Time per step is linear in the number
of samples
ƒ But: number needed may be large
ƒ This is how robot localization
works in practice
Particle Filtering: Time
ƒ Each particle is moved by sampling its
next position from the transition model
Particle Filtering: Observation
ƒ Slightly trickier:
ƒ We don’t sample the observation, we fix it
ƒ This is similar to likelihood weighting, so
we downweight our samples based on
the evidence
ƒ This is like prior sampling – samples are
their own weights
ƒ Here, most samples move clockwise, but
some move in another direction or stay in
place
ƒ This captures the passage of time
ƒ If we have enough samples, close to the
exact values before and after (consistent)
ƒ Note that, as before, the probabilities
don’t sum to one, since most have been
downweighted (they sum to an
approximation of P(e))
4
Particle Filtering: Resampling
ƒ Rather than tracking weighted
samples, we resample
ƒ N times, we choose from our weighted
sample distribution (i.e. draw with
replacement)
ƒ This is equivalent to renormalizing the
distribution
Robot Localization
ƒ In robot localization:
ƒ We know the map, but not the robot’s position
ƒ Observations may be vectors of range finder readings
ƒ State space and readings are typically continuous (works
basically like a very fine grid) and so we cannot store B(X)
ƒ Particle filtering is a main technique
ƒ [DEMOS]
ƒ Now the update is complete for this
time step, continue with the next one
SLAM
ƒ SLAM = Simultaneous Localization And Mapping
ƒ We do not know the map or our location
ƒ Our belief state is over maps and positions!
ƒ Main techniques: Kalman filtering (Gaussian HMMs) and particle
methods
ƒ [DEMOS]
DP-SLAM, Ron Parr
5