Skill Learning in Telerobotics
using Hidden Markov Model
by
Gary Holness
Skill Learning
•
•
•
•
•
•
Human performance stochastic
Repeated trials of same task different
Something about the task
Uncovering “nature” of data
Most likely performance rejecting noise
Stochastic methods perfect
Why HMM?
• Double stochastic process
- observable process (motion data)
- hidden process (mental state/intent)
• Parametric model with incremental update
• Observations as “symbols”
• Unifying framework for perception and
action
• Likely human performance from measured
activity
Software architecture
•Pre-processing to extract observation “symbols”
•Algorithm on real-data or simulation
SM2 configuration
•7 DOF
•6 DOF free flying hand controller provide control
input
HMM experiment
• Orbit replacement unit (ORU)
• Nut driver in gripper
• Model action under teleoperator control as
HMM
• Position/trajectory in Cartesian space
• Position/trajectory in Joint space
• Velocity/trajectory in Cartesian space
HMM Experiment cont’d
• Observable symbols: trajectory
• States: subtasks
• Special case on states: time index
increases left-to-right (Bakis model)
• Fewer parameters than ergodic HMM
• 100 trajectories recorded and scored
Position trajectory in Cartesian
space
•Forward algorithm used for scoring: P(O|)
•Trajectory 60,90 better than average
•Score increase w.r.t iteration  model improvement
Position trajectory in joint space
•Iteration 77 best score
•Velocity in Cartesian, iteration 49 is best
What was good?
•Use mathematical framework for which many
statistical tools already exist
•Integrating framework
•Software engineering
•Rigorous: make sound statements about
experiment (not just “it works therefore its proven”)
•Clearly laid out design
What was bad?
• Poor initialization: Baum-Welch can converge to
local maxima
• (not problem of experiment)
• Simplification in left-right HMM
• (understandable why they did it)
• Independence assumption among r.v. in
R-dimensional observation vector
• (joints non-independent)
Why do I care?
• Learning in HMM for ergodic case
• Choose right “features” as observation
symbols
• Make use of 80-years of statistical tools
• Beautifully engineered research artifacts
• Framework for skills transfer and re-use
• Transition among HMMs still an HMM:
– lends itself to hierarchical descriptions
Conclusion
Representation, representation, representation