Neural Robot Control
Cornelius Weber
Hybrid Intelligent Systems
University of Sunderland
Talk at Nottingham Trent University, 8th December 2004
on the occasion of returning the MI competition trophy
Collaborators: Mark Elshaw, Alex Zochios, Chris Rowan and Stefan Wermter
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Contents
• Visual cortex & reinforcement network for docking
• Cortex self-imitation network for docking
• Imitation networks for multiple actions:
1-stage/2-stage hierarchical network
• Outlook
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Contents
• Visual cortex & reinforcement network for docking
• Cortex self-imitation network for docking
• Imitation networks for multiple actions:
1-stage/2-stage hierarchical network
• Outlook
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Example Task: Docking
Docking Architecture
Information Flow
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Docking Architecture
Training (1/3)
unsupervised training
generative model
sparse distributed coding
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
V1 Receptive Fields
(training result)
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Comparison of
Response Characteristics
linear
sparse
competitive
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
winner
Attractor Network:
Competition via Relaxation
weight profile
activation profile
activation update
y(t+1) = f (Wlat y(t))
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Docking Architecture
Training (2/3)
supervised training,
attractor network
for pattern completion
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Docking Architecture
Visual System
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Docking Architecture
Training (3/3)
reinforcement training
actor-critic model
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Contents
• Visual cortex & reinforcement network for docking
• Cortex self-imitation network for docking
• Imitation networks for multiple actions:
1-stage/2-stage hierarchical network
• Outlook
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Mirror Neuron
Docking Architecture
Information Flow
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Mirror Neuron
Docking Architecture
Training
unsupervised training
generative model
distributed coding
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Mirror Neuron
Docking Architecture
Training
supervised training,
attractor network
for prediction
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Mirror Neuron Self-Imitation
Docking Architecture
Information Flow
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Basal Ganglia vs. Motor Cortex
Basal ganglia units are active during early task acquisition
but not at a later stage (rat T maze decision task).
early:
late:
Jog et al. (1999)
Science, 286, 1158-61
Basal Ganglia ≙ state space?
Motor cortex might take over BG function via self-imitation.
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Docking via Mirror Neurons
Contents
• Visual cortex & reinforcement network for docking
• Cortex self-imitation network for docking
• Imitation networks for multiple actions:
1-stage/2-stage hierarchical network
• Outlook
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Simulated Robot Environment
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Imitation Model Choice
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Areas of Motor- and Language
Representations
motor units
forward
individual unit’s
receptive fields
in hidden area
back
left
right
language units
‘go’
‘pick’
‘lift’
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
all
Areas of Task-Specific Activations
‘go’
‘pick’
‘lift’
Production:
Recognition:
Activations agree with the Somatotopy-of-Action-Words Model.
‘go’
‘pick’
‘lift’
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Language Instructed
Imitative Behaviour
‘go’
‘pick’
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
‘lift’
Imitation Model Choice
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Neuron’s Receptive Fields in HM Area
forward
backward
left
motor units
4 SOM-area units
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
right
Conclusion for Imitation Network
A neural network as a generative model for sensory stimuli
• generates interactive action sequences
• allows for context dependent interactive action sequences
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Contents
• Visual cortex & reinforcement network for docking
• Cortex self-imitation network for docking
• Imitation networks for multiple actions:
1-stage/2-stage hierarchical network
• Outlook
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Outlook (1/2): Object-Background
Separation for Enhanced Object
Learning
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Outlook (2/2): Docking Range
Extension by Neural Coordinate
Transformations
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme