Teaching an Agent by Playing a Multimodal Memory Game:
Challenges for Machine Learners and Human Teachers
AAAI 2009 Spring Symposium: Agents that Learn from Human Teachers
March 23-25, 2009, Stanford University
Byoung-Tak Zhang
Biointelligence Laboratory
School of Computer Science and Engineering
Cognitive Science, Brain Science, and Bioinformatics
Seoul National University, Seoul 151-744, Korea
[email protected]
http://bi.snu.ac.kr/
Talk Outline

Multimodal Memory Game (MMG)

Challenges for Machine Learners

Challenges for Human Teachers

Toward Self-teaching Cognitive Agents
2
© 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
Toward Human-Level Machine Learning:
Multimodal Memory Game (MMG)
But, I'm getting married tomorrow
Well, maybe I am...
I keep thinking about you.
But,I'm
I'mwondering
getting married
And
if we tomorrow
made a mistake giving up so fast.
Well,
maybe
I am...
Are
you
thinking
about me?
I
keep
thinking
about
But if you are, call me you.
tonight.
And I'm wondering if we made a mistake giving up so fast.
Are you thinking about me?
But if you are, call me tonight.
Text
Hint
Image
Image
Hint
Sound
Image-to-Text Generator
(I2T)
Machine Learner
Text
Text-to-Image Generator
(T2I)
© 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
Text Generation Game (from Image)
Image
Sound
Text
I2T
Learning
by Viewing
T
Game
Manager
Text
Hint
T2I
4
© 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
Image Generation Game (from Text)
Image
Sound
Hint
I2T
Learning
by Viewing
I
Game
Manager
Text
Image
T2I
5
© 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
Characteristics of MMG Game








Interactive
Multimodal
Long-lasting
Hard to learn
Scalable data
Humans as teachers
Difficulty controllable
Learning by imitation (viewing and watching)
6
© 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
Three Approaches

Learning Architecture
 Model

Learning Strategies
 Algorithms

Teaching Strategies
 Humans
7
© 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
Methods of Machine Learning

Symbolic Learning






 Version Space Learning
 Case-Based Learning

Neural (Connectionist) Learning
 Multilayer Perceptrons
 Self-Organizing Maps
 Hopfield Networks

Evolutionary Learning




Evolution Strategies
Evolutionary Programming
Genetic Algorithms
Genetic Programming
Probabilistic Learning

Bayesian Networks
Boltzmann Machines
Hidden Markov Models
Deep Belief Networks
Hypernetworks
Other Machine Learning
Methods





Reinforcement Learning
Decision Trees
Boosting Algorithms
Kernel Methods (SVM)
PCA, ICA, LDA etc.
Learning with Hypernetworks
The hypernetwo rk is defined as
x1
x2
[Zhang, DNA12-2006]
x15
H  ( X , S ,W )
X  ( x1 , x2 ,..., xI )
S   Si ,
x3
Si  X , k | S i |
i
x14
W  (W ( 2 ) , W ( 3) ,...,W ( K ) )
Training set :
x4
x13
D  {x ( n ) }1N
The energy of the hypernetwo rk
E ( x ( n ) ;W )  
1
1
w(i i2 ) x (i n ) x (i n )   w(i 3i )i x (i n ) x (i n ) x (i n )  ...

2 i1 ,i2 1 2 1 2
6 i1 ,i2 ,i3 1 2 3 1 2 3
The probabilit y distributi on
x5
x12
x6
P(x ( n ) | W ) 
1
exp[   E (x ( n ) ;W )]
Z(W )

1

1
1
exp   w(i i2 ) x (i n ) x (i n )   w(i 3i )i x (i n ) x (i n ) x (i n )  ...
12
1
2
Z(W )
6 i1 ,i2 ,i3 1 2 3 1 2 3
 2 i1 ,i2


K 1

1
exp 
w(i ik )...i x (i n ) x (i n ) ...x (i n ) ,

12 k
1
2
k
Z(W )
 k  2 c(k ) i1 ,i2 ,..., ik

x11
where the partition function is
x7
x10
x8
x9
K 1
(k )
(m) (m)
(m) 
Z(W )   exp  
 wi1i2 ...ik x i1 x i2 ...x ik  9
 k  2 c(k ) i1 ,i2 ,..., ik

x( m )
© 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
How to Learn from Image-Text Pairs
10
© 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
How to Generate Image from Text
11
© 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
Image-to-Text Translation Results
Query
Matching & Completion
Answer
I don't know
don't know what
know what happened
I don't know what happened
There's a
a kitty in
…
in my guitar case
There's a kitty in my guitar
case
Maybe there's something
there's something I
…
I get pregnant
Maybe there's something I
can do to make sure I get
pregnant
12
© 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
Text-to-Image Translation Results
Matching & Completion
Query
Answer
I don't know what happened
Take a look at this
There's a kitty in my guitar case
Maybe there's something I can
do to make sure I get pregnant
13
© 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
Further Challenges
Challenges for Machine Learners









Incremental learning
Online learning
Fast update
One-shot learning
Predictive learning
Memory capacity
Selective attention
Active learning
Context-awareness

Persistency
 Concept drift
 Multisensory integration
15
© 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
Challenges for Human Teachers

Getting feedback
 Sequencing examples
 Identifying the weak points
 Choosing problems
 Controlling parameters
 Evaluating progress
 Estimating difficulty
 Generating new queries
 Modeling the effect of
learning parameters

Catching environmental change
 Minimal interactions
 Multimodal interaction
16
© 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/
Conclusion

Multimodal memory game (MMG)
 Highly-interactive lifelong learning scenario
 Challenges current machine learning techniques

Challenges for machine learners
 More attentive, active behavior
 Rather than parameter fitting, passive adaptation

Human partners
 More active role in interacting with the agents

The future: Self-teaching cognitive agents
 Cognitive learning agents that teach themselves
= Active learning agents + cognitively-aware human teachers
 Design new queries and test their answers by interacting with humans
17
© 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/