Biorobotics Workshop
Cognitive Developmental
Robotics (CDR):
What's CDR and how?
Minoru Asada
Graduate School of Engineering, Osaka University
JST ERATO Asada Synergistic Intelligence Project
13-15 December, 2010
Multipurpose Hall, Bibliotheca Alexandrina
Outline of my talk
1. Cognitive Developmental Robotics
1. Human development
2. What’s cognitive developmental robotics?
3. Fetus/neonate development simulation
4. Vocal imitation
2. Is paradigm shift possible?
1. Mirror Neuron System Connects Physical
Embodiment and Social Entrainment
2. From self/other identification to self/other
separation
What’s going on in the womb (1)
Emergence of fetal movements and sense (Brain figures on the top are
from Figure 22.5 in [Purves et al., 08], emergence of movements is from
Figure 1 in [Vries et al., 84], and fetal senses are from
Infant development and learning targets
M / behaviors / learning targets
--------------------------------------------------------------------------------------------------------------------------------------------------
5 / hand regard / forward and inverse models of the
hand
6 / finger the other’s face / integration of visuo-tactile
sensation of the face
7 / drop objects and observe the result / causality and
permanency of objects
8 / hit objects / dynamics model of objects
9 / drum or bring a cup to mouth / tool use
10 / imitate movements / imitation of unseen movements
11 / grasp and carry objects to others / action
recognition and generation, cooperation
Facets of development
Social interaction
Self-organization
・ DoF and motion
・ self-exploration
・ categorization
・ sensori-motor
integration
Set of constraints
symbol
index
icon
Incremental
process
Neuromodulation, value,
and synaptic plasticity
Dynamic interaction
External world
(other agents, objects)
Based on [Lungarella_Metta_Pfeifer_Sandini03]
Outline of my talk
1. Cognitive Developmental Robotics
1. Human development
2. What’s cognitive developmental robotics?
3. Fetus/neonate development simulation
4. Vocal imitation
2. Is paradigm shift possible?
1. Mirror Neuron System Connects Physical
Embodiment and Social Entrainment
2. From self/other identification to self/other
separation
What is Cognitive Developmental Robotics?
• Cognitive developmental robotics [1] (in
short, CDR) aims at understanding human
cognitive developmental process by
synthetic or constructive approaches, and
its core idea is "physical embodiment”
that enables information structuring
through interactions with the environment,
including other agents.
[1] M. Asada et al., "Cognitive developmental robotics:
a survey," IEEE Transactions on Autonomous Mental
Development, 1(1):12–34, 2009.
Design issues of CDR
• From a viewpoint of artifact design:
１． What kinds of capabilities or structures should
be embedded? → the design of selfdeveloping structures inside the robot's brain
２． How to set up the environment so that the
robots embedded therein can gradually adapt
themselves to more complex tasks in more
dynamic situations? → the environmental
design
３． What’s the temporal development structure?
 Discussion on the issues of underlying
neuromechanism and behavioral
verification are needed. [Asada_et_al., 09]
Approaches of CDR
A: construction of computational model of cognitive
development
1)
2)
3)
hypothesis generation: proposal of a computational
model or hypothesis based on knowledge from existing
disciplines
computer simulation: simulation of the processes
difficult to implement with real robots such as physical
body growth
hypothesis verification with real agents (humans,
animals, and robots), then go to 1)
B: offer new means or data to know human
developmental process → mutual feedback with A
1)
2)
3)
measurement of brain activity by imaging methods
verification using human subjects or animal ones
providing the robot as a reliable reproduction tool in
(psychological) experiments
A view of social brain
prMFC
arMFC
oMFC
Orbital region of the MFC
(oMFC) represents and
updates the value of
possible future outcomes.
Posterior region of the rostral MFC
(prMFC) is involved in representing
and continuously updating the
value of possible future actions in
order to regulate behavior.
Anterior region of the rostral MFC
(arMFC) comprise roughly three
different categories: self-knowledge,
person knowledge and mentalizing.
Amodio, D. M. and Frith, C. D. (2006) Meeting of minds: the medial
frontal cortex and social cognition. Nat. Rev. Neurosci., 7, 268-277.
Synergistic Intelligence Project
Understanding
Emergence of
Human intelligence
Design for
symbiotic
humanoids
Perso-SI
Yasuo
Kuniyoshi’s
Physio-SI
group
Motor emergence through
Koh
Hosoda’s
Dynamic Interaction with Synergistic
group
environment
based on
Intelligence
Artificial Muscles
Synthetic understanding
Cognitive development
by Imitation Science
Socio-SI
SI-mechanism
Toshio Inui’s
group
Hiroshi
Ishiguro’s
group
Understanding and
Realization of
communication with
Androids
Verification of Computational
Models by brain functional imaging
and human/animal experiments
SI-group structure
Physio-SI
Perso-SI
SI-Mechanism
Socio-SI
Amodio, D. M. and Frith, C. D. (2006) Meeting of minds: the medial
frontal cortex and social cognition. Nat. Rev. Neurosci., 7, 268-277.
Towards the principle of cognitive development
• Rethinking "embodiment" or more correctly
"Shintaisei ( 身体性 )" explained as event that
the body specifies the interaction between
active agent and environment, and its contents.
That is also the infrastructure to form the
cognition and behaviors by providing the
structure to the interaction.
• Representation and architecture
– Individual development: more neuroscientific
representation and more computational
Body representation &
Dynamic Motion
walking
running
jumping
Spatial
perception
allocentric
co-system
whole body
motion & assist
object permanency
visuo-spatial
rep.
standing
with support,
early
walking
crawling
rolling over
holding during
neonate & infant
functional
development
ATR 知能ロボティクス研究所開発
egocentric co-system
perception
sensory
motor mapping
voluntary
motions reflection
To intension
understanding
Vowel imitation
higher
cognition
Joint attention
body scheme & tool use
Fetus
Simulation
Sympathy development
Responsive gaze
Future image: development
of communication
From emergence of social behavior through interactions with caregiver to development of communication
Physio-Synergistic Intelligence
• From dynamic connection
between body structure and
environment with artificial
muscles to understanding
emergence of human
intelligence
– Antagonistic pair of pneumatic
actuators enables biped robots to
walk on unstructured terrain and
also to run and jump.
– Seamless transition between
dynamic motions
[Hosoda G]
From motor development to cognitive one
[Prof. Koh Hosoda G]
Outline of my talk
1. Cognitive Developmental Robotics
1. Human development
2. What’s cognitive developmental robotics?
3. Fetus/neonate development simulation
4. Vocal imitation
2. Is paradigm shift possible?
1. Mirror Neuron System Connects Physical
Embodiment and Social Entrainment
2. From self/other identification to self/other
separation
Question?
Infants’ imitation ＝　AIM-model
( Active Intermodal Matching Model )
（ A.N.Meltzoff and M.K.Moore 1977 ）
how do infants code the perceived
and produced human acts within a
common framework which enables
infants to detect equivalences
between the features of the different
modality ??
Organ Identification
When the robot touches its own body….
→ Spatial and temporal integration
of each sensory information
Visual sense
Tactile sense
Proprioceptive sense
Fetal Sensori-motor
Development
[Kuniyoshi and Sangawa 06]
Yasuo Kuniyoshi and Shinji Sangawa, Early Motor
Development from Partially Ordered Neural-Body Dynamics -Experiments with A Cortico-Spinal-Musculo-Skeletal Model,
Biological Cybernetics, vol. 95, no. 6, pp. 589-605, Dec., 2006.
Fetal Brain Development (1)
[Sangawa & Kuniyoshi 06]
Body and brain
Fetal Brain Development (2)
[Kuniyoshi et al., 08]
see new video
Fetal Brain Development (3)
[Sangawa & Kuniyoshi 06]
Fetal Brain Development (4)
[Sangawa & Kuniyoshi 06]
Fetal Brain Development (5)
[Kinjo & Kuniyoshi 09]
Visuo-Tactile Face
Representation through Selfinduced Motor Activity
[Fuke, Ogino, Asada 07]
Fuke Sawa, Masaki Ogino, and Minoru Asada. Body
image constructed from motor and tactle images with
visual information. International Journal of Humanoid
Robotics, Vol.4, No.2, pp.347-364, 2007.
Simulation
Arm : 5 degrees of
freedom
Tactile units : 21×21
Hand cells : 36
<The robot can detect…>
1. Angles of its arm
2. Position of the hands
in the camera image
of the ？
3. Arrangement
The ID of the nearest
units
？ cell oftactile
the hand
A neural network model for
image formation of the face
through double-touch without
visual information
[Song, Inui, Takemura 09]
Wei Song, Toshio Inui, and Naohiro Takemura.
2nd International Conference on Cognitive
Neurodynamics 2009(ICON'09), 2009.
Schematic diagram of the model
tactile
sensor
of face
tactile
sensor
of finger
shape coding
with
Gabor filters
onion-like
mapping
SI (face)
SOM with
Gaussian-like
local connections
VIP
location
pathway
shape
pathway
SOM with
Gaussian-like
local connections
SI (finger)
Hebbian
connections
visual
area
[Song, Inui, Takemura 09]
Location pathway
• When a part of a fetus’s face is
touched by his/her hand, the
tactile receptors on the facial skin
are activated. Topological
information about touched area
on the face is transmitted to, and
represented in, the VIP (ventral
intraparietal area) via an onionlike map in the primary
somatosensory cortex (SI)
(Moulton et al., 2009).
• Connections between SI and VIP
as well as VIP and visual cortex
finger area
are learned with the selforganizing map (SOM).
S1
map
corresponding
representation of
onion-like map
[Song, Inui, Takemura 09]
Shape pathway
• Concavo-convex information of the
face within the finger surface is
encoded by Gabor filters in SI
according to the physiological
findings (Bensmaia et al., 2008).
There are sixteen kinds of filters
with different orientation selectivity
(see right fig.) The population of
finger SI units codes shape
information for each location within
this filter bank.
[Song, Inui, Takemura 09]
Results and Discussion
Log(r+1)
• After SOM, VIP units acquired
facial somatotopic mapping
(Fig. 5) similar to that seen in
human fMRI data (Sereno and
Huang, 2006).
• Right figure shows the acquired
facial image in the visual
cortex. This image appears
when all the SI (finger) units
are activated.  it is a potential
image formed by a number of
finger tactile inputs. Retinocortical mapping is well known
as a log-polar transformation of
the retinal image (e.g.,
Shwartz, 1977)
A
B
C
q
Learning result
[Song, Inui, Takemura 09]
Issues:
• Myowa and Takeshita (2006): mouth opening
reflection when fetuses (25w) hear their
mothers’ voice.
• Oyabu (2008): Lip motion is exceptional while
body motion and sensing function are
compatible two months after the birth.
• Go and Konishi (2008): Neural infrastructure
for neonatal imitation is under sub-cortical
structure.
 To what extent and how much innate?
Body representation &
Dynamic Motion
walking
running
jumping
Spatial
perception
allocentric
co-system
whole body
motion & assist
object permanency
visuo-spatial
rep.
standing
with support,
early
walking
crawling
rolling over
holding during
neonate & infant
functional
development
ATR 知能ロボティクス研究所開発
egocentric co-system
perception
sensory
motor mapping
voluntary
motions reflection
To intension
understanding
Vowel imitation
higher
cognition
Joint attention
body scheme & tool use
Fetus
Simulation
Sympathy development
Responsive gaze
Future image: development
of communication
From emergence of social behavior through interactions with caregiver to development of communication
Outline of my talk
1. Cognitive Developmental Robotics
1. Human development
2. What’s cognitive developmental robotics?
3. Fetus/neonate development simulation
4. Vocal imitation
2. Is paradigm shift possible?
1. Mirror Neuron System Connects Physical
Embodiment and Social Entrainment
2. From self/other identification to self/other
separation
Visual attention by saliency
leads cross-modal body
representation
[Hikita, Fuke, Ogino, Asada 07]
Mai Hikita, Fuke Sawa, Masaki Ogino, and Minoru
Asada. IEEE 7th International Conference on
Development and Learning, 2008.
Dynamic body representation (1)
•
Bimodal neuron ： body image extension by
tool use. [Iriki_et_al,96,01]
Visual
Proprioceptive
Tactile
•
•
Body scheme → unconscious, dynamic
process of body control.
Body image → conscious representation of
[Stamenov, 2005]
self body.
Dynamic body representation (2)
Tool-use by the Japanese macaque [Iriki et al. 1996]
The activities of bimodal neurons
change after training of tool-use.
Before tool-use
After tool-use
The receptive field was extended to the tool.
? How such representation is acquired ?
Dynamic body representation (3)
Results: Experiment with a real robot
The connection weights
CB2 ( Minato et al. 2007)
The activation of the integration map
With hand
With a tool
Outline of my talk
1. Robotics and RoboCup Initiative
1. What’s RoboCup?
2. Video Clips
2. Cognitive Developmental Robotics
1. Human development
2. What’s cognitive developmental robotics?
3. Fetus/neonate development simulation
4. Physical interaction
5. Vocal imitation
3. RoboCity CoRE
Body representation &
Dynamic Motion
walking
running
jumping
Spatial
perception
allocentric
co-system
whole body
motion & assist
object permanency
visuo-spatial
rep.
standing
with support,
early
walking
crawling
rolling over
holding during
neonate & infant
functional
development
ATR 知能ロボティクス研究所開発
egocentric co-system
perception
sensory
motor mapping
voluntary
motions reflection
To intension
understanding
Vowel imitation
higher
cognition
Joint attention
body scheme & tool use
Fetus
Simulation
Sympathy development
Responsive gaze
Future image: development
of communication
From emergence of social behavior through interactions with caregiver to development of communication
Analysis of Physical HumanRobot Interaction for Motor
Learning with Physical Help
Shuhei Ikemoto, Takashi Minato, and
Hiroshi Ishiguro
2008 IEEE-RAS International Conference on
Humanoid Robots, 2008
Background
• Flexibility of robot joints
– simple control system
– physical interaction
• How to evaluate physical
interaction?
– the result (success or failure)
– subjectively good interaction
for human.
Which parameters lead
subjectively good physical interaction
Human assists robot uprising
Procedure
1. The human pulls up the robot’s hands.
2. The Robot reacts and shrinks back.
3. The Robot
kicks floor.
4. The human
assists robot
uprising
Human assists robot uprising
CB2 just switches own desired postures only
twice.
Cross-Correlation Value
posture change norm
• Time-window:
approx. 1.5sec
• lag:
-1.5 sec ～ 1.5sec
Human data
are shifted
Robot data
are shifted
• Robot moves after human
started moving
• The profiles of both norm
data become similar each
other
time
time
cross-correlation
value
lag
Movies
Expert
Beginner (smooth)
Beginner (nonsmooth)
Beginner (failed)
Successful Result
Expert
Beginner (success)
An expert can start an interaction from more
synchronized state because he can
predict robot’s motion
Failed Result
Beginner (failed)
Beginner (nonsmooth)
An interaction successfully done needs
continuous positive correlation,
and the lag should be smaller
Results
• An expert starts an interaction from more
synchronized states.
• An expert is supposed to predict robot’s movements.
• An interaction successfully done needs continuous
positive correlation, and the lag should be smaller.
• In failure interaction, that feature doesn’t appear in
the correlation space.
Physical Interaction Learning:
Behavior Adaptation in
Cooperative Human-Robot Tasks
Involving Physical Contact
[IEEE RO-MAN 2009, 2009]
Physical human-robot interaction(1)
• We are focusing on physical human-robot
interaction
• In human-human interaction, there are
several kinds of physical interaction
– Physical interaction between a mother and a baby
– The baby can stand up despite she still cannot
stand up by herself
Physical human-robot interaction(2)
• Main technical difficulties in this challenge
are:
– How to ensure that safety is always guaranteed
– How to assure that the robot reacts appropriately
to the force applied by the human instructor
– How to improve the robot’s behavior using a
machine learning algorithm in the PHRI
Flexible joints
are necessary!
Learning to switch target pauses
• Robot motions and
changes cause human’s
motions and changes.
Vice versa?
Switching rule
PHRI
modulation
s2
DB
success
s1
:Posture space
Human-in-the-loop Approach (1)
• In order to refine
the robot’s
motion, improving
the human’s
motion is required
The human bears a part of the
dynamics as an actor of the interaction
The human bears a part of the
learning system as a critic of the
interaction
– The purpose is
to improve not
only the robot’s
motion but also
the human’s
motion, in other
words, to
improve the
interaction
Human-in-the-loop Approach (2)
Which is the target pause during
motions
52D Pause Space
Reduction by PCA
1
・・
・
52 → 2
Colors change when the target
pauses change
2
Gaussian Mixture Model:
EM
Algorithm
where
These trajectories can be realized
with human’s physical assistance
Results (1)
Before
learning
After
learning
Before
learning
After
learning
Results (2)
Mapping from Facial
Expression to Internal State
based on Intuitive Parenting
[Watanabe, Ogino, Asada 07]
Ayako Watanabe, Masaki Ogino, and Minoru Asada.
Mapping Facial Expression to Internal States Based on
Intuitive Parenting. Journal of Robotics and
Mechatronics, Vol.19, No.3, pp.315--323, 2007
Introduction (I)
• Future robots entering into our daily life are
expected to be endowed with the ability of
sympathy.
• Understanding the developing process to
acquire the sympathetic ability is one of the
interesting mysteries in developmental
psychology.
– imitation during motor babbling [Breazeal
et al., 2005] ?
Introduction (II)
• Parents often mimic and emphasize their facial
expression of certain emotional stated presumed
from their children’s facial expression
[ Papousek and Papousek. 1987 ]
Intuitive Parenting
Introduction (III)
• Intuitive parenting is thought to help children to
develop their sympathetic ability [Gergely and
Watson, 1999]
Visual space
for face
Internal state
relation
stimulus
• Modeling a developmental process of
sympathy through intuitive parenting
System Overview
Internal state (I)
• The internal state of the robot consists of two
kinds of variables, arousal-sleep axis and
pleasure-displeasure axis. Arousal
[Russel1980 ]
Surprise
Afraid
Frustrated
Excitement
Happy
Stress
Displeasure
Sad
Natural
Depression
Bored
Elated
Sleepy
Sleep
Pleasure
Calm Content
Relaxed
Facial expression
• The robot changes its facial expression according
to the internal state, S.
• Two structural variables “curving and releasing”
and “inclination” are related to the internal state.
[Yamada, 1993]
Learning (I)
• Facial expression of the caregiver is
categorized by self organizing map (SOM)
• Mapping between face SOM and internal state
is learned by Hebbian.
∆w =α
f ij (V)gkl (S)
kl
ij
f ij (V )
Face SOM
g kl (S)
Internal space
f ij ( V ) = e
−ν |V −V ij | 2
g kl ( S ) = e
−ρ |S − S kl |2
Experimental Setup
Display
Camera
Microphone (sound)
Keyboard (touch sensor)
Learning experiment
４： human facial expression
Human facialtoexpression
corresponding
robot state
and expression
１： External stimulas
Robot facial
expression
３： facial expression
SOM of human
facial expression
Hebbian Learninfg
a
２： change of internal state
by the external stimulas
Internal state space
p
Experimental result
• The result of mapping between caregiver’s
facial expression and self internal state
• The caregiver’s
faces are
associated with
the internal
state, and the
distribution is
almost same as
the Russel’s two
dimensional
emotional model.
Experimental result
Sympathetic communication after learning
• Sympathetic
facial expression
is possible by
introducing the
effect of the
correspondence
of internal state
and other’s facial
expression.
Discussion (1)
• In brain, which region is responsible for
sympathy?  [Singer et al. 2004]
• Using fMRI, brain activity while volunteers
experienced a painful stimulus and compared it
to that elicited when they observed a signal
indicating that their loved one. Bilateral anterior
insula (AI), rostral anterior cingulate cortex
(ACC), brainstem, and
cerebellum were activated
when subjects received pain
and also by a signal that a
Discussion (2)
1. The pain area is stimulated first using selfsensor input.
2. Another sensor input of a different modality
(e.g. visual information from a caregiver) is
partly projected on to the same area.
3. Intuitive parenting strengthens the connection
between the subject’s feeling based on the
pain sensation of self and the feeling toward
that of others based on visual information.
4. Sensor input from different modalities
suppress each other, finally forming a
Discussion (3)
• It seems that the emotional state of self and
others are represented in the different area of
the same region. What switches self and
others?
Vowel Acquisition by Maternal
Imitation
[Yoshikawa et al., 03] etc.
Yuichiro Yoshikawa, Minoru Asada, Koh Hosoda,
and Junpei Koga. A Constructivist approach to
infants' vowel acquisition through mother-infant
interaction. Connection Science, Vol.15, No.4,
pp.245--258, 2003.
Introduction
•
Vowel Imitation between Agents with Different
Articulation Parameters by Parrot-like
Teaching
– Infants seem to acquire (imitate)
phonemes:
• without any explicit knowledge about the
relationship between their sensorimotor
system and phonemes, and
• without a capability to reproduce the
adult’s sound as they are.
• How can robots do that?
Human Vocalization [Deacon 98]
• Vocalization  the
interaction of the oral and
respiratory tracts 
special association with
midbrain systems.
• To organize vocalization 
coordinated activation of
the cluster of motor
neurons that control the
muscle of breathing, the
tension of the larynx, and
the movement of the oral
and facial muscles. the
motor neurons controlling
all of these are located in
the upper brain stem.
Human Vocalization [Deacon 98]
• Two evolutionary shifts producing increasing
cortical control over motor output from brain stem
articulatory and vocal systems. These shifts were
produced by an increase in the proportions of the
cerebral cortex in comparison to these brain stem
structures.
Cortex-brain stem projection
Non-primate Primate
Human
A constructivist approach
• The purpose To build a robot that
acquires the vowels of a human caregiver
• Design issues:
– What kind of mechanism should be
embedded?
– What should be the behavior of the
caregiver?
Robot’s
mechanism?
Caregiver’s
behavior?
Observations in human infants
• Infant’s speech-like cooing tends to make its
mother utter [Masataka and Bloom’94].
• Maternal imitation of infant's cooing (i.e.,
parrot-like vocalization) increases vocalization
rates of a three-month-infant [Pelaez-Noqueras
’96] .
infant cooing
maternal
imitation
Conjectures
 It reinforces infants’
speech-like cooing.
 It helps to find the
correspondence
between cooing and
phonemes.
The robot
Output sound
Sound
source
Silicon tube
Microphone
5
Motors
deforming
Formant extractor
Articulation
vector
PC
F1
F2 F3
F4
…..
Frequency [Hz]
Formant
vector
What’s Formant Space?
• Resonant frequency
changes depending
on the shape of
vocal tract.
• Vocal feature for
vowel
discrimination.
• Non-human
primates and birds
utilize as perceptual
Formant distribution of Japanese
average
cues
[Fitchfemale
2000]
A model of interaction
Randomly
The caregiver
cooing
motors
Parrot-like
teaching
The robot
Random
articulator
Articulation
layer
Auditory
layer
microphone
Learning
module
Learning mechanism
• Clustering the articulation parameters and the
formant vectors by the SOM algorithm.
• Connections are updated based on Hebbian
learning.
Articulation
random
articulator
articulation layer
vector
cooing
SOM
Parrot-like
teaching
trigering
input
Formant
extractor
Hebbian
learning
SOM
Formant
vector
auditory layer
Acquired vowels
• The acquired vowels can be interpreted as
Japanese vowels.
/a/,
/i/,
/
u/,
/e/
Result: how it acquire vowels?
/i/
PC 2
/e/
/a/
/u/
PC 1
The articulation vectors corresponding to
the variation of the caregiver’s vowels
•
The vowel /o/ is not acquired due to the difference in
shape of vocal tracts.
• There is “arbitrariness” in correspondence.
Introducing subjective criteria
• A “subjective” criterion: more facile
articulation is better.
– less torque ( ctrq )
– less deformation change ( cidc )
Articulation layer
Basic Hebbian rule
∆ wij ∝ aif a mj
Modified Hebbian rule
Auditory layer
∆ wij ∝ η (ctrq , cidc )ai a mj
f
Result: effect of subjective criteria
Comparing the distribution of the articulation
vectors corresponding to the variation of the
caregiver’s vowels
/i/
PC 2
PC 2
/i/
/e/
/a/
/u/
PC 1
Basic Hebbian rule
/e/
/u/
/a/
PC 1
Subjective criteria
Subjective criteria can reduce arbitrariness.
Lip shape imitation
[Miura et al, 2006]
Visual imitation , too!
Caregiver’s Automirroring and Infant’s
Articulatory Development
Enable Vowel Sharing
i e
u
a
o
i e
u a
o
Hisashi Ishihara*,
Yuichiro Yoshikawa**, and Minoru Asada**,*
* Grad. Sch. of Eng., Osaka University, Japan
** Asada Synergistic Intelligence Project, ERATO, JST
[ICDL08,EpiRob09]
Infant’s Vowel Development
• Sharing process of perceptual & behavioral primitives
between a caregiver and her infant across their
different bodies
– Physical quantities of their producible vowels are
different [Vorperian & Kent.’07]
– Infants’ audition [Polka & Werker.’94] & articulation [Kuhl &
Meltzoff.’96] adapt to mother-tongue (become native)
• Dynamic process including
intrapersonal interaction
Social engagement
Perceptual
& social interactionof caregiver
[Kuhl et al.’08]
developmen
Articulatory
development
Caregiver’s Sensorimotor Magnets Lead Infant’s
Vowel Acquisition through Auto Mirroring
[Ishihara et al, 2008]
• A method that aids unconscious guidance in
mutual imitation for infant development based
on a biasing element with two different kinds of
modules.
1. The normal magnet effect in perceiving heard vocal
sounds as the listener’s own vowels (perceptual
magnet) and also includes another magnet effect for
imitating vocal sounds that resemble the imitator’s
vowels (articulatory magnet).
2. What we call “auto mirroring bias,” by which the heard
vowel is much closer to the expected vowel because
the other’s utterance is an imitation of the listener’s
Caregiver’s Sensorimotor Magnets Lead Infant’s
Vowel Acquisition through Auto Mirroring
How humans imitate the sound?
2nd Form
ant [m
el ]
Synthesi zed
Voi ces
Imitated
im
i tati onVoices
/i /
/u/
/e/
/a/
/o/
1st Form
ant [m
el ]
[Ishihara et al, 2008]
Psychological experiment
for auto-mirroring bias
[Wakasa et al., unpublished]
Cnt. G
Imi (Cnt)
Exp. G
Imi (Exr)
Hypothesis
AutoMirroring
bias
Imi (Cnt) < Imi (Exp)
Voice-Feature
Space
Voice-Feature
Space
Psychological experiment
for auto-mirroring bias
[Wakasa et al., unpublished]
Voice Feature
Difference
[Hz]
1st set
3rd set
※significant level ５％
Cnt. Exp.
０．０３６
p-value
# of Data
2nd set
０．０１４
０．４５４
Cnt. G
15
15
14
Exp. G
11
14
14
Significant difference
Auto-mirroring bias exists!
No significant
difference
Concentration
level down
Caregiver’s Sensorimotor Magnets Lead Infant’s
Vowel Acquisition through Auto Mirroring
[Ishihara et al, 2008]
Caregiver’s Sensorimotor Magnets Lead Infant’s
Vowel Acquisition through Auto Mirroring
[Ishihara et al, 2008]
Remarks
• Our model reproduced some phenomena in vowel
sharing process of infants
– Rapid separation of infant vowel clusters
– Motherese
– Gradual acquisition of native vowels
• Caregiver’s affirmative biases guided infant vowels to
be native through mutual imitation
• Infant articulatory immaturity (noise) contributed vowel
cluster separation and enables acquiring native
vowels
• Stretching motherese, exaggerating vocalizations,
might caused by caregiver’s underestimation of infant
vocalizations
Outline of my talk
1. Cognitive Developmental Robotics
1. Human development
2. What’s cognitive developmental robotics?
3. Fetus/neonate development simulation
4. Vocal imitation
2. Is paradigm shift possible?
1. Mirror Neuron System Connects Physical
Embodiment and Social Entrainment
2. From self/other identification to self/other
separation
Our goal is ….
• To build a model of human’s cognitive
development processes.
• One promising approach is “Cognitive
Developmental Robotics” that aims at not
simply filling the gap between brain science and
developmental psychology but more
challengingly at building a new paradigm that
provides new understanding of ourselves and at
the same time new design theory of humanoids
symbiotic with us.
• But, not yet!
Is a paradigm shift possible? (1)
• Analysis based approach from a God’s
viewpoint faces with twofold difficulties when it
focus on humans as living things.
1.Biology  more and more microscopic
views with various kinds of levels and
representations such as those in cell biology
or molecular biology.
2.Understanding human beings as social
agentshard or insufficient under a single
scientific paradigm, therefore
Interdisciplinary approach seems essential.
Is a paradigm shift possible? (2)
• What is a sufficient condition?
• Is it impossible by integrating the existing scientific
disciplines?
• Is CDR completely independent from them?
• Of course not! By involving them, CDR should give its
significance by prospecting the limits of the existing
scientific disciplines.
• In this context, the issue to be attacked is “interaction”
between neurons or brain regions or individual
persons.
• Communication, a kind of interaction between
subjective agents may involve the language
Is a paradigm shift possible? (3)
• To summarize the CDR approach,
1)integrate the knowledge, evidences, and findings
(utilize the existing paradigms and synthesize them),
 not to deny the existing disciplines but to involve
them. Therefore, CDR researchers should have the
minimum amount of knowledge in these disciplines
2) build a model or a hypothesis that have no contradiction with the existing disciplines or resolve the
contradiction or controversial issues,  a key point for
the CDR researchers to hit on an idea that reflects the
integrated knowledge in 1) and,
3) Find a new factor that provides a solution to mystery
Is a paradigm shift possible? (4)
• The common issues are body representation, rhythm
and timing, multimodal input/output (vision, auditory,
touch, somatosensory, motion, vocalization etc.), selfother separation, sociality acquisition, and so on.
• If CDR can provide the constructive and unified form
of the representation that can explain and
simultaneously design the cognitive development of
these issues, instead of representing them separately,
this may lead to the creation of a new value of the
paradigm shift.
• To enable this, the studies of developmental
disorders in addition to the studies of normal children
may help the unified model construction of cognitive
Outline of my talk
1. Cognitive Developmental Robotics
1. Human development
2. What’s cognitive developmental robotics?
3. Fetus/neonate development simulation
4. Vocal imitation
2. Is paradigm shift possible?
1. Mirror Neuron System Connects Physical
Embodiment and Social Entrainment
2. From self/other identification to self/other
separation
Acknowledgement
• Group Leaders and Research Contributors :
–
–
–
–
–
–
–
Prof. Koh Hosoda (Osaka Univ.)
Prof. Yasuo Kuniyoshi (Univ. of Tokyo)
Prof. Hiroshi Ishiguro (Osaka Univ.)
Prof. Toshio Inui (Kyoto Univ.)
Dr. Masaki Ogino (Osaka Univ., body image)
Dr. Yuichiro Yoshikawa (JST ERATO, vowel imitation)
Ph.D student Hisashi Ishihara (Vowel imitation, Affetto)
Thank you for your attention!
Please come to Osaka!
Everything robotics and
cognition is there!