OBJECTIVE METHODS OF SOCIAL COGNITIVE
NEUROSCIENCE FOR SOCIAL ATTUNEMENT
AGNIESZKA WYKOWSKA
Engineering Psychology, Luleå University of Technology
&
Institute for Cognitive Systems, Technical University Munich
[email protected]
WHY METHODS OF SOCIAL COGNITIVE
NEUROSCIENCE?
2
WHY METHODS OF SOCIAL COGNITIVE
NEUROSCIENCE?
• objective measures to examine fundamental mechanisms (often implicit)
of human sensing and human cognition in human-robot interaction
2
WHY METHODS OF SOCIAL COGNITIVE
NEUROSCIENCE?
• objective measures to examine fundamental mechanisms (often implicit)
of human sensing and human cognition in human-robot interaction
• designing and adapt robot behaviour so that it is well attuned to those
mechanisms
2
WHY METHODS OF SOCIAL COGNITIVE
NEUROSCIENCE?
• objective measures to examine fundamental mechanisms (often implicit)
of human sensing and human cognition in human-robot interaction
• designing and adapt robot behaviour so that it is well attuned to those
mechanisms
“Imitating”: robots
behave similarly to how
humans would behave
Social attunement
Neuro-cognitive,
behavioral and
physiological
measures
2
THE METHODS
3
S O C I A L AT T U N E M E N T
JOINT ATTENTION
– AT T E N D I N G T O W H E R E O T H E R S AT T E N D
4
J O I N T AT T E N T I O N - G A Z E C U E I N G
POSNER CUEING
5
J O I N T AT T E N T I O N - G A Z E C U E I N G
POSNER CUEING
5
J O I N T AT T E N T I O N - G A Z E C U E I N G
POSNER CUEING
T
5
J O I N T AT T E N T I O N - G A Z E C U E I N G
POSNER CUEING
T
Validly cued target
5
J O I N T AT T E N T I O N - G A Z E C U E I N G
POSNER CUEING
T
Validly cued target
5
J O I N T AT T E N T I O N - G A Z E C U E I N G
POSNER CUEING
T
Validly cued target
T
5
J O I N T AT T E N T I O N - G A Z E C U E I N G
POSNER CUEING
T
Validly cued target
T
Invalidly cued target
5
J O I N T AT T E N T I O N - G A Z E C U E I N G
POSNER CUEING
GAZE CUEING
T
Validly cued target
T
T
Invalidly cued target
5
GAZE CUEING AND ROBOTS
Karolinska Directed Emotional Faces database (KDEF,
Lundqvist, Flykt & Öhman, 1998)
(c) LSR, TU München
TASK: DISCRIMINATE LETTER T/F
Wiese, Wykowska, Zwickel & Müller (2012, PloS One)
6
GAZE CUEING AND ROBOTS
Wiese, Wykowska, Zwickel & Müller (2012, PloS One)
until response
Factors:
• cue type (robot vs. human)
• validity (valid vs. invalid)
!
Figure!2.!Size!of!Gaze!Cuing!Effects!in![ms]!as!a!function!of!Cue!Type!
GAZE CUEING: RT valid - RT invalid
(Human,!Robot)!and!Instruction.!Instruction!1:!Human!–!Robot!(Exp.!
1),!Instruction!2:!Human!E!Human!like!Robot!(Exp.!2),!Instruction!3:!
Puppet!–!Robot!(Exp.!3).!!
!
7
GAZE CUEING AND ROBOTS
Wiese, Wykowska, Zwickel & Müller (2012, PloS One)
until response
Factors:
• cue type (robot vs. human)
• validity (valid vs. invalid)
!
Figure!2.!Size!of!Gaze!Cuing!Effects!in![ms]!as!a!function!of!Cue!Type!
GAZE CUEING: RT valid - RT invalid
(Human,!Robot)!and!Instruction.!Instruction!1:!Human!–!Robot!(Exp.!
1),!Instruction!2:!Human!E!Human!like!Robot!(Exp.!2),!Instruction!3:!
Puppet!–!Robot!(Exp.!3).!!
!
7
GAZE CUEING AND ROBOTS
until response
• Within-participants instruction manipulation
• Physically identical stimuli across conditions
Wykowska, Wiese, Prosser & Müller (2014, PLoS One)
8
GAZE CUEING AND ROBOTS
-2 µV
Valid
O1/O2/PO7/PO8
Invalid
100
200
300
400 ms
2 µV
Wykowska, Wiese, Prosser & Müller (2014, PLoS One)
9
GAZE CUEING AND ROBOTS
Human-controlled
-2 µV
Valid
O1/O2/PO7/PO8
Invalid
100
200
300
400 ms
2 µV
Wykowska, Wiese, Prosser & Müller (2014, PLoS One)
9
GAZE CUEING AND ROBOTS
Human-controlled
-2 µV
Valid
O1/O2/PO7/PO8
Invalid
100
200
300
400 ms
2 µV
Wykowska, Wiese, Prosser & Müller (2014, PLoS One)
9
GAZE CUEING AND ROBOTS
Human-controlled
-2 µV
Valid
O1/O2/PO7/PO8
Invalid
100
200
300
400 ms
2 µV
Wykowska, Wiese, Prosser & Müller (2014, PLoS One)
9
GAZE CUEING AND ROBOTS
Human-controlled
-2 µV
Valid
O1/O2/PO7/PO8
Invalid
100
200
300
400 ms
2 µV
P1
Wykowska, Wiese, Prosser & Müller (2014, PLoS One)
9
GAZE CUEING AND ROBOTS
Human-controlled
-2 µV
Valid
O1/O2/PO7/PO8
Invalid
100
200
300
400 ms
2 µV
P1
Wykowska, Wiese, Prosser & Müller (2014, PLoS One)
9
GAZE CUEING AND ROBOTS
-2 µV
Valid
O1/O2/PO7/PO8
Invalid
100
200
300
400 ms
2 µV
P1
Wykowska, Wiese, Prosser & Müller (2014, PLoS One)
9
GAZE CUEING AND ROBOTS
Pre-programmed
-2 µV
Valid
O1/O2/PO7/PO8
Invalid
100
200
300
400 ms
2 µV
P1
Wykowska, Wiese, Prosser & Müller (2014, PLoS One)
9
GAZE CUEING AND ROBOTS
Pre-programmed
-2 µV
Valid
O1/O2/PO7/PO8
Invalid
100
200
300
400 ms
2 µV
P1
Wykowska, Wiese, Prosser & Müller (2014, PLoS One)
9
GAZE CUEING AND ROBOTS
Pre-programmed
-2 µV
Valid
O1/O2/PO7/PO8
Invalid
100
200
300
400 ms
2 µV
P1
Wykowska, Wiese, Prosser & Müller (2014, PLoS One)
9
GAZE CUEING AND ROBOTS
-2 µV
Valid
O1/O2/PO7/PO8
Invalid
100
200
300
400 ms
2 µV
P1
Wykowska, Wiese, Prosser & Müller (2014, PLoS One)
9
GAZE CUEING AND ROBOTS
-2 µV
Valid
O1/O2/PO7/PO8
Invalid
100
200
300
400 ms
2 µV
P1
Wykowska, Wiese, Prosser & Müller (2014, PLoS One)
EARLY ATTENTIONAL PROCESSES modulated by higher-order social cognition
9
A P P L I C AT I O N T O
THERAPEUTIC DOMAINS
10
GAZE CUEING - THE CASE OF AUTISM
DO PATIENTS WITH ASD ALSO FOLLOW GAZE OF HUMANS MORE THAN
GAZE OF ROBOTS?
Wiese, Müller, & Wykowska (2014). Using a gaze-cueing paradigm to examine social cognitive mechanisms of individuals with
autism observing robot and human faces. LNCS, 8755, 370-379. DOI: 10.1007/978-3-319-11973-1_38
11
J O I N T AT T E N T I O N , R O B O T S A N D
AUTISM
EXPERIMENTAL DESIGN
Test:
Early social communication scale
Training: 6 sessions
Kajopoulos, .., & Wykowska (2015). LNAI, 9388. DOI: 10.1007/978-3-319-25554-5_30
In collaboration with Tan Yeow Kee and social robotics team A*STAR Singapore
12
J O I N T AT T E N T I O N , R O B O T S A N D
AUTISM
EXPERIMENTAL DESIGN
Kajopoulos, .., & Wykowska (2015). LNAI, 9388. DOI: 10.1007/978-3-319-25554-5_30
In collaboration with Tan Yeow Kee and social robotics team A*STAR Singapore
13
J O I N T AT T E N T I O N , R O B O T S A N D
AUTISM
RESULTS: RESPONDING TO JOINT ATTENTION
(NO EFFECT ON INITIATION OF JOINT ATTENTION)
Kajopoulos, .., & Wykowska (2015). LNAI, 9388. DOI: 10.1007/978-3-319-25554-5_30
In collaboration with Tan Yeow Kee and social robotics team A*STAR Singapore
14
NEXT STEPS
EEG: EXAMINING WHICH STAGES OF PROCESSING
ARE TARGETED BY THE ROBOT-ASSISTED THERAPY
-2 µV
Valid
O1/O2/PO7/PO8
Invalid
100
200
300
400 ms
2 µV
15
CONCLUSION
METHODS OF COGNITIVE NEUROSCIENCE (WHICH
INCLUDES WELL DESIGNED EXPERIMENTAL SETUP)
ALLOW FOR TARGETING SPECIFIC MECHANISMS OF
THE HUMAN BRAIN
—> WELL-TAILORED THERAPIES TO THE NEEDS OF
USERS
16
OBJECTIVE METHODS OF SOCIAL COGNITIVE
NEUROSCIENCE FOR SOCIAL ATTUNEMENT
AGNIESZKA WYKOWSKA
Engineering Psychology, Luleå University of Technology
&
Institute for Cognitive Systems, Technical University Munich
[email protected]