Emotion in Interaction:
Embodied Conversational Agents
Catherine Pelachaud
IUT de Montreuil
Université Paris 8
1
Emotion in Interaction:
Humaine

Three domains of study
– Perception domain: how certain aspects related to perception
may influence agent’s actions
– Interaction domain: how to create relations between users
and agents; how the agent can provide feedback
– Generation domain: how to show expressive behaviours
consistently and naturally across modalities
2
Perception and Attention

Non obtrusive agent: based on the
interest shown by the addressee decides
– when to start a communication
– how long to maintain it
– when to finish it

Perception of attentive and emotional
behaviors
– where to look at
– how to interpret other’s behaviors
3
Perception and Attention

Endow an ECA with the capabilities to:
– have a visual attention and perception model
– show attention and interest
– attract the attention of others
4
Interaction

Interaction: two or more parties exchange
messages.

Interaction is by no means a one way
communication channel between parties.

Within an interaction, parties take turns in
playing the roles of the speaker and of the
addressee.
5
Interaction

Speaker and addressee adapt their
behaviors to each other
– Speaker monitors addressee’s attention and
interest in what he has to say
– Addressee selects feedback behaviors to show
the speaker that he is paying attention
6
Interaction







Agent as adressee ought to have the capabilities:
attention: pay attention to the signals produced by
speaker to perceive, process and memorize them
perception: of signals
comprehension: understand meaning attached to
signals
internal reaction: the comprehension of the
meaning may create cognitive and emotional
reaction
decision: decision to communication the internal
reaction
generation: display behaviors
7
Interaction

Providing feedback is a fundamental aspect of
engagement

Backchannel: signals of
–
–
–
–
–
12345-
perception
attention -- interest
understanding
attitude: belief (T/F); like (T/F);
acceptance
positive/negative
 any combination of the above

– pay attention but not understand; understand but not
believe
8
Expressive Behaviors

Behaviors encode:
– content information (the ‘What is
communicating’)
– expressive information (the ‘How it is
communicating’)
highly synchronized
 multimodal

9
Expressive Behaviors
 Basic emotions
– Anger, fear, sadness, happiness, disgust,
surprise
– Universally recognized (Ekman)
– Basic emotion = family of related states
(Ekman 75)
 Real
life emotions are often complex
and involve several emotions
10
Expressive Behaviors

Blends may occur either as
– superposition of emotions
– masking of one emotion by another emotion
– quick succession of different emotions
– suppression / overacting of an emotion
Superposition of 2 emotions: evaluation of
an event can happen under different
angles
 Mask an emotion by another one :
consideration of social context

11
Examples

Disappointment Masked by Joy
Disappointment

joy
Disappointment masked
by joy
Superposition of Sadness and Joy
Sadness
Joy
Sadness and Joy
12
Callas

Creation of a virtual agent able to
– perceive and be attentive
– be aware of and adapt to the surroundings
(physical and social context)
– interact with users/other agents
– communicate and listen
– show ‘real’ emotions
– use expressive gestures
13
Callas
Extension and refinement of the results
obtained in Humaine
 Real-Time integrated system
 Integration of virtual and real perception
models
 Real situations

14