Artificial Intelligence and Multi-Agent Systems
Laboratory
University Politehnica of Bucharest
Affective techniques for emotion
simulation
PhD Research Report nr. 1
PhD Student: Mihaela-Alexandra Puică
AI-MAS Laboratory
Computer Science and Engineering Department
University ”Politehnica” of Bucharest
Supervisor: Prof. Adina-Magda Florea
AI-MAS Laboratory
Computer Science and Engineering Department
University ”Politehnica” of Bucharest
1
Contents
1 Introduction
3
2 Emotion theory
5
2.1
Emotion classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
2.2
Emotion theories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
2.2.1
James-Lange . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
2.2.2
Canon-Bard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
2.2.3
Schachter-Singer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
2.2.4
Cognitive appraisal theory . . . . . . . . . . . . . . . . . . . . . . . . .
9
3 Emotion recognition
11
3.1
Facial expression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.2
Gestures and posture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.3
Speech . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.4
Physiological parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.5
Multimodal emotion recognition
. . . . . . . . . . . . . . . . . . . . . . . . . 14
4 Emotion simulation
15
4.1
OCC theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.2
Emotional BDI models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
5 Proposed architecture
25
5.1
Agent Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
5.2
Emotions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.3
Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.4
Personality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
6 Conclusions
6.1
34
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2
Chapter 1
Introduction
In the final speech of the 1940 movie ”The Great Dictator”, Charlie Chaplin said: ”We
think too much and feel too little. More than machinery we need humanity.”. More than
half a century later, scientist begun to turn research direction in artificial intelligence from
strictly rational to a more human approach. This new approach was inspired by studies in
psychology, neurology and cognitive science which shown that emotions have a central role
in human intelligent behaviour.
Oficially, the new research direction appeared in 1997 when Rosalind Picard published the
book that gave the name and the scope of this reasearch field: Affective Computing, which is
”computing that relates to, arises from, or deliberately influences emotions” [Picard, 2000].
In other words, Affective Computing may be regarded as Artificial Intelligence that uses
emotions to display intelligent behavior.
Affective Computing combines knowledge from several research domains to develop systems that display human intelligence. Such systems are able to better understand the
context and cope with the situation they are facing. The basic element that switches the
focus from strict rationality to a smoother reasoning process is the emotion. Emotions are a
type of feelings that people experience, but their mechanisms are not completely understood.
There is no sharp boundary between what is and what isn’t emotion. Usually, emotions are
explained in contrast to some related concepts, like moods, attitudes or personality traits.
These are distinguished through time constraints: emotions are ”behavioral dispositions that
persist for seconds or minutes” [Gratch and Marsella, 2004], moods are states with similar
effects, but that last for hours or days, attitudes can stretch over months and years and personality traits ”reflect relatively stable behavioral tendencies” [Gratch and Marsella, 2004]
[Ekman, 1994] [Moshkina and Arkin, 2009].
Nevertheless, advances in human sciences make it certain that the role of emotions in
effective decision making cannot be ignored. In psychology, Daniel Goleman defines emotional intelligence and argues its importance in day-by-day life [Goleman, 1996]. It is to be
noticed that emotional intelligence can be improved in time, while mathematical or logical
3
intelligence is something that one is born with, the IQ being constant over the years. Antonio Damasio obtained interesting results through studying patients who had damages in the
limbic system (the part of the brain that deals with memory, perception and emotion). He
thus dicovered that such patients, although their IQ was high enough, they had problems
in dealing with dayly routine: they could hardly make up their mind, and they were unable
to recognize bad decisions, repeating them all over again [Damasio, 1994].
Therefore, human intelligence cannot be achieved without modeling emotions, for which
we must first understand what emotions really are and what are their internal mechanisms:
what triggers them and how they influence behavior. Human sciences answered these questions through emotion classification and emotion theories. We will talk about these in
Chapter 2. Using this knowledge in combination with artificial intelligence algorithms and
agent modeling techniques, we are able to develop a more efficient and realistic agent model,
capable of acting in a dynamic, nondeterministic environment.
Regarding applications, affective computing is focusing on two main directions: emotion
recognition and emotion simulation. The first direction has the goal to detect user emotional state, which further helps the system in changing its behavior or the environment
accordingly. Emotion recognition can be usefull in Ambient Intelligence scenarios or even in
other domains, not related to intelligent machine development. For example, an intelligent
system for AmI could project amusing images on the walls of a room where a person is being
sad. A psychologist could rely on automatic software to establish people’s emotions during
a longer period of time instead of watching hours of recordings. In publicity, one could see
the effects of their commercial on their customers through observing their affective states.
Emotion recognition is a large research field and several methods have been developed for
this purpose. We will talk about them in detail in Chapter 3.
The other direction will also have a great impact in future intelligent systems. Its goal is
to simulate emotions and their effects into agents. There is a debate here whether machines
could really feel emotions, but we should concentrate more on the beneficial effects of emotions over decision making and work on achieving them. There are many systems developed
and some have been implemented, prooving their efficiency. Chapter 4 will discuss some
of them in more detail, highlighting their advantages and disadvantages, limitations and
directions where there can be worked on.
After reviewing existing litarature, a new architecture will be presented in Chapter 5, one
that tries to take what is best from the other models regarding the purposes it is built for.
This architecture will account for agent personality and will aim for improved performance
and efficient resource usage.
In the last Chapter conclusions will be drawn and directions for future work will be
indicated.
4
Chapter 2
Emotion theory
Emotions have been discussed since ancient times, but they were considered undesirable,
negatively influencing reasoning. They are still viewed so by many people nowadays, but
in the scientific community perspectives on emotions started to improve since 19th century,
when Charles Darwin stated that emotions were a result of natural selection, helping people
survive over the history [Darwin, 1872]. Since then, research in emotion increased, various
theories being developed regarding emotion categories or emotion elicitation.
2.1
Emotion classification
Because emotions display various tipologies in what concerns their expression or their elicitation, depending on the personality of the person or on their culture, researchers have
classified them using different criteria: a stateless-like type of classification (following either
a discrete model or a dimensional model) or a statefull-like type of classification.
The first criterion refers to emotions-as-are, without considering the context or the time
sequence they appear in. The discrete model is similar to the model of colors, where basic
colors blend to create new colors. Siilarly, there are basic and complex emotions. Basic
emotions are claimed to be innate, and therefore universally felt and recognized by all people,
while complex emotions appear by mixing two or more of the basic emotions. Darwin’s
theory that emotions evolved through natural selection sustains the idea of basic emotions
common to all cultures. Ekman also supports this claim by studies that show that basic
emotions are expressed using similar configurations on the face which any person is able to
read, provided their environment, education or culture [Ekman, 1999]. These emotions are
happiness, sadness, fear, anger, disgust and surprise. He later adds other emotions
like contempt, shame or satisfaction, but the first six enumerated are equally mentioned by
other researchers. For example, Plutchik created a model known as ”Plutchik’s wheel of
emotions” [Plutchik, 2001] which consits of eight basic emotions arranged as four pairs of
opposites, their combinations that give the complex emotions and their variants depending
on their intensity. Figure 2.1 shows the wheel of emotions. The eight basic emotions
suggested by Plutchik are joy versus sadness, trust versus disgust, fear versus anger
5
and surprise versus anticipation.
Figura 2.1: Robert Plutchik’s wheel of emotions
The dimensional model uses a two-dimensional space, where one axis represents the
valence of the emotion (pleasant-unpleasant) and the other displays the level of arousal
(anxiety-boredom). This model, called the circumplex model of affect, was first developed
by James Russel [Russell, 1979] [Russel, 1980] and later adapted by others like Pieter Desmet
[Desmet, 2002] and Trevor van Gorp [van Gorp, 2006]. Fig. 2.2 shows eight emotions placed
in a circular order in this space. The valence axis is defined by pleasure-misery, while the
arousal axis by arousal-sleepiness. The other four emotions simply define the contents of
each quadrant.
Figura 2.2: James Russell’s circumplex model of affect
The second criterion takes into consideration the way an emotion is generated: cognitively or non-cognitively. Thus, the distinction is made between primary emotions and
secondary emotions [Damasio, 1994]. This sorting is based on neurological evidence of the
6
functions of different parts of the brain. The main elements in this context are the cortex
and the limbic system. The cortex is the most studied - given that is much more accessible,
being situated closer to the scalp. It contains the visual cortex and the auditory cortex,
thus the conclusion that the cortex handles all perception. The limbic system is situated
below the cortex and it is the place of attention, memory and emotion. For a long time it
was believed that the two components of the brain are completely independent, but recent
studies by Antonio Damasio [Damasio, 1994], Joseph LeDoux [LeDoux, 1996] and Cytowic
[Cytowic, 1996] show that cognition and affect are much more related and highly influence
eachother. The main difference between cortex and limbic system is that the first one
is slower, but more accurate, while the latter is faster, but more imprecise (the
”quick and dirty” path). Consequently, when a stimulus is perceived, it reaches first
the limbic system, which outputs the primary emotion. Then, the same stimulus reaches
the cortex, which processes it connecting physiological responses with cognitive appraisal
of events and outputs the secondary emotion. For example, if one sees an object approaching rapidly, fear will instantly arise, which will determine quick moves to avoid the object.
This kind of emotion is innate, hard-wired into the human body, and thus non-cognitively
generated. That is why it is called ”primary emotion” and its primary role is in survival.
Rosalind Picard recognizes the importance of primary emotions stating that ”it is clearly
beneficial for our survival that fear can hijack our brain and cause us to jump out of the
way of a rapidly approaching object before we can consciously perceive and analyze that
a bus is about to hit us” [Stork, 1996]. On the other hand, secondary emotions ”play an
especially important role in decision making, even in decision making that appears to be
purely rational” [Stork, 1996]. Marvin Minsky agrees that emotions have a crucial role in
decicion making: ”I don’t think you can make AI without subgoals, and emotions is crucial
for setting and changing subgoals” [Stork, 1996].
2.2
Emotion theories
Saul Kassin defines emotion as ”a feeling state characterized by physiological arousal, expressive behaviors and a cognitive interpretation” [Kassin, 2004]. The exressive behavior is
the specific action (body, face, language) displayed by a person who experiences the emotion, action which helps the others recognize their emotion. The other two are the main
components that elicit the affective state. Physiological arousal refers to the bodily changes
that happen when the stimuli is present: pulse quickened, breathing rate increased, profuse sweating and so on. Cognitive interpretation deals with understanding the situation,
appraising it in terms of its influence on the subject. The main question that theories of
emotion try to answer is which of the two components goes first.
2.2.1
James-Lange
One of the earliest theories of emotion that is still discussed today is that of James-Lange.
This was developed independently by William James in 1884 [James, 1884] and Carl Lange in
1885. This theory claims that physiological arousal comes first, and then the emotion. More
7
precisely, the emotion is the result of bodily responses to the event (and not their cause).
James’ thesis was that ”the bodily changes follow directly the PERCEPTION of the exciting
fact, and that our feeling of the same changes as they occur IS the emotion”. He explained
that ”we feel sorry because we cry, angry because we strike, afraid because we tremble”,
and not the opposite, which is what common sense would suggest. The implications of this
theory is that without physiological arousal, no emotion is felt. Fig. 2.3 shows the claims if
the James-Lange theory.
Figura 2.3: James-Lange theory of emotion
2.2.2
Canon-Bard
This implication was one of the main criticism of the James-Lange theory. In 1927, Walter
Cannon rejected this assumption after conducting a series of experiments which proved that
emotion can appear without the brain knowing about the bodily responses (the connection
between the two was cut off) [Cannon, 1927]. Another reason to not accept the James-Lange
theory was that the same physiological arousal happens in different emotions (for example
angry or excited). Philip Bard agreed with Cannon and they continued their experiments
until they concluded that affect and bodily responses are independent. Therefore, they
presented the theory that when an event occurs, emotion and phisiological arousal take
place simultaneously and independent from one eachother (see Fig. 2.4). Both theories they
say nothing about the role of cognition in these processes, but while James-Lange theory
defines emotions as the feeling of physiological arousal, Cannon-Bard theory provide no
mechanisms of how emotions appear.
Figura 2.4: Cannon-Bard theory of emotions
8
2.2.3
Schachter-Singer
More insight about this was taken by two researchers, Stanley Schachter and Jerome Singer,
who developed their own theory in 1962 [Schachter and Singer, 1962]. Their theory is called
Two-factor theory of emotion, because it states that both physiological arousal and cognition
take part in the apparition of emotion. Thus, when a stimuli is present, the bodily changes
appear, and the subject tries to explain these changes according to the context they are
in. This is shown in Fig. 2.5. Schachter and Singer proved their theory by making an
experiment where subjects were injected either with adrenaline or with a saline solution,
then they were either explained, misinformed or said nothing about the side-effects, and
eventually they were put in a happy or angry situation. Their state was measured by two
methods: self report and third-party observation of their behavior. Subjects were not told
about the real purpose of the experiment. The adrenaline induces arousal, but the saline
solution has no effect on the physiological parameters, so it is used just as control measure.
The results confirmed the statements of their theory.
Figura 2.5: Schachter-Singer theory of emotions
2.2.4
Cognitive appraisal theory
A more recent theory is the cognitive appraisal theory. Its development is attributed mainly
to Magda Arnold and Richard Lazarus. They argue that when an event is triggered the
first step is to cognitively assess the event in terms of each one’s beliefs, goals, standards
and attitudes. This process is commonly referred to as ”appraisal” and is mainly concerned
with how a person perceives the environment they live in. The relevance of perception in
everyday life has been noticed not only by psychologists, but also by people working in other
domains. The writer Anais Nin said that ”The world is not as it is, but as we perceive it to
be”. After appraisal, the next step is ”coping”, which deals with the action(s) that an agent
has to take in order to adapt to the new situation. The main claim of this theory (which
is also the main criticism) is that emotions appear only through cognitive appraisal, even
without the physiological arousal to happen. Fig. 2.6 shows this.
Resuming, we note that Lazarus specifies two types of appraisal:
9
Figura 2.6: Cognitive appraisal theory of emotions
• primary appraisal, which consists in assessing the event in the light of what metters
for self [Smith and Kirby, 2009]; this includes:
– motivational relevance - if the event is relevant to one’s goal;
– motivational congruence - if the event is consistent or inconsistent with one’s goal;
• secondary appraisal, which consists in assessing what one can do to cope with that
situation; there are two posibilities:
– problem-focused coping - what can one do to change (external) situation
– emotion-focused coping - what can one do to change (internal) perception
This first description of appraisal theory has been later developed in various directions
by other researchers, but the basic idea of appraising and coping is still ongoing. Cognitive
appraisal emphasizes the role of cognition in the apparition of emotions. It must capture
various possible evaluations of the context for an accurate interpretation and it must cope
with the dynamic nature of emotions to be able to quickly adapt to new changes.
10
Chapter 3
Emotion recognition
Rafael Calvo and Sidney D’Mello made a comprehensive review on the methods currently
used to recognize human emotions [Calvo and D’Mello, 2010]. Affect can be detected from
facial expressions, speech, posture and gestures or physiological parameters such as heart
rate, skin conductance, respiratory rate or temeprature. Although physiological parameters
sould be the most effective ones, given that they have precise values which can be matched to
specific emotional states, they may also alter the actual emotion felt, increasing the arousal
when the measuring device is attached to the subject. On the other hand, face, voice or
gesture detection interfere less, because the subjects may not be aware of the measuring
devices.
There are several methods developed to recognize human emotions, depending on the
forms of the sentic modulation - what Rosalind Picard defines as the influence of emotion
on bodily expression [Picard, 2000]. People have different ways of showing their emotions,
willingly or unwillingly, more or less apparent to others. Usually the more apparent forms
are easier to control by the subject, while the less apparent forms are harder to control. In
the first category there are facial expressions, voice intonation, posture and gestures. The
second category includes respiration, heart rate, temperature, perspiration, blood presure,
muscle action potentials [Picard, 2000].
3.1
Facial expression
Facial expressions are among the most important form of sentic modulation for the recognition of emotions by humans. In a conversation, the first thing that one observes is the
other’s face. And if we are to believe Ekman, we can recognize any of the basic emotions
(but not only the basic ones) from the corresponding facial expression because each emotion
has its own specific configuration of the face muscles [Ekman, 1994]. There are of course
some exceptions, when subjects control their muscles to hide their true feelings, because the
situation requires it (either in a business context or culture determined). Nevertheless, in
most of the situations the face can betray the real feelings. Therefore, face recognition is
one of the most studied topic in emotion recognition.
11
In psychology, Paul Ekman did a lot of research in emotion recognition through facial
expressions and, together with Wallace Friesman, he developed a system that objectively
classifies expressions, called Facial Action Coding System [Ekman and Friesen, 1978]. This
system measures facial configuration using Action Units (AU), which are individual movements of the elements on the face (eyes, brows, nose, lips). The FACS system defines about
45 codes for the action units of the face muscles, plus another 25 codes for head or eye
movements. Ekman developed (together with his colleagues) a training tool for teaching
people how to recognize micro expressions and also wrote a book explainig what emotions
do different muscle movements betray.
In artificial intelligence, researchers developed various methods to automatically recognize emotions from facial expressions. Generally, they use either neural networks or genetic
algorithms. Databases with facial expressions of different emotions have been created and
are used in supervised learning. As Rafael Calvo and Sidney D’Mello note in their review
[Calvo and D’Mello, 2010], the majority of the systems recognize only basic emotions and
they do not operate in real time.
According to [Hebe et al., 2005], current emotion recognition systems using facial expressions recognize between 2 to 7 emotion categories, with an accuracy of 83% - 98%.
3.2
Gestures and posture
Besides facial expressions, gestures and posture are another form of sentic modulation that
completes the visual image of the subject. While in the first case the target was given only
by the facial muscles, we are now interested in the whole body and arms position. Though,
facial expressions have the advantage that they express the same emotions everywhere in
the world. Gestures on the other hand are very different, and one gesture could have oposed
significance in two different cultures. The advantage over facial expression is that body
posure cand be detected at much longer distances.
Body posture is a good method to interpret the interest level in a context where the
subject is sitting (for example in a learning environment, where a student is standing on a
chair and studying at a computer). Pressure sensors are used that detect the point where
the pressure is focused, being thus able to say if the subject is leaning back or sitting upright
- a sign of relaxation or great interest respectively.
For body postures standing, Rudolf Laban presented a notation system used for analyzing
human movement, now called Labanotation [Badler and Smoliar, 1979]. Human body has
numerous points of freedom, there are hundreds of different configuration of the body. The
Labanotation considers the extremities and the articulations (hand, wrist, elbow, shoulder,
hip, knee, ankle, foot) and specifies the position of each joint with respect to a coordinating
system. In addition to recognizing body movements, the Labanotation may also help in
developing animations of the human body movements.
12
According to [Calvo and D’Mello, 2010], [Mota and Picard, 2003] was among the first
works in automatic body posture emotion recognition, using a Body Preasure Measurement
System (BPMS) from Tekscan company. It recognized interest level with an accuracy of
about 82%. Their work was extended in [D’Mello and Graesser, 2009] to recognize 5 emotion
categories (boredom, confusion, delight, flow and frustration) having an accuracy of 70% 83%.
3.3
Speech
Voice intonation spans the audio part of affect recognition. It is also a very important step in
recognition of emotions by humans and other animals. For example, if your pet dog upsets
you, you will probabily yell at it in anger. The dog won’t understand your words, but will
realize that you are mad. This is true in all the situations where verbal communication is
the case (wheather the listener is a person or an animal).
Affect is transmited through speech not (only) by what is said, but by how it is said.
There are several characteristics, like pitch or volume, that encode emotion into speech. For
example, high pitch is a sign of arousal, and it could be understood as anger or fear; a lower
intensity of the voice is a sign of lowe arousal, which could mean sadness or disgust. In
general, sadness and anger are the easiest to recognize, whie disgust is the hardest.
Regarding accuracy of current affect recognition through voice, the results are similar
to those for body posture. [Dellaert et al., 1996] uses 17 voice features and detects 4 emotion categories with an accuracy of 80%. [Petrushin, 1998] did a comparison between human speech recognition and automatic speech recognition, obtaining similar results - about
65%. Scherer claimed that human ability to recognize emotions from speech is about 60%
[Scherer, 1996]. A more recent tool has been developed at Rochester University. It uses 12
features of speech and has an 81% accuracy [Yang et al., 2012].
3.4
Physiological parameters
Humans cannot recognize other people’s emotional state through their psychological parameters, because special devices are needed to measure them. But because of their accurate
return values, they are very reliable in automatic recognition. Te physiological parameters
measured are in fact electrical signals from brain, heart, skin and muscles. Table 3.1 shows
the physiological parameters usually measured to detect emotions.
There has been an extensive reasearch in affect detection using physiological parameters,
and the results are encouraging. [Calvo and D’Mello, 2010] compared 12 studies using one
or the other of the methods, classifying 3 up to 8 emotions and with an average accuracy of
80%.
13
Physiological parameter
brain activity
heart rate
skin conductance
muscles activity
Measuring method
EEG (Electroencephalography)
fMRI (functional Magnetic Resonance Imaging)
EKG (Electrocardiogram)
EDA (Electrodermal Activity)
GSR (Galvanic Skin Response)
EMG (Electromyogram)
Tabela 3.1: Physiological parameters and their measuring methods
3.5
Multimodal emotion recognition
The new trend in emotion recognition is to use more then one method of the ones presented
above. That is beacuse an emotion is generally expressed through more than one communication channel. For example, anger can be recognized through the frowning facial expression,
high pitch and fast speech rate, extensive gestures and faster heart rate. Although the benefits are theoretically recognized, there are not many implementation of this technique, as
Rafael Calvo and Sidney D’Mello note in their review [Calvo and D’Mello, 2010].
According to [Calvo and D’Mello, 2010], there are 3 methods to combine the information
obtained through different methods:
• data fusion - performed at the lowest level, on raw data; not very useful because signals
must be compatible (can’t sum up apples with pears)
• feature fusion - performed on the set of signals coming from a certain device; usually
an average or standard deviation are computed on the set and then these results are
combined further
• decision fusion - performed at the highest level - after obtaining the affective state
resulted from each classifier, they must be merged to output the global affective state;
the results from each device should be more or less similar, otherwise priorities should
be used to decide upon the final result
The few implemented systems do not have very conclusive results. In some of them the
reported accuracy for multichannels is higher than for single channels, but in others, one
of the channels had the same accuracy as after the information fusion, which means that
the other channels are redundant. Further studies should be done in order to improve the
results.
14
Chapter 4
Emotion simulation
There are several theoretical models of emotions, but these are rather form a pshychological
point of view, they not very clearly specified from a computational point of view. To
implement an affective agent, more detailed models are requiered. Therefore, computational
models of emotions have been developed. There are plenty of them and we only discuss a
small part in this paper, but most of them use a cognitive approach to simulate emotions.
4.1
OCC theory
The most proeminent model of the 20th century is that developed by Andrew Ortony, Gerald
Clore and Allan Collins in 1988, generally known as the OCC model [Ortony et al., 1988].
They claim emotions are a result of cognitively appraising an event or a situation in the
light of one’s goals, standards and attitudes. Thus, there are three aspects of the world that
contribute to experiencing emotions:
• events and their consequences on itself and on others - these factors can generate
pleased or displeased emoions;
• agents and their actions - these factors can generate approval or disapproval emotions;
• objects and their aspects - these factors can generate liking or disliking emotions.
Overall, Ortony, Clore and Collins differentiate among 28 types of emotions (22 concrete
and 6 abstract). Their model has widely influenced other systems, event though it hasn’t
dominate psychological community [Gratch and Marsella, 2004], and it still presents interest
to AI researchers.
Affective Reasoner [Elliot, 1992] is a project developed by Clark Elliot for his PhD in
1991 which is based on the OCC model. He implemented a platform for reasoning about
emotions aimed at testing psychological theories of emotions. The platform is a multi-agent
world in which agents are endowed with affective states and are able to reason about other
agent’s emotions and emotion-induced actions. The project uses a set of 24 emotion types
15
based on that of [Ortony et al., 1988].
In 1996, Scott Reilly finished his PhD thesis which presents the OZ project [Reilly, 1996].
In this project, the focus is on interactive drama and specifically on interactive characters
that have to perform believable social agents in the provided setting. Interactive drama is
a system where a human user plays the part of a character in an interactive story-based
simulation. Reilly provides a general framework that allows building emotional agents. It
incorporates the Tok agent architecture which, in its turn, embeds the Em emotion architecure. To test his work, Reilly developed seven believable social and emotional agents which
act in three different simulations.
Another model that uses OCC, but accounts for the dynamic nature of emotions is
PETEEI [Seif El-Nasr et al., 1999]. The agent that follows this architecture learns form its
experience through four types of learning:
• learning about events - the agent learns about the probability of an event; using Qlearning and computing a desirability measure, the agent is likely to experience a
certain emotion;
• learning about the user - the agent learns typical sequences of actions (patterns) performed by the user and is able to later anticipate what is their next most probable
action
• learning about pleasing and displeasing actions - the agent learns what actions are
pleasant or unpleasant for the user
Jonathan Gratch and Stacy Marsella goes beyond OCC. They use Lazarus model of
appraisal and coping to develop their EMA architecture (EMotion and Adaptation) [Gratch and Marsella, 2004].
The EMA agent is evaluating the personal-environment relation in terms of beliefs, desires,
plans and intentions, creating specific appraisal frames. Based on the features extracted,
the appraisal frames are mapped into individual emotions, which are then integrated into
an emotional state. This further influence the decision to be taken in order to cope with the
current situation.
Some researchers ([Adam et al., 2009], [Meyer, 2006], [Dastani and Meyer, 2006] or [Steunebrink, 2010])
take another approach to modeling emotions. They consider modal logic to be appropriate
for representing agents’ mental attitudes and to reason about them. Hence, they create a
logical formalization of emotions using what is called Logic of Emotions. Althouh it seems
contradictory to associate logic with emotions, the aforementioned researchers prove that
this approach is reasonable and feasible. The psychiological part of their work is founded
on the OCC theory of emotions.
Thus, Adam, Herzig and Longin [Adam et al., 2009] give definitions for 20 emotions
of the 28 presented in the OCC model, having the aim of formalizing it as faithful as
possible. But, even thought they claim to focus on the triggering conditions of the emotions,
Steunebrink reproaches them that they mix emotion triggering with emotion expression
[Steunebrink, 2010].
16
In his PhD thesis, Bas Steunebrink formalizes all the 28 emotions in the OCC model,
making a clear difference between emotion elicitation, emotion expression and emotion regulation. His work takes inspiration from the work of Jonh-Jules Meyer and Mehdi Dastani
[Meyer, 2006] [Dastani and Meyer, 2006], who formalize only four emotion types (happiness, sadness, anger and fear) based on the psychological work of Keith Oatley and Jennifer
Jenkins [Oatley and Jenkins, 1996].
4.2
Emotional BDI models
Another approach is to have BDI agents include emotions in their architecture. BDI is known
to be a suitable and wide-accepted model for intelligent agents [Rao and Georgeff, 1991]. It
is based on the practical reasoning that humans usually perform [Bratman et al., 1988].
That is, reasoning directed toward action, not the classical logical formalisms. Practical
reasoning implies deliberation (the agents decides what are the goals it wants to obtain) and
means-end reasoning (the agent decides the actions it has to perform to achieve its goals). By
mixing emotions in this architecture, the agents beliefs, desires and intentions are influenced
by emotions. Several researchers embraced this approach [Florea and Kalisz, 2004] (BDE
architecture), [Pereira et al., 2005], [Jiang et al., 2007] (EBDI architecture), but work is not
over yet and it is still an interesting research direction, suitable for future development.
Early work was done by Lin Padgham and Guy Taylor in the ’90s in Australia [Padgham and Taylor, 1997].
Their system has four main modules: cognitive, emotional, behavioral and system management. The cognitive module is exactly dMars, an implementation of the BDI architecture
done by Australian Artificial Intelligence Institute (a successor of PRS - Precedural Reasoning System and a predecessor of Jack Intelligent Agents platform). The behavioral module
handles the graphics of the system, synchronizing the abstract actions executed by dMars
with the concrete actions displayed in the environment.
The emotional module processes the emotions felt by the agent. The three modules interact only through the fourth module - system management. The emotional model describes
causes and effects of emotions, but also contains agent personality. Causes include threats
and opportunities or success and failure of goals, all associated with motivational concerns
specific to each agent. Emotions can affect behavior directly (through instantaneos actions
- influence on the behavioral module) or indirectly (through reasoning - add, delete or reprioritize goals - influence on the cognitive module). The model considers pairs of opposite
emotions, each emotion gauge having a neutral, positive and negative point. Each agent has
a given treshold for the emotional gauge, and when this is passed, the belief that the agents
feels that emotion is asserted. The treshold is a characteristic of the agent, thus definig its
personality. Agent personaliy is also defined by their motivational concerns and by the decay
rate of the emotions. Therefore, the main problem of this architecture is that emotions do
not influence reasoning process directly, but are considered only as beliefs. Nevertheless,
the first step of integrating agent emotions and BDI agent model into the same system was
done. Consequent research goes into more detail on the relationship between emotions and
the reasoning process.
17
In 2005, a conceptual model of an Emotional BDI agent was built by David Pereira, Eugenio Oliveira, Nelma Moreira and Luis Sarmento from University of Porto [Pereira et al., 2005].
The aim of this work was to identify the disadvantages of the BDI model and to overcome
them. The main problems associated with BDI architecture were found to be the lack of
information about resource bounds, the problem of agent reconsideration, and the lack of
other human-like mental states. The solution for each of these was to introduce resources,
capabilities and emotions respectively, three concepts that would influence the reasoning
process. Therefore, in addition to the regular components of a BDI architecture, the emotional BDI architecture contains resources, capabilities, a module for sensing and perception
and a module for managing the emotional state. All of the available resources, agent capabilities and agent emotional state influence the processes in the BDI model: belief revision,
option generation, filter and execute. While resources and capabilities are simply ”internal
representations of the means that an agent has to execute upon its environment” (concrete
means and abstract plans of action, respectively), the emotional state manager is an entire
module that generates the emotions felt by the agent. Nevertheless, the emotional state
manager is just skechted, it is not fully specified. Thus, the authors only mention the main
characteristics that this module should include: a well defined set of Artificial Emotions
(emotions that can be computationally implemented), various triggering events and a decay
rate for each of them. Although the details of the emotional module are left for further
consideration, the idea that this architecture introduces, namely having agent resources and
capabilites, is worth to be considered.
The next year, but independent of [Pereira et al., 2005], Van Dyke Parunak, Robert Bisson, Sven Brueckner, Robert Matthews and John Sauter, at Altarum Institute, Michigan,
USA, have created a model of emotions for situated agents applicable in the particular
situaton of a combat [Parunak et al., 2006]. Their aim is to ”simulate large number of combatants very rapidly” (faster than real-time). For this, the agent model that the combatants
will follow has to be computationally efficient, such that it allows a rapid reasoning process.
The architecture includes two reasoning processes: appraisal and analysis. In addition to
the BDI components, it also contains dispositions, emotions, triggers and tendencies. The
appraisal process assesses the beliefs in the context of agent disposition and returns the
emotion felt by the agent. The beliefs are mapped to digital pheromones that inform the
presence of other agents or objects in the environment and they act as triggers for emotions. The analysis process selects the intentions based on beliefs, desires and emotions.
Here, emotions impose a tendency on intentions. As noted before, the architecture is very
specific, and therefore so are their components. The combat situation is represented by red
units (enemy), blue units (friendly) and green agents (civilians). The beliefs (pheromones
sensed) are: RedAlive, RedCasualty, BlueAlive, BlueCasualty, GreenAlive, GreenCasualty,
WeaponsFire, KeySite, Cover and Mobility. The desires are: ProtectRed, ProtectBlue, ProtectGreen, ProtectKeySites, AvoidCombat, AvoidDetection and Survive. Selected emotions
are fear and anger, found to be ”the most crucial emotions for combat behavior”. Being
very specific, it does not suit well to our purposes.
A more refined version of an emotional BDI agent was developed in 2007 by Hong Jiang,
18
Jose Vidal and Michael Huhns at University of South Carolina [Jiang et al., 2007]. They
reference the three papers discussed above, so they are aware of the previous work and try to
improve it. The novelty of their architecture is that it considers both primary and secondary
emotions, as well as three sources for beliefs: perception, communication and contemplation.
In what concerns emotions, the primary ones are generated immediately after an event occurs
or a message is received; they are instinctive and reactive and help taking fast decisions.
If more time is available, further deliberation is conducted, and secondary emotions are
generated; they can replace primary emotions and are used to refine the decision making
process. The main focus of the EBDI model is to ”apply emotions to agent’s decision
making” and this is shown in Fig. 4.1.
Figura 4.1: Main characteristics of the EBDI model
The authors do not agree with the introduction of resources and capabilities by [Pereira et al., 2005].
They see them as independent of emotions and note that it is not shown exactly how emotions influence beliefs and decision making. Instead, they solve the problem of resource
boundery by having priorities for each of the beliefs, desires, intentions and emotions and
by deleting those with lowest priority when memory is full. On the other hand, they leave
the problem of reconsideration to be treated by the plan execution function, considering
that it depends on the specific problem. The authors also reproach to [Pereira et al., 2005]
that they do not highlight the differences between the emotional agent and a simple rational
agent. In contrast, they have implemented their architecture and tested it in Tileworld,
where both emotional agents and non-emotional agents live simultaneously. Results show
that the emotional agent has better results than the others. The authors highlight the importance of intentions in the practical reasoning process: they drive means-end reasoning,
they constrain future deliberation, they are influenced by and they influence the emotions.
The work of [Jiang et al., 2007] is clearly motivated and well structured, from the conceptual model up to the running scenario. The main concern of the authors is decision making:
how do emotions affect decision making and how can we use this knowledge to apply it in
intelligent agents. It is a good starting point for our work.
The most recent paper discussing emotional BDI architectures is that of Hazaël Jones,
Julien Saunier and Domitile Lourdeaux from France [Jones et al., 2009]. It refers both
19
[Jiang et al., 2007] and [Parunak et al., 2006]. The authors reproach to the former that it
misses personality and physiology aspects and to the latter that is limited, modeling only
two emotions in relation with two personality aspects. Their main motivation lies in the
context of global security, and their goal is to simulate a crisis situation in a virtual reality
environment. The agents in this system follow an architecture which is based on BDI, but
in addition contains emotions, personality and physiology. The authors state that these are
key concepts in the context of crisis management, but they do not motivate their options.
The set of parameters that they consider relevant in a crisis situation are shown in Fig. 4.2.
Figura 4.2: Concepts necessary in handling crisis situation
Beliefs are acquired through perception, but there are three perception functions: sensing
the environment, communication and pshysiology. Emotions are of two types - primary and
secondary - and there is a different update function for each of them. The paper presents the
agent control loop and the update functions, and illustrates the decision making process with
an algorithm execution example on a scenario which involved escaping from a fire. Having
a predefined focus on global security, the architecture is very specific. One should note
that physiology parameters are not necessary in a non-crisis scenario. In such a situation,
and where performance is the main focus, resources could be a more suitable component to
consider.
In parallel with the above mentioned systems, where later models take into consideration
the earlier ones, two more architectures were developed, both in the topic of emotional BDI
models, but eachone independent of previous works and of one another.
The first one (in arbitrary order) is that of David Hernandez, Oscar Deniz, Javier Lorenzo
and Mario Hernandez from Spain [Hernandez et al., 2004]. The authors developed a modular architecture, named BDIE, which contains four sperate modules: Perceptual System
(mapped to Beliefs), Motivational System (mapped to Desires), Behavior System (mapped
to Intentions) and Emotional System (mapped to emotions). The latter module influences
all the other modules and is influenced by the perceptual system and by the motivational
one. Emotions modify the perceptual process and, when primary emotions become active,
they activate certain goals. Additionally, emotions influence the action selection algorithm
by narrowing the set of options available to execute and also produces affective behavior
20
like facial expressions. An important factor that increases the architecture’s modularity is
the fact that the emotional module is not directly connected to intentions, but it does so
through goals. A characteristic of the BDIE architecture is represented by the two level
beliefs, evaluators and emotions. The first level beliefs are acquiered through perception.
The first level evaluator performs an affective appraisal of the first level beliefs, giving the
primary emotions (fear, surprise). If one of these is active, control passes to the planning
algorithm. Otherwise, second level beliefs are infered from first level beliefs and then the
second level evaluator performs a cognitive appraisal of the second level beliefs and returns
secondary emotions (happiness, sadness, anger). There are third level beliefs and tertiary
emotions (shame and pride), but only at a conceptual level, not implemented in the actual
architecture. The emotional space is a continuous two dimensional space divided in emotional sectors; the current emotion is obtained by matching valence and arousal values and
to the corresponding sector. Desires comprise goals (state of affairs which needs pro-active
behavior to be achieved) and homeostatic variables (state of affairs that requires pro-active
behavior if they are not achieved). The Behavior System contains the planning algorithm,
which is interchangeable. However, no details are given about the deliberation or means-end
reasoning processes, the paper doesn’t explain how the agent chooses its desires and intentions. The experiments conducted are simple and don’t require such considerations: the only
desire is survival and the only behavior displays facial expressions baed on some pictures
shown (which induce emotions into the agent). The modular structure of the system can be
appointed as the most interesting aspect of the BDIE architecture.
The second architecture is called BDE and was developed by Adina Florea and Eugenia
Kalisz [Florea and Kalisz, 2004]. The start point on creating this architecture is behavior
anticipation: their thesis is that emotions contribute to behavior anticipation, which in turn
contributes to making an agent more realistic. Scott Reilly makes a clear separation in his
PhD thesis [Reilly, 1996] between realistic and believable characters: realistic are those characters that mimic the reality as accurate as possible, while believable refers to characters
that may alter the reality, but allow users that watch or interact with them to suspend their
disbelief. An unrealistic character can be believable, for example a talking animal. Consequently, [Florea and Kalisz, 2004] are interested in simulating realistic human reasoning
and behavior. They note that there are several moderators that influence human behavior,
including emotions and personality, but they only consider emotions in their model. In addition, they show that an agent has an emotional state, which is the integration of several
emotions that an agent has following several events. The integration is said to be done using
a rule-based approach, but the rules are said to be under study. Also, the authors account
for emotion decaying, but also for emotional memory; this is valid in the case of strong feelings, when particular powerful emotions remain in memory, even if they are momentarily
decayed. To pass from the generality of the emotion reasoning functions presented as part
of the general BDE architecture, the paper specifies the emotion eliciting conditions and the
influence of emotional state on behavior for seven emotions: satisfaction, joy, hope, sadness,
anger, fear and disappointment. Fig. 4.3 shows in detail these interactions, emphasizing the
cognitive appraisal process: the events are evaluated based on the agent’s beliefs, desires
21
and intentions, emotions appearing as a result of this process. Further, emotions influence
the beahvior of the agents also through its beliefs, desires and intentions, by continuously
revising them and taking the next decisions accordingly.
Figura 4.3: Emotion Eliciting Conditions and Influence of Emotional State on Behavior
The paper focuses on the influence of emotions on deliberation and means-end reasoning,
leaving the planning process for future analysis. The architecture presented is not very complex. In contrast, the EBDI model [Jiang et al., 2007] takes into account both primary and
secondary emotions, so two apprasal process must take place, one faster and one slower (but
more accurate), and also a refinement of the decision making process, based on secondary
emotions. Although this mechanism is closer to the human mechanism, the BDE model
appears to be easier to implement, and its simplicity makes it more appropriate in static (or
not very dnamic) environments.
TABLE 4.1 summarizes the characteristics of the presented models. In what follows, we
discuss in more detail the similarities and differences between them.
First of all, one can note that each paper is concerned with different issues:
• [Pereira et al., 2005] - BDI improvement
• [Parunak et al., 2006] - faster-than-real-time simulation of a combat situation with
large number of combatants
• [Jiang et al., 2007] - improved decision-making
• [Jones et al., 2009] - handling crisis situations
• [Hernandez et al., 2004] - performance, human-computer interaction
• [Florea and Kalisz, 2004] - behavior anticipation
22
Components
Model
[Pereira et al., 2005]
Belief
Desire
Intention
Emotion
Other
- acquired through perception & inference
- one revision function,
distinct algorithms (depending on resources, capabilities and emotional
state)
- mapped to digital pheromones
- revised through distinct
algorithms, depending on
resources, capabilities and
emotional state
- revised through distinct
algorithms, depending on
resources, capabilities and
emotional state
- Emotional State Manager - not detailed
- resources
- capabilities
- Sensing and Perception
Module - rules that bind
stimuli to concepts
- wants
- constant over time
- acquired through perception, communication &
contemplation
- three revision functions
- not influenced by emotions
- OCC (result of appraisal
process)
- triggered by beliefs, depending on disposition
- primary
- secondary
- Disposition, Emotion,
Trigger, Tendency
[Jiang et al., 2007] (EBDI)
[Jones et al., 2009]
BDI)
- acquired through perception, communication &
physiology
- one revision function,
three perception functions
- first level (acquired through perception)
- second level (aquired
through first level belief
revision)
- third level (aquired through second level belief revision - not implemented)
- acquired through perception & inference
- one revision function
- options
- influenced by beliefs, intentions, personality and
physiology, but not by
emotions
- goals (to be achieved)
- homeostatic variables (to
be maintained)
- result of analysis process
- tendency imposed by
emotions
- highly important
- influenced by emotions,
desires & intentions
- state of affairs that the
agent has commited to
achieve
- filtered options
- influenced by beliefs, desires, intentions, emotions
and physiology, but not by
personality
- separate module, contains the planning algorithm, which can be replaced
- connected with emotions
through goals
- primary
- secondary
- Personality,
Physiology
-primary & secondary
- emotional space divided
in emotional sectors
-
- course of action to be taken in order to achieve desires
-
- emotion intensity - given
by:
— desire’s preference
— desires’s plausability
— event’s expectability
- emotion integration rule-based approach
[Parunak et al., 2006]
(DETT)
(PEP-
[Hernandez et al., 2004]
(BDIE)
[Florea and Kalisz, 2004]
(BDE)
- influenced only by the
emotional state
OCC
emotional state
decay
emotional memory
-
Tabela 4.1: Comparison of existing emotional BDI systems
Secondly, there are also obvious differences with respect to the architecture components.
[Jiang et al., 2007] considers three different functions for acquiring beliefs: through percepts sensed in the environment, through messages received from other agents and through
contemplation. [Pereira et al., 2005], [Parunak et al., 2006], [Hernandez et al., 2004] and
[Florea and Kalisz, 2004] don’t take into consideration communication, but they have only
one belief revision function that includes both new percepts and current beliefs. In fact,
contemplation refers to updating current beliefs based on what is perceived in the environment. [Jones et al., 2009] also considers one revision function defined over current beliefs
and new percepts, but these new percepts can be obtained through sensing the environment,
receiving communication messaged and through physiology parameters (that define agent
health).
Next, the reasoning process happens differently. [Parunak et al., 2006] consider desires to
be constant over time, while in [Florea and Kalisz, 2004] emotions may influence them (but
only emotions, no other component). Starting from this, the deliberation process takes the
desires, beliefs, emotions and current intentions, and returns the updated intentions (which
can be executable actions or can be composed of other intentions and/or executable actions).
The means-end reasoning process structures the intentions into a plan - a set of actions to
be executed. For [Florea and Kalisz, 2004] and [Parunak et al., 2006], desires are predefined
and need to be filtered only if they are inconsistent. A set of consistent desires forms the
goals of the agent. [Pereira et al., 2005], [Jiang et al., 2007] and [Jones et al., 2009] consider
23
Emotion,
desires as options that need to be constantly generated based on current desires and beliefs.
In [Jiang et al., 2007] and [Jones et al., 2009], options and filter (the functions that generate
desires and intentions respectively) represent the deliberation process; next, the meansend reasoning is done by the plan function, which structures intentions into plans. In
[Pereira et al., 2005] on the other hand, the first process that occurs is generate options,
which is responsible for means-end reasoning; this generates desires and intentions that
hierarchically flow from abstract to concrete, until executable actions are obtained. Then,
the deliberation process (filter) chooses from these intentions, the ones that the agent will
be commited to.
What all the papers have in common is the focus on emotion influence on deliberation
and means-end reasoning. How the affective state affects the planning algorithm is not a
priority for any of the authors.
The commitment strategy is yet less discussed, althought the trade-off between reconsideration and the degree of commitment (or otherwise said, between reactive (event driven)
and pro-active (goal driven) behavior) is an important aspect of practical reasoning. From
the agent control loops presented it can be infered that [Florea and Kalisz, 2004] uses an
open-minded strategy (the agent is commited to an intention as long as it is still believed possible), while [Pereira et al., 2005], [Jiang et al., 2007] and [Jones et al., 2009] use a
single-minded strategy (the agent is commited to an intention until it is either realized or
not possible).
[Pereira et al., 2005], [Jones et al., 2009] and [Florea and Kalisz, 2004] are theoretical
papers. The first one describes only a conceptual architecture, which lacks an actual structure of the Emotional State Manager module and also some concrete examples or scenarios.
The other two tend to be more concrete: the former presents an example of the algorithm execution that illustrates the decision process, while the latter exemplify the emotion
generation and influence processes for a set of seven basic emotions. On the other hand,
[Parunak et al., 2006], [Jiang et al., 2007] and [Hernandez et al., 2004] are more practical:
they implement their agents and show experimental results that prove the functionality of
their model. Thus, [Jiang et al., 2007] runs both emotional and non-emotional agents in
Tileworld, showing that the EBDI agent has a better performance; [Hernandez et al., 2004]
implement their architecture on a robotic head which changes its facial expression based on
color and luminance of an image. Each of the two papers creates a general architecture,
which is then tested on some scenario. But [Parunak et al., 2006] create an architecture
which is specific to a combat situation, mapping to concrete beliefs, desires and emotions;
they also present some experimental results of the DETT agent model performing in two
different scenarios (given by two distinct projects).
24
Chapter 5
Proposed architecture
Now that we have reviewed several architectures based on beliefs, desires, intentions and
emotions, we can take advantage of all these contributions to build an improved agent
model that takes into consideration all the issues presented. Nevertheless, we will focus on
a general architecture which enhances agent performance, simulates human mechanisms of
internal resource usage and accounts for different agent personalities.
5.1
Agent Model
The EBDI model [Jiang et al., 2007] is the closest work to ours: our main goals are agent
performance and resource usage; the focus of [Jiang et al., 2007] is on decision making and
taking the right decisions improves agent performance. But we are also interested in resource usage, while [Jiang et al., 2007] ignore it, considering it useless and disregarding
[Pereira et al., 2005] for including resources and capabilities in their model.
The elements that we aim to integrate in our architecture are:
• PERCEPTS
– anything that comes from the environment: stimuli or messages from other agents
– influenced by emotions
• BELIEFS
– acquired from percepts
– revised to account for current beliefs and new percepts
– influenced by emotions
• DESIRES
– goals received by the agent at design time
– constant over time
• OPTIONS
25
– alternatives to accomplish the desires
– generated based on current beliefs and intentions
– influenced by emotions
• INTENTIONS
– options that the agent has commmited to
– revised based on current intentions, beliefs and options
– influenced by emotions and available resources
– open-minded commitment - agent is commited to the intention as long as it is
not achieved yet, it is not believed impossible to achieve and it is still a goal for
the agent
• EMOTIONS
– primary & secondary emotions
– primary emotions may determine instinctual behavior
– secondary emotions influence cognitive processes and available resources
– fixed set of eotions for each scenario
• PERSONALITY
– two axes: extrovert-introvert, pshychologically stable-unstable
– four types: sanguine, choleric, melancholic, phlegmatic
• RESOURCES
– maintained in a structure that gives access to them selectively, based on emotions
– fixed set of resources for each scenario
Fig. 5.1 ilustrates the proposed architecture.
There are several processes that take place within the agent control loop, all of which are
precisely shown in Fig. 5.2. In what follows, we will explain in detail this process, walking
step by step through the internal mechanism of the agent.
First, the agent perceives the stimuli in the environment. These can be the occurence of
an action, a change in the state of the world or a message sent by another agent. Emotions
influence the perception processes, an event being perceived with a positive bias when in a
positive mood and with a negative bias when in a negative mood. The perceive function is
therefore:
perceive : Env × E → P
Percepts are quickly appraised at an affective level and may determine the experience of
primary emotions. The primary emotions update process peu also accounts for the intentions
of the agent and is influenced by its current emotions:
peu : P × E × I → E
26
Figura 5.1: Proposed architecture
When the intensity of the new emotions passes a certain treshold , primary emotions
may be responsible for the agent executing some predefined reactive behavior. For example,
strong fear can determine a sudden fall back or maybe, on the contrary, a sudden stroke on
what caused the fear (depending on the personality). Percepts give the context and available
resurces determine the action that is to be taken, because an instinctual reaction has to do
with something that the agent already has or knows. If it doesn’t know a default behavior,
it has to reason about it, so no reactive actions are taken. The react function returns a plan
which generally contains only one action:
react : E × P × AR → π
We have seen that primary emotions generates an instinctual behavior. It may prove
to be based on wrong premises, but it is also very useful in survival situations, when quick
decisions are necessary. In the figure, the input for the react function has dotted lines to
higlight that the process is not guaranteed to happen at every step, but it is a matter of
strong, sudden emtions.
27
1. D = D0 ;
2. while true do
3.
P ← perceive(Env, E);
4.
E ← peu(P, E, I);
5.
if intensity(E) > then
6.
π ← react(E, P, AR);
7.
P ← perceive(Env, E);
8.
end if
9.
B ← brf (P, B, E);
10.
E ← seu(B, E, I, AR);
11.
12.
AR ← ru(R, E);
O ← analyzer(B, D, I, E);
13.
I ← f ilter(O, B, E);
14.
π ← plan(I, AR);
15.
while not (empty(π)
|| succeeded(I, B)
|| impossible(I, B)) do
16.
α := head(π);
17.
execute(α);
18.
π := tail(π);
19.
P ← perceive(Env, E);
20.
E ← peu(P, E, I);
21.
if intensity(E) > then
22.
π ← react(E, P, AR);
23.
P ← perceive(Env, E);
24.
end if
25.
B ← brf (P, B, E);
26.
E ← seu(B, E, I, AR);
27.
AR ← ru(R, E);
28.
if reconsider(I, B) then
29.
O ← analyzer(B, D, I, E);
30.
I ← f ilter(O, B, E);
31.
end if
32.
if not sound(π, I, B) then
33.
π ← plan(I, AR);
34.
35.
end if
end while
36. end while
Figura 5.2: Agent control loop
28
After the initial instinctual behavior, the percepts influence belief revision. This process
must account for both current beliefs and new percepts, acting like a truth maintenance
system which outputs consistent beliefs. Current emotions also influence this process:
brf : P × B × E → B
After belief revision, secondary emotions are being computed. These emotions are the
result of a cognitive appraisal of the situation. This appraisal process takes into consideration
not the raw percepts, but the newly revised beliefs. In addition, it is influenced by the
available resources. Intentions and current emotions still count in the secondary emotion
update function seu:
seu : B × E × I × AR → E
But emotions in its turn influence the available resources that an agent perceives, a
process that models human resource usage: when experiencing fear, one can run faster or
be stronger. Thus, at a given moment, depending on the affective state, an agent can have
access to different resources. The resources update ru function is defined by:
ru : R × E → AR
Next follows the two processes specific to practical reasoning, deliberation (modeled through analyzer and filter functions) and means-end reasoning (modeled through plan function).
The analyzer function determines the available options that the agent has to accomplish its
desires, based on its beliefs, intentions and emotions.
analyzer : B × D × I × E → O
The filter function chooses from the available options those that the agent will be commited to. The selected options are the agent intentions. Beliefs give the context in which
this decision is made, while emotions help speed up the process by eliminating options that
could lead to unwanted affective states.
f ilter : O × B × E → I
After deciding on the intentions, the agent must plan on how to achieve them, taking
into consideration available resources:
plan : I × AR → π
In the end, the plan is ready to be executed:
execute : π → Env
But during this process, the agent still has to pay attention to the stimuli in the environment. If the plan contains only one action (typical case of the reactive behavior), it
29
can simply execute it. But when it contains several actions, the environment can change its
state during execution, so the agent should not omit to reconsider its plan or its intentions.
Thus, the agent goes on with its plan as long as its intentions are not accomplished yet and
they are believed to be achievable. Moreover, in the light of new events, the intentions could
no longer be necessary to be achieved, so the agent should consider droping them. This is
done in the in the reconsider function:
reconsider : I × B → true, f alse
If this functions returns true, then the agent restarts the deliberation process, returning
renewed intentions. This approach is known as open-minded commitment strategy relative
to intentions. In addition to intention commitment, the agent must also be commited to the
plan adopted, otherwise it could simply stop executing it. But at every step the agent also
must test the soundness of the plan in relation to the intentions, and replan if necessary:
sound : π × I × B → true, f alse
5.2
Emotions
Emotions are the result of appraising an event occuring in the environment. The model we
will use in our architecture is the circumplex model of affect, where affect is represented in a
2D emotional space determined by the valence and arousal axes (see Section 2.1 in Chapter
2). Consequently, we thus can assume that an emotion is a point in this 2D space given
by the pair (v, a), where both v and a are rational numbers within given intervals: v is the
value of the valenece and a is the value of the arousal.
Following the OCC model of emotions [Ortony et al., 1988], an event is appraised in
terms of beliefs, desires and intentions, returning a certain score regarding its valence (positive or negative) and arousal (the intensity of emotion felt). The name of the emotion is less
important, its role is just to make the emotion understandable by humans. Nevertheless, we
will define a set of emotions for each scenario developed, each emotion occupying a certain
surface in the valence-arousal space.
We believe the circumplex model of affect to be an appropriate choice for modeling
our agent because it gives the possibility of having different sets of emotions on different
scenarios, depending on how the events are affecting the agent. Even if there is an accepted
set of basic emotions, they are not always suitable in every context; for example, in a fire
scenario it is less likely that one will feel joy. Most probably, a drama scenario may activate
emotions in the upper left quadrant, while a romance scenario may activate emotions in the
lower right quadrant.
30
5.3
Resources
Resources are an important component of this architecture. They have a central role in
decision making because decisions are based on the available resources that the agent knows
it has. Our aim is to model the mechanisms that humans use regarding resource usage in
critical situations. It is well known that in crucial contexts, people suddenly may appear to
have more power than they usually do. Thus, one may run at a certain speed when their
affective state is in the comfort zone. But in a threatening situation, the emotional intensity
heightens as a result of trying to escape from the threat. The arousal opens new paths in the
person’s capabilities, making them able to run faster. This is exhausting, so if not necessary,
it is pointless to waste all the energy in a single action, but if the situation requires it, it is
desirable. The same goes with agents.
In our architecture, we define three types of resources, each being easier or harder to
access. Each type forms a set of resources which can be accessed in different conditions.
The A type of resources are the most accessible ones. They are available in common, nonthreating situations and thus they can always be used by the agent. Type B defines resources
that the agent can reach only in situations which heightens its emotional intensity over a
given threshold. And the C type of resources are used in vary rare ocasions, when the
agent is aroused by dangerous situations which threaten agent survival. Fig. 5.3 shows the
relation between arousal and resource accessibility. The figure is drawn in correlation with
the Yerkes-Dodson law of arousal and performance to highlight the fact that more resources,
if used properly, enhance performance.
Figura 5.3: Agent accessibility to its resources
The picture depicts the emotion intensity tresholds which determine the type of resources
the agents has access to. Thus, if we define by ei the emotion intensity, and by p the
performance, we have the following cases:

access to resources of type A
0 ≤ ei ≤ ei1 ⇒
p ≤ p1
31

access to resources of type A & B
ei1 < ei ≤ ei2 ⇒
p1 < p ≤ p2

access to resources of type A & B & C
ei2 < ei ≤ ei3 ⇒
p2 < p ≤ pmax

access to respurces of type A & B
ei3 < ei ≤ ei4 ⇒
p1 < p ≤ p2

access to resources of type A
ei ≥ ei4 ⇒
p ≤ p1
5.4
Personality
Personality is defined by the innate characteristics plus the learned habits. The habits are
acquired through experience, being influenced by the society in which one lives. Innate
characteristics describe what is called temperament and this is what we focus on - we
model personality through the four well known temperament types: sanguine, choleric,
melancholic and phlegmatic. These are represented by the four areas delimited by two axes
on a 2D surface: extraversion axis (extrovert/introvert) and neuroticism axis (psychologically
stable/unstable). We chose this model because of its simplicity and of a correspondence
that can be made between the two axes and the parameters in our agent architecture. The
mapping is shown in Fig. 5.4 and is explained below.
Figura 5.4: The four temperaments
• introvert/extrovert axis - associated with arousal (emotion intensity)
– introvert - low arousal values are sufficient for the agent to have a good performance (according to Yerkes-Dodson law)
– extrovert - high arousal values are necesary for the agent to have a good performance (according to Yerkes-Dodson law)
32
• psychologically stable/unstable axis - associated with decay rate of emotions
– stable - low decay rate - agent has a rather coherent affective state
– unstable - high decay rate - agent quickly moves from one emotional state to
another
Therefore, the four temperament types have the following characteristics:
Temperament
Sanguine
Extroversion
extrovert
Neuroticism
stable
Choleric
extrovert
unstable
Melancholic
Phlegmatic
introvert
introvert
unstable
stable
Characteristics
-high emotion intensity
-low decay rate
-high emotion intensity
-high decay rate
- decay rate
-low emotion intensity
-low decay rate
Tabela 5.1: Personality modeled through temperament
33
Chapter 6
Conclusions
In this report we made a review of the current state of the art in affective computing. We
started from the very beginning, explaining the context and motivation for our work in
this field, and then we presented the two main directions that research is taking: emotion
recognition and emotion simultaion. We presented in detail affect detection techniques and
then presented just a few of the multitutide computational emotional models that were
developed up to present.
We believe emotional BDI models are an appropriate model for our purposes and therefore we insisted with a detailed presentation of seven papers that draw on emotional BDI
agents. The first one is just a first step in this direction, in which emotions are not treated as
independent concepts, but as other type of beliefs. The reasoning process is the same with
that in the BDI model, with the addition that in [Padgham and Taylor, 1997] the agent may
have beliefs about its feelings. The next six papers present architectures which encapsulates
emotions as main components in the reasoning process. We have detailed the similarities
and the differences between them and we have emphasized the advantages and disadvantages of each of them. Some of these papers describe a general architecture, able to function
on various scenarios, but others design the architecture fitted for a concrete scenario known
apriori.
As for our purpose, we are interested in designing an agent with increased performance
and improved resource usage, in a context that is not previously established. Therefore,
we were specifically attracted by the EBDI model [Jiang et al., 2007], because of its proved
improved performance of the emotional agents over the non-emotional agents; also, the model
of [Pereira et al., 2005] presented interest for their inclusion of resources in the architecture
as independent concepts and last, but not least, the cognitive appraisal theory applyed
on beliefs, desires and intentions, as described in [Florea and Kalisz, 2004]. We saw the
other three papers to be more specific-oriented, with the architectures adapted to concrete
scenarios.
34
Consequently, in the second part of this paper we present in detail our architecture: the
functions that are to be executed at each step, how are the emotions going to be represented and appraised, how are the resources goint to be represented and used, and lastly
what personalities will the agents have and how are these going to influence their behavior
concretely. It is a model built after analyzing several emotional BDI models, looking at their
advantages and disadvantages and taking the most relevant components and methods from
each of them, such that eventually the newly designed agent will accomplish the two goals
initially established: performance and efficient resource usage.
In the near future we plan to implement an agent using the new proposed architecture
and to develop some scenarios in which this agent could act. The goal is to find out if our
model has indeed a better performance and improved resource usage, as we aim to. We will
have to run different tests on at least two scenarios, changing the parameters - personality
type and conditions for resource accessibility - and to observe what are the results obtained.
In addition to working at the computational model developed, we plan on doing a little
research on emotion recognition, using Microsoft Kinect’s new Face Tracking SDK to detect
facial expressions and then to interpret them according to Ekman’s Action Unit codes.
35
6.1
Acknowledgments
This work is supported by University Politehnica of Bucharest and POSDRU Grant No.
POSDRU/107/1.5/S/76813.
36
References
[Adam et al., 2009] Adam, C., Herzig, A., and Longin, D. (2009). A logical formalization
of the occ theory of emotions. Synthese, 168(2):201–248.
[Badler and Smoliar, 1979] Badler, N. and Smoliar, S. (1979). Digital representations of
human ovement. Computing Surveys, 11(1):19–38.
[Bratman et al., 1988] Bratman, M., Israel, F., and Pollack, M. (1988). Plans and resourcebounded practical reasoning. Computational Intelligence.
[Calvo and D’Mello, 2010] Calvo, R. and D’Mello, S. (2010). Affect detection: An interdisciplinary review of models, methods, and their applications. IEEE Transactions on
Affective Computing, 1(1):18–37.
[Cannon, 1927] Cannon, W. (1927). Bodily Changes in Pain, Hunger, Fear and Rage - an
account of recent researches into the function of emotional excitement. D. Appleton and
Company.
[Cytowic, 1996] Cytowic, R. (1996). The Neurological Side of Neuropsychology. MIT Press.
[Damasio, 1994] Damasio, A. (1994). Descartes’ Error: Emotion, Reason and the Human
Brain. Gosset/Putnam Press.
[Darwin, 1872] Darwin, C. (1872). The expression of the emotions in man and animals.
London: John Murray.
[Dastani and Meyer, 2006] Dastani, M. and Meyer, J.-J. C. (2006). Programming agents
with emotions. In Proceedings of the 17th European Conference on Artificial Intelligence
(ECAI06), page 215219.
[Dellaert et al., 1996] Dellaert, F., Polzin, T., and Waibel, A. (1996). Recognizing emotion
in speech. In Proceedings of International Conference on Spoken Language Processing,
page 19701973.
[Desmet, 2002] Desmet, P. (2002). Designing Emotions. PhD thesis, Delft University of
Technology.
[D’Mello and Graesser, 2009] D’Mello, S. and Graesser, A. (2009). Automatic detection of
learner’s affect from gross body language. Applied Artificial Intelligence, 23:123–150.
37
[Ekman, 1994] Ekman, P. (1994). The Nature of Emotion: Fundamental Questions, chapter
Moods, Emotions and Traits. Series in Affective Science. Oxford University Press.
[Ekman, 1999] Ekman, P. (1999). The Handbook of Cognition and Emotion, chapter Basic
Emotions, pages 45–60. John Wiley & Sons Ltd.
[Ekman and Friesen, 1978] Ekman, P. and Friesen, W. (1978). Facial Action Coding System:
Investigator’s guide. Consulting Psychologist Press.
[Elliot, 1992] Elliot, C. (1992). The Affective Reasoner: A process model of emotions in a
multi-agent system. PhD thesis, Northwestern University.
[Florea and Kalisz, 2004] Florea, A. M. and Kalisz, E. (2004). Behavior anticipation based on beliefs, desires and emotions. International Journal of Computing Anticipatory
Systems, 14:37–47.
[Goleman, 1996] Goleman, D. (1996). Emotional Intelligence: Why It Can Matter More
Than IQ. Bloomsbury Publishing PLC.
[Gratch and Marsella, 2004] Gratch, J. and Marsella, S. (2004). A domain-independent
framework for modeling emotion. Journal of Cognitive Systems Research.
[Hebe et al., 2005] Hebe, N., Cohen, I., and Huang, T. (2005). Handbook of Pattern Recognition and Computer Vision, chapter Multimodal Emotion Recognition. World Scientific.
[Hernandez et al., 2004] Hernandez, D., Deniz, O., Lorenzo, J., and Hernandez, M. (2004).
Bdie: a bdi like architecture with emotional capabilities. In Papers from AAAI Spring
Symposium: Architectures for Modeling Emotion: Cross-Disciplinary Foundations.
[James, 1884] James, W. (1884). What is an emotion? Mind, (9):188–205.
[Jiang et al., 2007] Jiang, H., Vidal, J., and Huhns, M. (2007). EBDI: An architecture for
emotional agents. In Proceedings of the 6th International Joint Conference on Autonomous
Agents and Multiagent Systems (AAMAS’07), pages 38–40.
[Jones et al., 2009] Jones, H., Saunier, J., and Lourdeaux, D. (2009). Personality, emotions
and physiology in a bdi agent architecture: the pep -¿ bdi model. In Proceedings of WI-IAT
- IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent
Agent Technology, volume 2, pages 263–266.
[Kassin, 2004] Kassin, S. (2004). Psychology. Prentice Hall, 4 edition.
[LeDoux, 1996] LeDoux, J. (1996). The Emotional Brain. Simon&Schuster.
[Meyer, 2006] Meyer, J.-J. C. (2006). Reasoning about emotional agents. International
Journal of Intelligent Systems, 21(6):601619.
[Moshkina and Arkin, 2009] Moshkina, L. and Arkin, R. (2009). Beyond humanoid emotions: Incorporating traits, attitudes and moods. In Proceedings of 2009 IEEE Workshop
on Current Challenges and Future Perspectives of Emotional Humanoid Robotics, IEEE
International Conference on Robotics and Automation(ICRA 2009).
38
[Mota and Picard, 2003] Mota, S. and Picard, R. (2003). Automated posture analysis for
detecting learners interest level. In Proceedings in Computer Vision and Pattern Recognition Workshop, volume 5, pages 49–54.
[Oatley and Jenkins, 1996] Oatley, K. and Jenkins, J. (1996). Understanding Emotions.
Blackwell Publishing, Oxford, UK.
[Ortony et al., 1988] Ortony, A., Clore, G., and Collins, A. (1988). The cognitive structure
of emotions. Cambridge University Press.
[Padgham and Taylor, 1997] Padgham, L. and Taylor, G. (1997). A system for modelling
agents having emotion and personality. In Proceedings of PRICAI Workshop on Intelligent
Agent Systems: Theoretical and Practical Issues, Lecture Notes in AI 1209, pages 59–71.
Springer-Verlang.
[Parunak et al., 2006] Parunak, H. V. D., Bisson, R., Brueckner, S., Matthews, R., and
Sauter, J. (2006). A model of emotions for situated agents. In Proceedings of 5th International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS’06),
pages 993–995.
[Pereira et al., 2005] Pereira, D., Oliveira, E., Moreira, N., and Sarmento, L. (2005).
Towards an architecture for emotional BDI agents. In IEEE Proceedings of the 12th
Portuguese Conference on Artificial Intelligence (EPIA’05), pages 40–47. Springer.
[Petrushin, 1998] Petrushin, V. (1998). How well can people and computers recognize emotions in speech? In Proceedings of AAAI Fall Symposium, pages 141–145.
[Picard, 2000] Picard, R. (2000). Affective Computing. MIT Press.
[Plutchik, 2001] Plutchik, R. (2001). The nature of emotions. American Scientist.
[Rao and Georgeff, 1991] Rao, A. and Georgeff, M. (1991). Modeling rational agents within
a BDI-architecture. In Proceedings of Knowledge Representation and Reasoning(KR&R91), pages 473–484.
[Reilly, 1996] Reilly, S. (1996). Believable Social and Emotional Agents. PhD thesis, Carnegie Mellon University.
[Russel, 1980] Russel, J. (1980). A circumplex model of affect. Journal of Personality and
Social Psychology, 39(6):1161–1178.
[Russell, 1979] Russell, J. (1979). Affective space is bipolar. Journal of Personality and
Social Psychology, 37(3):345–356.
[Schachter and Singer, 1962] Schachter, S. and Singer, J. (1962). Cognitive, social and psychological determinants of emotional state. Psychological Review, 69(5):379–399.
[Scherer, 1996] Scherer, K. (1996). Adding the affective dimension: A new look in speech
analysis and synthesis. In Proceedings of International Conference on Spoken Language
Processing, page 18081811.
39
[Seif El-Nasr et al., 1999] Seif El-Nasr, M., Ioerger, T., and Yen, J. (1999). PETEEI: A pet
with evolving emotional intelligence. Technical report, Texas A&M University.
[Smith and Kirby, 2009] Smith, C. and Kirby, L. (2009). Putting appraisal in context:
Toward a relational model of appraisal and emotion. Cognition and Emotion, 23(7):1352–
1372.
[Steunebrink, 2010] Steunebrink, B. (2010). The logical structure of emotions. PhD thesis,
Utrecht University.
[Stork, 1996] Stork, D., editor (1996). HAL’s legacy: 2001’s Computer as Dream and Reality. MIT Press.
[van Gorp, 2006] van Gorp, T. (2006). Emotion, arousal, attention and flow: Chaining
emotional states to improve human-computer interaction. Master’s thesis, University of
Calgary, Faculty of Environmental Design.
[Yang et al., 2012] Yang, N., Muraleedharan, R., Kohl, J., Demirkol, I., Heinzelman, W.,
and Sturge-Apple, M. (2012). Speech-based emotion classification using multiclass svm
with hybrid kernel and threshholding fusion. In 2012 IEEE Workshop on Spoken Language
Technology. http://www.rochester.edu/news/show.php?id=5072.
40