Behavior Planning for Lifelike Characters and Avatars
1
Behavior Planning for Lifelike Characters and Avatars
Riccardo Antonini1
1
Consorzio Roma Ricerche, Università di Roma “Tor Vergata”, Via O. Raimondo, 8,
00173 Rome, Italy
[email protected]
Abstract. In virtual worlds the behavior of avatars is normally goverened
directly by the user input. In order to relieve the user from the burden of giving
too many inputs preserving at the same time the possibility of expressing
himself by means of the avatar we have devised an evolutionary procedure that
allows the avatar to evolve its own behavior. The avatar has a repertoire of
behaviors, only the behavior successful in bringing the avatar in contact with
other avatars for a sufficient time will survive and reproduce into a new
repertoire.
1
Introduction
The research described in this paper has been carried out as a part of the i3 Project
Amusement-Esprit project 25197.
In the Amusement Project one of the goals is to help people interact at distance using
virtual worlds. The quality of interaction is strictly linked to the possibility of
expressing oneself with a rich facial and body language. This rich language has to be
created and up to now the avatars needed special programming for the
implementation of gestures and all the user could do was to trigger a preprogrammed
gesture. We have divided the problem into two parts: the automatic generation of
avatar gestures that is discussed in [1] and the automatic association of a given
situation to an appropriate gesture.
2
State of the Art
There has been a number of experiments on putting some sort of “artificial life” into
avatars in particular, and into artificial creatures in general. The evolution of artificial
creatures (that in all of the following cases are not avatars per se) has been studied for
example by Karl Sims [8] and Steve Grand [5]. Here we will briefly analyze those
two works.
The idea of the adaptability of genetically-inherited autonomous behavior has, in the
work of Sims and Grand, two different meanings. Both meanings differ from the one
adopted in our work.
Grand’s creatures inherit a neural structure capable of (Hebbian) learning.
Behavior Planning for Lifelike Characters and Avatars
2
Sims's creatures, on the other hand, inherit neural structures that are capable only of a
behavior that is fixed for any given individual through all its life span. In other words:
one has to wait for the next generation in order to have behavioral changes.
Besides what has already been said, we can take those two examples, Grands and
Sims, as paradigms of what Darwin [2] called artificial (human) selection and natural
selection respectively. It could be also observed, incidentally, that the world of
Grand [5] is bi-dimensional whereas the world of Sims [9] is three-dimensional.
Despite this minor difference, both share a great emphasis on linking the internal
physical structure model with the behavioral model.
It also has to be taken into account that, sometimes, in virtual world literature, such as
[7], computer game characters, which show just adaptive autonomous behavior, are
referred to as “artificial life creatures.” We will use, here, the term “artificial life
creature” in a stronger sense only, referring only to a creature that has some sort of
genetic evolutionary capability for behavior.
In conclusion, we have not found in the field any agent which exists in a multi-user
virtual environment as an avatar and also is capable of evolvable behavior. Until now,
these two characteristics appeared to be mutually exclusive.
3
Theoretical Foundations
The theoretical foundation that underlies our work hypothesis stems from the study of
neo-Darwinism and in particular from the position of Richard Dawkins [3]. His most
relevant observation, as far as our work is concerned, is that selection acts on the
effects of the phenotypic structure (rather than on the structure itself). The beautiful
example reported by Dawkins [3], about the evolution of the dam construction
behavior of a beaver, shows that the selection is not on the physical structure, but on
the effects of the physical structure. It appears necessary then that the normal
GENOTYPE-PHENOTYPE PAIR be extended to what we may call the TRIPLET
GENOTYPE-PHENOTYPE-TECHNOTYPE. We share Dawkin’s view that the
natural selection does NOT select structures but the USES of the structure.
In the case of artificial agent’s behavior the above implies that an algorithm modelling
the evolution of such behavior should be largely independent from the structure
modelling the said behavior.
4 Avatar’s
Estimation
Intelligent
Behavior
As
Adaptive
Model-Free
Our approach to “intelligent behavior" is based on the chapter “Intelligent Behavior
As Adaptive Model-Free Estimation” found in [6].
In Kosko’s view (the notation is slightly different from Kosko's), a behavior (b),
defined as a correspondence between a given situation (or stimulus) (s  S) and a
response (r  R), can be modeled as a function b that maps S onto R, being S the set
of all the possible Stimuli, and R the set of all the possible Responses. That is:
b:SR
Again, regardless of the complexity of the dynamics linking the input (situation) to
the output (response), at the most abstract level, it is possible to think of a behavior as
Behavior Planning for Lifelike Characters and Avatars
3
a mapping between pairs of vectors (s, r). The correspondence is usually obtained by
means of a neural network or a fuzzy system.
One of the advantages of this approach is that it is model-free. This is particularly
useful given the complexity of the task of modelling intelligent behavior. Modeling
intelligent behavior, moods and emotions is extremely interesting, but it is out of the
scope of this paper.
The term “model-free” should not be taken too literally. What is really model-free is
the b function. Some sort of modeling (dimensions of input/output spaces, for
example) is obviously necessary.
5
How Evolution Arises from Avatars Interaction
Avatars who are in our Active Worlds’ world “Amuse” and “Amuse 2” move driven
by users instructions (via mouse or other pointing devices) or by internal logic or
both, depending on functioning mode selected. We have defined three different
modes called mode 1, 2 or 3 respectively. Except for debugging and experimental
purposes their mode is not shown to the user.
Regardless of the functioning mode, the pointing device acts only as global moving
device. The user can direct only the physical displacement of its avatar not the
movement of its single parts ( for example, the expression of “moods” via a
predefined, but evolvable gesture or the walking style pattern). The Behavior (i.e.
response to a given situation) is completely determined by both the internal state of
the avatar and the situation. I.e. If I move towards another avatar it is up to my
avatar, to decide whether or not to greet another avatar and how.
5.1 Mode 1 (Puppet Mode)
In this Mode, avatars are driven by user’s pointing device command in order to
interact with other avatars. Evolution is still possible for the avatars functioning in
this mode but it will not show until their mode is switched to an higher mode.
Anyhow, they will influence other avatar's both behavior, in the short term, (i.e.
they make other avatars react in a certain way) and the evolution of behavior in
the long term (i.e. they way they react to a given situation). Provided the others
are functioning in modes 2 and 3, the influence of mode 1 on mode 2 and of mode
1 on mode 3 will be soon visible. This is mainly a test and a training mode.
5.2 Mode 2 (Ro-Bot Mode)
In Mode 1 the Avatar accepts no direct pointing device input. It acts according to its
internal state and the interaction with other avatars.
Behavior Planning for Lifelike Characters and Avatars
4
5.3 Mode 3 (Dog Mode)
In this Mode the user merely suggests by means of a pointing device what the avatar
should do (i.e. pointing towards a given direction or goal). The avatar in turn
interprets the command and decides how to do the task (in other terms he can “choose
how to get there, how to walk and what to do in the meantime”). In principle its
freedom could arrive even up to the point of ignoring the user’s command. We have
not implemented such a mode insofar. We have dubbed it high 3 mode. We have
implemented only low 3 mode up to now.
A more formal discussion of the internal model follows.
5.4 Formal discussion of the internal model
The generic internal model is illustrated in figure a. In order to explain the workings
of the internal model, we will describe it from the point of view of a generic user.
The user sees the avatar’s physical appearance, the displacement and rotation of its
whole body inside the environment. Such displacements and rotations are called
movements. Moreover, the user can see the rotation of the various body segments of
the avatar. A given set of such rotations is called a gesture. During a session in a
Virtual World what the user perceives it as a combination between movements and
gestures. The user, in turn, can interact with the environment and the avatar by means
USR
POS AV
It
=
K
……… 1
POS AV
……… K
POS AV
M
USR = USR INPUT
USR  I t USR
It   (USR, USR)
K
K
It
Ot
IO
K
OG
SO
St
IS
K
a
Gt
K
SS
CK1
St
K
 X
Y
 1   X
P 
K
Behavior Planning for Lifelike Characters and Avatars
5
of a generic pointing device.
The model has also an internal state that, for the sake of simplicity, we have limited in
this first part of the Amusement project to the following variables: x, y, z, p (the three
cartesian coordinates of the avatar’s position plus the angle of pointing). The model
accepts two sub-sets of inputs: the user inputs (USR), and the other avatar position
and pointing (USR). The transition from one state to the next is regulated by the clock
CK1. The CK1 frequency is usually a sub-multiple of the framerate. The framerate is
the frequency at which all the scene rendering is recalculated. Typically the framerate
is 10fr/s. The CK1 frequency is usually 1 Hertz, i.e. 1 every 10 frames. The outputs
are generated from the inputs by means of four fundamental functions called IO, SO,
IS, SS. The gestures are selected from an ordered, finite cardinality, set by the output
by means of the function of OG. The output from IO and SO, and from IS and SS, is
compounded by means of the operator
that has not to be necessarily a sum.
The structure of IO, SO, IS, SS can be very varied and we will discuss only the
implementations that are important in the cases of our application.
Before proceeding we will introduce some little modifications to the model adopted in
fig a. The relevant schematic diagram is illustrated in fig a'.
In this new version the two sub-sets of the input USR and USR have been made
Ot
USR
IO
USR
SO
S tK
a’
d
  
t
 
Gt
K
OG
IS
SS
St
K 1
K
X
Y 
 X
P 
CK1
explicit. Moreover, the following transforms have been applied, taking into account
only the "horizontal" coordinates x and z. The vertical y coordinate is ignored.
The variable t is the total time of interaction and is taken summing the CK1 intervals
where d and  are defined as follows:
  p2  p1
Behavior Planning for Lifelike Characters and Avatars
d  ( x 2  x1 ) 2  ( z 2  z 1 ) 2
5.5 Modes in detail
PUPPET
MODE 1
The simplest case of all is the
one we have called mode 1
(puppet mode). In this case SG
and SS are null. The link
between the output and the input
is, hence, direct. The next state,
that is the next avatar position
and pointing, is specified directly
from the user input as well as the
the predefined gesture to be
performed.
In mode 2 the avatar is, in
reality, a fully independent
agent usually called a BOT
(from the well known word
“robot”). The BOT is fully
independent and the user has no
direct way to interact with it.
The output and, hence, the
relevant gestures, are function
of the state only, by means of
SO. The next state (in practice
the next avatar position and
pointing), is calculated as a
function SS of the position of
all the other avatars and/or the
present avatar position.
In low mode 3 the user
specifies directly the next avatar
state (in practice the new
avatar’s coordinates), but the
gesture is calculated using the
state of the avatar.
USR
OG
b
(RO)BOT
MODE 2
USR
OG
SO
SS
c
CK1
LOW MODE 3
USR
SO
USR
OG
d
CK1
6
Behavior Planning for Lifelike Characters and Avatars
7
HIGH MODE 3
In high mode 3 the user can
still specify directly the next
USR
OG
avatar state. But, since now
SO
USR
the function SS is present, the
new state is a combination of
both the user input and the
SS
avatar’s “will”.
The OG block has been made
explicit in all modes for the
following important reasons.
CK1
The OG is, among other
things, the memory in which
the gestures are stored, regardless on how they have been triggered and/or generated.
The generation of gestures by means of evolutionary techniques has been described in
[1].
The SO block is responsible of one the most important parts of the avatar model: the
link between a situation, identified with what we have defined here as the “state of the
avatar”, and a gesture. We will call this association between the situation and the
gesture a "behavior" according to [6]. The SO block implements b.
e
6
The Evolution of the SO block
The SO block implements the procedures illustrated in the following allowing the
behavior b to evolve. We have a population of b. Which one will be activated in a
given situation s? We have associated to any individual b a number in the range of
[0,1] called c, for choice. Now we have a set of pairs [b,c] that we call a repertoire .
Formally
 = {[bi,ci]} = {i }
i = 1…M
M is the cardinality of  and can be viewed as the size of the population. Since we
have different generations of  it may be useful to use a superscript index to indicate
the generation. The size of the populatin may vary at each hth generation.
 h = {[bih,cih]} = {ih}
i = 1…Mh; h = 1…
In theory there is no limit to the number h of generations albeit, for pratical purposes,
we reset the counter h from time to time.
Of all the ih only the one with the highest c will be active, the others remain dormant.
At Each kth time the active ih,k , let us call it ah,k , a for active, has the chance to be
expressed, it will have its cih,k increased or decreased as follows:
cih,k = f ( cih,k + s )
Behavior Planning for Lifelike Characters and Avatars
8
Where f can be any sigmoid function like the one in figure f, and s is the “success
figure”. We have calculated s as a function
f
of the state S:
f
s =  ( d, , t ) =  ( S )
+1
We have experimented with various .
If ah,k is not successful one of its brothers
behavior ih,k takes over.
From time to time the population 
reproduces into  h+1. In order to maintain
the cardinality of  under control and, most
important, to allow the more successful to
s
h
reproduce
more,
the
number
Nih = Mh+1 of offspring of each ih , is
controlled by the following function g
as also illustrated in figure g:
-1
g
Nmax
N =  g (c) 
To summarize we have two basic
0
fundamental rules:
Rule 1 = express only the b with the
highest c
Rule 2 = reproduce more the b with the highest c
7
g
c
1
0
Discussion
The difference between the evolution of the behavior discussed in this paper and an
adaptive behavior is profound. The first difference is that we don’t have only one
behavior, that yes, can be adaptive, instead we have a population of behaviors. One of
the most important consequences of this difference is that when a behavior fails, there
is a whole population of behaviors that can take over. Those “brother” behaviors may
well be called “alternatives.” The second difference is that the behavior “individuals”
not only reproduce using a copy-mutation mechanism but they can actually breed
among themselves using, for example crossover procedures. This is very important in
the case that the “alternatives” have, at least from time, to time the possibility to
became active and hence to be tested for fitness. In order to allow this we have
slightly modified the computation of c, allowing dormant behavior to became active
sometimes. In practice we “weighted” c with a random function in [0,1]. It is
important to note also that the “success figure” s is a measure of the fitness of any
given behavior but it does not feed back directly into the behavioral model as a
reinforcement does. The mutations in the behavior happens independently of. Only
the selection procedure takes into account the “fitness figure”. Our approach is
consistent with a Darwinian view of the emergence of complexity. In this paper we
refrained, on purpose, from using the word meme as defined in [4]. We did this for
Behavior Planning for Lifelike Characters and Avatars
9
the simple reason that the very simple behavior shown by our avatars, albeit very
effective for the visual impact of their humanoid appearance, should not be compared
with any behavior that we may want to call intelligent in a broader sense.
8
References
1.
Antonini R.: Implementing An Avatar Gesture Development Tool As A Creative
Evolutionary Collaborative System, Proceedings Of Aisb '99 Springer, (1999)
Darwin C.: L'origine Delle Specie, Newton Comp., (1973)
Dawkins R.: The Extended Phenotype, Oxford University Press, (1982)
Dawkins Richard: Blind Watchmaker, Longman, (1986)
Grand S.,Cliff D.,Malhotra A.: Creatures: Artificial Life Autonomous Software
Agents For Home Entertainment, Autonomus Agents 97 - Acm Press, (1997) 2229
Kosko, Bart.: Neural Networks And Fuzzy Systems : A Dynamical Systems
Approach To Machine Intelligence, Prentice Hall, (1992)
Maes P., B73: Artificial Life Meets Entertainment:Lifelike Autonomous Agents,
Communications Of The Acm, Vol. 38,11 (1995) 108-114
Sims K.: Evolving Virtual Creatures, Acm Transactions On Computer-Human
Interaction, (1994) 15-22
Sims K.: Evolving 3d Morphology And Behaviour By Competition, Artificial
Life, Vol. 1 (1995) 353-372
2.
3.
4.
5.
6.
7.
8.
9.