Spatially Distributed Naming Games
Luc Steels y
SONY Computer Science Laboratory Paris
6 rue Amyot, 75003 Paris
France
VUB AI Laboratory
Pleinlaan 2, 1050 Brussels
Belgium
Angus McIntyre
SONY Computer Science Laboratory Paris
6 rue Amyot, 75003 Paris
France
The paper investigates the dynamical properties of spatially distributed naming games. Naming games are interactions between two agents, a speaker and a
hearer, in which the speaker identiesan object using a name. Adaptive naming
games imply that speaker and hearer update their lexicons to become better
in future games. By engaging in adaptive naming games, a coherent shared vocabulary arises through self-organisation in a population of distributed agents.
When the agents are spatially distributed, diversity can be shown to arise, and
changes in population contact lead to language changes.
KEYWORDS : self-organization, evolutionary linguistics, language contact,
language change.
y Email: '[email protected]'. More information about
http://www.csl.sony.fr/> and <http://arti.vub.ac.be/>.
<
our research is available at
2
Steels et al.
1. Introduction
It is a fact that natural languages evolve at all levels: New sounds may get
introduced or the sound system of a language may start shifting, new words
and meanings may enter the lexicon existing meanings may expand or contract,
new syntactic categories may emerge, syntactic structures may shift, features
expressed through morphology may shift to a syntactic expression, etc. These
changes may occur very rapidly, particularly in the case of contact languages
such as pidgins and creoles (Arends et al., 1995). There may also be periods of
relative stability, even in contact languages (Sebba, 1997). All these changes take
place against a background of continuously changing populations (the linguistic community is in constant ux due to the in- and outow of members) and
changing environments and thus new sources of meaning (new artefacts, new
institutions, new types of social relations, etc.).
If we restrict linguistic investigations to a single (idealised) speaker-hearer
in a homogeneous language community who knows and speaks the language
perfectly (Chomsky, 1975), it is unclear why languages should change. It is even
more of a puzzle if we assume that language is to a large extent innate, which
implies that true innovations can only occur through genetic mutations and
subsequent spreading in the population (Pinker, 1994). With respect to the timescales observed in the emergence and spreading of linguistic innovations, genetic
mutations are simply too slow. Moreover, the 'success' of a linguistic innovation
is dependent on whether the group as a whole has adopted it. A mutation in
a single agent does not give this agent any immediate advantage, and is thus
unlikely to be favoured by natural selection.
On the other hand, if we accept that languages are invented and transmitted culturally, language change becomes self-evident. During transmission, and
particularly during learning, variations will automatically arise. These variations
may become conventionalised as they propagate to the rest of the group. Language learning, which is at the heart of the transmission process, then provides
important clues about what kind of language changes are to be expected and conversely, changes in languages give us clues about the kind of learning processes
that must be active. For example, language learners (as listeners) may overinterpret linguistic behavior, in the sense that they may erroneously believe that
certain behavior is rule-governed. If learners then use this over-interpretation in
subsequent language production, it can become a new norm. An empirical study
of the kind of over-interpretation that has taken place may give us a clue towards
the kind of learning strategies that were at work.
The cause of language changes may be internal to the language (Labov, 1994).
For example, if a new sound starts to propagate, it may cause a chain shift of
changes in existing sounds so that distinctiveness is retained in the total system.
Other causes of language change are external (Nichols, 1992). They come from
contact between languages due to increased communication between speaker
communities, large-scale bilingualism causing mutual inuence, the splitting of a
community into separated groups allowing subsequent language divergence, etc.
This paper focuses on language change due to changes in the dynamics of the
underlying populations, particularly in the communication structure between
linguistic communities as determined by spatial location. A very basic linguistic
Spatially Distributed Naming Games
3
behavior is investigated: the naming of objects with the aim of developing a common lexicon. The learning/construction procedure is kept as simple as possible
so as to focus on the issue of language contact itself. The agents are spatially
distributed so that the impact of changes in a community's structure on language
can be studied. Even with many simplifying assumptions, we can see interesting
phenomena emerge which cannot be explained by focusing on single speakers
without learning capability.
The rest of the paper is in three parts. Section 2 introduces the linguistic
behavior and the learning process of a single agent. In section 3 the typical
behavior of such a system is studied without language contact. Finally, in section
4 an experiment related to the dynamics of language contact is discussed.
2. The Naming Game
The naming game (Steels, 1996) is a specic kind of language game which
is played between two agents. One agent (the speaker) identies an object by
using a name. The game succeeds if the other agent (the hearer) agrees that this
name is appropriate for the object identied. A naming game is adaptive when
both participants in the game are able to change their rules (i.e. their wordobject associations) to be more successful in future games. The naming game is
strongly related to other models for cultural lexicon acquisition as proposed by
(Oliphant, 1996).
The naming game is a theoretical model for studying essential characteristics
of language use and language formation, similar to the use of cellular automata as
formal models of computation. In particular, naming games allow us to study the
creation, propagation, and evolution of linguistic conventions. Naming is admittedly the simplest possible linguistic device imaginable but a lot can be learned
that carries over to the creation, propagation, and evolution of more complex
linguistic mechanisms such as phonologies or syntactic constraints. On the other
hand, certain issues such as the emergence of ambiguity, multiple interacting
levels, the grounding in real-world sensing or actuation, etc., cannot be studied
with the very basic model presented here. Such issues can be studied with various extensions of the model as discussed in other papers. For example, Steels
(Steels, 1997) describes games which relate perceptual features with words, Steels
and Vogt (Steels and Vogt, 1997) discuss how language games can be grounded
on physically embodied robots, and Steels and Kaplan (Steels and Kaplan, 1999)
discuss the complex semantic dynamics in grounded robotic systems.
2.1. Definition of the Naming Game
We assume a set of agents A of size NA where each agent a 2 A has contact
with a set of objects of size NO O = fo0; :::; o g. Some or all of the objects are
shared between the dierent agents. A word is a sequence of letters drawn from a
nite alphabet. The agents are all assumed to share the same alphabet. A lexicon
L is a dynamical relation between objects, words and two quantities: The number
of times the relation has been used and the number of times the relation was
successful in use. Each agent a 2 A has his own lexicon L O W N N
which is initially empty. There is the possibility of synonymy and polysemy: A
a
n
a
a
a
4
Steels et al.
single word can be associated with several objects and a given object can be
associated with several words. An agent a 2 A can now be dened as a pair
a = hL ; O i.
A naming game N = hs; h; oi is an interaction between two agents: a speaker
s and a hearer h about a topic o which is an object, o 2 O . The interaction
proceeds as follows:
a
a
s
1 The speaker randomly selects a topic out of his set of objects. The speaker
draws the attention of the hearer to the topic. In a natural conversation
this could be by pointing or prior linguistic exchanges.
2 The speaker s encodes the topic o through a word w. The word chosen is the
most successful word w where ho; w; u; si 2 L . A word w1 is more successful
than a word w2 i ho; w1; u1; s1 i 2 L , ho; w2; u2; s2i 2 L , u1 u2 0
and either s1 =u1is2 =u2 or s1 =u1 = s2 =u2 and u1iu2 . If several words are
equally successful, a random word is selected from the most successful set.
3 The hearer h decodes the word w to derive a set of objects O such that
o 2 O =) ho; w; u; si 2 L
4 A naming game is successful i o 2 O .
s
s
s
h
h
h
h
Several naming games involving dierent agents can be going on in parallel
because interactions are always local.
An adaptive naming game is a naming game N = hs; h; oi as dened above,
with changes to the lexicon of both the speaker and hearer as a side eect of
the game. This means that we have to introduce a temporal dimension in the
denition of an agent. An agent a at time t is dened as a = hL ; O i. A
time point corresponds to an event in which two agents play a language game.
If the speaker at time t uses a word w where ho; w; u; si 2 L and if the game
ends in success, then ho; w; u + 1; s + 1i 2 L +1 and ho; w; u + 1; s + 1i 2 L +1 .
If a word is used by speaker or hearer but some failure has occured, then use is
incremented but not success: ho; w; u + 1; si 2 L and ho; w; u + 1; si 2 L +1 .
The other changes dening the state of an agent at time t + 1 are as follows:
1. Missing object (step 1 fails).
The object chosen as topic by the speaker is not shared by the hearer: o 62 O .
In that case, the game ends in failure but the hearer acquires knowledge
S about
the object with an object propagation probability p : O +1 = O fog.
2. The speaker does not have a word (step 2 fails).
In other words there is no ho; w; u; si 2 L . In that case, the game fails but the
speaker may (randomly) create a new word w0 and associate this Swith the object
in his lexicon with a word creation probability p : L +1 = L fho; w0; 1; 0ig.
3. The hearer does not know the word (step 3 fails).
In other words there is no ho; w; u; si 2 L . In that case, the game ends in
failure but the hearer
S may extend his lexicon with a word absorption probability
p : L +1 = L fho; w; 1; 0ig.
4. The topic is not decoded by the hearer (step 4 fails).
In other words, o 62 H. In that case the game ends in failure, but the hearer
may acquire theS word for his lexicon with a word absorption probability p :
L +1 = L fho; w; 1; 0ig
t
a;t
a;t
s;t
s;t
h;t
s;t
h;t
h;t
o
h;t
h;t
s;t
c
s;t
s;t
h;t
a
h;t
h;t
a
h;t
h;t
Spatially Distributed Naming Games
5
If there is no explicit change as dened above, then the denition of an agent
a remains the same: a +1 = hL ; O i
t
a;t
a;t
2.2. Limitations and assumptions of the naming game
The naming game as presented is almost trivially simple. It is not our intention
to suggest it as a 'realistic' model of what happens in language formation. We
do not claim that members of an emerging linguistic community spend their
time playing naming games with each other in order to construct a language.
Rather, we claim that activities similar to the naming game may form a part
of the linguistic interactions that take place between dierent members of a
community, or between parents and children. A mother who shows a child an
object and says its name is playing a kind of naming game with the child, even
though this is merely one among many possible kinds of interaction that may
take place during the child's acquisition of language.
The naming game is a deliberate simplication, which is intended to illustrate
or represent linguistic interactions rather than to accurately model them. Our
main interest is in the mechanisms by which shared conventions may spread
through a community, and the phenomena associated with this process. The
simplicity of the naming game means that we can study these mechanisms and
phenomena without the distractions and distortions introduced by a { probably
vain { attempt to construct a highly 'realistic' model of linguistic interaction.
The naming game also involves a number of assumptions. It is important to
make these explicit, so that we can either justify them { with reference either to
the real world or to our particular focus of interest - or investigate the eects of
relaxing or discarding our assumptions.
One major assumption made in this paper is that communication, both linguistic and non-linguistic, is reliable and unambiguous. Relaxation of this assumption
is explored in (Steels and Kaplan, 1998), which shows that agent communities can
in fact cope with high degrees of uncertainty, and that shared conventions may
both emerge and persist under conditions of imperfect communication. Other assumptions relate to the capacity of the agents' memories { assumed to be reliable
and eectively innite. This assumption could also easily be relaxed, by imposing
a limit on vocabulary sizes, introducing 'noise' into the agents' memories, and
so forth. Informal exploration of this issue using a pruning mechanism that discards under-utilised words (thus eectively reducing lexicon size) suggests that
the same phenomena described in this paper { convergence on shared vocabularies { will take place even when the agent's mental capacities are constrained
in this way.
The naming game model also assumes that participants share the 'rules' of
the game and a common interest in playing it. Their communication abilities are
strong enough for them to coordinate their involvement in the game { including
the signalling of success or failure { but not strong enough to allow them to
resolve problems by dialogue. In defence of this assumption we would say that
it is not our intention in this paper to explore the emergence of coordinated
activity, as other work, particularly (Noble, 1998), has already done so. The
question of how agents may come to engage in particular interactions is a very
valid one, and one that would be well worth exploring experimentally. However,
6
Steels et al.
100%
90%
Communicative Success
80%
70%
60%
50%
40%
30%
20%
10%
0%
0
2500
5000
7500
10000
Games
Figure 1: Simulation of closed system with 20 agents and 20 objects. Communicative
success is shown on the y-axis per 25 games. A rapid increase in communicative success is
achieved for the rst 5000 games, after which progress levels o to reach complete average
success after 10000 games.
in this paper we are primarily interested in the formation of shared linguistic
conventions once the basic interactional protocols have been established, rather
than in the mechanisms that lead to the establishment of such protocols.
3. Behavior
3.1. Behavior of Closed Naming Systems
Let us now examine the behavior of naming games if played out in succession
among a set of agents, where the choice of speaker and hearer is randomly chosen.
First we look at a closed naming game system with the following parameters:
NA 20
NO 20
p
0.1
p
0.75
A typical simulation result after 10,000 games is seen in Figure 1. There is a
rapid increase in a shared lexicon as seen from the communicative success.
An inspection of the lexicon of the invididual agents is quite interesting. Let
us focus on agent a255. The size of the lexicon is 45, which means that more
than twice the number of words strictly required to cover the set of 20 objects
c
a
Spatially Distributed Naming Games
7
have been either created or acquired by this agent. Here is the object-lexicon
of agent a255. For each object we show the possible words followed by the use,
success and score. The words are ordered according to the success criterion used
by the agent to decide which word to prefer.
Obj255: BIPA[34,34,1.00] DUPO[1,1,0.00] NUJO[0,0,0.00]
Obj254: BEGAGO[22,22,1.00] GU[29,21,0.72] DE[2,2,0.00]
BADO[1,1,0.00] NEKIKO[0,0,0.00] MIPA[0,0,0.00]
GUGIGE[0,0,0.00]
Obj253: KUJO[33,33,1.00] TUGAJA[10,5,0.50] TI[0,0,0.00]
BU[0,0,0.00]
Obj252: TEPI[21,21,1.00] GUDEBA[8,4,0.50] DU[4,4,0.00]
GIMO[1,1,0.00] JOJEJI[0,0,0.00]
Obj251: DEGUPI[3,3,0.00]
Obj250: GONIDO[44,43,0.98] NAPO[1,1,0.00]
Obj249: MI[37,36,0.97] JOTUBE[4,1,0.00] BUKEJI[0,0,0.00]
Obj248: DADUTO[1,1,0.00] BAPOTU[0,0,0.00]
Obj247: KOMIJA[30,29,0.97] KEGUPI[2,2,0.00]
JODU[0,0,0.00]
Obj246: GUKU[12,12,1.00] MEBAJI[35,27,0.77]
GITIDO[1,1,0.00] PANE[0,0,0.00] JEGO[0,0,0.00]
Obj245: TA[12,2,0.17] NE[1,1,0.00]
Obj244: JAJOKI[27,27,1.00] NAPENA[15,15,1.00]
MIDE[13,4,0.31]
Obj243: TUTU[23,23,1.00] GUTINE[0,0,0.00]
Obj242: GUMIPI[21,21,1.00] DUKUJO[11,7,0.64]
BE[2,2,0.00] KIGETO[1,1,0.00] GODU[0,0,0.00]
Obj241: TIDU[28,28,1.00] MIPO[14,11,0.79]
NUDO[1,0,0.00]
Obj240: PU[52,51,0.98]
Obj239: PIBU[10,10,1.00] GITOBU[26,22,0.85]
BIGE[2,2,0.00] MIMA[0,0,0.00]
Obj238: MOMUMA[41,39,0.95] PI[1,1,0.00]
DUMONE[0,0,0.00] MA[0,0,0.00]
Obj237: NI[28,28,1.00] JAKO[29,20,0.69] TO[2,2,0.00]
JUMUTE[1,1,0.00] PUTA[0,0,0.00]
Obj236: KUJU[9,9,1.00] NO[30,25,0.83] KU[0,0,0.00]
POJI[0,0,0.00]
We observe that only a few objects have a single word (one of them is Obj240).
For all others there is synonymy, with dierent words used to denote the same
object. Strict synonymy is generally held to be rare in natural language: in
our articial language, the apparent synonyms are mostly redundant, with one
form clearly dominating the others, and this dominance by a single word will
be reinforced as the games proceed. Application of a simple pruning mechanism
based on a criteria such as recency or frequency of use can then eliminate the
unused synonyms.
8
Steels et al.
Let S be the average synonymy in the lexicon of an agent a, which is in this
case S 1 = 3:40.
In our simulations, homonymy and polysemy are very much rarer. In fact,
polysemy, in which a single lexeme may have multiple meanings, is eectively
absent in these simulations, as the only meanings in the world consist of atomic,
unanalysed objects, allowing no possibility of the kind of analogical formations
or meaning drift that give rise to polysemy. By contrast, polysemy is very much
present in other simulations we have constructed, such as those described in
(Steels and Kaplan, 1999).
Homonymy (following Lyons (Lyons, 1981) we take homonymy to be the case
in which two or more unrelated lexemes have the same form but distinct meanings, e.g. the English words "bank" and "bat") is possible but rare, occurring
only as a result of accidental generation of identical forms. This is seen in the
following word-lexicon for the same agent. For each word the possible objects
are printed, followed by the use and success counters. There are no homonyms
in this vocabulary.
a
a
JAJOKI:
KEGUPI:
JODU:
GODU:
DE:
NI:
KUJO:
MIDE:
DUPO:
DADUTO:
GUDEBA:
NUJO:
TUGAJA:
MIPO:
GUKU:
BAPOTU:
GU:
KIGETO:
NE:
POJI:
JAKO:
PUTA:
BU:
GONIDO:
BIGE:
DUMONE:
DEGUPI:
JOJEJI:
TEPI:
GITIDO:
BADO:
Obj244[27,27],
Obj247[2,2],
Obj247[0,0,],
Obj242[0,0],
Obj254[2,2],
Obj237[28,28],
Obj253[33,33],
Obj244[13,4],
Obj255[1,1],
Obj248[1,1],
Obj252[8,4],
Obj255[0,0],
Obj253[10,5],
Obj241[14,11],
Obj246[12,12],
Obj248[0,0],
Obj254[29,21],
Obj242[1,1],
Obj245[1,1],
Obj236[0,0],
Obj237[29,20],
Obj237[0,0],
Obj253[0,0],
Obj250[44,43],
Obj239[2,2],
Obj238[0,0],
Obj251[3,3],
Obj252[0,0],
Obj252[21,21],
Obj246[1,1],
Obj254[1,1],
GUGIGE:
JEGO:
MIPA:
KUJU:
GIMO:
NUDO:
PIBU:
GUTINE:
NO:
MI:
BUKEJI:
TUTU:
BE:
TO:
TA:
MA:
PANE:
NEKIKO:
MIMA:
BEGAGO:
NAPENA:
NAPO:
JUMUTE:
MOMUMA:
KOMIJA:
PI:
KU:
DU:
TIDU:
BIPA:
JOTUBE:
Obj254[0,0]
Obj246[0,0]]
Obj254[0,0]
Obj236[9,9]
Obj252[1,1]
Obj241[1,0]
Obj239[10,10]
Obj243[0,0]
Obj236[30,25]
Obj249[37,36]
Obj249[0,0]
Obj243[23,23]
Obj242[2,2]
Obj237[2,2]
Obj245[12,2]
Obj238[0,0]
Obj246[0,0]
Obj254[0,0]
Obj239[0,0]
Obj254[22,22]
Obj244[15,15]
Obj250[1,1]
Obj237[1,1]
Obj238[41,39]
Obj247[30,29]
Obj238[1,1]
Obj236[0,0]
Obj252[4,4]
Obj241[28,28]
Obj255[34,34]
Obj249[4,1]
Spatially Distributed Naming Games
9
20
18
16
Word Score
14
12
BIPA
NUJO
DUPO
10
8
6
4
2
0
0
250
Games
500
This gure shows the dierent competing words ("bipa", "nujo", "dupo") in the
population for the same object. After a certain set of games, there is a convergence to a
single word ("bipa").
Figure 2:
DUKUJO: Obj242[11,7],
MEBAJI: Obj246[35,27],
TI:
Obj253[0,0],
GUMIPI: Obj242[21,21]
GITOBU: Obj239[26,22]
PU:
Obj240[52,51]
The rarity of homonymy in our simulations can be explained by the fact that
homonymy can only occur when a particular word is independently created more
than once. The probability that this will happen is low and depends on the word
formation process: the size of the alphabet and the typical word length. We would
expect homonymy
We dene H the homonymy in the lexicon of an agent a to be equal to the
percentage of the words that have more than one possible referent. H 1 = 0.
Let us now compare the lexicons of dierent agents. An agent understands
many more words than it preferentially uses. Consequently average communicative success can reach 100% without the lexicons of the agents being the same.
However we typically see a convergence towards a single word as illustrated in
Figure 2 and thus the lexicons of the dierent agents will gradually become equal.
a
a
The convergence that is seen here is a special case of self-organisation in the
sense of spontaneous formation of dissipative structures by the amplication
and damping of random uctuations (Prigogine and Stengers, 1984). It is a well
known mechanism for achieving coherence without central control, and has been
amply demonstrated in several biological systems for explaining pattern formation (Meinhardt, 1982) or in insect societies (Deneubourg, 1977). It applies here
as follows: agents randomly couple words to meanings, and engage in communi-
10
Steels et al.
100%
90%
Communicative Success
80%
70%
60%
50%
40%
30%
20%
10%
0%
0
2500
5000
7500
10000
12500
Games
The population now consists of 50 agents and 50 objects. After scaling, we see
the same curve shape as in 1 but now 12500 games were needed to reach total average
communicative success.
Figure 3:
cation. They record the success of a particular word-meaning pair and preferably
use that in future communications. This establishes a positive feedback mechanism: a word that is used a lot will have a high communicative success and will
hence be used even more.
Changes in the main parameters of the system lead to the following observations:
1 Increase in the size of the population NA will obviously increase the number
of games that is needed before communicative success (and coherence) is
reached. This is seen in Figure 3 which shows a series of language games
with a population of 50 agents.
2 Increase in the number of objects NO will likewise increase the number
of games, as shown in 3. Repeating the simulations with progressively
larger agent populations or object sets leads to longer times to reach communicative success, but the phenomena described in this paper appear
to hold irrespective of population size. Kaplan, Steels and McIntyre (Kaplan et al., 1998) demonstrate that complete communicative success can be
achieved even for large - 1000 agents or objects - populations or meaning
sets. The time taken to reach 100% success increases as a function of population size, with the success curve assuming a more pronounced sigmoid
shape as the population is increased. This type of curve, with a at initial
section followed by a steep climb followed in turn by a gradual levelling o,
is commonly seen in models of growth, and has been widely observed in the
Spatially Distributed Naming Games
11
100%
90%
Communicative Success
80%
70%
pa=0.10
pa=0.25
pa=0.5
pa=0.75
pa=1
60%
50%
40%
30%
20%
10%
0%
0
10000
20000
30000
40000
50000
Games
Figure 4: The gure plots the result of language games with 20 agents and 20 objects,
each time varying the word absorption rate. As this rate increases, the time needed to reach
communicative success increases. Note that the size of the gameset (1000 in this gure but
25 in gure 3) determines the amount of detail of the graph. Smaller variation is smoothed
in these graphs because the size of the game set is larger.
propagation of linguistic conventions in human populations (Kroch, 1989;
Niyogi and Berwick, 1995).
3 An increase in the probability of word creation p has more subtle eects.
We redo the rst experiment (with 20 agents and 20 objects) but change
p = 0:75. It takes noticeably longer to reach communicative success. More
importantly, the agents create many more words S 1 = 6:15 compared to
3:40 previously. A typical example is the following set in one agent for one
object:
c
c
a
Obj701: MAPO[32,32,1.00] KEMI[25,15,0.60] JEDA[3,3,0.00]
DA[2,2,0.00] JAGEPO[2,2,0.00] JOKE[1,1,0.00]
BUTAJU[0,0,0.00] BI[0,0,0.00]
4 An increase in the probability of word absorption p has an eect as seen in
Figure 4. As the word absorption rate decreases { in a real world situation,
this might be due to the time needed to learn the form of the word or
increased background noise { it takes longer to reach steady communicative
success.
a
12
Steels et al.
50
100%
Size of population (agents)
75%
30
50%
20
25%
10
Population Size
Communicative Success
0
0
12500
25000
Communicative success (%)
40
0%
50000
37500
Games
Figure 5: The gure plots the result of open naming games starting with a population
of 20 agents and increasing the population with probability eA = 0 00025. The (average)
communicative success per 500 games is shown on the y-axis. The population is able to
cope with the continuous extension.
p
:
3.2. Behavior of Open Naming Systems
We now turn to an investigation of the naming game as an open system. This
means that there is an in- and outux of agents and objects. We can study both
aspects at the same time by making the set of objects equal to the set of agents,
i.e. the agents only talk about each other. The set of agents and objects now have
a temporal dimension: A denotes the set of agents at time t of size NA; t and
O denotes the set of objects to which a has access at time t. For the purposes
of the experiments below, A = O for every a 2 A .
Let p A, the entrance probability, denote the probability that a new agent may
enter population A and p A, the departure probability, the probability that a
new agent may leave the population A. The entrance of a new agent has two
eects: (1) The new agent has no lexicon yet. It has to acquire the lexicon that
exists already in the population and may introduce new words himself. (2) The
new agent also constitutes a new object, so the others have to invent and agree
upon a new word to name the new agent. The departure of an agent has also
two eects: (1) The agent can no longer be the topic of a conversation and so a
word falls into disuse. (2) The agent takes with him a particular lexicon and so
the collective memory diminishes.
Here are the result of some simulations showing the eect of dierent entrance
and departure probabilities. 5 shows the eect with a low p A = 0:00025. We see
that each time when the population extends the communicative success drops
but the agents recuperate by developing the necessary vocabulary to successful
name the new agents.
t
a;t
t
a;t
t
e
d
e
Spatially Distributed Naming Games
90
13
100%
70
75%
60
50
50%
40
30
25%
20
Communicative success (%)
Size of population (agents)
80
Population
Communicative Success
10
0
0
12500
25000
37500
0%
50000
Games
Figure 6: The gure starts with a population of 20 agents which increases with probability
A = 0 001 to reach around 80 members at the endpoint. The (average) communicative
success per 500 games is shown on the y-axis. The population is not able to cope with the
continuous expansion.
pe
:
The next experiment, shown in Figure 6, shows the eect with a high entrance
probability p A = 0:001. Now the population is not able to keep up with the
inux of new agents (and hence new objects)
Next we look at the departure probability. Figure 7 shows the eect with a
departure probability p A = 0:001. Agents leave the population but this does
not cause a loss in communicative success because the remaining agents have an
easier task (there are less objects) and the lexicon is suciently preserved in the
rest of the population.
In natural populations there is usually a positive entrance and departure rate.
Figure 8 and Figure 9 study the impact of this ux. Figure 8 shows what happens with low probabilities. Although the members of the population change and
therefore also the objects they talk about change, there is no loss in communicative success which means that the population is able to cope both with in- and
outux of agents and in- and outux of objects.
Figure 9 shows what happens with higher entrance and departure probabilities.
Every time the population of agents (and thus of objects) goes up there are losses
in average communicative success. When the population decreases the group
catches up but this may not be sucient to reach total communicative success.
e
d
14
Steels et al.
100%
20
16
75%
14
12
10
50%
8
Population Size
Communicative Success
6
25%
Communicative success (%)
Size of population (agents)
18
4
2
0%
50000
0
0
12500
25000
37500
Games
The gure starts with a population of 20 agents which decreases with departure
probability d A = 0 001. The (average) communicative success per 500 games is shown on
the y-axis. Although the population dies out, there is no loss in communicative success.
Figure 7:
p
:
35
100%
75%
25
20
50%
15
Population Size
Communicative Success
10
25%
Communicative success (%)
Size of population (agents)
30
5
0%
0
0
12500
25000
37500
Games
Figure 8:
pd
The gure has positive probabilities for entrance e A = 0 0005 and departure
A = 0 0005. The population is able to cope with the ux.
:
p
:
Spatially Distributed Naming Games
100%
60
75%
40
50%
30
20
25%
Communicative success (%)
Population Size
Communicative Success
50
Size of population (agents)
15
10
0%
50000
0
0
12500
25000
37500
Figure 9: The gure has higher positive probabilities for entrance e A = 0 01 and departure d A = 0 01. The population is no longer or barely able to cope with the ux. A rich
dynamics is seen.
p
p
:
:
4. Language contact
4.1. Spatial Structures
It is clear that in the evolution of natural languages the spatial distribution
of the agents plays a very large role (Nichols, 1992). This can also be modeled
computationally and mathematically by locating the agents in a 2-dimensional
space. In our experiment, the space is taken to represent a physical space, thus
modeling geographical distribution of agents. It might equally well represent a
social, genetic or economical space. We could even perform experiments in a
multi-dimensional space representing several of these alternate dimensions.
Each agent a has x,y coordinates a , a . d is the trigonometric distance
between two agents a and b:
x
y
a;b
q
d = (a ? b )2 + (a ? b )2
(4.1)
A spatial distribution is randomly generated, modulated by a clustering tendency. A typical example of a distribution is given in Figure 10.
The probability with which two agents a and b interact is based on their
respective distance, and an interaction factor m , which determines the weight
of the distance. If the interaction factor increases, the role of distance becomes
higher and interactions tend to reect the spatial clustering more. Based on this
parameter we can study the evolution of language when communications between
the communities increase. Our simulations show that average communicative
success evolves as before (see Figure 11). However this obscures the interesting
phenomena which can only be seen when inspecting and comparing the languages
a;b
x
x
y
a
y
16
Steels et al.
Figure 10: The gure shows the spatial distributionof a set of 20 agents. There is clustering
around three centers.
100
90
Communicative success (%)
80
70
60
50
40
30
20
10
0
0
2500
5000
Games
7500
10000
The gure shows the evolution of communicative success in a group of 20
agents clustered in three communities. Success is never total because there are occasional
interactions with members of the other community, however intercluster communication
reaches total success.
Figure 11:
Spatially Distributed Naming Games
17
of the dierent agents. Inspection of the agent lexicons reveals that agents will
develop a stable language within their cluster. But they will also develop a second
language, an interlingua which is weaker but shared among the dierent clusters.
This interlingua will become stronger as more agents interact between clusters.
Thus we observe language diversity due to the spatial distribution but at the
same time occasional cognates as well as a second interlingua.
The following vocabularies illustrate this point clearly. The rst vocabulary is
taken from an agent from the leftmost cluster in Figure 10:
obj3419: MUGU[38,36,0.95] NINU[6,3,0.50] PI[7,3,0.43]
JI[3,2,0.00] TU[2,0,0.00]
obj3418: KUTUME[16,15,0.94] BU[3,1,0.00] NI[1,0,0.00]
obj3417: BAPI[23,22,0.96] KI[21,17,0.81]
DAMUTI[1,0,0.00]
obj3416: BIMOGU[43,42,0.98] BA[3,0,0.00]
obj3415: JITI[36,34,0.94] GATO[11,7,0.64]
obj3414: DODINE[26,23,0.88] PIBO[6,5,0.83]
GIJE[1,0,0.00] PUPETO[1,0,0.00]
obj3413: JA[6,4,0.67] JO[3,2,0.00]
obj3412: GEMA[28,27,0.96] DO[15,5,0.33] GAPU[1,0,0.00]
obj3411: MU[49,48,0.98] GITE[4,3,0.00] PAKU[2,1,0.00]
obj3410: NA[16,15,0.94] NUGUGE[21,19,0.90]
PA[5,4,0.80] NE[3,0,0.00]
obj3409: PE[51,48,0.94] DA[2,1,0.00]
obj3408: KUDE[48,43,0.90] NADO[3,2,0.00]
obj3407: MEPABO[38,37,0.97] JABETO[17,12,0.71]
DI[1,0,0.00]
obj3406: BO[35,33,0.94] BABIGE[11,9,0.82]
obj3405: TINE[47,46,0.98]
obj3404: NEBU[42,41,0.98] ME[12,10,0.83]
obj3403: MOMA[44,43,0.98] TI[3,0,0.00]
NUDO[3,0,0.00] KENE[3,2,0.00]
obj3402: GO[59,58,0.98] TA[1,0,0.00]
obj3401: NUGINI[31,30,0.97] GI[18,15,0.83]
MAJIBA[4,0,0.00]
obj3400: KUBE[43,38,0.88] GUTIDA[4,3,0.00]
MOKO[1,0,0.00]
This is the vocabulary for an agent taken from the rightmost cluster in Figure
10:
obj3419: NINU[32,26,0.81] MUGU[2,1,0.00] PI[1,0,0.00]
obj3418: NI[36,34,0.94] KUTUME[5,4,0.80]
MOKO[5,4,0.80] MEKAMI[2,1,0.00]
obj3417: KI[38,37,0.97] BAPI[1,0,0.00] KE[1,0,0.00]
obj3416: BIMOGU[35,34,0.97] BA[12,10,0.83]
PIPEBE[1,0,0.00]
obj3415: JITI[30,29,0.97] GATO[13,6,0.46]
18
Steels et al.
obj3414: DODINE[23,22,0.96] PUPETO[16,9,0.56]
PIBO[4,2,0.00] GIJE[3,1,0.00]
obj3413: JO[27,26,0.96] JI[19,17,0.89] JA[1,0,0.00]
obj3412: GEMA[42,38,0.90] GAPU[8,7,0.87]
obj3411: GITE[42,37,0.88]
obj3410: PA[40,35,0.87]
obj3409: DA[37,32,0.86] NINE[7,5,0.71]
obj3408: NADO[24,23,0.96] KUDE[10,8,0.80]
PUTO[1,0,0.00]
obj3407: MEPABO[40,36,0.90] JUNIPE[8,6,0.75]
DI[1,0,0.00]
obj3406: BABIGE[42,39,0.93] BO[3,2,0.00]
obj3405: TINE[71,64,0.90]
obj3404: NEBU[38,37,0.97] ME[1,0,0.00]
BUKUGO[1,0,0.00]
obj3403: KENE[27,24,0.89] MOMA[17,14,0.82]
NUDO[4,0,0.00] KOKO[1,0,0.00]
obj3402: GO[19,18,0.95] TA[5,1,0.20]
obj3401: GI[9,8,0.89] MATU[20,17,0.85]
PUMONI[7,0,0.00]
obj3400: GUTIDA[33,26,0.79] KUBE[4,3,0.00]
Some words (for example "TINE" for obj3405) are shared. But most of the
time there are at least two words. One word is used preferentially inside the
cluster, the other is known but preferentially used by members of another cluster.
Thus the word "MUGU" for obj3419 is preferred for the rst object in the rst
cluster and known but not preferentially used by the agent in the second cluster.
Conversely, "NINU" is preferred for the same object by the agent in the second
cluster, although he also knows "MUGU".
When the interaction factor increases, we see further dierentiation because
there is less communication between clusters. When it is decreased, we see more
coherence because there is more intercluster communication. Thus we can eectively tune divergence or coherence based on probability of interaction between
communities (clusters) of agents.
4.2. The Dynamics of Language Contact
This brings us to the core subject of the present paper. Given a spatial distribution and a variable degree of interaction, we can study the impact of increased
communication and hence more intense language contact on clusters of language
communities.
The experiment runs as follows. Agents in three dierent clusters (a, b, and c)
interact and evolve a lexicon for each cluster. The interaction between clusters
is initially very weak (see Figure 12). At some point in time the intercluster
communication is increased drastically. At rst there is a drop in communicative
success but then total communicative success is again reached.
This general evolution hides however the more interesting developments. Figure 13 shows the evolution of coherence CA for each cluster (a, b, c) separately
;t
Spatially Distributed Naming Games
19
Communicative success (%)
100%
75%
50%
start of strong
interactions
25%
0%
0
2000
4000
6000
8000
10000
Games
Figure 12: Evolution of average communicative success in a Group of agents with rst
weak and then strong interactions. The average communicative success per 25 games is
shown at each point.
and also for the total set of agents. As long as the agents have relatively little contact, total coherence is low although coherence of each cluster is high. Coherence
starts to increase with increased contact. Coherence in each cluster diminishes
somewhat because the agents in the cluster are in the process of accomodating
to the total set.
Figure 14 shows with the same scale the evolution of the equality measure
EA1 A2 between clusters a and b, a and c, and b and c. We see that the equality
reaches a plateau before the increase in contact but then starts to climb. This
means that the languages of the dierent groups are in the process of merging
due to the increased language contact.
Finally, mutual understanding or bilingualism is shown in the next gure (Figure 15). The moment language contact increases we see a rapid increase in understanding, i.e. in how far a prefered word of one agent is also known to the
other agent.
Thus we see that increased contact causes at rst a rapid increase in bilingualism, then a gradual mixing of the languages, and, if the contact continues, an
evolution towards complete coherence. The more rapid the contact is increased,
the faster the three phases can be observed. Interestingly enough the communicative success of the whole system (intra-cluster and inter-cluster communication)
is hardly aected, as shown in Figure 12.
These various phases are also seen in natural languages (Thomason and Kaufman, 1988). A common phenomenon is 'borrowing' of lexical items between adjacent language communities. Examples of this include central Belgium, where
two language communities (French and Flemish) co-exist and large groups of
the population have become bilingual. In these cases we see a strong mutual
;
;t
20
Steels et al.
100%
Coherence
75%
c-tot
c-a
c-b
c-c
50%
25%
0%
0
1000
2000
3000
4000
5000
6000
Games
The gure shows the evolution of coherence plotted for every 25 games. We
see that initially each individual cluster reaches a high level of coherence, whereas the total
set of agents has a much lower coherence. When contact between the groups is increased,
global coherence begins to rise steadily.
Figure 13:
100%
axb
axc
bxc
Sharing
75%
50%
25%
0%
0
1000
2000
3000
4000
5000
6000
Games
Figure 14: The gure shows the evolution of how many prefered words are shared between
groups of agents. This starts to increase slowly after language contact is increased (at the
1500 game mark).
Spatially Distributed Naming Games
21
100%
Understanding
75%
50%
axb
axc
bxc
25%
0%
0
1000
2000
3000
4000
5000
6000
Games
Figure 15: Evolution of intercluster understanding or bilingualism. A rapid increase is
seen after agent contact is increased (at the 1500 game mark).
inuence on the lexicons of both communities. Thus a cup will be called "kop"
in Dutch but "tas" in Flemish, inuenced by the French "tasse". This occurs
despite the existence of a homonymous form "tas", meaning bag in Dutch as
well as Flemish.
A less-balanced situation may occur if one group is numerically or politically
stronger than another. A subpopulation that has been exposed due to invasion
or migration to a larger and more dominant language community may progressively or radically shift their lexicon towards the language of the majority group,
even though parts of the speakers' own language, such as the grammar, may remain intact. Such phenomena are observed in the formation of pidgin and creole
languages: Sebba (Sebba, 1997) describes the concept of a lexier, typically the
language of the dominant group, which constitutes the target towards which the
evolving language develops.
4.3. Further work
Clearly, there is a great potential for further experimentation. Experiments
which could usefully be conducted include:
1 Population sizes could be varied to see in how far population domination
leads also to language domination.
2 Other factors could be made to inuence the probability of inter-cluster
communication. For example, the social distance could have the same stratication eect as spatial distance.
22
Steels et al.
3 Rather than increasing language contact, we could decrease it, showing
that the population diversies.
4 The populations in the cluster experiment above are closed. Another obvious experiment is to let them vary at specic rates, so that the languages
in each cluster are moving targets.
All these experiments reect situations that occur in human language populations and can thus be compared to language evolution phenomena that have
been observed to accompany population changes.
5. Conclusion
We believe that language is a complex adaptive system which self-organises
and achieves coherence even if the agents or possible meanings change. Language
evolves partly due to its own internal dynamics and is transmitted culturally and
not genetically. These hypotheses dier substantially from the notion that language is primarily transmitted genetically through an innate language acquisition
device embodying universal grammar, and hence originated due to genetic evolution (Pinker, 1994). Our work resonates strongly with research on historical
linguistics (Nichols, 1992), sociolinguistics (Labov, 1994), and the study of language contact and language formation as observed in pidgin and creole languages
(Arends et al., 1995).
We propose that spatially distributed language games provide theoretical frameworks and associated simulation tools to study the issues raised in these elds.
For example, our experiments demonstrate that language convergence does not
necessarily have to be caused by a common origin. The fact that languages share
vocabularies, as investigated in statistically oriented research towards cognates
(shared words) (Ruhlen, 1994), can be the result of interaction between clusters of language users, and is not necessarily the result of a universal common
Ursprache as comparative linguists commonly assume. It has also been demonstrated that diversity is not necessarily due to a progressive mutation of an
existing language when a subgroup has become isolated, but can be explained
in terms of the independent evolution of lexicons in isolated clusters. Relative
coherence in such cases follows only after subsequent contact between clusters.
It should be possible to gain useful results in this area by feeding the simulations with data from human population change and empirical data on lexical
evolution.
The main goal of the paper was to show through a series of simulation experiments that naming game systems, particularly open and spatially distributed
games, exhibit a rich and interesting dynamics. This paper has only scratched
the surface and many more variations can and will be examined, including age
structure, bilingualism, spatial migrations, etc. Moreover it is clearly possible to
develop a mathematical theory of naming game systems, so that the relations between the dierent parameters become understood, similar to comparable studies
in cellular automata or evolving populations. This remains work for the future
and we hope other researchers will be encouraged to join us in this exciting
venture.
Spatially Distributed Naming Games
23
6. Acknowledgement
This work was nanced and carried out at the Sony Computer Science Laboratory in Paris. We are indebted to Toshi Doi and Mario Tokoro to be able to
work in this superbly productive environment. The BABEL software tool under development by Angus McIntyre (McIntyre, 1998) at SONY CSL-Paris has
proved invaluable for eciently exploring the issues discussed in this paper.
References
Arends, J., Muysken, P. and Smith, N. Pidgins and creoles: an introduction . Amsterdam:
Benjamins (1995).
Chomsky, N. Reections on Language . New York: Pantheon (1975).
Deneubourg, J.-L. Application de l'ordre par uctuations a la description de certaines etapes
de la construction du nid chez les termites. Insectes Sociaux 24 (2), 117{130 (1977).
Fasold, R. and Schirin, D. Language Change and Variation . Washington, DC: Georgetown
University Press (1989).
Kaplan, F., Steels, L. and McIntyre, A. An architecture for evolving robust shared communication systems in noisy environments. In: Proceedings of Sony Research Forum 1998 .
Tokyo (1998).
Kroch, A. Function and grammar in the history of english. In: (Fasold and Schirin, 1989),
pp. 133{172.
Labov, W. Principles of Linguistic Change. Volume 1: Internal Factors . Oxford: Blackwell
(1994).
Lyons, J. Language, Meaning and Context . Sussex, UK: Fontana (1981).
McIntyre, A. Babel: A testbed for research in origins of language (1998).
Meinhardt, H. Models of Biological Pattern Formation . London: Academic Press (1982).
Nichols, J. Linguistic Diversity in Space and Time . Chicago and London: The University of
Chicago Press (1992).
Niyogi, P. and Berwick, R. The Logical Problem of Language Change . Tech. rep., MIT AI
Laboratory and Center for Biological and Computational Learning, Department of Brain
and Cognitive Sciences (1995). Technical Report AI Memo 1516/CBCL Paper 115.
Noble, J. Evolved signals: Expensive hype vs.conspirational whispers. In: Proceedings of Alife
VI (Adami, e., ed.), pp. 358{367. Cambridge, MA.: The MIT Press (1998).
Oliphant, M. The dilemma of saussurean communication. Biosystems 1{2 (37), 31{38 (1996).
Pinker, S. The Language Instinct . London: Penguin Books (1994).
Prigogine, I. and Stengers, I. Order Out of Chaos . New York: Bantam Books (1984).
Ruhlen, M. On the Origin of Languages. Studies in Linguistic Taxonomy . Stanford, CA:
Stanford University Press (1994).
Sebba, M. Contact Languages: Pidgins and Creoles . Basingstoke,UK: Macmillan Press (1997).
Steels, L. Self-organizing vocabularies. In: Proceeding of Alife V (Langton, C., ed.). Nara,
Japan (1996).
Steels, L. Constructing and sharing perceptual distinctions. In: Proceedings of the European Conference on Machine Learning (van Someren, M. and Widmer, G., eds.). Berlin:
Springer-Verlag (1997).
Steels, L. and Kaplan, F. Stochasticity as a source of innovation in language games. In:
Proceedings of Articial Life VI (Adami, C., Belew, R., Kitano, H. and Taylor, C., eds.).
Los Angeles: MIT Press (1998).
Steels, L. and Kaplan, F. Situated grounded word semantics . Tech. rep., Sony Computer
Science Laboratory Paris (1999). Submitted to Publication.
Steels, L. and Vogt, P. Grounding adaptive language games in robotic agents. In: Proceedings
of the 4th European Conference on Articial Life (Harvey, I. and Husbands, P., eds.).
Cambridge, MA: The MIT Press (1997).
Thomason, S. and Kaufman, T. Language Contact, Creolization, and Genetic Linguistics .
Berkeley, CA: University of California Press (1988).