Consensus under Constraints: Modeling the
Great English Vowel Shift
Kiran Lakkaraju1 , Samarth Swarup2 , and Les Gasser3,1
1
Department of Computer Science, University of Illinois at Urbana-Champaign
2
Virginia Bioinformatics Institute, Virginia Tech
3
Graduate School of Library and Information Science, University of Illinois at
Urbana-Champaign
Abstract. The emergence and coordination of language is a domain
which is ripe for exploration through large scale agent based social simulations. In particular, the study of norms and their emergence can shed
light on the process by which linguistic norms emerge in societies.
However, current work on norms mainly ignores three aspects critical to
the study of language: (1) large normative option spaces; (2) complexly
structured, interacting and mutually constraining option states; and (3)
ambiguous (e.g., noisy) and incomplete (e.g., sampled) knowledge of others’ current normative states and preferences. Our general objective is
to explain how norms can emerge in settings with all of these characteristics. The benefit to studying norm emergence in such situations would
be to allow for the modeling of more complex social phenomena.
We view Language as a model problem for the study of norm emergence,
by simultaneously serving as a domain in which to apply ABSS and also
provide insight into the necessary next steps to model complex social
phenomena.
In this work we focus on the emergence of norms in complexly structured
option states where choices of options can interact with and constrain
each other. As a concrete instantiation we use agent based social simulation to study the Great English Vowel Shift (GEVS). Through our
simulations we are able to investigate the roles of constraint space complexity on norm emergence and also lend support to a hypothesis on the
emergence of the GEVS.
1
Introduction
It is clear that language changes at many levels. New meaning and concepts are
added to a language constantly; grammatical structures fall out of use (which
can cause great problems when interpreting historical documents [1]); new words
replace others; and finally even the pronunciation of letters changes – this is
called sound change [2].
Linguistic change occurs because of internal and external (social) factors.
Errors in spelling and pronunciation can catch on and become the norm; constraints on articulation and perception of sounds can influence how sounds are
pronounced; and even cognitive capabilities can influence grammatical structure.
Social factors have a tremendous impact on language change. “Pidgins” and
“creoles” are types of languages that emerge from the contact between two societies in certain situations. Words and phrases are often stolen wholesale as
people of different languages intermingle. Even the pronunciation of words can
change as a function of the social status of speakers.
Due to the interaction between internal and external factors, and the enormous number of factors that can impact language change Agent Based Social
Simulations are an increasingly useful technique for understanding these important phenomena.
We view language as a type of norm. Loosely speaking, norms are collective behavioral conventions that have an effect of constraining, structuring, and
making predictable the behaviors of individual agents—that is, they are conventions that can exert a normative force that shapes individual agent behaviors—
removing some behavioral possibilities from consideration and encouraging others—
without a centralized “enforcement agency.”
Models of norm emergence can be applied to linguistic cases. However, the
language case introduces three intricacies that enrich the general study of norms:
Large Normative Option Space Certain aspects of a language’s lexicon can
change independently from its grammar - for example, words in an established grammatical category (“noun,” “verb”) can be added to or deleted
from the language without changing its grammar. Thus the space of possible
options for interpreting an utterance changes multiplicatively with changes
to the lexical and/or grammatical complexity. For this reason, there is an
enormous number of possible combined lexical/grammatical interpretations
for a given utterance when the language is even moderately complex. A
population must converge on a shared lexical and grammatical convention
within this very large space.
Complexly Structured Option States A language has many interactions
between its elements. For example, lexical (word-meaning) and structural
(syntactic, grammatical) features of language interact: choosing a grammar
will establish certain lexical constraints, and conversely determining a lexical role for a word will constrain the possible grammars that fit that interpretation. Thus language as a normative case is quite different from other
convention phenomena studied earlier (e.g. [3]) that have a simple binary
normative option space.
Ambiguity and Imperfect Knowledge Agents have no direct access to others’ preferred linguistic choices (normative states). Instead they must infer
other agents states on the basis of limited linguistic samples (utterances)
possibly coupled with a limited number of shared situations from which to
“ground” semantic choices. In general, this limits agents to imperfect knowledge of collectively-preferred normative options, creating ambiguity when
they try to converge. Again, this differs significantly from binary option
spaces in which knowledge of other agents’ states can be inferred perfectly:
¬1 = 0 and ¬0 = 1.
In this work we focus on the emergence of norms in complexly structured
option states where choices of options can interact with and constrain each other.
As a concrete instantiation we use agent based social simulation to study the well
known phenomena of sound change in language (such as the Great English Vowel
Shift in England or the Northern Cities Vowel shift in the U.S. [2]). We show that
through a simple protocol where agents imitate the linguistic behavior of other
agents while minimizing phonetic constraints we can produce interesting sound
change phenomena. These preliminary results point towards future work where
more complex models of norm emergence can inform and illuminate processes of
linguistic change.
Our objective is two fold; (1) to show the efficacy of Agent Based Social
Simulation in modeling linguistic phenomena and to shed light on linguistic
hypotheses; (2) to provide insight into issues of norm emergence in complexly
structured option spaces.
2
The Great Vowel Shift
Language changes continuously on all levels. At various points in time linguists
have recognized large, significant changes in a language due to a variety of factors.
One such changes is the known as the Great English Vowel Shift (GEVS).
The GEVS took place roughly from the middle of the fifteenth century to the
end of the seventeenth century. It was a change in the pronunciation of certain
vowels; “the systematic raising and fronting of the long, stressed monopthongs
of Middle English” [4]. For example, the pronunciation of the word “child” went
from [čild] (“cheeld ”) in Middle English to [č@Ild] (“choild ”) to [čAIld] (“cha-ild ”)
in Present Day English.
The GEVS is often seen as an example of a chain shift – a situation where one
vowel changes pronunciation, thus “creating space” for another vowel to “move
up”. This causes a chained shift for a set of vowels – starting from a change by
one vowel [2].
Figure 1 is a graphical depiction of the GEVS. The trapezoid is an abstract
representation of the vowel space – the space of possible pronunciations of a
vowel. The symbols represent vowels (in the standard International Phonetic
Alphabet notation). Note that the topology of the vowel shift is linear, and thus
we can model it in a linear array.
It is still not clear how the GEVS happened. Some theories suggest purely
internal, linguistic factors; while others suggest that the interaction between
different social groups with regional dialectal variations caused a shift [4–6].
Our objective in this work is to evaluate the latter theory using Agent Based
Social Simulation. We will show that a chain shift can occur as a consequence of
the interaction of two forces. Agents, on the one hand, try to increase communicability with each other when they interact by trying to align their pronunciations,
but, on the other hand, are constrained by a phonetic differentiability require-
ment, i.e. they have to maintain sufficient spacing between their phonological4
categories to be able to distinguish the vowels from each other. The interplay of
these two processes leads to a chain shift.
i
@I @U
aI aU
e
E
æ
a
u
o
O
Fig. 1. The space of vowels, and the shift that occurred during the GEVS.
Modeling shifts in pronunciation is interesting because sounds can interact
with and constrain each other. For instance, two vowels that sound very similar
will not effectively distinguish words – and this will cause phonological merging
of the two vowels. We can view this as a constraint – vowels should be somewhat
different in pronunciation from each other in order to effectively distinguish
themselves.
Our goal is to model how vowels can shift due to interaction between agents
with different pronunciations. Our preliminary analysis will set the stage for
further exploration of these phenomena. In general, many linguistic situations
can be modeled as coming to consensus on a set of constrained variables. The
current work is one of the simplest examples that serves to illustrate a general
model of agreement under constraints.
Our work differs from others by explicitly focusing on modeling the process
of chain shifts. [7] has interesting work on the evolution of a human like vowel
system through imitation games; however it focuses on the end state of the vowel
system and not on modeling chain shifts.
[8] evaluates Trudgill’s theory of language change in New Zealand through
simulation. This work focuses on the propagation of linguistic variants in a population. Our model is similar, but we focus not only propagation, but on how
different variants interact with each other.
2.1
Language Model
Our focus in this paper is on vowel shifts, and thus we will create a simple
phonological model of language. A language consists of 5 vowels. We represent a
vowel as an integer between 0 . . . (q − 1). This corresponds, abstractly, to some
measure of the first formant of the vowel. A formant is one of the principal
frequency components of a sound. The lowest frequency component is called
4
Phonetics refers to the study of the (continuous) sound signal; phonology refers to
the study of the (discrete) categories into which this signal is perceptually mapped.
the “first formant”, the second lowest is the “second formant”, etc. Due to the
low resolution of human perception the first two formants chiefly disambiguate a
vowel [7]; for simplicity we focus on a single formant model, following Ettlinger[9].
Agents have the capability to change their vowels based on interaction with
others. This models people interacting with each other and learning new words/phrases
and even changing their pronunciation to increase communicability with each
other. We simulate interaction between people via language games [10, 11] –
interactions in which agents exchange knowledge of their language with each
other. Since we are concerned only with vowels here, we abstract an interaction
as follows.
A language game consists of two agents, a speaker (as ) and a hearer (ah ).
One of the five vowels is chosen as the topic of the game. The speaker then communicates to the hearer the value of this vowel. The hearer changes the value
of its own vowel based on the hearers value of the vowel, and based on the constraints below. Note that this seemingly unrealistic interaction is an abstraction
of a more natural interaction, where an agent utters a word that contains the
vowel which we are calling the topic above. Further, it is generally possible to
guess the word from context, even if the vowel pronunciations of the two agents
disagree. Thus, we assume that the hearer knows both which vowel the speaker
intended to utter, and which one (according to the hearer’s vowel system) he
actually uttered. The hearer then changes his own vowel system based on this
information and the constraints below.
In this work we focus on interactions where agents want to change – that
is, whether intentionally or unintentionally hearer agents modify their vowels.
In real interactions there are a whole host of social forces that could inhibit
change – such as wanting to maintain a separate social identity. While this is an
extremely important question, we do not attempt to address those issues in this
simple model.
Phonetic Differentiation Constraint The vowels (in a vowel system) should
have pronunciations that are different enough to allow reasonable differentiation between them. Intuitively, if two vowels are very similar they will
be mistaken for each other. To increase communicability vowels should be
different from each other.
Ordering Constraint There is a total ordering on the vowels. This means,
e.g., that two vowels cannot swap their locations.
These constraints, along with the model, are formally specified below. In
addition to these constraints, we assume that the agents are boundedly rational,
which means that there is only a limited amount of change an agent can do to
its vowel system per time step. Intuitively, it does not make sense for a person
to change their entire vowel system after every interaction.
More formally, our model is specified by the tuple: < A, V, D, f > where:
– A = {a0 , a1 , . . . an−1 } are a set of n linguistic agents.
– V = {v0 , v1 , . . . , vm−1 } be a set of m variables where each variable represents
a vowel. In this work we only consider the case of m = 5. Each agent has its
own set of variables which represents its pronunciation.
– D = {0, 1, . . . (q−1)}. Each variable can take a value from D which represents
the frequency of the first formant of the vowel. In this model we look at a
set of discrete frequencies labeled 1 through 30.
– f : D × D → {0, 1}. f models the phonetic differentiation constraint. See
below.
f is parametrized by d (0 < d < q) which implements the phonetic differentiation constraint. We define f as:
0 if |x − y| − 1 ≥ d
f (x, y) =
(1)
1 otherwise
f (x, y) = 0 when two vowels cannot be differentiated: their pronunciations
are too similar.
Let xi denote the state of agent ai i.e. the settings of its variables. xi,j denotes
the value of variable vj for agent ai . We use xi to denote the value of variable
vi when the agent does not matter.
Let
X
Fx =
f (xx,i , xx,j )
(2)
i,j
be the local cost function. We use the term cost to describe the number of
pairs of vowels that violate the phonetic differentiation constraint. Fx determines,
for each agent ax , the cost of its current state. We say a state is a minimal state
when the local cost of the state has been minimized. Note that every agent uses
the same cost function.
Let
n−1
X
F =
Fx
(3)
x=0
be the global cost function; the total number of pairs of vowels that are phonetically ambiguous over the entire population.
We say the system is coordinated and minimized when the the following
conditions are met:
Coordination Condition ∀i,j xi = xj
Minimization Condition F is minimized.
We also use the term fully solved when both conditions are met. When one
or both of the conditions are not met we say the system is coordinated and
unminimized, uncoordinated and minimized, and finally uncoordinated and unminimized. We also refer to these three situations as the system being partially
solved.
The ordering constraint means that: ∀i xi < xi+1 .
At every time step a language game is played:
1. Choose two neighboring agents as and ah who are referred to as the speaker
and hearer respectively.
2. A random variable vt is chosen as the topic of the game. Let xs,t and xh,t
be the value of vt for the speaker and hearer respectively.
3. The hearer runs the iterative update algorithm updateVariable (vt , xs,t , E).
E represents the bounded rationality of the agents in terms of a maximum
amount of “effort” an agent can undertake per time step. While this an important
and interesting parameter to modify we leave it to future work to explore the
impact of varying bounds on effort. We set effort to a large amount so that it
does not affect the system dynamics.
The algorithm by which the hearer agent modifies its vowels is presented
in Algorithm 1. The essential idea is that the hearer only modifies its vowel
if the modification does not violate the phonetic differentiation constraint. For
instance, suppose ah has value xh,t = 5 and xh,t+1 = 10 with d = 4 and plays
a language game as who has value xs,t = 6. The value of vt would not change
for ah , as it would violate the phonetic differentiation constraint. In addition, if
there is not enough effort the change does not occur as well.
Algorithm 1 Hearer update algorithm.
1: procedure updateVariable(vt , xs,t , E)
2:
dir ← sign (xs,t − xh,t )
3:
z ← t + dir
0
4:
xh,t ← changeVariable(vt , xs,t , E, dir )
0
5:
if |xh,t − xh,z | − 1 ≥ d then
0
6:
xh,t ← xh,t
7:
end if
8: end procedure
9: function changeVariable(vt , to, E, dir )
10:
return min{to, xh,t+dir − dir , xh,t + dir · E}
11: end function
2.2
Number of Minimal States
The optimal solution is for all agents to converge upon a minimal state, this
increases the communicability since the vowels will be phonetically differentiated.
A clear indication of the complexity of this is the number minimal states which
depends upon the values of n, q, and d. Let N (n, q, d) be the number of minimal
states, then
n+z
N (n, q, d) =
(4)
n
where z = q − (n − 1)(d + 1) − 1.
Left justified minimally compact state
1
2
3
4
v0
5
6
7
v1
8
9
10
11
8
9
10
11
v2
Two minimal states
1
2
3
4
5
6
v0
1
7
v1
2
3
4
v0
5
v2
6
7
8
9
10
11
v1
v2
Fig. 2. Example construction of minimal states with n = 3, q = 11 and d = 2. The
blue squares are “extra space”.
Number of minimal states when variables are in sequence.
100000
log(N(n,30,d))
10000
1000
100
10
n=2
n=3
n=4
n=5
1
1
2
3
4
5
6
7
8
9
10
d
Fig. 3. Log-Linear plot of the number of minimal states for q = 30 and n = 2 . . . 5 and
d = 1 . . . 10.
We derive N (n, q, d) by first counting the amount of space taken by the
left justified minimally compact state. Imagine the variables positioned on a 1D array labeled from 1 to q.Then the left justified minimally compact state is
where all the variables are as close as possible to the left side of the domain. The
extra space to the right can be allocated to each variable without violating the
phonetic differentiation constraint.
z = q − (n − 1)(d + 1) − 1 is the amount of
remaining space, and n+z
is the number of ways to distribute that space over n
n
variables. Figure 2 shows an example of a left justified minimally compact state
and two minimal states.
Figure 3 shows the number of minimal states for q = 30, n = 2 . . . 5, d =
1 . . . 10. For all values of n there is an exponential decline as d increases. The
rate of decline increases as n increases, because more variables have to “fit into”
the vowel space.
3
Simulation Results
We want to study how a chain shift can occur in a population. To do this, we
are going to empirically evaluate two situations:
1. From an initially random condition, where the agents have randomly chosen
assignments of vowel positions (that respect the ordering constraint but not
the phonetic differentiation constraint), we run the simulation to show that
they can attain consensus on a fully solved configuration.
2. From a fully solved configuration, where all agents have the same state that
minimizes the local cost, we modify a subset of the agents to have a slightly
different, but minimal, state. This models sudden immigration of a new
population. Does this cause a chain shift in the pronunciation of vowels?
Simulation Parameters The simulations below had the following settings:
n = 1000, m = 5,q = 30, d = 4. To simulate social hierarchies we array the
agents on a scale-free graph. We use scale-free graphs because they have been
shown to model many other real-world phenomena, such as actor-collaboration
graph [12]. We used the extended Barabasi-Albert scale free network generation
process [13]. The parameters were m0 = 4, m = 2, p = q = 0.4.
3.1
Consensus to a fully solved configuration
In the first simulation, we initialize the population randomly, as described above,
and follow algorithm 1 for the hearer in each language game. The figure shows
time on the x-axis, and the average value of each vowel position on the y-axis.
We see that the lines corresponding to the vowels become completely flat as the
simulation progresses, and then stay that way. This demonstrates the emergence
of a stable state. Further, the vowel positions are widely separated, which shows
that the cost function is being minimized.
3.2
A New Subpopulation
What happens when a new population of individuals is introduced? In this experiment we replaced 30% of the population with a new population that were
converged on a different state. Initially, all agents had state [0, 5, 10, 15, 20]. The
introduced population had state [5, 10, 15, 20, 25], which is an overlapping but
different minimal state. The entire vowel system is shifted by five positions with
respect to the existing vowel system.
The new population replaced the lowest degree 30% of nodes in the graph.
The new population is introduced in iteration 1000, and shows up as a sudden
jump in the average value of the vowel positions in figure 5.
Interactions between the new population and existing agents immediately
start to cause a shift in the vowel positions. However, the vowels don’t start
shifting all together – the first vowels to shift are v0 and v4 , which move in
Average values of variables vs. time
30
25
Average value
20
15
10
5
v0
v1
v2
v3
v4
0
0
5e+06
1e+07
1.5e+07
2e+07
2.5e+07
3e+07
Time (Measured in Iterations)
Fig. 4. Emergence of a consensus fully-solved configuration.
Average values of variables vs. time
30
25
Average value
v4 initial Avg.
20
v3 initial Avg.
15
v2 initial Avg.
10
v1 initial Avg.
5
v0
v1
v2
v3
v4
v0 initial Avg.
0
0
5000
10000
15000
20000
Time (Measured in Iterations)
Fig. 5. Initiation of the vowel shift.
25000
30000
opposite directions. These are followed in turn by v3 and v2 (in the direction
of v4 ), while v1 ends up staying more or less stable. Figure 6 shows the long
run behavior of the same experiment. We see from this figure that eventually
the entire population converges on a new stable state [0, 5, 15, 20, 25], which is a
combination of the vowel systems of the two populations. Further, the emergence
of this new vowel system occurs through a chain shift, in two directions – v0
moves down, while v4 , v3 , andv2 move up, in that order.
Average values of variables vs. time
30
25
Average value
v4 initial Avg.
20
v3 initial Avg.
15
v2 initial Avg.
10
v1 initial Avg.
5
v0
v1
v2
v3
v4
v0 initial Avg.
0
0
500000
1e+06
1.5e+06
2e+06
2.5e+06
3e+06
3.5e+06
4e+06
4.5e+06
5e+06
Time (Measured in Iterations)
Fig. 6. Coordination to a new state with a new introduced population.
4
Discussion
As discussed earlier, the vowel space in figure 1 can be “unfolded” into a linear
array, in which the a → æ→ E→ e → i → @I→ aI shift is a movement to the left,
and the O → o → u → @Ú→ aÚ shift is a movement to the right. This matches,
qualitatively, the movements observed in the experiments above, where some of
the vowel positions shift up (to the left), and some shift down (to the right).
There are several competing explanations for the GEVS. One of the main
contenders is that, after the Black Death there was a mass immigration into
South-Eastern England, and the contact between the immigrants and locals
led to a sudden change in vowel system. We have shown that this is a viable
explanation, as it requires only the minimal assumptions that people tend to
“accomodate” during interactions, i.e. they change their way of speaking to
adapt to the other participant in the interaction, and that this accomodation is
very limited, i.e. that they do not change their entire vowel system at once.
5
Conclusion
The study of the emergence of norms in populations is an important aspect
of social simulation. Norms are the means by which a society autonomously
structures itself. In “contact situations”, i.e. in cases where two populations come
into contact with each other, or where a segment of a population undergoes rapid
replacement, norms can change abruptly too. Such abrupt changes can have the
effect of marking major transitions in society, such as the transition from Middle
English to Modern English.
The study of language norms is a difficult undertaking, however, because language as a system consists of an intricate web of relationships and constraints
between several variables. We have presented an approach to study consensus
under constraints, and have shown that we can model phenomena of sound
change such as vowel shifts. However, the same approach can, in principle, also
be applied to any situation where we have a system of constrained variables over
which consensus must be achieved. Note that this is different from a distributed
constraint satisfaction scenario, where the variables are distributed over a population of agents. Here each agent has the same set of variables, and has to
coordinate with other agents so that they all converge on the same solution to
the problem.
There is much work that can be done to extend the current work. An account
of the theoretical underpinnings of this framework remains to be developed, as
well as extensions to other aspects of language, such as syntax. The idea of
consensus under constraints is general enough, though, that we are confident
that these goals can be achieved.
6
6.1
Appendix
Albert-Barabàsi Extended Model
The Albert-Barabàsi extended model depends upon four parameters; m0 is the
initial number of nodes, m(≤ m0 ) is the number of links that are added or
rewired every step of the algorithm, p is the probability of adding links, and q is
the probability of rewiring an edge (p + q = 1). The algorithm to generate the
network is as follows. Start with m0 isolated nodes, and at each step perform
one of these three actions:
1. With probability p add m new links. Choose the start of the link uniformly
randomly and the end point with distribution:
ki + 1
j (kj + 1)
Πi = P
(5)
where Πi is the probability of selecting the ith node and ki is the number of
edges of node i. This process is repeated until m new links are added to the
graph. If m links cannot be added we add as many as possible.
2. With probability q rewire m edges. Pick uniformly randomly a node i and
link lij between node i and node j. Delete this link and choose another node k
according to the probability distribution Πi with the constraints that k 6= i, j
and lik does not already exist. Add the link lik .
3. With probability 1 − p − q add a new node with m links – the new links with
connect the new node to m other nodes chosen according to Πi .
References
1. Baron, D.E., Bailey, R.W., Kaplan, J.P.: Brief for professors of linguistics and
english dennis e. baron, ph.d,. richard w. bailey, ph.d. and jeffrey p. kaplan, ph.d.
in support of petitioner. Amicus Brief
2. Hock, H.H., Joseph, B.D.: Language History, Language Change, and Language
Relationship: An Introduction to Historical and Comparitive Linguistics. Mouton
de Gruyter (1996)
3. Shoham, Y., Tennenholtz, M.: On the emergence of social conventions: modeling,
analysis, and simulations. Artificial Intelligence 94(1-2) (July 1997) 139–166
4. Lerer, S.: Inventing English: A Portable History of the Language. Columbia
University Press (2007)
5. Leith, D.: A Social History of English. 2nd edn. Routledge (1997)
6. Perkins, J.: A sociolinguistic glance at the great vowel shift of english. Papers in
Psycholinguistics and Sociolinguistics. Working Papers in Linguistics (22) (1977)
7. de Boer, B.: Self-organization in vowel systems. Journal of Phonetics 28 (2000)
441–465
8. Baxter, G., Blythe, R., Croft, W., McKane, A.: Modeling language change: An
evaluation of trudgill’s theory of the emergence of new zealand english. Language
Variation and Change (To Appear)
9. Ettlinger, M.: An exemplar-based model of chain shifts. In: Proceedings of the
16th International Congress of the Phonetic Science. (2007) 685–688
10. Wittgenstein, L.: Philosophical Investigations. Blackwell Publishing (1953)
11. Steels, L.: Emergent adaptive lexicons. In: Proceedings of the Fourth International
Conference on Simulating Adaptive Behavior. (1996)
12. Barabasi, A.L., Albert, R.: Emergence of scaling in random networks. Science 286
(1999) 509
13. Albert, R., Barabàsi, A.L.: Topology of evolving networks: Local events and universality. Physical Review Letters 85(24) (2000) 5234–5237