Diss. ETH No. 19639
Decision making in models
of social interaction
A dissertation submitted to the
ETH ZURICH
for the degree of
DOCTOR OF SCIENCES
presented by
HANS-ULRICH STARK
Diplom-Verkehrswirtschaftler, Technische Universität Dresden
born September 23, 1977
citizen of Germany
accepted on the recommendation of
Prof. Dr. Dr. Frank Schweitzer, examiner
Prof. Dr. Dirk Helbing, co-examiner
2012
Contents
I
Introduction
2
1 Modeling and analyzing social interaction
3
1.1
“Micromotives and macrobehavior” . . . . . . . . . . . . . . . . . . . . . .
4
1.2
The Weidlich-Haag approach . . . . . . . . . . . . . . . . . . . . . . . . . .
5
1.3
Dynamical fixed points and their stability . . . . . . . . . . . . . . . . . .
7
1.3.1
A one-dimensional flow . . . . . . . . . . . . . . . . . . . . . . . . .
8
1.3.2
Multi-dimensional flows . . . . . . . . . . . . . . . . . . . . . . . .
10
2 Decisions through social influence
12
2.1
Decision theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12
2.2
Models of social influence . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14
2.3
The voter model
18
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3 Decision making in (evolutionary) game theory
3.1
23
Classical game theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
23
3.1.1
Definitions and terminology . . . . . . . . . . . . . . . . . . . . . .
23
3.1.2
Coordination- vs. cooperation problems
. . . . . . . . . . . . . . .
29
3.1.3
Symmetrical 2×2 games – the framework . . . . . . . . . . . . . . .
32
3.2
Experimental game theory . . . . . . . . . . . . . . . . . . . . . . . . . . .
35
3.3
Evolutionary game theory . . . . . . . . . . . . . . . . . . . . . . . . . . .
37
3.3.1
Evolutionary dynamics . . . . . . . . . . . . . . . . . . . . . . . . .
39
3.3.2
The “conundrum” of cooperation . . . . . . . . . . . . . . . . . . .
45
ii
4 Social influence in evolutionary game theory
47
II
52
Models and analyses
5 Slower is faster: Fostering consensus formation by heterogeneous inertia 53
5.1
(Social) Inertia in the voter model . . . . . . . . . . . . . . . . . . . . . . .
54
5.2
Preliminary considerations . . . . . . . . . . . . . . . . . . . . . . . . . . .
56
5.3
Results of binary inertia . . . . . . . . . . . . . . . . . . . . . . . . . . . .
59
5.4
Results of multiple inertia states . . . . . . . . . . . . . . . . . . . . . . . .
68
5.5
Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
76
5.6
Outlook: inertia-effects in another nonlinear voter model . . . . . . . . . .
79
6 Social influence in social dilemmas
84
6.1
Homogeneous, dynamic random networks . . . . . . . . . . . . . . . . . . .
85
6.2
Evolutionary dynamics with imitators and contrarians . . . . . . . . . . . .
86
6.3
Co-evolution in the spatial Prisoner’s Dilemma . . . . . . . . . . . . . . . .
89
6.4
The role of imitation in social dilemmas . . . . . . . . . . . . . . . . . . .
95
6.4.1
Motivation and literature . . . . . . . . . . . . . . . . . . . . . . . .
95
6.4.2
The evolution of cooperation through imitation . . . . . . . . . . .
98
6.5
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
7 Dilemmas of partial cooperation
116
7.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
7.2
Symmetrical 2×2 games . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
7.3
Derivation of PCD in repeated games . . . . . . . . . . . . . . . . . . . . . 123
7.3.1
Partial cooperation in repeated games . . . . . . . . . . . . . . . . 123
7.3.2
From social dilemmas to PCD . . . . . . . . . . . . . . . . . . . . . 124
7.4
Derivation of PCD in evolutionary games . . . . . . . . . . . . . . . . . . . 126
7.5
Evolution of partial cooperation . . . . . . . . . . . . . . . . . . . . . . . . 129
7.6
Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
iii
III
Conclusion
135
Appendix: Transformations of the standard cooperation model
iv
140
Preface It has always been a hard task to explain in two or three sentences what the
research of my PhD thesis is about. Fortunately, there is room for some more sentences
here and I want to take the opportunity of this Preface to do my best in explaining this
as untechnically as possible.
As is obvious from the title, this thesis deals with “social interaction”, which is maybe the
most intuitive part: at least two individuals encounter each other and interact in any arbitrary way. Such individuals can be humans or animals and the kind of interaction ranges
from talking, helping, and fighting to rather indirect interactions like consumers reacting
to special supermarket offers, animals using the same food resources, or a species producing a biological side product which supports another species. These examples already
indicate a number of relevant research fields, e.g. social sciences, psychology, economics,
management, biology, and ecology, thereby pointing out the broad interdisciplinarity of
the research topic.
In the behavioral sciences, one would then formulate questions like:
(i) How do human beings decide/behave in certain situations and what are their reasons,
motivation, and decision mechanisms?
But this is not the kind of question which is addressed in this thesis. In fact, the knowledge
gained by behavioral sciences is used in order to investigate questions of the type:
(ii) Given the behavior of individuals, which consequences arise from the combination of
individual behaviors for a community of many interacting individuals?
In many cases, the consequence of social interaction is not just the sum of individual
actions, but might be interdependent on each other. Let me give a simple example: if a
fire emerges during a theater performance, people should get out of the theater as fast as
possible. However, if all the people do their best to do so, the result can be a fatal panic
situation, i.e. the worst outcome where the evacuation can get stuck. This example is also
suited to illustrate that careful investigations of this type may lead to counter-intuitive
results: it can theoretically and practically be shown that deliberately placing obstacles
in front of an emergency exit can help avoiding fatal clogging situations. In other words,
putting obstacles in front of an emergency exit can improve the evacuation during an
emergency event (see Helbing et al. (2005a) for details and other results).
“Modeling” social interaction is very important for the research in this example, but also
in many other cases. By modeling I mean the abstraction of real social behaviors and
interactions into mathematical equations that capture the most important characteristics.
v
Solving these equations analytically or by means of numerical computer simulations, one
gains a quantitative understanding of causes and effects under the conditions covered
by the equations. Another advantage is that one can efficiently change these conditions
and investigate differences between different scenarios. Such an approach also allows to
investigate questions of the type:
(iii) How can we understand and explain facts in our environment, that seem to be contradictory to current knowledge?
Such a question is closely connected to the previous ones. If one understands the main
driving forces leading to the status quo, a lot is learned about the behavior of subjects
under consideration (type (i) questions). This in turn improves the models of social
interaction, which then are used to answer questions of type (ii).
In this thesis, questions of the type (ii) and (iii) will be discussed. In the models, social
individuals are characterized by one or more certain attributes and a behavior for the
interaction with other individuals. To illustrate this, let me mention a simple example: the
famous segregation model of Thomas C. Schelling (Schelling, 1978). There are two types of
individuals that differ in one ethnical trait or, for example, in their religious beliefs. They
all live in one area - maybe a small town - and the arrangement of houses is simplified to a
regular grid, where houses are located at the intersections of lines. Hence, every house is
surrounded by four other houses, but not all of them are inhabited. As a simple behavioral
rule, it is assumed that an individual moves to a random, empty house if more than half
of its neighbors do not share its belief. With this simplistic model, Schelling described
how uncoordinated actions (individuals move to a random node) can lead to a state with a
high level of global ordering (segregation). Whereas Schelling initially analyzed this model
on a sheet of paper, other, still simplistic models require more quantitative involvement.
In combination with the fact that there are parallels between social individuals and the
physical interaction of atoms or other particles, the interdisciplinarity of this thesis’ subject
is even more broad. Especially in the last decade, a vital exchange of ideas and models
between the sciences mentioned above and mathematics, physics, chemistry, and computer
science has been and still is developing. One of the results is the emergence of a new
scientific branch called socio- or econophysics, where methods of physics are applied to
social or economic systems. The content of the present work does also exhibit this kind of
interdisciplinarity.
Finally, let me introduce the specific kinds of decision making within the social interaction
that are investigated in this thesis – “social influence” and “strategic decisions”, where
the latter refers to following an arbitrary strategy in the game theoretic sense, and the
vi
former is restricted to individuals’ reactions to observed behavior. The difference lies in
the characteristics of the individuals that are considered in the models. To some extent,
the considered individuals undergo a cultural evolution in the course of this work: they
start as individuals with “zero confidence”. They do not consciously decide for a specific
alternative, but only imitate other individuals. Imitation is a very simple form of decision
making and when I speak of imitation throughout this work, I mean the most primitive
form of imitation: an individual does not imitate because it tries to copy a successful
behavior, but it just imitates regardless of success considerations. One example is “herding
behavior”, which might have its origin in the absence of relevant information or cognitive
abilities. Moreover, these “primal” individuals are also zero confident because they are
not tied to a once taken decision, but readily imitate another one. Let us say that these
individuals are entirely susceptible to social influence. There are a number of models (some
of them will be introduced in the main body of this thesis) dealing with such behavior
and the main conclusion is quite intuitive: individual tendencies to imitate each other
lead to highly homogeneous decisions among individuals. However, there are aspects that
are not straight-forward and one can even find counter-intuitive results that need to be
understood. At least one of such surprising results will be presented and carefully analyzed
within this thesis.
The considered individuals then develop a certain kind of confidence. Unlike before, they
take a decision for some reason and their readiness to switch decisions is reduced compared
to the zero confident individuals. Moreover, their confidence might evolve in time – an
individual that took one and the same decision many times already is assumed to be highly
confident about it and has an “inertia” to change to another decision. It is one of the main
aims of this thesis to thoroughly investigate how such behavior influences the theoretical
results of the respective models.
In a next “evolution step”, the individuals develop more sophisticated decision behavior.
They well distinguish between the different decision alternatives and their consequences
(success) and directedly decide for one of them. They might even be capable of memorizing
past interactions and developing adaptive decision behavior. I call this “strategic decision
making”, because considerations about the possibilities of interaction partners, or their
behavior in the past, might enter the decision making process of an individual. Such a
success-oriented behavior is also essential for the individuals, because there is competition
in their environment. The success of taken decisions is evaluated and compared with
other individuals. Only successful individuals are able to survive the competition and to
reproduce. In consequence, successful behavior can spread in the environment, whereas
unsuccessful behavior might face extinction. In my own view, it has been exciting to
realize that there are respective models capable of capturing the essential principles of
vii
natural selection in this way.
Finally, I will compare the nature of the “primal individuals” with the one of self-confidently
deciding individuals. Although it seems unfair, I will even put them together in the same
competitive environment and let them “struggle for life”, as Darwin has put it (Darwin,
1859). Unfair or not, it seems reasonable to assume that there is both zero-confident imitation behavior and more strategic decision making present in Nature. The results of this
experiment disclose surprising consequences of this coexistence and yield new insight into
one of the most discussed, interdisciplinary questions in current science: if cooperative
behavior is individually suboptimal, how can such behavior be richly present in todays
world? In fact, this is one of the questions Darwin left open, because, following his theory,
natural selection favors the most successful behaviors in the long run. In this work, some
pieces of evidence will be added in order to overcome the seeming discrepance between
cooperation and evolution.
The present thesis concludes with a more conceptual work on game theoretic decision
situations themselves. The modeling of the individuals’ “environment” is of course very
important, because the validity of theoretical results is restricted to the specifications of
the model. It is always a crucial, but not simple task to find a reasonable compromise
between keeping models simple (in order to derive precise conclusions) and making them as
realistic as possible. In this final part, I argue that with a different look at an established
methodology, one can cover a wider range of strategic decision situations and possibly
obtain more explanations of real phenomena. Besides some concrete results, this part
consists of a methodological argumentation and serves as an outlook on potential further
research in this field.
viii
Abstract Investigating social interaction systems is highly complex because no social
individual behaves exactly like another and, in many cases, the action of an individual i
causes effects on other individuals, that in return may act to cause effects on the behavior
of individual i. Therefore, we find the main sources of complexity; heterogeneity and the
existence of feedback loops. Both are sources of non-linear dynamics, which often lead to
unexpected, sometimes even paradoxical, results for dynamical systems. However, understanding the intrinsic, dynamical properties of social interaction systems is very important
for regulating such a system towards a desired state, or at least to avoid measures that
are disastrous for the social system. Moreover, there is a large overlap between social
interaction and the interaction between biological species or physical particles. Therefore,
especially in the last decade, there is growing interest in the interdisciplinary fields of
socio-physics (applying physical and mathematical laws to social interaction) and evolutionary game theory (applying models of strategic decision theory to the evolution of
social behaviors, biological species, and ecological populations). Both approaches aim at
a quantification and analysis of complex systems, which is only possible if the complexity is largely reduced. It results in a critical trade-off between complexity reduction and
the explanatory power of the analyzed model, which is an important part of the research
challenges in these fields.
In this thesis, we investigate social interaction systems from two perspectives. The first is
called “opinion dynamics”, which emphasizes the role of social influence in decision making. The second is evolutionary game theory, where the decisions are not only depending
on social influence, but might follow any decision rule. While the work within this thesis
contributes novel insight into both fields, the central aim is to reveal the large common
ground and to argue that knowledge gained in one field is very likely to be relevant for the
other field. Beginning with the field of opinion dynamics, we investigate a co-evolutionary
dynamics of imitation and conviction, where some individuals quickly imitate their neighbors, while others rather keep their own opinion. Furthermore, the behavior of individuals
steadily changes depending on their interaction history. We report on surprising results,
where a system with a higher level of conviction promotes the emergence of consensus
in comparison to a system where all individuals readily imitate. We then apply a very
similar co-evolutionary dynamics to evolutionary game theory, specifically to the evolution of cooperating and non-cooperating individuals. Since cooperating individuals always
provide help to others at their own cost and following the selective rules of evolution, noncooperating individuals are generally expected to displace cooperating individuals in the
course of time. It is one of the most relevant, scientific problems to find suitable explanations of the high level of cooperation in Nature and our cultural environment. With
the applied co-evolutionary rule, where some individuals are more influential than oth-
ix
ers and this characteristic again changes depending on the interaction history, we find
that this promotes the persistent survival of cooperating individuals, although initially
cooperating and non-cooperating individuals have the same level of influence. Therefore,
this co-evolutionary setup is a good example to illustrate the strong connection between
opinion dynamics and evolutionary game theory. In a further step, we combine the purely
imitational behavior of opinion dynamics and the more general decision behavior of game
theory together in a novel model of evolutionary game theory. Also in this approach we
find remarkable results, indicating that the pure existence of imitational behavior might
explain the evolutionary success of cooperative strategies. We therefore can conclude with
the proposition we started from: opinion dynamics and (evolutionary) game theory are
two perspectives on the same matter – social interaction systems.
x
Kurzfassung Die Untersuchung sozialer Interaktionssysteme ist komplex: Einerseits
verhalten sich keine zwei Individuen exakt gleich und andererseits hat eine Aktion eines
Individuums i Einfluss auf die Aktion eines anderen Indivduums j, was wiederum zu
einem Einfluss auf i führen kann. Damit sind die Hauptursachen von Komplexität vorhanden: Heterogenität und die Existenz von Rückkoppelungseffekten. Beides führt zu nichtlinearer Dynamik, die unerwartete, teilweise sogar paradoxe, Resultate hervorbringen
kann. Ein Verständnis der inhärenten dynamischen Eigenschaften sozialer Interaktionssysteme ist aber wichtig, um sie effizient regulieren zu können oder sie zumindest vor drastischen Schäden bewahren zu können. Zudem gibt es Parallelen zwischen sozialer Interaktion und der Interaktion zwischen biologischen Arten oder physikalischen Teilchen, was
besonders im letzten Jahrzehnt zu einem wachsenden Interesse an interdisziplinären Wissenschaften wie der Sozio-Physik (Anwendung physikalischer und mathematischer Methoden zur Beschreibung sozialer Interaktion) und der evolutionären Spieltheorie (Anwendung
von Modellen strategischer Entscheidungstheorie auf die Evolution sozialen Verhaltens, biologischer Arten und ökologischer Populationen) geführt hat. Beide Disziplinen streben
eine Quantifikation und Analyse komplexer Systeme an, wozu eine Verringerung der realen
Komplexität vonnöten ist. Daraus ergibt sich ein Widerstreit zwischen der notwendigen
Reduktion von Modell-Komplexität und der Aufrechterhaltung der Aussagekraft des untersuchten Modells, was einen entscheidenden Teil der wissenschaftlichen Aufgaben in
diesem Bereich darstellt.
In der vorliegenden Arbeit werden soziale Interaktionssysteme aus zwei Perspektiven betrachtet. Die erste ist die Untersuchung von “Meinungsbildungsprozessen”, bei welcher
die Rolle von sozialem Einfluss auf die Entscheidungen Einzelner hervorgehoben wird.
Die Zweite ist die der evolutionären Spieltheorie, wobei Entscheidungen nicht nur durch
sozialen Einfluss bestimmt werden, sondern jeglicher Art von Entscheidungsregel entspringen können. Während die Arbeit in dieser Dissertation neue Erkenntnisse für beide
Disziplinen hervorbringt, ist es das zentrale Anliegen, die gemeinsame Grundlage beider Gebiete darzustellen und herauszuarbeiten, dass Resultate in einem Gebiet mit großer
Wahrscheinlichkeit auch Relevanz für das jeweils andere Gebiet haben. Wir beginnen mit
Meinungsbildungsprozessen und untersuchen eine koevolutionäre Dynamik von Imitation
und Überzeugung, wobei einige Individuen bereitwillig ihre Nachbarn imitieren, während
andere eher ihre Meinung behalten. Außerdem ändert sich dieses Verhalten der Individuen in Abhängigkeit ihrer Interaktions-Historie. Wir beschreiben das überraschende
Ergebnis, dass ein System mit einem höheren Grad an Überzeugung die Enstehung von
Konsens unterstützt, wenn man es mit einem System von reiner, bereitwilliger Imitation
vergleicht. Anschließend wenden wir eine vergleichbare koevolutionäre Dynamik auf die
evolutionäre Spieltheorie an, im Speziellen auf die Evolution von kooperierenden und nicht-
xi
1
kooperierenden Individuen. Da Kooperierende jederzeit anderen Individuen auf ihre eigenen Kosten helfen und die Dynamik den selektiven Regeln der Evolution folgt, sollten die
Kooperierenden im Laufe der Zeit von nicht-kooperierenden Individuen verdrängt werden.
Es ist eines der überaus relevanten wissenschftlichen Problemstellungen herauszufinden,
wie das hohe Maß an Kooperation in unserer natürlichen und kulturellen Umwelt zu
erklären ist. Die koevolutionäre Dynamik, bei welcher einige Individuen einflussreicher
sind als andere und erneut die Ausprägung dessen von der Interaktions-Historie abhängt,
führt zu einer deutlich größeren Überlebenswahrscheinlichkeit für Kooperierende, obwohl
Kooperierende und Nicht-Kooperierende ursprünglich gleich viel Einfluss haben. Daher
ist dieser koevolutionäre Modell-Ansatz ein gutes Beispiel um die Verbindung zwischen
Meinungsbildungsprozessen und evolutionärer Spieltheorie zu veranschaulichen. In einer
weiteren Untersuchung kombinieren wir das rein imitative Verhalten von Meinungsbildungsprozessen mit dem allgemeineren Entscheidungsverhalten der Spieltheorie in einem
neuartigen Modell-Ansatz der evolutionären Spieltheorie. Auch in diesem Ansatz erhalten
wir beachtenswerte Resultate: Die bloße Existenz von Imitation kann unter Umständen
den evolutionären Erfolg von Kooperation erklären. Zusammenfassend kann man die
aufgestellte Prämisse dieser Arbeit wiederholen: Meinungsbildungsprozesse und (evolutionäre) Spieltheorie sind zwei verschiedene Perspektiven auf ein und denselben Zusammenhang – soziale Interaktionssysteme.
Part I
Introduction
The present thesis investigates models of social interaction from different perspectives.
This introductory Part consists of four Chapters. Chapter 1 aims at explaining and motivating the quantitative research approach taken in this work. In Chapters 2 and 3, we
mainly introduce the fields of opinion dynamics and (evolutionary) game theory, which
constitute the main subjects of this work. The first one is what we refer to as models of
“social influence” and the latter represent investigations on “strategic decision situations”.
Finally, Chapter 4 relates both fields to each other and reveals one of the goals of this
thesis: showing that both approaches mainly differ in their perspective on social interaction and using this proposition to investigate the impact of social influence on models of
evolutionary game theory.
Chapter 1
Modeling and analyzing social
interaction
Within this first Chapter, we aim at preparing the general ground for the discussions,
models, and analyses of the present thesis. We want to motivate the central approaches
applied in the thesis and address their difficulties and critical points. In particular, we
introduce the so-called micro-macro link, that forms the central fundament of all modeling
techniques within the presented material of Part II. Furthermore, based on an exemplary
model, we motivate and introduce the quantitative modeling approach taken within this
thesis and discuss and the ubiquitous challenge attached to it: finding a reasonable balance
between needed simplifications and remaining explanatory power of the models.
Let us first specify what exactly is meant by “social interaction”. A sociological lexicon
defines it as:
“Interaktion, soziale, [1] die durch Kommunikation (Sprache, Symbole,
Gesten usw.) vermittelten wechselseitigen Beziehungen zwischen Personen und
Gruppen und die daraus resultiende wechselseitige Beeinflussung ihrer Einstellungen, Erwartungen und Handlungen. [...]”
Lexikon zur Soziologie (Fuchs-Heinritz et al., 1994)
Freely translated by the author, this means that social interaction concerns the interactive
relations between individuals or groups, mediated via communication (language, symbols,
gestures, etc.) and the resulting, interactive influence on the individual attitudes, expectations, and actions.
3
4
1.1
Chapter 1. Modeling and analyzing social interaction
“Micromotives and macrobehavior”
Focusing on social interaction, it is less important how an individual reacts to a certain
situation and why it does so, but one is mainly interested in the resulting effects out
of the interaction between individuals. The particular interest of the present work is on
interaction outcomes that are not primarily intended and assessed by the involved social
entities (that generally will be called “individuals” in this work, but in special cases also
“voters” or “players”), but result from inherent (mathematical) properties of the social
system. When we speak of a (social) system here, in simple words, we mean the collectivity
of social interactions (that might be an emergent phenomenon itself). The heading of this
Section, which we borrowed from an excellent book by Thomas C. Schelling (Schelling,
1978), sketches the central theme of the present work: investigating macroscopic effects of
microscopic actions.
For this purpose, it is often necessary to reduce the complexity of real interaction scenarios to much more simple interaction models. Only then it is possible to quantitatively
investigate the influence of concrete factors on the dynamics of the system. In order to do
so, in this thesis methods of agent-based modeling, multi-agent simulation, statistical and
analytical methods, and evolutionary game theory are employed.
Schelling, in conclusion of his segregation model, wrote:
“The model that is described is limited in the phenomena it can handle because it makes no allowance for speculative behavior, for time lags in behavior,
for organized action, or for misperception. It also involves a single area rather
than many areas simultaneously affected. But it can be built on to accommodate
some of those enrichments.”
Thomas C. Schelling (Schelling, 1978)
The models and analyses of this thesis follow this modeling philosophy. We aim at a better
understanding of how the existence, structure, and dynamical properties of interactions
influence the decisions of individuals in the system. Every day in life basically consists of
a steady sequence of decisions to be taken by individuals. Such decisions can be conscious
and well elaborated, or they can be taken unconscious, following primitive instincts. A
decision is the result of a selection among alternatives. If many individuals have to take a
common decision, e.g. a board of directors’ appointing a chief executive officer, one speaks
of collective decision processes and social choice theory (Arrow, 1970; Suzumura, 1983).
Here, but also for most other decisions, it holds that an individual decision does not only
1.2. The Weidlich-Haag approach
5
affect the individual itself, but has effects on other individuals either. Therefore, in social
systems, economics, politics, etc. it is important to understand individual decision making
(i.e. on the microscopic level) and its effects on a system comprising many individuals
(the macroscopic level).
In sociology, economics, psychology, and political sciences, the individuals under consideration are human beings. However, social interaction is not restricted to human beings, but
relevant for biological species in general. Especially the so called “social insects”, like ants
and bees, but also swarming birds and fishes are good examples to illustrate the importance and consequences of interaction between individuals (Bonabeau et al., 1999; Choe
and Crespi, 1997). Therefore, models of social interaction are not necessarily tied to human investigations, but are frequently applied in biological and ecological sciences (Chave,
2001; Reichenbach et al., 2007).
In their book on “Growing artificial societies”, Epstein and Axtell develop an extensive
computer model with the aim to simulate the evolution of a whole society from bottom
up (Epstein and Axtell, 1996). Especially the way they relate their work to the social
sciences has many parallels to the approach taken in this thesis.
“We apply agent-based computer modeling techniques to the study of human social phenomena, including trade, migration, group formation, combat,
interaction with an environment, transmission of culture, propagation of disease, and population dynamics. Our broad aim is to begin the development
of a computational approach that permits the study of these diverse spheres
of human activity from an evolutionary perspective as a single social science,
a transdiscipline subsuming such fields as economics and demography. [...]
We believe that the methodology developed here can help to overcome [some
problems of social sciences]. This approach departs dramatically from the traditional disciplines, first in the way specific spheres of social behavior–such as
combat, trade, and cultural transmission–are treated, and second in the way
those spheres are combined.”
Joshua M. Epstein and Robert Axtell (Epstein and Axtell, 1996)
1.2
The Weidlich-Haag approach
In 1983, a fundamental basis of “sociodynamical” models has been formalized by Weidlich
and Haag (1983). The state of a system is described by a small set of global variables, the
6
Chapter 1. Modeling and analyzing social interaction
“socio-configuration”, and its change modeled by “equations of motion”. An important
example, also with respect to this thesis, is the master equation formulation. Master
equations are used to mathematically describe stochastic processes where the probability
of finding a system in a particular state cannot explicitly be written, but an equation for
its time evolution is given. It can be derived from the discrete “Markov” assumption,
according to which the state of a system at time t does only depend on the state at time
t − 1. Let P (s, t) denote the probability of finding a particular system in state s at time
t. If the Markov assumption holds,
P (s, t) =
X
P (s, t|s′ , t − △t) P (s′ , t − △t),
s′
where the sum runs over all possible states of the system s′ and △t is the size of the
discrete timestep. P (a|b) denotes the conditional probability of a when b is given. The
time-continuous Master equation describes the time evolution of P (s, t) for infinitesimal
small timesteps (the limit of △t → 0) and has the form
#
"
X
d P (s, t)
w(s|s′ , t) P (s′ , t) − w(s′ |s, t) P (s, t) , s 6= s′ ,
≡ Ṗ (s, t) =
dt
s′
where w(s|s′ , t) = lim△t→0 P (s, t + △t|s′ , t)/△t is the conditional transition rate from
s′ → s. This general master equation, which is a specific differential equation, can be
formulated in words: the probability of finding a system in state s at time t increases
with any “inflow” into state s from any other state s′ and decreases with any “outflow”
from state s to any other state s′ at time t. The inflow is determined by all conditional
transition rates w(s|s′ , t), multiplied with the respective occurrence probabilities P (s′ , t),
and the outflow analogously.
This formalism (where defining the transition rates specifies the respective sociodynamical
model), allows for many analytical insights into these dynamical systems. However, its
applicability is mostly restricted to systems with very few dynamical variables (like e.g.
in the “mean-field” limit, where the state of a system is described by an aggregated
variable) and a limited number of possible state realizations. When this is not given,
other modeling techniques can be employed. Agent-based modeling (ABM) and multiagent simulations (MAS) are such techniques where microscopic agents (individuals) and
their interactions among each other are defined and the macroscopic evolution of a system
is observed through stochastic simulations (e.g. “Monte Carlo” methods). Such models
often allow for a better understanding of the microscopic processes leading to macroscopic
consequences, but have the disadvantage that analytical results are, if at all possible, hard
1.3. Dynamical fixed points and their stability
7
to obtain. It results a critical tradeoff between the modeling principle “keeping it simple
and stupid (KISS)” and realism in the models.
“It can scarcely be denied that the supreme goal of all theory is to make the
irreducible basic elements as simple and as few as possible without having to
surrender the adequate representation of a single datum of experience.”
Albert Einstein (Einstein, 1934)
... or, as Einstein is often cited with respect to the above statement:
“Things should be as simple as possible, but not simpler.”
1.3
Dynamical fixed points and their stability
In the following Chapters, we will use differential equations in order to describe the respective model under consideration. These differential equations constitute a “dynamical
system” and it is in most cases helpful to find equilibrium states of the system – so-called
“fixed points” of the dynamics. As this is of interest in several parts of this thesis, we
shall explain the methodology in general:
Consider a dynamical system f (x, y, t) = x(t) + y(t), where all equations of motion are
given. They represent time-derivatives of the respective, time dependent state functions
x and y. This means, the time derivative of the system is given as f˙(x, y, t) = ẋ(t) +
ẏ(t), where the dot above the variables indicates time derivatives. Consider further that,
e.g. due to non-linear dependencies, the dynamical equations are not integrable, i.e. the
primitive functions x(t) and y(t) cannot be obtained. In this cases, one can at least learn
about the fixed points of the dynamics, i.e. states of the system where f˙(x, y, t) = 0 (also
called “stationary solutions”, there is no dynamics in such a point). A state of the system
f at time t is characterized by the values x and y at time t and, obviously, in a fixed point
these values have to satisfy either ẋ(t) = ẏ(t) = 0 or ẋ(t) = −ẏ(t).
However, having found a fixed point in x0 and y0 , a small perturbation in x or y might drive
the system out of the fixed point where the dynamical equations are non-zero. Therefore,
in addition to finding the fixed points, one is interested in their stability. Let us use
the notion of a “dynamical flow” of the system f (x, y, t). In our case, this flow is twodimensional because both x and y can change. In a fixed point f0 (x0 , y0 , t), there is no
dynamical flow. In any other state, there is a dynamical flow along either one or both
8
Chapter 1. Modeling and analyzing social interaction
dimensions. To evaluate stability of a fixed point, we consider a small perturbation of a
fixed point which means we find the system in a “vicinal” state f (x0 + ǫ1 , y0 + ǫ2 ), where
the absolutes of ǫ1,2 are very small. If the dynamical flows in this vicinity drive the system
into the considered fixed point, we call it stable and attractive (or asymptotically stable).
If the dynamical flows keep the system constantly within this vicinity, e.g. by oscillations,
the fixed point would be marginally stable, but not attractive. Whenever a dynamical
system exhibits exactly one stable fixed point, we can expect to find it in this state (or in
its vicinity) after a sufficient time of evolution. If a system exhibits more than one stable
fixed point, its dynamical outcome depends on the initial state. Note that a system with
a finite state space must possess at least one stable fixed point, as we will see later on.
If the dynamical flows in the complete vicinity of a fixed point drive the system further
away from it, it is called unstable or repellent. Finally, a fixed point can also be attractive
in a part of its vicinity, and repellent in another part. This is called a saddle point and
means that, dependent on the initial condition, a system might evolve into this state
or depart from it. Let us be more precise by considering a specific example of “onedimensional flows” and, afterwards, its general translation to “multi-dimensional flows”,
i.e. dynamical systems with one or more dynamical variables, respectively.
1.3.1
A one-dimensional flow
Assume that the state of a system is fully described by one variable x. The dynamical
equation shall be
dx (t)
≡ ẋ = x3 − x.
dt
(1.1)
The fixed points are found by setting ẋ = 0. There are three values of x fulfilling this
condition, namely x1 = −1, x2 = 0, and x3 = 1. These are the fixed points of the
dynamical system constituted by Eq. (1.1). To find out the stability, we linearize the
dynamics around the fixed points, i.e. we derive ẋ after the only state variable x and
obtain ẋ′ = 3 x2 − 1. We evaluate this derivative at each fixed point and the sign of this
result indicates stability or not. The argument can easily be understood by visualizing it,
as done in Fig. 1.3.1: Consider a positive slope (derivative) of ẋ at a fixed point, e.g. the
one of x1 = −1. This simply means ẋ < 0 left of the fixed point and ẋ > 0 right of the
fixed point. Hence, the dynamics at a value of x < x1 further decreases x, while a slightly
higher value lets x increase further, i.e. fixed point x1 is unstable. The same holds for x3
where ẋ′ = 2. The reversed scenario holds when the derivative is negative, like the one at
fixed point x2 . It is therefore a stable and attractive fixed point and its attractor region
1.3. Dynamical fixed points and their stability
9
is bounded by the two unstable fixed points. A derivative equal to zero (not present at
a fixed point in our example) would indicate a saddle point. The dynamics of Eq. (1.1)
can be analyzed as follows: The final state of the system depends on the initial value of x.
If initially −1 < x < 1, the stable fixed point will be reached after some time of system
evolution and the system will remain in this state. For another initial condition, x diverges
(x → ±∞).
6
4
dx(t) / dt
2
0
−2
−4
−6
−2
−1
0
x
1
2
Figure 1.1: The dynamical function of Eq. (1.1) dependent on the state x. Arrows indicate
the direction of the dynamics in the vicinity of the three fixed points.
Let us briefly mention two extensions of the just described dynamics. The first one regards
a statement from above regarding the necessity of a stable fixed point in a limited state
space. If we assume a restricted state space for our example, e.g −2 < x < 2, the system
cannot diverge, but only run into this boundary states. Obviously, these are then also
stable fixed points – a small (valid) perturbation causes a dynamics driving the system
back to the boundary. Note that the applicability of the linearization is restricted in such
a case, because of the discontinuity of the dynamical function.
As a second extension, let us assume another parameter in the dynamics, e.g. assume
ẋ = x3 − x + a, which vertically shifts the function in Fig. 1.1. Of course, the values of
fixed points and their stability now depend on a. Furthermore, the general characteristics
of the dynamical system might change at some values of the parameter: whereas we found
a stable fixed point if a = 0, for a = 2 the system invariably diverges. Such qualitative,
parameter dependent changes in the dynamics are called bifurcations (see e.g. Redner
(2001); Strogatz (1994) for a classification of bifurcations).
10
1.3.2
Chapter 1. Modeling and analyzing social interaction
Multi-dimensional flows
Here, we assume n dynamical variables x1 , ..., xn and their evolution functions
ẋ1 = f1 (x1 , ..., xn )
(1.2)
ẋ2 = f2 (x1 , ..., xn )
..
.
(1.3)
ẋn = fn (x1 , ..., xn ).
(1.5)
(1.4)
At first, we have to compute the fixed points, i.e. all sets of (x1 , ..., xn ), such that ẋi = 0
holds for all i ∈ {1, ..., n}. The stability analysis follows the same reasoning as above, but
is a little more involved. Like before, we want to find out what happens if a system in
a fixed point is perturbed. In the example above, we considered only one dimension of
dynamical flows (the x-axis) and the perturbation could only happen along this dimension
(left or right of the fixed point). Here, we have n dimensions for the dynamics to move
along and for each dimension two possible directions of movement (in- or decreasing the
respective variable). Hence, to find out about stability, we have to take into account all
possible partial derivatives. We do this by building the Jacobian matrix

∂ ẋ1 /∂x1 ∂ ẋ1 /∂x2 ... ∂ ẋ1 /∂xn


 ∂ ẋ2 /∂x1 ∂ ẋ2 /∂x2 ... ∂ ẋ2 /∂xn 
.

J=
..

.


∂ ẋn /∂x1 ∂ ẋn /∂x2 ... ∂ ẋn /∂xn

(1.6)
The n eigenvectors of this matrix stand for one dimension in the state space each and the
associated eigenvalues indicate the direction of the dynamical flow along this dimension.
Note that every possible line of movement can be derived as combination of all eigenvectors,
a so called superposition of the eigenvectors. Therefore, the n eigenvalues at a fixed point
inform us about the dynamical flows in the n-dimensional vicinity of that fixed point just
as the only derivative in our one-dimensional example from above: if all eigenvalues of
the Jacobian at a fixed point are negative, the fixed point is stable and attractive (from
every direction, the dynamical flows drive the system into this fixed point). If there are
positive and negative eigenvalues, we have a saddle point. Exclusively positive eigenvalues
indicate an unstable fixed point (a repellor). It can also happen that more than one fixed
point form a line (or a plane) of fixed points, which would get apparent by at least one
eigenvalue that equals zero.
1.3. Dynamical fixed points and their stability
11
The same classification holds for complex eigenvalues. Non-zero imaginary parts indicate
oscillations in the dynamics, but the sign of the real parts indicate whether the fixed
point attracts or repels (i.e. whether the oscillations shrink or grow) in the same way as
described above. 1 If the eigenvalues are complex and the real parts equal zero, this gives
rise to oscillations that are constant in time, i.e. the fixed point neither attracts nor repels
the dynamics.
1
Note that complex eigenvalues have equal real parts and for the imaginary parts only the signs are
opposite.
Chapter 2
Decisions through social influence
2.1
Decision theory
This thesis investigates the influence of social interaction on aggregated outcomes of the
dynamics within a group of individuals. Although we basically focus on quite primitive
behavioral rules and our investigations are not necessarily focused on the interaction between human beings, let us first mention some major concepts of (human) decision theory.
First, there is a general distinction between normative and descriptive decision theory; the
former seeks to understand how individuals should decide if they were rational (a term
that we shall explain more in detail below), and the latter observes and aims at explaining
how individuals actually do decide in certain circumstances.
Rationality involves the existence of utility- or preference functions, in order to normalize
the consequences (or success) of decisions. For example, the personal value of 100 Swiss
Francs surely differs for people of significantly different incomes. With this normalization
of absolute values, one generally defines “more is always better” and individuals act with
the aim of maximizing their utility. These preferences and the associated decisions have
to fulfill some propositions in order to be considered rational:
• Transitivity
The preferences of individuals are consistent, i.e. if an individual prefers alternative
A over B and alternative B over C, it also prefers alternative A over C. This
proposition follows directly from the definition of utility.
• Invariance
Decisions of individuals are independent of the representation of decision situations.
If, for example, alternative A is presented prior to alternative B or the other way
12
2.1. Decision theory
13
round must not influence the decision.
• Independence of irrelevant alternatives
If from the alternatives (A, B) an individual decides for A, the same individual must
not choose B from the alternatives (A, B, C).
The term rationality goes even beyond – mostly it also involves the assumption of complete
and perfect knowledge, i.e. all information relevant for the decision are available to the
individual. In other cases, it is sufficient to assume exact knowledge about the probabilities
of several alternative circumstances in order to allow for a rational decision. Especially in
classical economic theory, the models assume a perfectly rational, economic actor, which is
often called “homo oeconomicus” (see also Anderson (1991); Eisenführ and Weber (2002);
Simon (1955); Smith (1991); Tversky and Kahneman (1986)).
It is easy to criticize the rationality assumptions: in most situations, people do not have
all relevant information available, their capabilities to analyze all available information are
restricted, future consequences of decisions are subject to uncertainty, and, most importantly, we humans do not even decide consistently or rationally given the information we
have. One example: we arrive at the theater and realize that we forgot our ticket, that
we paid 100 Swiss Francs for, at home. Asked if we would buy a new one, most of us
would answer “no”. If we did not already have a ticket, but while queueing to buy one we
realize that we lost a 100 Swiss Francs note on the way to the theater, most of us would
still buy the ticket (Eisenführ and Weber, 2002). The economic decision is the same in
both cases, but somehow it is not the same to us. Besides this rather illustrative example,
much more empirical evidence has been collected to conclude that rationality is not well
suited to explain human decision behavior (Fehr and Gächter, 2002; Henrich et al., 2001;
Kagel and Roth, 1995).
The concept of bounded rationality, introduced by Herbert Simon (Gigerenzer and Selten,
2001; Simon, 1956, 1972), takes into account the limitations to information availability
and information processing. It has to be distinguished from irrationality in the sense of
making decisions that are harmful to the individual. It rather means that individuals
do not optimize before making the decision. For example, if many decision alternatives
are available, optimizing requires comparative effort and takes time. For this reason,
individuals stop the optimizing process at some point and make a (hopefully) “nearoptimal” decision. Simon named this “satisficing”, which refers to satisfaction with a
sufficient, but not optimal decision.
Another approach within the framework of bounded rationality is to consider decision
making processes as following heuristics (Gigerenzer and Selten, 2001; Gigerenzer and
14
Chapter 2. Decisions through social influence
Todd, 1999; Tversky and Kahneman, 1974). Instead of trying to find an optimal solution
for a certain decision situation, the whole situation is simplified to some important features.
This simplified situation can be analyzed much faster and, depending on the heuristics, a
more or less successful solution be found.
Decisions under social influence clearly belong into the bounded rational category of models. For the game theoretic part of this thesis, we will generally assume that individuals
strive for maximizing their utility or that an evolutionary force favors individuals with a
higher utility values. Furthermore, the decision situations under consideration are mostly
very simple, there is only a small number of decision alternatives with clearly specified
consequences. In some parts of the argumentation, perfect information is also assumed.
There, rationality will play a role, but only for a first assessment of the respective situation
(serving as a kind of benchmark). The most important findings and conclusion do not rely
on the rationality assumption.
This short overview over some concepts of decision theory is aimed at introducing part
of the terminology used throughout this work. However, as mentioned before, there are
fundamental differences in the research approach taken in the works cited above and the
context of this thesis. The main points of differentiation are:
• While decision theory seeks to understand and explain human behavior, we apply
different behavioral rules in our interaction models in order to investigate the dynamical properties and the effects of social interaction.
• In our decision situations, we exclusively consider interdependent decisions, i.e. the
decision of one individual has consequences for and is influenced by other individuals.
• Depending on the specific decision rule under consideration, the individuals in the
models of this thesis are not necessarily humans.
2.2
Models of social influence
As discussed in the previous Section, there are many reasons to assume bounded rational
rather than rational decisions in the real world. If comparing all decision alternatives is
costly, non-rational decision making might be more efficient. One natural form of nonrational decision making is imitation, i.e. trusting in: “what is good for you cannot be
bad for me”.
“The following general statement, offered as a summary of many prior studies as they apply here, explains why imitation occurs: Nonrational decision
2.2. Models of social influence
15
making occurs because it economizes on decision-making resources.”
Mark Pingle (Pingle, 1995)
Putting this argument further: in the absence of information, imitation might be the
best decision to be taken. Consider an individual in a burning building with dense wads
of smoke. The individual has no idea which direction to choose in order to leave the
building. Following some other individuals seems then the best option because there is a
chance that these individuals do have some information on the next exit. However, with
slightly changed assumptions, the opposite can be true. If there are too many individuals
heading in the same direction and there is evidence of more than one exit, it might be
the best to choose another direction in order to avoid clogging at the commonly chosen
exit. What these options have in common is that there is contentwise symmetry between
the alternatives – the individual does not know where an exit is and, with respect to this,
every direction is as good as another. Therefore, the decisions are entirely dependent on
the decisions of others and independent of the alternatives as such.
We refer to this kind of decision making as decisions by social influence.
“Social influence is defined as change in an individual’s thoughts, feeling,
attitudes, or behaviors that results from interaction with another individual or
a group.”
The Blackwell Encyclopedia of Social Psychology (Rashotte, 2007)
This is related to “observational-” or “social learning” in social psychology (Bandura,
1977; Miller and Dollard, 1998). There, the consequence of observation is not necessarily
imitation, but can also be the opposite if the observed consequences of a behavior are
negative. In general, such learning theories require the anticipation of consequences as
motivation to learn, which is not necessary for the social influence mechanisms assumed
in this work.
However, we can classify the effects of social influence into (i) imitation/herding/persuasion,
(ii) repulsion/avoidance/contrarian behavior, and (iii) compromising/averaging, as done
by Helbing (1995). However, the main focus here and in the literature is on positive social
influence, i.e. imitational processes. Positive social influence and imitation is closely connected to majority rules or majority voting. If interaction occurs between more than two
individuals, imitation is not straight-forward because an individual might observe different
16
Chapter 2. Decisions through social influence
behaviors. In that case, following the behavior of the majority represents positive social
influence.
Imitational behavior can originate from different sources: different from compliance or
obedience, which assumes a form of authority or power to coerce individuals to behave
like others, conformity refers to the mere desire to not behave differently from others.
Experiments with human subjects illustrated how strong, and sometimes dangerous, such
tendencies can grow: Asch (1951) showed that people sometimes deny the truth if all
other people in a group state something that objectively is wrong (conformity). Regarding compliance and obedience, the experiments of Milgram (1974) and Zimbardo (2008)
are remarkable. In Milgram’s setup, people were told to punish other people with electric
shocks whenever they did mistakes in remembering words. As the frequency of mistakes
increased, many test persons were ready to shock other people with life-threatening voltage, just because they were told to and although the other individual was just another
test person.1 In the experiment of Zimbardo, 24 test persons were divided into prisoners
and guards of a simulated jail. They were told to play these roles for two weeks with all
associated consequences regarding power of the guards and restrictions for the prisoners.
This order was taken so serious by the test persons, that the experiment had to be stopped
after a few days. Initially homogeneous test persons internalized their roles up to a level
that the experimenter could not take the responsibility any longer.2
Across the social sciences, social influence has been subject to investigations from many
different fields and with different specifications. In a relatively recent review, Mason et al.
(2007) conclude four different, fundamental dimensions on which social influence models
differ:
(a) the pattern of connectivity and influence assumed among individuals,
(b) the treatment of attitudes or behavioral responses as continuous versus discrete,
(c) the presence or absence of individual differences in private information, and
(d) the assumptions made about the assimilative versus contrastive directional efforts of
social influence.
The focus in (a) lies on network effects within social systems and accounts for the fact,
that not only direct influence between two individuals matters, but also influence via indi1
Of course, the individual to be threatened was not a test person, but an actor who did not receive a
real electric shock. However, this was not realized by the real test persons.
2
This experiment was also taken as basis of the novel “Black box” by Mario Giordano and the German
movie “Das Experiment” by Oliver Hirschbiegel.
2.2. Models of social influence
17
rect connections between individuals occurs (see also Wasserman (1994)). This has latest
become obvious since Milgram (1967) formalized the phrase of a small-world problem,
following which every person on Earth is connected with any other person via “six degrees
of separation” on average (Guare, 1990). Dimension (b) differentiates between continuous and discrete decisions or behaviors upon which social influence works. An example
of continuous behaviors stems from the theory of reasoned action (Fishbein and Ajzen,
1975), where the relative strength of an individual intention to perform a certain behavior (“behavioral intention”) is taken into account. On the other hand, discrete decisions
play an important role e.g. for investigations on innovation diffusion, like in the model
of Granovetter (1978). In dimension (c), Mason et al. (2007) distinguish between models
where social influence is the only source of influence, like in (Granovetter, 1978), or where
there are also other sources of input for the individuals, as for example in the theory of
reasoned action (Fishbein and Ajzen, 1975) or in models of swarm intelligence (Kennedy
et al., 2001). Finally, dimension (d) reflects the already mentioned topic of imitational
processes versus contrarian behavior. While assimilative or imitational processes are extensively discussed within this thesis, the seceder model of Dittrich et al. (2000) serves as
example to show how group formation can take place as consequence of local tendencies to
be different. For a much more comprehensive overview of theories and models regarding
social influence, we point the interested reader to the review of Mason et al. (2007).
In order to understand the intrinsic properties of systems with strong social influence,
a number of models have been developed that take the spread of opinions as sample
application. Early approaches in the social sciences showed that the existence of positive
social influence (i.e. imitation behavior) tends to establish homogeneity (i.e. consensus)
among individuals (Abelson, 1964; French, 1956). The “voter model”, rigorously defined
by Liggett (1995), confirms these results. As we will see later on, it models dynamics
between two discrete opinions (e.g. pro or contra), where individuals tend to imitate
the opinion of an interaction partner. The same holds for models with a continuous
opinion space (Deffuant et al., 2000; Hegselmann and Krause, 2002), where an individual’s
opinion is one position on a continuous spectrum – mostly the unit interval between zero
and one – and an individual moves its opinion in the direction of an interaction partner
(compromising). There, however, it was also shown that a selection of interaction partners
(“bounded confidence”) can lead to stable diversity of opinions, even when considering
positive social influence. Bounded confidence means that an individual does not interact
with any other individual, but only with those whose opinion is not too distinct from
their own one, i.e. opinions within a certain “confidence interval”. Such results are in line
with dynamical models of social impact theory, as introduced by Latané (1981), Latané
and Wolf (1981) and Latané (1996). There, the fundamental argument is a spatial one:
18
Chapter 2. Decisions through social influence
interaction between near-by individuals is more likely than long-range interactions. This
was shown to create clearly separated clusters of homogeneous behavior and to diminish
but stabilize the share of individuals applying the minority behavior.
While modeling details for the voter model are introduced below, let us only mention some
of the most important models (a more elaborated review of the field and its models can
be found in Castellano et al. (2007)): the Ising model of statistical mechanics originally
constitutes a spin system explaining order-disorder phase transitions in ferromagnets (Binney et al., 1993). In the model of Weidlich (1971), it found first application in a social
context, but many other models apply the analogy between energy minimization of spins
and human tendencies to avoid social friction (see also the Galam model (Galam et al.,
1982)). One of them is the voter model (Liggett, 1995), which will extensively be discussed in Chapter 5. Opposing to e.g. the voter model, the Sznaijd model (Behera and
Schweitzer, 2003; Stauffer et al., 2000) assumes an inside-out influence where (two) likeminded individuals are able to spread their opinion to other individuals. If the state space
of opinions cannot sufficiently be described by (two) discrete opinions, there foremost are
two models with a continuous opinion space: the Deffuant model (Deffuant et al., 2000)
and the Hegselmann-Krause model (Hegselmann and Krause, 2002). They differ in that
the first one considers bilateral interactions whereas in the latter one an individual interacts with all other individuals within its confidence interval. For a recent survey of
continuous opinion models with bounded confidence, see Lorenz (2007).
2.3
The voter model
Consider a number of individuals that can have one out two possible opinions about a
certain topic, e.g. whether to vote ’yes’ or ’no’ in the next referendum. The topic might
be complex and individuals are not very decided on it. Hence, they repeatedly discuss
about it with some other individuals that they are connected to. Of course, individuals are
rather connected to others that they have some similarities with and so we assume that
the arguments of others sound convincing enough to adopt the same opinion. Based on
this, admittedly rather simplistic scenario, one is interested in the dynamical properties
of such a system.
In the original voter model (Dornic et al., 2001; Holley and Liggett, 1975; Liggett, 1995),
N “voters” are positioned at the sites of a regular, d−dimensional lattice. This means
that every voter exclusively interacts with a fixed set of nearest neighbors, e.g. the voters
at the four directly neighboring sites in a regular, 2-dimensional lattice. Each voter i has
one of two possible opinions at a time, σi (t) = ±1, which might change every time the
2.3. The voter model
19
voter interacts with one of its neighbors. A timestep in the evolution of the system of N
voters consists of N update events. In an update event, one voter is picked at random and
adopts the opinion of one randomly selected neighbor. Thus, the probability that voter i
adopts opinion σ, that will be denoted by WiV (σ), is equal to the frequency of opinion σ
in its neighborhood. Hence,

X
1
σ
WiV (σ, t) ≡ WiV (σ|σi , t) = 1 +
σj (t) ,
2
k

(2.1)
j∈{i}
where k is the number of neighbors each voter has, and {i} is the set of its neighbors.3 The
second term within the brackets yields the average opinion in the neighborhood, which
ranges from −1 to +1. Addition by 1 and division by 2 transfers this average opinion
into a probability between 0 and 1. As stated above, if 3 out of 4 neighbors have opinion
+1, the probability to randomly choose a +1 neighbor is 0.75, which is identical to the
probability to adopt this opinion. For this reason, the model is also called “linear voter
model”, because the adoption probabilities depend linearly on their frequencies (compare
with (Cox and Durrett, 1991; Schweitzer and Behera, 2009)).
Here, it gets obvious why the voters in this model are said to have “zero confidence”: their
own opinion σi does not enter Eq. (2.1), i.e. it does not have any influence on a voter’s
decision. Note that this equation can also be applied to networks of different topology, as
we will do later on.
For the voter model, the 2-dimensional regular lattice was found to be a critical dimension,
i.e. depending on the dimension of the system, its dynamical properties change. Only in
systems with dimension 2 or below, coarsening takes place. This means that starting
from a completely disordered state, clusters of likeminded individuals form and grow over
time. Due to the probabilistic nature of the transition probabilities (2.1), these clusters
are neither spatially static nor separated from each other by sharp boarders. Instead, they
exhibit strong fluctuations and move around in the system (compare with Fig. 2.1). This
“critical coarsening without surface tension” is an important characteristics of the voter
model – it defines its own dynamical universality class (Dornic et al., 2001). However,
modifying the dynamics, e.g. implementing a third, intermediate opinion (Castellò et al.,
2006), or including memory effects (Dall’Asta and Castellano, 2007), might change this
fluctuation driven dynamics into curvature driven dynamics (with surface tension).
3
Eq. (2.1) represents the node update rule of the voter model. There is another specification – the link
update – in which instead of one voter, two connected voters are chosen randomly for opinion update (a
link with its two end nodes). If this link is active, i.e. it connects two voters with different opinions, one
of the voters (chosen randomly) switches its opinion, thereby making the link “inactive”.
20
Chapter 2. Decisions through social influence
Finite voter model systems reach consensus (one of the two absorbing states where all
voters have either opinion +1 or -1) in a finite time (compare with Fig. 2.1). The time to
reach consensus, Tκ , depends on the size of the system and the topology of the neighborhood network. For regular lattices with dimension d = 1, Tκ ∝ N 2 , for d = 2, Tκ ∝ ln N ,
and for d > 2, Tκ ∝ N . The coarsening of systems with dimension d ≤ 2 drives the system
into consensus even in the thermodynamic limit, i.e. when the system size goes to infinity.
In systems of any dimension larger than 2, no coarsening takes place and consensus might
not be reached in the thermodynamic limit (Slanina and Lavicka, 2003).
Another important feature of the voter model is the conservation of magnetization (Castellano et al., 2003; Frachebourg and Krapivsky, 1996; Suchecki et al., 2005a), as compared
to other prototypical models, such as the Ising Model with Kawasaki dynamics (Gunton
et al., 1983). Let A(t) (B(t)) be the global frequency of voters with opinion +1 (−1)
at time t. The average opinion of the system (also called “magnetization” analogous to
studies of spin systems in physics) can be computed as
M (t) = A(t) − B(t).
(2.2)
In the mean-field limit, that we will study in more detail later on, we assume that the
change of the opinion of an individual voter only depends on the average frequencies of
the different opinions in the whole system. Therefore, we replace the local frequencies in
Eq. (2.1) by global ones, which leads to the adoption probabilities W V (+1| − 1, t) = A(t)
and W V (−1| + 1, t) = B(t). For the macroscopic dynamics, we can compute the change
in the global frequency of one opinion as
A(t + 1) − A(t) = W V (+1| − 1, t) B(t) − W V (−1| + 1, t) A(t)
= A(t) B(t) − B(t) A(t)
≡ 0,
(2.3)
i.e. the frequency of each opinion is conserved for every state of the system. Departing
from mean-field calculations, the magnetization conservation generally holds. In a single
simulation run, a voter model system changes its magnetization due to fluctuations and
eventually reaches consensus. However, ruling out these fluctuations, we find that the
exit probability of an opinion (or fixation probability, i.e. the probability that an initial
frequency of an opinion leads to consensus in this opinion) is exactly as high as the
initial frequency of that opinion. The considerations of this conservation law will play an
important role in the investigations of the following Sections.
2.3. The voter model
21
(a) t = 0
(b) t = 1200
(c) t = 3600
(d) t = 4600
(e) t = 6900
(f) t = 7100
Figure 2.1: Exemplary time evolution of the voter model on a 2-dimensional, regular
lattice, where every voter interacts with its 4 nearest neighbors. The system consists of
N = 10, 000 voters and the evolution time t is measured in generations, i.e. 10,000 single
updates correspond to 1 generation.
The order parameter, most often used in the voter model, is that of the average interface
density ρ. It gives the relative number of links in the system that connect two voters with
different opinions and can be written as
ρ(t) =
1XX
1 − σi (t) σj (t) .
4 i
j∈{i}
(2.4)
22
Chapter 2. Decisions through social influence
Because of its simple structure, the voter model allows for many analytical calculations
(Liggett, 1995; Redner, 2001) and, therefore, serves a comprehensive understanding of the
dynamics involved. Application areas of the voter model include coarsening phenomena
(Dornic et al., 2001), spin-glasses (Fontes et al., 2001; Liggett, 1995), species competition (Chave, 2001; Ravasz et al., 2004), and opinion dynamics (Holyst et al., 2001).
Based on the voter model, investigations were conducted to study interesting emergent
phenomena and relevant applications. Such works comprise the possibility of minority
opinion spreading (Galam, 2002; Tessone et al., 2004), dominance in predator-prey systems (Ravasz et al., 2004), forest growth with tree species competition (Chave, 2001),
and the role of bilingualism in the context of language competition (Castellò et al., 2006).
The question of consensus times and their scaling for different system characteristics was
particularly addressed in several studies (Castellano et al., 2003; Liggett, 1995; Redner,
2001; Sood and Redner, 2005; Suchecki et al., 2005b).
Chapter 3
Decision making in (evolutionary)
game theory
At difference with models of opinion dynamics, in the following we will consider a wider
range of decision behavior of individuals rather than imitation, avoidance, and compromise.
In this Chapter, we will first introduce some fundamental concepts of classical game theory;
its origin, different specifications, aims, and terminology. While formal, mathematical
definitions of games and concepts can be found in a number of introductory books (Aubin,
1979; Fudenberg and Tirole, 1991; Güth, 1999; Kreps, 1990; Luce and Raiffa, 1957; Osborne
and Rubinstein, 1994), we will focus on contentwise explanations that are aimed at allowing
a better understanding of the discussion below. After mentioning important questions
addressed by non-cooperative game theory and providing examples of social and economic
systems, two fields basing on classical game theory are introduced: experimental game
theory and evolutionary game theory. Within this thesis, we mainly deal with evolutionary
game theory. Concepts and analyses of classical game theory are introduced and refered
to at several places in the present work, but always with the aim of analyzing the outcome
of evolutionary dynamics applying particular game theoretic models.
3.1
3.1.1
Classical game theory
Definitions and terminology
Decision making in game theory generally follows a strategy, where the space of strategies
covers all possible decision behaviors – from simple (e.g. random or always deciding the
same) to complex (e.g. history dependent and/or anticipating), from stupid to smart (e.g.
23
24
Chapter 3. Decision making in (evolutionary) game theory
involving learning mechanisms). By common definition:
“A strategy defines a set of moves or actions a player will follow in a given
game. A strategy must be complete, defining an action in every contingency,
including those that may not be attainable in equilibrium. For example, a
strategy for the game of checkers would define a player’s move at every possible
position attainable during a game. Such moves may be random, in the case of
mixed strategies.”
Dictionary of game theory (Shor, 2005)
First we have to note that therefore also decisions through social influence are strategic,
because the (re-)action of an individual is at every time defined by the observed action
of interaction partners. However, the available strategies in models of social influence are
restricted to those being a function of the interaction partner’s decisions. Strategies in
the general sense of game theory can be any kind of decision plan, including the repeated
application of one and the same action or random decisions. The conceptual difference
of decisions by social influence compared to general strategic decision making is that
individuals cannot assign a specific decision to any decision point, but the decision will
always be dependent on the action applied by the interaction partner. In this Chapter we
consider strategic decision making in the general sense, i.e. individuals can choose from
the whole strategy space. To summarize:
• Decisions by pure social influence are taken frequency dependent and utility independent and the decision alternatives themselves are symmetric, i.e. apart from
the comparison with interaction partners, there is no difference for the individuals
between one or the other decision alternative.
• Strategic decisions in general can be taken frequency independent, i.e. the decisions
are not necessarily symmetric for the individuals. This allows for utility maximization in the strategy selection process of the individuals, dependent on the available
information and analytic capacity.
• Decision making by social influence is a subset of general strategies, where information on utility might not be available, individuals are not able to process these
information, or utility considerations do not play a role because of the absence of
competition.
3.1. Classical game theory
25
In 1944, John von Neumann and Oskar Morgenstern first applied their own mathematical
framework to interdependent decision making in economic environments (Von Neumann
and Morgenstern, 1944). This work is nowadays widely reckoned as the hour of birth of
game theory.
A game, in the sense of game theory, is defined by the number of players (N ≥ 2),
the complete set of decision alternatives to be chosen among by each player (at least 2
alternatives per player), and the utility (referred to as payoff ) for every player in every
possible outcome of the game. In order to analyze the games, we have to assume that
every player is rational and that rationality is common knowledge, i.e. any player i knows
that all players are rational; all players know that i knows that all players are rational;
i knows that all players know that i knows that all players are rational; ... and so on.
Furthermore, all individuals have perfect information about the rules and payoff structure
of the game. A complete decision plan, that assigns a decision to every possible state of
the play, is called the strategy of a player.
Let us give some simple examples in order to clearly distinguish between games in colloquial
language and the situations covered by game theory:
• Roulette in the casino does not fall into the definition of games. There are usually
many players on the table, but their decisions do not influence each other – everybody
plays for himself against the casino and the odds of a player are independent of the
decisions of other players. Given the rules of roulette, the casino is the interaction
partner, but it has no strategic choice. It is chance that decides how much is won
by whom.
• The case is not so simple when considering an ordinary lottery. The situation is
almost the same as for roulette, but now players can try to maximize their payoff
in case they win. Their chances to win is still independent of other decisions, but
the amount that can be won depends on how many other players won in the same
category (e.g. the category of 5 correct numbers – all players with 5 correct numbers share the jackpot of this category). Therefore, it makes sense to consider the
decisions of others when deciding for a set of numbers.
• A straight-forward example of games fulfilling the definitions of game theory is the
famous board game chess, where 2 players have a strategic interaction and the success
of a player depends on its own decisions and the decisions of the opponent.
• Whereas chess is a game in which exclusively the playing skills of players decide
the winner, this is not prerequisite to this extent. The game “rock-paper-scissors”
26
Chapter 3. Decision making in (evolutionary) game theory
is another example of a strategic game. Two players have to decide for one of the
symbols rock, paper, or scissors. Rock wins over scissors, scissors wins over paper,
and paper wins over rock. Despite of mental abilities, there are no skills that help
winning, but still it is one of the strategic interactions in the sense of game theory.
Figure 3.1: Presentation of the rock-paper-scissors game in extensive form (top) and in
normal form (bottom). In the upper picture, big, black circles represent “decision nodes”
where the respective player has to decide for one of its alternatives. The dashed ellipse
indicates an “information set”: a player does not know which of the decision nodes within
one information set is its current one. Therefore, the example game is a simultaneous one.
The small, black circles at the ends of branches indicate the possible outcomes of the game
and the payoffs are given in brackets: the first one for player 1 and the second one for
player two. The lower picture contains exactly the same information about the game in
form of a payoff bimatrix.
3.1. Classical game theory
27
There is a substantial difference between cooperative- and non-cooperative game theory. In
cooperative game theory (see e.g. Luce and Raiffa (1957); Osborne and Rubinstein (1994)),
players are allowed to have preplay discussions in order to coordinate their strategies. To
be precise, players have the possibility to conclude binding contracts. This means that
some players arrange their strategies with each other and the agreements are enforceable
by the contract. In cooperative game theory, one is mainly interested in which coalitions
will form, can be maintained, and are joined by which players of the game. The present
work exclusively deals with non-cooperative game theory, where every player decides on
its own without discussing strategies with others. In fact, it is sufficient to assume that
there is no possibility for the players to conclude binding contracts.
In order to explain the terminology used in the respective parts of the thesis, let us briefly
mention some other differentiations:
• Simultaneous vs. sequential games
Straight-forwardly, if all players decide at the same time (with identical information),
we call it a simultaneous game, otherwise a sequential one.
• Games in normal form vs. extensive form
Every game can be presented in a decision tree, i.e. in extensive form. A game in
normal form is presented in a bimatrix of payoffs and can therefore only consider 2
players and simultaneous games (see Fig. 3.1 for a comparison of one and the same
game in both forms).
• Symmetrical vs. asymmetrical games
If the initial situation (chances and risks, i.e. the payoff structure) is identical for
all players, it is a symmetrical game, otherwise an asymmetrical one.
• Constant- (e.g. zero-) sum games vs. variable-sum games
In a zero-sum game (e.g. the game in Fig. 3.1), any positive payoff gain of a player is
“paid” by other players with negative payoff. There is no gain or loss in the “system”
of all players. Accordingly, any constant system gain or loss can be distributed by a
constant-sum game. In a variable-sum game, the gain or loss of the whole of players
can vary between the different outcomes of the game.
The rock-paper-scissors game illustrated in Fig. 3.1 is a non-cooperative, simultaneous,
symmetrical, zero-sum game with two players. Therefore, it can be presented both in
extensive and in normal form. Since the game is symmetrical, without loss of information
it would be sufficient to specify the payoffs of player 1 only (as we will do in such cases from
28
Chapter 3. Decision making in (evolutionary) game theory
now on). The games mainly discussed in the present work are similar to that example,
but mostly variable-sum games.
Let us turn our attention to some analysis concepts that are particularly important for
the investigations reported below:
• Nash equilibrium
Certainly, one of the most important solution concepts of game theory is the equilibrium state defined by John F. Nash Jr., which was awarded the Nobel prize in
economics in 1994 (Nash, 1950, 1951). It states that a solution1 is an equilibrium
outcome of a game if none of the players has an incentive to unilaterally deviate from
this solution. A strategy of a player is called best answer if this strategy maximizes
the player’s payoff for a particular strategy combination of all other players. Hence,
a Nash equilibrium consists of best answers only. If all best answers of an Nash equilibrium are unique, i.e. any player would definitely get less payoff when unilaterally
deviating, it is called a strict Nash equilibrium. The Nash theorem states that every
game possesses at least one such equilibrium in mixed strategies2 . In pure strategies,
a game might possess no Nash equilibrium (like in the rock-paper-scissors example),
a unique one, or multiple Nash equilibria (see below for examples).
• Dominance
A strategy that is best answer to all strategy combinations by the others is called
a payoff dominant strategy. If this strategy is always the unique best answer, it
is called strictly payoff dominant. A strategy is called risk dominant (or maximin
strategy) if its minimal possible payoff (worst case for the player) is higher than the
minimal possible payoff of any other strategy.
• Pareto efficiency
This is the most common efficiency criterion for solutions of a game. A solution A
is called Pareto efficient if there is no other solution B that (i) yields a higher payoff
for at least 1 player and (ii) yields at least equal payoffs for all players. If such a
solution B exists, one would speak of a Pareto dominance of B over A and A is Pareto
deficient. In the example of Fig. 3.1, every solution is Pareto efficient. Note that
there is no dependency between this efficiency criterion and a Nash equilibrium: a
Nash equilibrium can be Pareto efficient or not and a Pareto efficient solution might
be an equilibrium or not.
1
A final outcome of a game, i.e. a set of strategies of all players, is also called a solution.
The strategies considered so far are also called pure strategies. A mixed strategy consists of probabilities for choosing any possible pure strategy. It is a randomized strategy where an agent does not decide
for one of its pure strategies, but assigns a probability to all of them and lets chance decide.
2
3.1. Classical game theory
29
• System optimality
We introduce the notion of system-optimal solutions in the sense of Schelling’s collective total (Schelling, 1978). In a system optimum, the sum of all players’ payoffs
are the highest possible. In economic terms, one can also regard this the welfare
maximum. We will use this notion as an efficiency criterion that is hardly used in
the literature so far, but will be crucial to the investigations presented in Chapter 7.
Note that a system optimal solution invariantly is Pareto efficient. It is a corollary
of the Pareto efficiency definition that a Pareto dominating solution B has a higher
system payoff than A. Therefore, system optimality of A precludes the existence of
a Pareto dominating solution B, which makes A Pareto efficient.
3.1.2
Coordination- vs. cooperation problems
Particularly in extensive form, game theory can abstract complex strategic situations
presenting enormous challenges not only for the considered players, but also for researchers
trying to analyze the game. A number of such examples can be found in Güth (1999), for
instance games dealing with the problem of insurances and labor markets.
However, it is maybe the simplest class of games that often is the base of investigations,
namely games with only 2 players and 2 choice alternatives each, where both players
have identical roles (symmetrical 2×2 games). In Section 3.1.3, we will provide a detailed
classification of such games when assuming that payoffs in different situations are never
exactly equal. For the moment, let us have a look at some simple examples to illustrate
strategic conflicts of coordination and cooperation.
Since the games are symmetrical, we will specify them by providing a 2 by 2 payoff matrix
which contains the payoff for the player deciding for a row-strategy, given the columnstrategy of the “opponent” player. In a game with the payoffs
A B
A
B
!
1
0
0
1
%
,
(3.1)
players face a coordination problem. The game possesses 2 equivalent Nash-equilibria
and players have to synchronize their strategies in order to achieve one of the 2 desired
outcomes (A, A) or (B, B). Another coordination problem arises in the game
30
Chapter 3. Decision making in (evolutionary) game theory
A B
A
B
!
0
1
1
0
%
,
(3.2)
where players now have to asynchronously coordinate their strategies to reach a mutually
profitable outcome, e.g. (A, B). Therefore, we refer to it as asynchronous coordination
problem 3 . In both games (3.1) and (3.2), players have no conflict of interest, but only a
problem due to the lack of communication possibilities. This is not anymore entirely true
for the game
A B
A
B
!
2
1
0
1
%
.
(3.3)
This game also has 2 Nash equilibria, but only (A, A) is both Pareto- and system-optimal.
This means that there is no coordination problem since it is clear to both players that
strategy A leads to an outcome that yields the highest possible payoff for both players.
However, by slightly relaxing the strict rationality assumption, it is conceivable that one
player might want to win over the other, which only is possible by choosing B. Moreover, it
is sufficient to assume one player that expects the other to think in this way. Then, strategy
B would stand for risk-averseness because it ensures at least a payoff of 1 (“maximin
strategy”). We therefore refer to this scenario as a basic cooperation problem, where
cooperative behavior would mean to abandon such competition considerations.
As a fourth of such basic problems, let us have a look at the game
A B
A
B
!
0
1
0
0
%
.
(3.4)
To some extent, the problems of coordination and cooperation are both present in this
scenario. The only way for the players to obtain any payoff is to coordinate either in
solution (A, B) or (B, A). But there is only 1 payoff point at stake, i.e. in addition to the
coordination problem, one player would have to be cooperatively enough to abandon his
chance of receiving the payoff. Here, a clear conflict of interest arises. We refer to this as
a partial cooperation problem, which will play a crucial role in Chapter 7.
3
In the literature, it is mostly called “anti-coordination” problem. This might be misleading because
players effectively have to coordinate their strategies.
3.1. Classical game theory
31
The so called Prisoner’s Dilemma game is surely the most prominent example of game
theory. It is a paradigmatic cooperation problem and its history is closely tied to the
founding of game theory and Nash’s work on equilibria in games. It started with experiments carried out by Merril Flood and Melvin Dresher in 1950 at the RAND corporation.
Although they used different payoff values and their game was not even symmetric, the
essence of their experiment is captured by the matrix
C
D
!
C
D
R
T
S
P
%
,
(3.5)
with the following specifications (although this matrix is usually used for the Prisoner’s
Dilemma, we will later on use the same variables for any symmetrical 2×2 game, but
partially refrain from the following semantics): if both players choose to cooperate (C),
each gets the “reward”, e.g. R = 3 Dollar. If both do not cooperate, called defection D,
they get a “punishment” of P = 1 Dollar each. However, if only one player cooperates, he
leaves with nothing (“sucker”, S = 0) while the defector receives the “temptation” payoff
T = 5 cent. Hence, the matrix reads
C
D
!
C
D
3
5
0
1
%
.
This game possesses the dominant strategy defection, leading to the strict Nash equilibrium
(D, D) – a real dilemma because if the players would cooperate, both would receive 3 cents
instead of 1 cent. In general, the Prisoner’s Dilemma game is defined by T > R > P > S
and some references additionally require 2R > S + T .
When preparing a lecture on the very new idea of game theory, the supervisor of Nash,
Albert W. Tucker, illustrated the game by the following story and named it Prisoner’s
Dilemma: two suspects are caught by the police and interrogated separately. Both have
the option to confess the crime or not to do so . The police has no clear evidence against
them and offers that if only one suspect confesses, he will get free while the other is
sentenced the maximum imprisonment. However, if both confess, both are imprisoned
slightly below maximum. In contrast, their crime could not be proven if both would not
confess. Then, the only consequence would be some more time of remand4 .
4
The information regarding the history of the Prisoner’s Dilemma game are collected from various
sources, e.g. Holt and Roth (2004); Rasmusen (2001).
32
3.1.3
Chapter 3. Decision making in (evolutionary) game theory
Symmetrical 2×2 games – the framework
In the previous Section, we have seen that symmetrical 2×2 games can serve as simple
models to investigate strategic problems of coordination and cooperation. They will be the
model of choice in the respective part of the present thesis. Therefore, a full classification
of such games is presented in the following.
Let us base on the parameters in Eq. (3.5), i.e. we have two strategies (C, D) and the different combinations lead to one of four payoff values (R, S, T, P ). Since we will elaborate
on different symmetrical 2×2 games, it is important to define which strategy is regarded
“cooperative” and which “defective”. For the sake of convenient readability, we will use
the following simplification throughout this Section:
Let us only consider an encounter of different strategies leading to the payoffs S and T
(“partial cooperation”). In such a situation, the strategy which yields the lower payoff is
regarded the cooperative strategy (C), and the other one the defective strategy (D).
This definition has no contentwise consequences, it is one possible way of commonly naming
the different outcomes in all the games. Where it makes sense to speak of cooperative
behavior, this simplification yields the correct naming of strategies. For the other games,
it is maybe the most useful way to name the strategies likewise. In all the cases, cooperation
means to risk losing against the other player and defecting means holding the chance to
end up with a higher payoff than the other. Therefore, this approach puts more weight on
a player’s relative payoff with respect to the co-player’s payoff. With these specifications
in mind, we use the same variables commonly used for the Prisoner’s Dilemma game for
any symmetrical 2×2 game.
Let us briefly comment on our definition of T > S: it differs from the one taken in
other works, where mostly R > P is fixed. However, there is no restriction regarding the
considered games. As we will see below, we address the same (ordinally distinct) games as
the ones discussed elsewhere (Hauert, 2002; Helbing et al., 2005b; Rapoport, 1967; Stark
et al., 2008a; Tanimoto and Sagara, 2007). The reason for our differing approach will
become clear in Section 7: it leads to an integrative representation of all symmetrical
2×2 games where the connection between neighboring games can be explained (Fig. 7.2).
Having made the points of Section 7, we will come back to this issue in the respective
Discussion (Section 7.6).
A game is defined by the number of players, their set of strategies, the sequence of decisions to be taken by the players, and the payoffs for all players and for every possible
strategy-combination. The class of games described here is one of the simplest: two players decide between two alternative strategies. The strategic situation is identical for both
3.1. Classical game theory
33
players, (e.g. the one for the row-player in the matrix of Eq. (3.5)). After they decided
simultaneously, they receive a payoff depending on their own strategy and the strategy of
the other player.
As stated above, we find it convenient for the reader if we use the same variables commonly
used for the Prisoner’s Dilemma game also for an arbitrary symmetrical 2×2 game. Hence,
we impose the label “cooperative” to that strategy that yields the lower payoff in an
encounter of different strategies, i.e. T > S always holds.
We assume that the absolute payoff values are not decisive for the strategic situation, but
only the ranking of them (we will qualify this point later on). Since we defined T > S,
which eliminates equivalent rankings, one can discern 12 ordinally distinct games. Fig. 3.2
conveniently visualizes the phase space of symmetrical 2×2 games in a coordinate system
and includes exemplary payoff matrices.
Each of the 12 rectangular or triangular parcels of the coordinate system (separated by
full lines) host one ordinal payoff ranking. Within this classification scheme, we find the
prominent Prisoner’s Dilemma game, which we already have introduced in Section 3.1.2.
This game is characterized by a strict Nash equilibrium that is not Pareto efficient and,
depending on the payoff values, system-optimal or not. Let us briefly introduce the single
games (compare with the payoff matrices of Fig. 3.2):
The game of Chicken is also often called “Hawk-Dove” game or “Snowdrift” game. One
of the stories is about 2 teenagers and their test of courage. They arrange that they
frontally approach each other by cars and that they turn to the right if they want to avoid
a collision. Of course, the one that chooses this avoidance option is perceived as “chicken”
and the other one will get the respect of the teenagers. However, a frontal collision of the
cars could be fatal and is, therefore, the worst outcome. If both turn, they survive and
none of them loses his face against the other. Similar in spirit, the Hawk-Dove story is
about moderate and escalating fight-strategies in animals and the Snowdrift game about
shoveling snow from a car or waiting for the other person to do this. The game possesses
2 Nash equilibria in pure strategies, namely the partial cooperation solutions. Depending
on the payoff values, these equilibria are system-optimal or not.
Battle-of-the sexes (not in the asymmetric version of Dawkins (1989)) sketches a situation
where a couple arranged to meet in the evening, but they forgot on what they agreed
on. One of them preferred to visit a sports event and the other had an opera visit as
preference. However, both would rather abandon their own preference than spending the
evening alone. If both abandon their preference, they yield the worst case of visiting the
unpreferred event alone. The best is a coordination where one of them chooses her/his
preference and the other other one abandons her/his preference. This game also possesses
34
Chapter 3. Decision making in (evolutionary) game theory
Figure 3.2: Classification of symmetrical 2×2 games according to payoff rankings. For
each area a respective payoff matrix (in the form of matrix (3.5), with T > S) is given.
Nash equilibria are marked by bold payoff numbers. Parcels separated by solid lines denote
different rankings of the payoff values. Two-dimensionality is achieved by fixing T > S
and classifying ordinal differences only.
2 pure, system-optimal Nash equilibria in the partial cooperation solutions.
Leader denotes a similar situation, but now the worst case is if both chose their preferred
event – maybe because then they start to doubt their relationship. The Nash equilibria
are the same as in the previous game.
The 4 games exemplified so far represent the four archetypes of Rapoport (1967) (“Martyr”, “Exploiter”, “Hero”, and “Leader”).
In the game Stag Hunt (see e.g. Skyrms (2004)), 2 hunters have to decide whether to hunt
a stag or a rabbit. The stag requires both hunters to be slain – if only one goes for the
stag he will not succeed whereas the other gets the rabbit. If they hunt together, they
3.2. Experimental game theory
35
can either share the stag or have to compete for the rabbit. This game is also interesting
because it possesses 2 pure Nash equilibria in the synchronous solutions (C, C) and (D, D).
Only (C, C) is system-optimal, but (D, D) has the advantage of being risk-dominant.
The game of Route Choice reflects important characteristics of (vehicular- or data-) traffic systems and was named and experimentally investigated in Helbing et al. (2005b)
and Stark et al. (2008a). Independent of these works, the same situation was also included in the experimental setup of Kaplan and Ruffle (2007). A reasonable story could
be that 2 controllers have to send a large truck convoy along the same highway and know
about this fact. They have to decide whether to send the convoy on this highway anyway
or to use another route over different highways. If they both send it on the same path,
the highway will be overloaded and traffic jams lead to costly time delays. However, one
route is much shorter and, therefore, making the detour would most probably take more
time than queuing on the short route. Like the Prisoner’s Dilemma, this game has 1 strict
Nash equilibrium that might be system-optimal or not.
Deadlock is very similar, but now there is an additional advantage of being the first to
arrive. Therefore, R > S because being the last is worse than needing more time. The
Nash equilibrium is the same like in the Route Choice game.
The remaining games Harmony I and II and the game Own Goal are similarly trivial
in the sense that they all have a strict Nash equilibrium that is Pareto efficient, system
optimal, and risk-dominant. Only Harmony I gained some attention in the literature due
to the fact that there the equilibrium lies in cooperation. It is also referred to as “Byproduct mutualism” because the cooperative act does not only benefit the recipient, but
also the donor (see e.g. Bergstrom and Lachmann (2003); Clutton-Brock (2002); Hauert
(2002)).
3.2
Experimental game theory
In the previous Section about classical game theory, we mentioned the necessary assumptions underlying the solution concepts; rationality, common knowledge, and perfect information. Moreover, these assumptions also imply that each individual is capable of storing
and optimally processing all the information available. It is obvious that these assumptions
are quite strong and that, even when restricting the analysis to human beings, individuals are not fully rational, not always all the necessary information is available, and if so,
individuals differ in their abilities to memorize information and in their analytical skills.
Hence, game theory serves us as important theory about strategic decision making, but
its ability to explain individual decision behavior has to be questioned.
36
Chapter 3. Decision making in (evolutionary) game theory
In order to validate theory, experimental game theory (or experimental economics) analyze
real human decision behavior under laboratory conditions. In most of the cases, test
persons are invited into a computer lab, get instructions on how to play a game against
one or more other test persons, and take decisions which lead, depending on the game
and the decisions of the other(s), to payoff scores. In order to simulate relevant decision
situations, where the individuals try to optimize the outcome of their decisions, it is
important to appropriately generate an incentive structure for the test persons, such that
they are motivated to try their best to be successful. Such incentives can potentially have
different forms, in most of the cases test persons obtain a success-dependent, monetary
reward, where payoff scores are converted into a disbursement.
Although such techniques allow to test theoretical predictions under relaxation of theoretical assumptions, each experimental design still contains assumptions and, most importantly, the validity of results of such experiments significantly depends on the quality of
the experimental design. This quality is influenced by many factors: the selection of test
persons, the selection of information to the test persons, the clarity of these information
(did every test person understand the instructions correctly?), the design of the rules has
to be appropriate for the questions the experiment is aimed to answer, and many other
factors. The importance of these design factors cannot be overemphasized and we would
like to point the interested reader to the “Handbook of experimental economics” by Kagel
and Roth (1995), which constitutes a comprehensive survey of results, but also of the
methods to design and conduct laboratory experiments.
While there is a vast array of relevant and interesting insight into human decision behavior
resulting from experimental research, we here want to address only one general finding,
which also serves as motivation for some investigations contained in this thesis: human beings consistently decide cooperatively in experiments. One of the most suited experiments
to test the hypothesis of selfish, payoff maximizing, and rational humans is the ultimatum
game (UG), which has been conducted innumerous times with different specifications. The
basic ultimatum game is rather simple: one of two individuals receives a certain amount
of money, say 10 Swiss Francs, from the experimenter. This individual, the “proposer”,
has to make an offer to the other individual how to share this 10 Francs among the two
individuals. The other individual, “the responder”, then decides about the offer and has
two options: (i) the responder acccepts the offer, then the experimenter splits the 10
Francs and pays the individuals according to the offer; (ii) the responder rejects the offer,
then the experimenter keeps the 10 Francs and both individuals get nothing. The game
theoretical prediction is straight: the proposer will offer the smallest possible, non-zero
amount to the responder and the responder will accept this offer, because its alternative is
to receive nothing. However, the empirical evidence is significantly different. Even if test
3.3. Evolutionary game theory
37
persons do not know each other and will very likely never meet again, the average offer of
the proposer lies somewhere between 40 and 50% of the whole amount, while the rejection
rates even for more than minimum offers is larger than zero. Variants of the experimental
investigation of the UG, all of which confirm the mentioned result at least qualitatively,
comprise tests in different cultural environments by conducting the experiments in 15 different, small-scale societies (Henrich et al., 2001), raising the amount of money at stake
to a higher than usual level (Cameron, 1999), testing asymmetries in the experimental
setup (Kagel et al., 1996), or observing neural responses of the test persons (Sanfey et al.,
2003).
While this thesis does not contain experimental work, part of it is motivated by and
follows up on empirical work on the decision behavior of test persons in route choice situations (Helbing et al., 2005b; Stark et al., 2008a). Particularly with view on the design
of advanced traveller information systems and the optimal routing in data networks, experiments have been designed and conducted where individuals repeatedly had to decide
for one of two alternative routes. One route could be seen as a highway and the other as
rural road. While the highway in principle is faster, it might be congested (because too
many of the test persons decided for this route), in which case the rural road can even
be faster than the congested highway. Omitting the details, the decision situation to the
test persons was of the category “partial cooperation problems”, which was introduced
in Eq. (3.4). Besides the general result, that individuals were able to develop alternating
cooperation strategies in order to exploit the system optimal distribution within the “traffic system” in a fair manner, several treatments have been conducted to investigate the
optimal conditions and the influence of additional information on the decision behavior
of the test persons. Despite the interesting empirical finding that individuals are even
under difficult circumstances willing and able to cooperate, it turned out that this kind
of strategic decision situations contains relevant theoretical implications, which have not
been thoroughly discussed from a game theoretic point of view. Therefore, particularly
Chapter 7 deals with such situations and introduces “partial cooperation dilemmas”.
3.3
Evolutionary game theory
Natural evolution works on populations of reproducing individuals (species). Differences
in the speed of reproduction are explained by differences in reproductive fitness and lead
to augmentation and extinction of species, i.e. selection. Mutations are small variations
between ancestor and offspring in the course of reproduction – mostly rare events of erroneous reproduction. These mutations allow for variation, thereby increasing competition
38
Chapter 3. Decision making in (evolutionary) game theory
in the “struggle for life”, as Charles Darwin has put it (Darwin, 1859). With these 2 ingredients (have a glance at the incipient quotation of Section 3.3.2 for a recent proposal of
a third fundamental process), evolution incorporates a fascinating optimization procedure
that allowed to create efficient, complex life forms, including human civilization.
How complex Nature and its laws might be, evolution can be described and analyzed by
rather simple mathematical tools, and game theory is one of them. Instead of individuals
which are free to choose between different strategies, evolutionary game theory assumes
the individuals to have one strategy “hardcoded” in their genetic material – the single
individual is restricted to the decisions following this strategy. Reproductive fitness is
the equivalent of payoff in classical game theory: individuals interact with each other
according to their hardcoded strategies, and the gain or loss in payoff contributes to their
own reproductive fitness, but thereby of course also to the reproductive fitness of their
genotype. Compared to classical game theory, two extensions are most important: (i)
starting from any combination of applied strategies, we analyze the evolutionary dynamics
towards an equilibrium state or between equilibrium states, and (ii) the main solution
concept (equivalent to that of the Nash equilibrium) is the concept of evolutionarily stable
strategies (ESS), which will be explained below (Maynard Smith, 1982; Maynard Smith
and Price, 1973).
One of the the longstanding questions in evolutionary biology, which over the time also
entered other scientific domains like economics, social sciences, physics, etc. and will be
of particular importance also in this thesis, is that of the emergence and maintenance of
cooperative behavior in competitive environments. The methods of evolutionary game
theory (see e.g. Friedman (1991); Hofbauer and Sigmund (1998); Nowak and May (1992);
Szabó and Fáth (2007); Taylor and Jonker (1978)) have since Maynard Smith and Price
(1973) been applied to innumerous investigations regarding the evolution of cooperation in
biology (Maynard Smith and Szathmáry, 1995; Nowak, 2006b), social sciences (Fehr and
Fischbacher, 2003; Henrich et al., 2003), and economics (Friedman, 1991; Gintis, 2005;
Kreps, 1990). The advantage of evolutionary considerations is that the strong assumptions
of rationality and common knowledge, key in classical game theory, are not necessary any
more.
“Evolutionary game theory diverges from classical game theory only when
it comes to the analysis of beliefs. In the classical theory, players have beliefs about one another which are grounded in, or at least consistent with, ideal
rationality and common knowledge; their strategy choices are rational in the
sense that they maximize subjectively expected utility, when subjective beliefs
are themselves rational. Evolutionary game theory does not require the ratio-
3.3. Evolutionary game theory
39
nality of beliefs.”
Robert Sugden (Sugden, 2001)
So far, we have introduced that evolution results from genetic inheritance in combination
with selection and mutation processes. The transmission process of successful behaviors
here is exclusively vertical, i.e. an evolutionarily positive effect for a successful behavior
can only occur in future generations, by a more successful replication of genes applying
the respective bahavior. Particularly with view on human beings, evolution might also
work through horizontal transmission, i.e. by individuals changing their (not genetically
encoded) behaviors. This might be induced by learning mechanisms, “trial and error”
behavior, strategic reasoning, or similar mechanisms. Another mechanism of cultural
evolution is the propagation of experience over generations, which is similar to genetic
evolution in that the transmission of behaviors is vertical, though not hereditary. At
difference with genetic evolution, the time scale of significant changes in cultural evolution
can be much faster than in genetic evolution. On one hand, horizontal transmission
does not require time costly reproduction, but can occur instantanously. On the other
hand, mutations in biology are rather seldom events of small changes, whereas in cultural
evolution radical shifts in the behavior of individuals are not unlikely.
In Chapter 6.3, we report on results from a modeling framework that arguments in terms
of cultural evolution. There, differences in the transmission of behaviors result from two
sources: (i) from different, co-evolving abilities of individuals to influence other individuals
and (ii) from payoff differences between the decision alternatives. In the other parts of the
thesis that deal with evolutionary game theory, the focus is rather on genetic eveloution and
the terminolgy accordingingly chosen. However, due to the generality of the investigated
models and mechanisms, most of the results and conclusions hold for any kind of fitness
or payoff dependent evolution.
3.3.1
Evolutionary dynamics
In this Section, we introduce one of the main dynamical systems to study evolutionary
game theory. On one hand, this serves as a good example of the general way of modeling,
on the other hand we will apply this kind of dynamics at several places in the thesis.
We consider the general symmetrical 2×2 game of Eq. (3.5). The whole population of individuals consists of species C and D, where individuals of each species do unconditionally
cooperate or defect, respectively. Assuming an infinitely large population and random
40
Chapter 3. Decision making in (evolutionary) game theory
interactions between individuals (a well-mixed population), one can write the expected
payoff, that is assumed to be equivalent to reproductive fitness, of a species as
πC = x R + (1 − x) S
= x (R − S) + S
πD = x (T − P ) + P,
(3.6)
where x denotes the global frequency of C individuals and 1 − x the global frequency of
D individuals. A mathematical expression for the fitness-dependent evolution of species
in a well-mixed, infinite population is the replicator equation (Hofbauer and Sigmund,
1998; Nowak, 2006a; Nowak and Sigmund, 2004), also called game dynamical equation.
Neglecting mutations, the dynamics is straight-forward: the population share of a species
depends on its relative fitness in the population. If species a has a higher fitness than
average, its share in the population fa will grow proportionally to that difference. On the
other hand, it will shrink in case it is less fit than average. This means, in general,
d fa (t)
≡ f˙a (t) = fa (t) (πa (t) − π̄(t)),
dt
(3.7)
where π̄ denotes the average fitness in the population. Eq. (3.7) is applied to all species in
the population and, therefore, replicator dynamics can involve arbitrarily many species. In
our two-species system, the state of the population is sufficiently described by x, because
the share of the D species is simply 1 − x. Hence, the whole replicator dynamics in this
case reads
ẋ = x (1 − x) (πC − πD )
= x (1 − x) (x (R + P − S − T ) + S − P ).
(3.8)
In order to find the fixed points of this dynamics, we set ẋ = 0 and find the three solutions
(i) x = 0
(ii) x = 1
(iii) x =
P −S
.
R+P −S−T
3.3. Evolutionary game theory
41
To find out which of them will be the evolutionary outcome, we need to know about the
stability of these fixed points. In a state close to a fixed point, will the dynamics drive
the system into this fixed point or away from it? Only in the first case, we can expect the
system to end up in this state. Applying the considerations of Section 1.3.1, we derive
d ẋ
≡ ẋ′ = −3 x2 (R + P − S − T ) + 2 x (R + 2P − 2S − T ) + S − P
dx
(3.9)
and conclude stability if ẋ′ < 0. Depending on the ranking of the payoff values (R, S, T, P ),
our dynamics can lead to the following scenarios (assuming 4 distinct values of R, S, T, P ):5
• Dominance of C: 1 unique stable fixed point in x = 1.
This occurs when R > T and S > P . In this case, species C dominates species D.
• Dominance of D: 1 unique stable fixed point in x = 0.
This occurs when T > R and P > S (like in the Prisoner’s Dilemma). In this case,
species D dominates species C.
• Bistability: 2 stable fixed points in x = 0 and x = 1 and 1 unstable fixed point in
x∗ = (P − S)/(R + P − S − T ).
This is the case if R > T and P > S and corresponds to the coordination problems.
• Coexistence: 1 stable fixed point at x∗ and 2 unstable fixed points at x = 0 and
x = 1.
The reversed bistability scenario, where T > R and S > P . This corresponds to our
asynchronous coordination problems.
We can also relate this discussion to the concept of evolutionarily stable strategies (ESS,
(Maynard Smith, 1982; Maynard Smith and Price, 1973)). A strategy (a species, a genotype) is called evolutionarily stable if a population, consisting almost only of individuals
applying this strategy, cannot be invaded by a few individuals with another strategy. In
our case of only two possible strategies in the system (C and D), this implies the following:
For C to be an ESS against D, the first of Maynard Smith’s conditions is that the payoff
of the solution (C, C) must be higher or equal to the solution (D, C), i.e. a D strategist
must not gain a higher payoff than an C strategist when playing against C. If equality
holds, an additional condition must be fulfilled: the payoff of (C, D) must be higher than
the payoff of (D, D), i.e. a C strategist must be able to exploit D strategists in this case.
If the first condition is a strict inequality or both conditions are fulfilled, strategy C is ESS
5
see also Hauert (2002); Nowak (2006b)
42
Chapter 3. Decision making in (evolutionary) game theory
against strategy D. Let us connect this with the above considerations regarding stable
fixed points and strict inequality of payoff values: strategy C is ESS if R > T . Strategy D
is ESS if P > S. If only one strategy is ESS, we find dominance of this strategy. If both
are ESS, we find bistability. Finally, the absence of an ESS leads to coexistence with a
proportion x∗ of individuals of species C and a proportion 1 − x∗ of individuals of species
D.
Fig. 3.3 uses color encoding to illustrate the evolutionary outcomes of Eq. (3.8) for all
possible payoff rankings: red areas indicate the stability of cooperation, blue areas the
stability of defection, light blue, green, yellow, and orange colors indicate an interior
stable fixed point. Note that in this Figure, we only consider stable fixed points under
the assumption of equal initial conditions, i.e. x(t0 ) = 0.5. Therefore, in the bistability
region (R > T ∩ P > S), we show the stable fixed point whose attractor region includes
the initial condition. Whereas here we only show the existence of this different dynamical
regimes, in Chapter 7 we will go into details of the symmetrical 2×2 games included in
Fig 3.3.
Let us again briefly comment on the illustration of games in Fig. 3.3: we use the classification scheme presented in Fig. 3.2, which differs from the commonly used one, e.g.
in Hauert (2002) and Helbing et al. (2005b). There, the payoff values R and P are kept
constant (with R > P ) and S, T are varied on the axes of the coordinate system. Of
course, the contained payoff rankings (games) are the same, but the spatial arrangement
is different. However, the region where all four dynamical regimes are neighboring each
other (around the coordinate R = T = 1, P = S = 0, investigations often are restricted
to this interesting region, e.g. in Santos et al. (2006) and Roca et al. (2008)) is similarly
existent in both representations. The reason why we use a different representation in this
thesis is that it better supports and illustrates the argumentation of Chapter 7 and it is
more convenient for the reader if one and the same representation is used throughout the
whole work.
For the analysis of infinite, well mixed populations, the results depicted in Fig. 3.3 will
be the reference scenario for the considerations in Chapter 6. In Fig. 3.4, we depict some
basic results of structured populations, but beyond that point the interested reader to two
comprehensive surveys, that contain these and many more results (Hauert, 2002; Roca
et al., 2008). Since, in such systems, analytical solutions are hard to obtain (if obtainable
at all), Fig. 3.4 shows results of numerical simulations. For comparison, panel (a) shows
the case of a fully connected network, i.e. the finite population analogon to Fig. 3.3. The
other panels show the cases of a random network in which every agent interacts with 8
neighbors (b) and a regular, 2-dimensional lattice with 4 (c) and 8 (d) nearest neighbors,
3.3. Evolutionary game theory
43
S = 0, T = 1
1
2
0.9
1.5
0.8
0.7
1
P
0.6
0.5
0.5
0.4
0
0.3
0.2
−0.5
0.1
−1
−1
−0.5
0
0.5
1
1.5
2
0
R
Figure 3.3: Analytically computed equilibrium fraction of cooperators according to replicator dynamics between cooperators and defectors in an infinite, well-mixed population.
In case of bistability (R > T ∩ P > S), the stable fixed point with the bigger attractor
region is displayed. Every R, P -coordinate constitutes one specific payoff matrix (a game)
with fixed values of S = 0, T = 1.
respectively. Compared to Fig. 3.3, the region around the point P = S, R = T is amplified,
because there the important effects happen.
What can been seen is that structure matters in the evolutionary dynamics and that
spatial structure, i.e. when the nearest neighbors of my nearest neighbors are second
nearest neighbors to me, increases the influence of cooperators in parts of the Prisoner’s
Dilemma, but acts detrimental to cooperation in parts of the Chicken game (Hauert, 2002;
Roca et al., 2008).
Let us finally remark on the algorithmic details applied in Fig. 3.4. The dynamics in such
simulations has two steps: (i) individuals in the system collect payoffs from interactions
with connected individuals (“neighbors”). This can be differently realized, e.g. such that
each individual interacts with one randomly chosen neighbor or such that each individual interacts with all of its neighbors. Then, (ii) individuals revise their applied decision
44
Chapter 3. Decision making in (evolutionary) game theory
0
0
−0.2
−0.2
−0.4
−0.4
−0.6
−0.6
−0.8
−0.8
−1
−1
−1.2
−1.2
−1.4
−1.4
−1.6
−1.6
−1.8
−1.8
−2
−2
1
1.2
1.4
1.6
1.8
2
2.2
2.4
2.6
2.8
3
1
1.2
(a) all-to-all
1.4
1.6
1.8
2.0
2.2
2.4
2.6
2.8
3
(b) random, k = 8
0
0
−0.2
−0.2
−0.4
−0.4
−0.6
−0.6
−0.8
−0.8
−1
−1
−1.2
−1.2
−1.4
−1.4
−1.6
−1.6
−1.8
−1.8
−2
−2
1
1.2
1.4
1.6
1.8
2
2.2
2.4
(c) 2-d, k = 4
2.6
2.8
3
1
1.2
1.4
1.6
1.8
2
2.2
2.4
2.6
2.8
3
(d) 2-d, k = 8
Figure 3.4: Simulation results: equilibrium fraction of cooperators after the evolution of
cooperators and defectors in a system of 400 agents. Considered are different population
structures: (a) fully connected networks; (b) random networks with 8 interactions per
agent; (c) 2-dimensional, regular lattices with 4 neighbors and (d) 8 neighbors, respectively.
Compared to Fig. 3.3, the most interesting region around P = S = 0, R = T = 1 is
magnified.
according to the observed success of other individuals.6 Also for this process, several realizations are applied in the literature, e.g. the Fermi rule, which will be introduced and
6
This explanation uses cultural evolution as terminology. However, the same algorithms hold for
genetic evolution, where the terminology would include dying individuals and the appearance of offspring
at empty sites of the network.
3.3. Evolutionary game theory
45
applied in Chapter 6.3, the Moran rule (Moran, 1962), where also underperforming decisions can be selected with low probability, or the so-called unconditional imitation process,
which deterministically choses the best performing decision from the neighborhood.7 For
the results of Fig. 3.4, we implemented the discrete equivalent of the replicator equation,
called replicator rule or proportional imitation rule (Hauert, 2002; Helbing, 1992; Roca
et al., 2008; Schlag, 1998). There, an individual imitates another individual only if it has
a higher fitness, and if so, it imitates with a probability proportional to the (positive)
fitness difference between the other individual and itself.
3.3.2
The “conundrum” of cooperation
“Evolutionary biologists are fascinated by cooperation. We think that this
fascination is entirely justified, because cooperation is essential for construction. Whenever evolution ’constructs’ a new level of organization, cooperation
is involved. The very origin of life, the emergence of the first cell, the rise of
multicellular organisms, and the advent of human language are all based on
cooperation. A higher level of organization emerges, whenever the competing
units on the lower level begin to cooperate. Therefore, we propose that cooperation is a third fundamental principle of evolutionary dynamics besides mutation
and selection.”
C. Taylor and M.A. Nowak (Taylor and Nowak, 2007)
Cooperative behavior is defined as an action that induces a fitness-benefit b to another
individual and a certain fitness-cost c to the cooperator himself. The payoff matrix
C
C
D
!
D
%
b − c −c
,
b
0
(3.10)
with b > c > 0, denotes a specific Prisoner’s Dilemma and is often used as the standard
cooperation model in evolutionary biology. It illustrates the puzzle of cooperation: cooperation is dominated by defection and, therefore, selection should lead to the extinction
7
It is important to note that also this “unconditional” imitation rule is different from what we discuss
in this thesis as plain imitation within the context of social influence. Whereas the latter is a behavior of
imitation without fitness or payoff dependence, the imitation rule in evolutionary game theory considers
fitness dependent imitation.
46
Chapter 3. Decision making in (evolutionary) game theory
of cooperative behavior. The question is then: how could cooperation lead to the “major transitions in evolution” (Maynard Smith and Szathmáry, 1995) and why are there
so many examples of cooperation in biology (e.g. single meerkats give alarm calls when
enemies approach, thereby exposing immediate danger to themselves, many animals help
other parents to breed offspring, for details and more examples see e.g. Clutton-Brock
(2002); Doebeli and Hauert (2005) and references therein)? Moreover, consider the evolutionary success of mankind and the fact that this success is based on the high level of
cooperation within human civilization.
Hence, even if not obvious in the first place, there must be mechanisms accounting for
the evolutionary success of cooperation, i.e. cooperation seems to induce an evolutionary
advantage to the individuals applying cooperative behavior. Many scientists from various
fields have addressed these questions in the last decades. Without any claim of completeness, some of the most important concepts shall be mentioned: Hamiltons inclusive fitness
theory has led to the concept of kin selection, which can explain cooperation among closely
related individuals (Hamilton, 1963); Trivers’ reciprocal altruism (Trivers, 1971) and Axelrod’s pioneering work on the iterated Prisoner’s Dilemma (Axelrod, 1984) emphasize
the importance of repeated interactions; Nowak and Sigmund’s concept of indirect reciprocity (Nowak and Sigmund, 1998) bases on reputation mechanisms that help directing
cooperative behavior at other cooperators, similarly to tag systems (Riolo et al., 2001)
and “green beard effects” (Dawkins, 1989; Hamilton, 1964; Traulsen and Nowak, 2007).
Other concepts comprise punishment (Fehr and Gächter, 2002), volunteering (Hauert et al.,
2002), network effects (Nowak and May, 1992; Santos et al., 2006, 2008), etc..
Based on Nowak (2006b), Taylor and Nowak (2007) implement five evolutionary concepts
into the general Prisoner’s Dilemma payoff matrix (3.5), i.e. they quantify the effects of
the different mechanisms by respectively modifying the basic payoff matrix. Using these
matrices, they are able to calculate the conditions for cooperation to prevail under natural selection within the particular evolutionary scenarios. We will go into the details of
these investigations later on in Chapter 7, where we discuss the characteristics of “partial cooperation dilemmas” and provide the evolutionary analysis with respect to these
mechanisms.
Particularly in Chapter 6.4 of this thesis, we will contribute to the unraveling of the
“conundrum” of cooperation by suggesting and anlyzing another mechanism to allow for
the evolution of cooperation. However, both Chapter 6 and Chapter 7 are focused on
investigations regarding the evolution of cooperation and disclose both new results on
the possibilty of cooperation to evolve (Chapter 6), and a novel modeling framework to
analyze cooperation dilemmas (Chapter 7).
Chapter 4
Social influence in evolutionary game
theory
In the previous Chapters, we discussed models of social interaction from two perspectives:
one in which the contentwise differences between the alternative decisions are neglected
and agents do only adapt to the decisions of their interaction partners – mostly they
tended to avoid being different from the others (imitational processes in models of opinion
dynamics). The second perspective was one in which agents experience explicit utility
consequences of their decisions (also dependent on the decisions of the others) and successful decision making has an advantage for the individuals in the evolution of a system
(evolutionary game theory). As consequent third perspective, we will investigate the effect of implementing decisions based on social influence in the competitive environments
of evolutionary game theory.
In fact, opinion dynamics and game theory are similar frameworks. Both consist of interacting individuals that hold one opinion/strategy out of a set of alternatives. In both
cases, social interactions influence the individual decisions and one is mainly interested in
the macroscopic outcome of such systems. Although the nature of interactions is quite
different (simple adaptation processes vs. the possibility of payoff maximization), both
are aimed at modeling aspects of social interaction. It is therefore not surprising that
many methods can be used in both frameworks (Hauert and Szabó, 2005; Szabó and Fáth,
2007). Moreover, microscopic imitation behavior in connection with spontaneous opinion
changes has been shown to lead to the replicator dynamics already more than a decade
ago (Helbing, 1992; Schlag, 1998). However, as regards content, both perspectives on social interaction are brought together rather seldomly, although we would argue that both
forms are present in real social systems.
To illustrate the commonalities and differences further, let us have a look at a specific
47
48
Chapter 4. Social influence in evolutionary game theory
example: the voter model actually is also a model of evolutionary game theory, although
a very boring one. Consider the “competition” (through the replicator equation) between
two species in a game defined by R = S = T = P = 0 (in Eq. 3.5). In this system,
there is no selection pressure as all species have an equal, constant fitness. Hence, every
state is a fixed point, because both species have the same rate of reproduction, which
preserves global frequencies. In finite systems (with the replicator rule), neutral drift
might lead to dominance of one species – which constitutes the voter model dynamics. In
order to build the bridge to more interesting evolutionary games, let us explain the same
matter differently: consider now the “coordination problem” described by Eq. (3.1), i.e.
R = P = 1, S = T = 0. There, individuals of the same species can mutually increase their
fitness, while an interaction between individuals of different species has no fitness effect. In
order to describe the voter model dynamics, instead of the replicator dynamics we have to
assume that the evolutionary dynamics has no fitness dependence, but is only frequency
dependent. One could assume a small poulation in a large system, where the required
resources are not scarce and both species can reproduce without selection pressure. After
some time of evolution, the whole population will reach a size at which resources start
to get scarce, i.e. competition occurs and the dynamics might assume the replicator
dynamics. This competitive system is then characterized by bistability 1 , where only the
two states of dominance of either species are stable fixed points. For finite systems, this
means that the absorbing dominance state is not reached by neutral drift, but much faster
by the directed evolutionary dynamics (see Fig 4.1).
These considerations show on one hand the large common ground by menas of the general
modeling framework, but on the other hand also the relevant difference: in general there is
no competition in opinion dynamics. Or, differently formulated, individuals are not capable of assessing payoff differences. If voters in the voter model try to establish consensus,
they should only imitate successful individuals, because they seem to be in a (local or
global) majority. This would lead to a much faster consensus in the system than their
unconditional imitation of any individual. The distinction between these two behaviors
is essential for this thesis: the imitation of successful decisions is what is referred to as
“imitation rule” in the game theoretic literature. However, it is important to note that
this is not equivalent to the “puristic” concept of imitation by social influence, that we
refer to when we speak of imitation in this work – particularly in Capter 6.4.
In the literature, one approach to combine both research fields was undertaken by Galeotti
and Goyal (2007). There, the population consists of two levels: one of economic actors and
one of social entities. The latter ones are interconnected through a social network where
1
Please refer to the distinction of dynamical scenarios in Section 3.3.1.
49
1
Wealth CG
Wealth VM
Magnetization VM
Magnetization CG
Magnetization (o) / Wealth (x)
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
10
1
10
2
3
10
10
Time (generations)
4
10
Figure 4.1: Simulation results of two systems with N = 100 individuals, where every
individual is connected to all other individuals (fully connected network). The magenta
colored symbols display the results of the voter model dynamics (as explained in the
text, without selection pressure). The blue symbols represent results from the replicator
dynamics in the coordination problem, i.e. in a game with R = P = 1, S = T = 0. Circles
show the development of the absolute magnetization in the system, a measure for the
homogeneity of opinions in the system, where the value zero indicates equal frequencies and
the value one consensus (M = |1−2x|, if x is the relative frequency of one opinion). Crosses
denote the “wealth” in each system in terms of the expected payoff of one individual
when playing the coordination problem game with a random other individual. Values are
averaged over 10,000 simulation runs for each system.
information is exchanged. The economic actors try to efficiently spread information in the
social network. For this purpose, they can apply different strategies leading to a particular
payoff. The focus of this work lies on the influence of different network characteristics on
efficient information strategies (e.g. advertisement). In a nutshell, however, strategic
interaction only takes place between the economic actors and the individuals subject to
social influence are merely part of the environment.
In the context of Chapter 6.4, we are specifically interested in the role of social influence
(as defined in Section 2) on evolutionary dynamics. Regarding our main model, we will
explain the straight-forward implementation of pure imitation behavior in the competition
50
Chapter 4. Social influence in evolutionary game theory
between cooperators and defectors and clearly relate it to similar, yet distinct approaches
in the literature. Most interestingly, the results presented in this Chapter show that
imitation is not able to dominate in a population, but due to its remarkable survival
abilities, it affects the evolutionary dynamics drastically. The novel approach here is to
combine the models of evolutionary game theory and of pure social influence by separately
considering both forms of decision making (decisions through pure social influence and
decisions completely independent of social influence) exposed to the same evolutionary
rules. Within this thesis, there are two fundamental models forming the starting points of
the investigations: the voter model for the social influence part and the replicator dynamics
applied to the evolution of cooperators and defectors in symmetrical 2×2 games for the
evolutionary game theory part. However, these two models are not only fundamental for
this thesis, but are also paradigmatic, interdisciplinary models for the scientific literature.
The approach in Chapter 6.4 is to straight-forwardly combine both models (put voters
together with cooperators and defectors) in order to address the question of how (positive)
social influence affects the evolution of cooperation.
While a more detailed motivation of assuming social influence decisions in evolutionary
dynamics is presented in Chapter 6.4, let us address some reasons for this assumption:
first, decisions based on social influence may reflect asymmetries in either the information
or the capabilities of individuals. If an individual is not able to reasonably or consciously
decide on its own, imitating its environment might be the only reasonable strategy. Second,
imitation can be seen as an unconditional form of reciprocal behavior, in close connection
to the literature on the “tit-for-tat” strategy or other forms of direct reciprocity (Axelrod,
1984; Fehr and Gächter, 1998; Nowak and Sigmund, 1993). Third, imitation might be
motivated by egalitarian considerations (Dawes et al., 2007; Fehr et al., 2008), as at least
for our symmetrical games, the same decisions lead to equal payoffs. Fourth, imitation
might be a consequence of the persuasive power of other individuals (e.g. “teaching”
capabilities (Szolnoki and Szabó, 2007)). Fifth, imitation is relevant because it is an
empirical observation in social, economical, and biological systems (e.g. “herding” or
“swarming” behavior (Bonabeau et al., 1999; Choe and Crespi, 1997; Helbing et al., 2000;
Kirman, 1993; Trotter, 2005)).
Based on this argumentation on the presence and relevance of decisions purely through social influence, i.e. independent of fitness or payoff considerations, we present in Chapter 6
different approaches to combine models of social influence with models of evolutionary
game theory. In a first approach to do so, we only assume imitators and contrarians,
i.e. individuals entirely susceptible to either positive or negative social influence, in all
evolutionary, symmetrical 2×2 games. While the results of these investigations are less
attractive in that they mirror the evolutionary outcome of the reference dynamics be-
51
tween cooperators and defectors (see Fig. 3.3), they might contribute to the explanation
why negative social influence plays are far smaller role in the social interactions of human
beings than herding and imitation does. In the main part of Chapter 6, we then investigate the already described mixture of imitational behavior with that of unconditional
cooperators and defectors in evolutionary games. Apart from this, we further connect the
opinion dynamics part of this thesis with the game theoretic part by testing a similar coevolutionary mechanism as proposed and discussed for the voter model in Chapter 5 in a
model of an evolutionary, spatial Prisoner’s Dilemma game with heterogeneous “teaching
abilities” by Szolnoki and Szabó (2007).
Part II
Models and analyses
This Part of the thesis is devoted to the specific models under consideration and the
results obtained by novel approaches. In Chapter 5, we introduce and investigate models
of social influence. In particular, extensions of the “voter model” are investigated with the
aim to account for individual memory effects in the dynamics of opinion spreading (see
also Stark et al. (2008b) and Stark et al. (2008c)). In Chapter 6 we turn to evolutionary
game theory, but in fact continue the investigations of Chapter 5 with a different focus.
On one hand, we apply a similar, memory dependent interaction mechanism to a model
for studying the evolution of cooperation and analyze the results of this modification (see
also Szolnoki et al. (2009)). On the other hand, we investigate the combination of purely
imitational and more self-confident decision behavior in a novel model of evolutionary
game theory and find that the inclusion of puristic social influence remarkably affects the
evolutionary dynamics in social dilemmas (Stark and Tessone, 2010). Finally, Chapter 7
focuses on the particular models of evolutionary game theory themselves. The framework
of symmetrical 2×2 games is particularly often applied because of its ability to abstract
different strategic decision situations by relatively simple models. Whereas the Prisoner’s
Dilemma and other social dilemmas are well studied, we will introduce another class of
dilemmas that represent a social dilemma only in repeated and evolutionary games (see
also Stark (2010)). Besides presenting explicit results, this Chapter 7 also serves as an
outlook on promising future research.
Chapter 5
Slower is faster: Fostering consensus
formation by heterogeneous inertia
In this Chapter, we will investigate the effects of implementing social inertia into one of
the paradigmatic models of opinion dynamics: the voter model, which has been introduced in Section 2.3. In our extension of the voter model, which we will introduce in
Section 5.1, voters are not “zero-confident” as in the standard voter model, but have a
kind of conviction regarding their own opinion which makes them inertial to change their
opinions. This inertia evolves with persistence time, i.e. the time elapsed since their last
opinion change. It seems natural that such an inertia would slow down the process of
consensus formation in the voter model. In remarkable disagreement with this hypothesis, we will present simulation results of and analytical insight into our main finding, the
“slower-is-faster” effect on reaching consensus through inertial voters. We investigate in
depth under which circumstances the effect can be observed and introduce a theoretical
framework, that allows for the understanding of the phenomenon. For the purpose of a
complete picture, we both present the results of a simpler model, where there are only two
different levels of individual inertia present in the system (a zero and a non-zero inertia
below one, Section 5.3), and a model with an almost continuous spectrum of inertia values (ranging between zero and a maximum value below one, Section 5.4). Although the
observed effect is qualitatively very similar in both model setups, its realization differs in
some aspects and it is worth presenting the whole analysis within this thesis. After concluding our investigations on the voter model, we propose further research on the inertia
mechanism in another dynamical model. Results of preliminary investigations allow the
conjecture of a related, but qualitatively different phenomenon caused by the dynamical
inertia mechanism.
53
54
5.1
Chapter 5. Slower is faster
(Social) Inertia in the voter model
Different from the standard voter model described in Section 2.3, we here consider that
voters additionally are characterized by a parameter νi ∈ [0, 1], an inertia to change their
opinion. The basic probabilities to choose an opinion are those from the voter model, i.e.
equal to the respective frequencies in a voter’s neighborhood. However, the probability
to chose another than the current opinion is multiplied by νi . Compared to the standard
voter model, this reduces opinion changes and increases the probability that a voter keeps
its opinion. Therefore, we have to distinguish between the probability that voter i changes
its opinion
Wi (−σi |σi , νi ) = (1 − νi ) WiV (−σi |σi )
(5.1)
and the complementary probability of sticking to its previous opinion
Wi (σi |σi , νi ) = 1 − Wi (−σi |σi , νi ).
(5.2)
In this setting, νi represents the strength of “confidence” that voter i has regarding its
opinion. WiV denotes the basic probabilities from the standard voter model.
In particular, we consider that the longer a voter has been keeping its current opinion, the
less likely it will change to the other one. Throughout this work, the linear relationship
νi = min(µ τi , νs )
(5.3)
is used. Here, τi denotes the persistence time of agent i, i.e. the number of update steps
since its last opinion change. The individual inertia increases by the growth rate µ until
it saturates at νs < 1. This saturation below 1 avoids trivial frozen states where agents
will never change their opinion again. Note that the results presented in the following do
qualitatively not depend on the exact form of function (5.3), as long as νi monotonically
increases with persistence time and saturates below 1.
For µ = 0, the standard voter model is regained. For values of µ > 0, our extension
differs from the standard voter model in that it considers the current opinion of voters
as an important decisive factor. The voters do not only act based on the frequencies
in their neighborhood, but take their own current opinion into particular account. In a
manner of speaking, we put some confidence on the originally “zero confident” voters.
This general idea can also be compared to the models of continuous opinion dynamics
(see Deffuant et al. (2000); Hegselmann and Krause (2002); Lorenz (2007)): already in the
5.1. (Social) Inertia in the voter model
55
basic continuous models, the current opinion of a deciding individual is of high importance.
More precisely, it is as decisive as the average opinion in the considered neighborhood
because the updated opinion is the average of both. The concept of bounded confidence
emphasizes this importance because individuals do only approach opinions that are not
too far away from their own current one. Therefore, bounded confidence can also be
interpreted as a kind of inertia that tend to let individuals keep their own current opinion.
However, the parameter regulating the confidence interval is generally kept constant in
time whereas, in our model, individuals change their decision behavior dependent on their
history.
While the standard voter model is also called linear voter model–because the probability to adopt an opinion is linearly dependent on the frequency of that opinion in the
neighborhood–the extended version of the voter model considered here is, apart from the
µ = 0 case, a nonlinear voter model (Cox and Durrett, 1991; Schweitzer and Behera,
2009). However, as the inertia values are heterogeneous and co-evolving together with the
individuals’ opinions in the system, it is non of the nonlinear voter models as described
by Cox and Durrett (1991) or Schweitzer and Behera (2009), but constitutes a dynamical,
nonlinear voter model.
As already mentioned, an important implication of this inertia mechanism is that, starting
from a homogeneous population of voters, a heterogeneity of decision behaviors emerges
and evolves in time. Such a heterogeneity, although without evolution and only with
one voter being different from all the others, was also investigated by Kacperski and
Holyst (1997). However, these authors do not assume a linear dependency of adoption
probabilities on their frequencies (like in the standard voter model), but one following an
S-shaped Fermi function (corresponding to Glauber dynamics, that we will compare our
model with in Section 5.6).
The inertia parameter ν, that reduces the probability of state changes, can have different
interpretations in the various fields of application of the voter model: it may characterize
molecules that are less reactive, the permanent alignment of spins in a magnet, etc. In
economics, changes may be discarded due to transition- or “sunk” costs (investments in
the current solution, that will be lost in case of a switch to another solution). In social
applications, there are at least two interpretations for the parameter ν: (i) within the
concept of social inertia, which deals with a habituation of individuals and groups to
continue their behavior regardless of possible advantages of a change, (ii) to reflect a
(subjective or objective) conviction regarding a view or an opinion. Originally, the latter
point served as a motivation for us to study the implications of built-in conviction in a
simple imitation model like the voter model. Will the systems, dependent on the level of
56
Chapter 5. Slower is faster
conviction, still reach a consensus state, or can we observe a (meta-) stable segregation of
opinions? How do the ordering dynamics and the emergent opinion patterns look like?
Our investigations focus on the average time to reach consensus, i.e. the number of
timesteps the system evolves until it reaches an equilibrium state in which all voters
have the same opinion. Taking into account the inertia introduced to the VM, we would
assume that the time to reach consensus shall be increased because of the slowed-down
voter dynamics. Counter-intuitively, we find that increasing inertia in the system can
decrease the time to reach consensus. This result resembles the “faster-is-slower” effect
reported in a different context by Helbing et al. (2000). In their work on panic situations,
they explain why rooms can be evacuated faster if people move slower than a critical value
through the narrow exit door. When individuals try to get out as fast as they can, this
results in clogging effects in the vicinity of the door, which decreases the overall evacuation
speed. Note that although the phrase “slower-is-faster” is appropriate for both findings,
our effect has to be clearly distinguished from the one described by Helbing et al.. In
their generalized force model, an individual increase in the desired velocity would have a
contrary effect on the microscopic level, i.e. all individuals would get slower and thereby
the macroscopic dynamics would be decelerated. In our case, microscopic changes produce
the counter-intuitive effect only on the macroscopic level.
5.2
Preliminary considerations
Before we go into the thorough investigations of the model, let us have a look at three
simplified models in order to obtain a first hypothesis on the effects of the time to reach
consensus in the voter model extended by social inertia. Generally, the expected time to
consensus, Tκ , can be found by an expected value formula: Let Pc (t) be the conditional
probability to reach consensus in the t−th update event if consensus has not been reached,
yet. The expected value is found by summing for every timestep the evolution time t
multiplied by the probability to reach consensus exactly at this time (which is Pc (t),
multiplied by the probability that no consensus was reached before t). Assuming that
there is no consensus in the initial condition (t = 0, otherwise simply Tκ = 0), we can
write
Tκ = Pc (1) +
∞
X
t=2
t Pc (t)
t−1
Y
(1 − Pc (τ )).
(5.4)
τ =1
The first term is simply Pc (1) since it is multiplied by t = 1 and our presumption that
5.2. Preliminary considerations
57
there is no consensus yet. All the other terms contain the factors t and the product of
complementary probabilities 1 − Pc (t) for all times before t (the probability that there is
no consensus at time t − 1).
Let us assume a system comprising N = 3 voters. Initially, the system is not in consensus,
which invariably means that two voters share the opinion and one has the opposite opinion.
According to the voter model dynamics, Pc = 2/9 at every update event t before consensus
is reached (probability of choosing the voter in minority as focal voter (1/3) times the
probability that it changes its opinion (2/3)). In this case, Eq. (5.4) reduces to
Tκ,0 = Pc +
∞
X
t Pc (1 − Pc )t−1 .
(5.5)
t=2
Therefore, Tκ,0 follows a geometric distribution with the expected value
Tκ,0 =
1
,
Pc
(5.6)
and yields the result 9/2 = 4.5 for the voter model dynamics. Let us now consider three
versions of our inertial voter model:
1. Homogeneous, static inertia:
If all voters have an equal, fixed inertia ν• , the time independent consensus probability reduces from Pc = 9/2 to Pc, ν• = (1−ν• ) 2/9. Since Pc is time independent before
consensus is reached, we can use Eq. (5.6) to calculate Tκ, ν• and, for 0 < ν• ≤ 1, we
obtain Tκ, ν• > Tκ,0 . For example, if ν• = 0.1, the expected consensus time would be
5 update events.
2. Dynamic, persistence time dependent inertia:
This is the scenario of our persistence-time dependent inertia mechanism. Here,
′
we need the inertia νi (t) of the voter in minority for the calculation of Pc (t) (the
conditional probability of consensus in this case), which now is time dependent.
′
!
Therefore, we have to use Eq. (5.4). To prove that also in this case Tκ ≥ Tκ,0 , we
have to show that
∞
X
t=2
′
t Pc (t)
t−1
Y
τ =1
′
[1 − Pc (τ )] ≥
∞
X
t Pc (1 − Pc )t−1 ,
t=2
′
using Eqs. (5.4) and (5.5) and the fact that Pc (1) = Pc .
58
Chapter 5. Slower is faster
′
First, we have to note that Pc (t) ≤ Pc for all timesteps t, i.e. the conditional
probability to reach consensus is maximal in a system without inertia and lower
in any system with non-zero inertia. Let us now describe the effect of the first
appearance of inertia at timestep s on the expected consensus time compared to the
Q
′
′
s−1
case with no inertia at s. We have Pc (s) < Pc , but τs−1
,
=1 [1 − Pc (τ )] = (1 − Pc )
i.e. the “exit probability” to reach consensus at time s is reduced by inertia. The
Q
′
′
difference Pc − Pc (s) is distributed over later timesteps via the term t−1
τ =1 [1 − Pc (τ )].
At time t = s + 1,
t
Y
′
′
[1 − Pc (τ )] = (1 − Pc )t−1 (1 − Pc )
τ =1
> (1 − Pc )t .
(5.7)
Assuming that inertia is only present at timestep s and vanishes again afterwards,
Eq. (5.7) also holds for any other timestep larger than s (“stochastic dominance”
Q
′
of the term tτ =1 [1 − Pc (τ )] over the term (1 − Pc )t for any t > s). On the other
hand, the conditional probability to reach consensus reassumes the value of a system
′
without inertia, i.e. Pc (t) = Pc for all t > s. Hence, for the event of non-zero inertia
′
only at one timestep, this proves a positive effect on the expected consensus time Tκ
compared to the case that this event would not have happened (Tκ,0 ).
This is a very special inertia mechanism, where inertia pops up one timestep and
vanishes afterwards. Now, we want to show that the same positive consensus time
effect appears for any inertia mechanism, including our persistence-time dependent
one. To show this, we consider the scenario from above as new reference scenario
and find out the consequence of another event of non-zero inertia after timestep
s. Obviously, the same argumentation holds and, therefore, proves a positive effect
on the consensus time in this case compared to the new reference case, which already implied a positive consensus time effect compared to the non-inertial voter
′
!
model. This induction can be continued arbitrarily and proves that Tκ > Tκ,0 for any
persistence time dependent inertia mechanism at work (q.e.d.).
3. Heterogeneous, static inertia:
Here, we consider that every voter has a static, but individual inertia value ν•, i . As in
′
the previous case, we need the inertia value of the minority voter to calculate Pc (t).
Although the individual inertia values are not time dependent in this scenario, the
expected inertia value of the minority voter changes by the evolution of the system.
In the first update step, due to the random initial condition, it is simply the average
5.3. Results of binary inertia
59
of inertia values in the system. Afterwards, the voter in minority (and the respective
inertia value) is not anymore random, but depends on the evolution of the system.
To write down these expected values is not convenient, but also not necessary. We
can use the proof of the previously considered scenario and extend it to an arbitrary
inertia mechanism, thereby including the present scenario. The restrictive equality
′
Pc (1) = Pc , with which we expressed that, in the first update step, no voter has
a non-zero inertia, can simply be neglected. The considered event of a non-zero
inertia value at only one timestep can also happen at t = 1 and will lead to the same
consequences as described above. Therefore, the proof above includes this scenario,
too.
Note that, in the course of calculating the effect on the expected consensus time in these
three scenarios, we achieved a proof for every possible inertial voter model. We have shown
that, in a system of N = 3 voters, where opinion updates follow the inertial voter model
dynamics (Eq. (5.1)), the expected consensus time invariably increases by: (i) the presence
of a non-zero inertia value at a single timestep, (ii) the number of such timesteps with a
non-zero inertia value, and (iii) the magnitude of the inertia values. In simple words, the
higher the level of inertia in such a system is, the longer the consensus process takes.
These considerations support the hypothesis that extending the voter model by any form of
inertial behavior will invariably lead to an increase of the expected time to reach consensus
– if consensus will be reached at all by the dynamics. Note that N = 3 is the highest
number of voters in the system where such simple calculations are possible. Already for
4 voters, one needs to involve a more detailed expression of the frequencies of opinions in
order to compute the values Pc (t). So far, there was a minority opinion with a frequency
of 1/3, and a majority opinion with frequency 2/3. With N = 4, there would also be the
possibility of equal frequencies, leading to Pc (t) = 0. In general, for N > 3, Pc (t) depends
on dynamic frequencies, which makes the computations much more complicated.
5.3
Results of binary inertia
Having analyzed the implementation of an arbitrary inertia mechanism in a small voter
model system, we now want to investigate the results of our persistent time dependent
mechanism in a multi-agent system, i.e. for N > 3. For the sake of simplicity, let us first
consider that the inertia growth rate is larger than the saturation value of inertia, i.e. in
µ ≥ νs holds in Eq. (5.3). Since we then only have one non-zero inertia value, we simply
denote it by ν and the respective function reads
60
Chapter 5. Slower is faster
νi (τ ) =
(
ν0 ,
ν,
if τ = 0
.
if τ > 0
(5.8)
At time t = 0, and in every timestep after voter i has changed its opinion, the persistence time is reset to zero, τi = 0, and the inertia has the minimum constant value ν0 .1 .
Whenever a voter keeps its opinion, its inertia increases to ν. We will study two distinct
scenarios later on: (i) fixed social inertia where ν0 = ν is a constant value for all voters.
(ii) ν0 < ν, a scenario in which inertia grows for larger persistence times.
It would be expected that including inertial behavior in the model would invariably lead
to a slowing-down of the ordering dynamics. We will show that, contrary to this intuition,
these settings can lead to a much faster consensus.
We performed extensive computer simulations in which we investigated the time to reach
consensus, Tκ , for systems of N voters. We used random initial conditions with equally distributed opinions and an asynchronous update mode, i.e. on average, every voter updates
its opinion once per timestep. The numerical results correspond to regular d−dimensional
lattices (von-Neumann neighborhood) with periodic boundary conditions, and small-world
networks with a homogeneous degree distribution.
In order to obtain a benchmark for the results of our model, we first consider the case of a
fixed and homogeneous inertia value ν0 = ν. In the limit ν → 0, we recover the standard
voter model, while for ν = 1 the system gets frozen in its initial state. For 0 ≤ ν < 1,
the time to reach global consensus is affected considerably; the systems still always reach
global consensus, but this process is decelerated for increasing values of ν. This can be
confirmed by computer simulations which assume a constant inertia equal for all voters
(see left panel in Fig. 5.1).
In the right panel of Fig. 5.1, we depict the evolution of the interface density ρ, as introduced in Eq. (2.4), for both the standard voter model and the inertial voter model with
ν0 = ν = 0.5. Differences between these cases can be seen in the very beginning and
at about 103 time steps, right before the steep decay of disorder in the system. There,
the ordering process is slower in the voter model with inertia than in the standard voter
model.
This behavior can be well understood by analyzing Eq. (5.1). There, we find that the ratio
between the opinion changes in the standard voter model and the inertial voter model is
given by W (−σ|σ, ν)/W V (−σ|σ, 0) = (1 − ν). Consequently, it is reasonable to infer
1
Note that the results of this Chapter are qualitatively robust against changes in the concrete function
νi (τ ). For more details on this, refer to the next Section
5.3. Results of binary inertia
61
0
10
5
simulations
theory
10
−1
ν =ν=0
10
<ρ>
κ
T
0
ν0 = ν = 0.5
4
10
0
10
−2
10
−1
10
−2
<ρ>(ν = 0,t)
10
<ρ>(ν = 0.5,t) / 0.5
−3
10
3
10
−3
−2
10
−1
10
ν0 = ν
0
10
10
0
10
0
10
1
10
1
10
2
3
10
10
2
4
10
3
10
10
timesteps
4
10
Figure 5.1: Left: Average time to consensus Tκ in the voter model with a fixed and
homogeneous inertia value ν0 = ν. The line corresponds to the theoretical prediction
Tκ (ν) = Tκ (ν = 0)/(1 − ν). Details are given in the text. Right: comparison of the
development of the average interface density ρ in the voter model and the model with
fixed inertia. Right, inset: collapse of the curves when the time scale is rescaled according
to t → t/(1 − ν). In both panels, the simulations results stem from averaging over 500
sample runs, where the system size is N = 30 × 30 and the voters are placed on a twodimensional, regular lattice.
that the characteristic time scale for a voter model with fixed inertia will be rescaled as
t → tV /(1−ν). As can be seen in Fig. 5.1, there is good agreement between this theoretical
prediction and computer simulations in both: the average time to consensus (see panel
(a)), and the time evolution of the interface density (inset of panel (b)). In conclusion:
analyzing a “benchmark” model with fixed inertia, the considerations of Section 5.2 are
confirmed.
As second benchmark, let us have a look at fixed, but heterogeneous inertia values within
the system. This means that voters have different inertia values, but these do not change
over time. Since heterogeneity endogeneously appears in our proposed model of persistence
time dependent inertia, this benchmark is important to obtain another reference with
the mere existence of heterogeneity in inertia values. For this purpose, we initialized
simulations of the voter model in fully connected networks, i.e. where every voter is
connected to all other voters. Every voter is assigned with a fixed, individual inertia
value. In panel (a), we see the average results of systems with N = 200 voters. We measure
the time-to-consensus in systems with different non-zero inertia values and with different
frequencies of individuals holding the non-zero inertia value. The results are similar to
those of the first benchmark: consensus is always reached, but with increasing the level of
62
Chapter 5. Slower is faster
1.4
2.8
10
1.3
T (ν) / T (0)
2.6
κ
10
1.2
κ
Average time−to−consensus T
κ
1.5
2.4
10
1.1
N = 25
N = 49
N = 100
N = 400
2.2
10
1
0
10
0
−1
10
−2
Frequency of voters with 10
non−zero inertia
−1
−2
10
(a)
10
Non−zero inertia value ν
10
0.9 −3
10
−2
−1
10
10
Maximum individual inertia ν
0
10
(b)
Figure 5.2: Average consensus times in simulations of the voter model embedded into
fully connected networks. (a) Surface of average consensus times when varying both the
non-zero inertia value ν and the frequency of voters holding the non-zero inertia value.
The system size is N = 200 and averages are obtained from 4000 simulation runs. (b)
Here, the inertia values are not binary, but continuous between 0 and the maximum inertia
value and all the voters have a non-zero inertia. Results are presented for different system
sizes N and the number of simulation runs varies between 10000 for the smallest system
size and 2000 for the largest system size.
inertia in the system (in either way) the time needed to reach the ordered state is generally
not decreasing, but increasing for larger levels of inertia in the system. In panel (b), we also
show results of continuous instead of binary inertia values, for different sizes of the voter
model system. Here, we assign each voter with a random inertia value (νi ) between zero
and the maximal inertia value (ν). Completing the investigations of benchmark models,
also these simulation results indicate that exogeneously induced inertia in the voter model
leads to the expected increase in consensus times.
We now turn our attention to the case where the individual inertia values evolve with
respect to the persistence time according to Eq. (5.8). Without loss of generality, we
fix ν0 = 0. Other choices simply decelerate the overall dynamics as described in the
previous subsection. Note that increasing ν increases the level of social inertia in the
voter population. Fig. 5.3 shows the average time to reach consensus as a function of the
parameter ν, namely the maximum inertia value reached by the voters when the system
is embedded in regular lattices of different dimensions. In Fig. 5.3, it is apparent that, for
lattices of dimension d ≥ 2, the system exhibits a noticeable reduction in the time to reach
consensus for intermediate values of the control parameter ν. We observe that there is a
5.3. Results of binary inertia
63
critical value of ν such that the average time to reach consensus has a minimum. Especially
compared to the results of the previous section, this result is against the intuition that a
slowing-down of the local dynamics would lead to slower global dynamics. Furthermore,
it is also apparent that the larger the dimension of the lattice, the more pronounced the
phenomenon is. Increasing the dimension of the system enlarges the region of ν, in which
consensus times decrease with increasing ν. This fact accounts for the main differences in
the shapes of the functions in panels (b-c) of Fig. 5.3.
(a)
7
(b)
10
6
10
6
10
5
10
10
Tκ
Tκ
5
4
10
4
10
3
10
3
10
2
10
0
0.2
0.4
ν
0.6
0.8
1
0
0.2
0.4
(c)
ν
0.6
0.8
1
0.6
0.8
1
(d)
5
4
10
10
4
Tκ
Tκ
10
3
10
3
10
2
10
2
10
0
0.2
0.4
ν
0.6
0.8
1
0
0.2
0.4
ν
Figure 5.3: Average time to reach consensus Tκ as a function of the maximum inertia value
ν. Panels (a)-(d) show the results for different system sizes in one-, two-, three-, and fourdimensional regular lattices, respectively. The results are averaged over 104 realizations.
The system sizes for the different panels are the following: (a) N = 50 (◦), N = 100 (△),
N = 500 (); (b) N = 302 (◦), N = 502 (△), N = 702 (); (c) N = 103 (◦), N = 153
(△), N = 183 (); (d) N = 44 (◦), N = 54 (△), N = 74 ().
Fig. 5.3(a) shows the results for a one-dimensional lattice, where the phenomenon is not
present at all. For this network topology, it is found that all the curves collapse according
to a scaling relation Tκ (ν, N ) = Tκ (ν)/N 2 , which is in accordance with the scaling of
64
Chapter 5. Slower is faster
5
10
4
Tκ
10
3
10
2
10
0
0.2
0.4
ν
0.6
0.8
1
Figure 5.4: Average time to reach consensus Tκ as a function of the maximum inertia
value ν in small world networks (see text for details). The symbols represent different
rewiring probabilities ω, starting with a 2-dimensional, regular lattice (ω = 0). The
curves corresponds to ω = 0 (◦), ω = 0.03 (△), ω = 0.1 (), and ω = 0.9 (♦). System
size is N = 302 and results are averaged over 104 realizations.
consensus times dependent on system size in the standard voter model (Tκ ∝ N 2 , as
introduced in Section 2.3).
In Fig. 5.4, we plot Tκ as a function of the maximum inertia value ν for different smallworld networks (Watts and Strogatz, 1998). The small-world networks are constructed as
follows: starting with a two-dimensional regular lattice, two edges are randomly selected
from the system and, with probability ω, their end nodes are exchanged (Maslov et al.,
2003). In this procedure, the number of neighbors remains constant for every voter. It
can be seen that the phenomenon of lower consensus times for intermediate inertia values
is also present in small world networks. Furthermore, increasing the rewiring probability
ω leads to larger reductions of the consensus times at the optimal value νc . This implies
that the formation of spatial configurations, such as clusters, is not the origin of this
slower-is-faster effect.
Finally, we show the results on a fully-connected network, i.e. where every voter has N − 1
neighbors. The results are shown in Fig. 5.5. As can be seen, the time to reach consensus
is significantly decreased for intermediate values of ν.
As already mentioned, the results of Figs. 5.4 and 5.5 indicate that the spatial clustering
5.3. Results of binary inertia
65
4
10
N = 100
N = 200
N = 1000
3
Tκ
10
2
10
0
0.2
0.4
0.6
0.8
1
ν
Figure 5.5: Average time to reach consensus Tκ as a function of the maximum inertia value
ν in fully connected networks of different size. Results are averaged over 104 realizations.
plays no important role for the voters’ aging2 and, therefore, for the qualitative behavior
observed. Hence, we will investigate the dynamics in the so-called mean-field limit, which
effectively means that all voters are connected with each other and the system is infinitely
large (N → ∞). For this purpose, we now use the global frequencies of opinions to
calculate the transition probability Wi (−σi |σi , νi ) in Eq. (5.1). As we will see, this allows
us to analytically approach the respective dynamics.
Let us first introduce the quantities aτ and bτ : a0 (t) (b0 (t)) as the fraction of voters with
opinion +1 (−1) and persistence time τi = 0, i.e. inertia state νi (0) = ν0 = 0, and a1 (t)
(b1 (t)) represent the fraction of voters with opinion +1 (−1) and persistence time τi ≥ 1,
i.e. inertia state νi (t) = ν. Thus, voters with opinion +1 that changed their opinion in
the last update step would contribute to the quantity a0 (t), without an opinion change
they would contribute to a1 (t). The global frequency of opinion +1, let us denote it by A,
at time t is given by
A(t) = a0 (t) + a1 (t).
(5.9)
Fig. 5.6 illustrates the possible transitions of voters from one fraction to another, visualizing the above described.
In the mean field limit, we can write down the equations for the change in the single
2
By aging we mean the possibility to build up higher persistence times that in turn lead to increasing
inertia values.
66
Chapter 5. Slower is faster
a0
a1
b0
b1
Figure 5.6: Illustration of the four fractions aτ and bτ and the possible transitions of a
voter.
quantities, i.e. the evolution equations, as
a0 (t + 1) − a0 (t) = W (+1| − 1, 0) b0 (t) + W (+1| − 1, ν) b1 (t)
− [W (−1| + 1, 0) + W (+1| + 1, 0)] a0 (t),
a1 (t + 1) − a1 (t) = W (+1| + 1, 0) a0 (t) − W (−1| + 1, ν) a1 (t),
(5.10)
(5.11)
where all quantities at time t + 1 only depend on quantities at time t, i.e. the system
only has a “memory” of 1 timestep. For this reason, our system represents a Markovchain model (Helbing, 1993; Van Kampen, 2007). For the expressions in Eqs. (5.10)
and (5.11), the global probabilities for changing opinion and for sticking to the current
opinion, respectively, are easily found by using Eqs. (2.1) and (5.1),
W (−1| + 1, 0) = W V (−1| + 1) = B(t),
W (+1| + 1, 0) = W V (+1| + 1) = A(t),
W (−1| + 1, ν) = (1 − ν) W V (−1| + 1) = (1 − ν) B(t),
W (+1| + 1, ν) = 1 − (1 − ν) W V (+1| + 1) = B(t) + ν A(t).
(5.12)
The missing expressions are analogous to the ones given above and can be derived by
consistently exchanging a ↔ b, A ↔ B, and +1 ↔ −1 in Eqs. (5.10), (5.11) and (5.12).
After some steps of straight-forward, but not very illustrative, algebra, we get for the
example of the +1 opinion
5.3. Results of binary inertia
67
a0 (t + 1) − a0 (t) = A(t) b0 (t) + (1 − ν) b1 (t) − a0 (t),
a1 (t + 1) − a1 (t) = A(t) a0 (t) + B(t) a1 (t) (ν − 1),
(5.13)
(5.14)
and the analogous equations for opinion −1
b0 (t + 1) − b0 (t) = B(t) a0 (t) + (1 − ν) a1 (t) − b0 (t),
b1 (t + 1) − b1 (t) = B(t) b0 (t) + A(t) b1 (t) (ν − 1).
The global frequency of the +1 opinion evolves as the sum of Eqs. (5.13) and (5.14)
which yields, again after some steps of straight-forward algebra, the change in the global
frequency
A(t + 1) − A(t) = ν [b0 (t) a1 (t) − a0 (t)b1 (t)] .
(5.15)
For ν = 0, i.e. the standard voter model, we obtain the general conservation of magnetization that we already have seen in Eq. (2.3). For ν > 0, everything depends on the
quantities aτ (t) and bτ (t). If there is no heterogeneity of social inertia in the system, i.e. if,
at some time, either a0 (t) + b0 (t) = 1 or a1 (t) + b1 (t) = 1, then there also is no dynamics in
the magnetization. The same holds if both products in the squared brackets of Eq. (5.15)
are equally high. This is true if A = B and the ratio of inertial voters is the same within
the two global frequencies, i.e. if a0 (t) = b0 (t).
In the remaining configurations of these four quantities, there is a dynamics in the magnetization of the system. This implies that, even if the global frequencies of the opinions
are the same (A = B = 0.5), we can find an evolution towards full consensus at one of
the opinions. Interestingly, the opinion whose frequency is increasing can be the minority
opinion in the system. In general, at every timestep opinion +1 has an increasing share
of voters in the system whenever its internal ratio of inertial voters satisfies the inequality
b1
a1
> .
a0
b0
(5.16)
Analogously, B increases if inequality (5.16) is reversed. However, the complete process
is nonlinear and, therefore, it is not possible to derive the final outcome of the dynamics
from Eq. (5.16).
68
Chapter 5. Slower is faster
Note that condition (5.16) is evidence of the important role of the heterogeneity of voters
on the dynamics in the system. More precisely, the main driving force of the observed
“slower-is-faster” effect is the voters’ heterogeneity with respect to their inertia.
In order to have an analytical estimation of the effect of social inertia on the times to
consensus, we initialize the system in a situation just after the symmetry is broken. In
particular, we artificially set the initial frequencies to differ slightly, i.e. we set a0 (0) =
1/2 + N −1 and, hence, b0 (0) = 1/2 − N −1 .3 Then we iterate according to Eqs. (5.13)
and (5.14). Furthermore, we assume that the consensus is reached whenever for one opinion
a0 (t) + a1 (t) ≤ N −1 holds.4 This is due to the fact that for a system of size N , if the
frequency of the minority state falls below N −1 , the absorbing state is reached. As we are
interested in the effect of different inertia-levels, we again use ν as control parameter and
compare the results with computer simulations of the identical setup of our inertial voter
model. In Fig. 5.7, the lines correspond to this theoretical analysis, where a qualitative
agreement can be seen with the simulation results (quantitative differences result from
finite size fluctuations in the simulations, i.e. from the fact that the numerical simulations
are run with finite system sizes, whereas the analytical solution assumed infinite systems).
5.4
Results of multiple inertia states
To be more general, we also present the results and analysis of our inertia mechanism
with more than two inertia states. For the sake of simplicity, the results presented here
assume that the individual inertia νi increases linearly with persistence time τi , µ being
the “strength” of this response, until it reaches a saturation value νs , i.e.
ν(τi ) = min (µ τi , νs ) .
Choosing νs < 1 avoids trivial frozen states of the dynamics 5 . The rate of inertia growth
µ determines the number of timesteps until the maximal inertia value is reached, denoted
as
3
We also calculated the theoretical predictions for breaking the symmetry in the other way, namely
by setting a0 (0) = 1/2 − N −1 and a1 (0) = N −1 . Here, again opinion +1 is favored, but this time just
by a higher fraction of inertial voters. The initial frequencies of opinions are equal. Qualitatively, this
procedure leads to the same theoretical predictions.
4
With the described initial condition, +1 can be the only consensus opinion.
5
The results presented here are qualitatively independent of the exact functional relation νi (τi ), as
long as a monotonously increasing function with a saturation below 1 is considered. This statement stems
from additional analyzes in the course of these investigations, e.g. using a multi nomial logit function.
5.4. Results of multiple inertia states
69
4
10
N = 100
N = 200
N = 1000
theory (N = 100)
theory (N = 1000)
3
Tκ
10
2
10
0
0.2
0.4
ν
0.6
0.8
1
Figure 5.7: Average time to reach consensus Tκ as a function of the maximum inertia value
ν in fully connected networks of different size. Symbols show the simulation results for
different system sizes (averaged over 104 realizations), lines the results of the theoretical
estimation (explained in the text).
τs = (νs /µ) .
Increasing µ increases the level of inertia within the voter population, thereby slowingdown the microscopic dynamics. Like in the discussion above, one would intuitively assume
an increase of the average time to reach consensus. Interestingly, this is not always the case
as simulation results of Tκ (µ) show for different network topologies (see Fig. 5.8). Instead,
it is found that there is an intermediate value µ∗ , which leads to a global minimum of Tκ .
For µ < µ∗ , consensus times decrease with increasing µ values. Only for µ > µ∗ , higher
levels of inertia result in increasing consensus times.
In panel (a) and in parts of panel (b) of Fig. 5.8, one can also clearly see a global maximum
in the consensus times depending on the control parameter µ. The fact that such a
maximum does not occur in fully connected networks and, especially, that it vanishes
in small-world networks for high rewiring probabilities, indicates that this phenomenon
results from spatial configurations in the respective systems. In contrast to this, the
minima occur for every of the investigated systems and we will focus on this very general
phenomenon in the following. Although it might be interesting to investigate the respective
spatial configurations leading to these maxima in the results, this will not be part of the
more general investigations conducted here.
70
Chapter 5. Slower is faster
(a)
(b)
(c)
3
10
4
10
4
10
5 T
κ
10
Tκ
10
10
2
10
10
1
4 10
10
1
10
2
3
10
10
10
10
4
0
−1
−2
−3
10
3
N
10
10
3
10
−1
10
µ’
0
10
1
10
2
2
2
10
10
10
2
Tκ’
−4
−3
10
−2
−1
10 µ 10
10
0
−4
10
−3
10
−2
10 µ 10
−1
0
10
10
−2
−1
10
µ
10
0
Figure 5.8: Average consensus times Tκ for varying values of the inertia slope µ and fixed
saturation value νs = 0.9. Sample sizes vary between 103 − 104 simulation runs. Filled,
black symbols always indicate the values of Tκ at µ = 0. (a) 2d regular lattices (ki = 4)
with system sizes N = 100 (◦), N = 400 (△), N = 900 (). The inset shows how
consensus time scales with system size in regular lattices at µ = µ∗ for d = 1 (⋄), d = 2
(×), d = 3 (⊳), d = 4 (⋆). (b) Small-world networks obtained by randomly rewiring a 2d
regular lattice with probability: (◦) pr = 0, (△) pr = 0.001, () pr = 0.01, (⋄) pr = 0.1,
(⋆) pr = 1. The system size is N = 900. (c) Fully connected networks (mean field case,
ki = N − 1) with system sizes N = 100 (◦), N = 900 (), N = 2500 (⋄), N = 104 (⋆).
Lines represent the numerical solutions of Eqs. (5.18), (5.19), (5.20) with the specifications
in the text. The inset shows the collapse of the simulation curves by scaling µ and Tκ as
explained in the text.
Before we analyze the dynamics similarly to the previous Section, let us provide some technical details of the simulation results: for a two-dimensional lattice, shown in Fig. 5.8(a),
we find that the value of µ, for which a system finds its minimum in the average consensus
times, scales with the system size as µ∗ ∝ 1/ ln N . Simulations of regular lattices in other
dimensions show that the non-monotonous effect on the consensus times is amplified in
higher dimensional systems. Being barely noticeable for d = 1, the ratio between Tκ (µ∗ )
and Tκ (µ = 0) (i.e. the standard voter model) decreases for d = 3 and d = 4. We further
compare the scaling of Tκ with system size N for the standard and the modified voter
model. For regular lattices of different dimensions (d), we have for the standard voter
model
• d = 1: Tκ ∝ N 2
• d = 2: Tκ ∝ N log N
• d > 2: Tκ ∝ N
5.4. Results of multiple inertia states
71
Note that for d > 2, the systems do not always reach an ordered state in the thermodynamic limit. In finite systems, however, one finds the above written scaling relation.
In the modified voter model, we instead find that Tκ (µ∗ ) scales with system size as a
power-law, Tκ (µ∗ ) ∝ N α (see inset in Fig. 5.8(a)); where α = 1.99 ± 0.14 for d = 1 (i.e., in
agreement with the standard voter model); α = 0.98 ± 0.04 for d = 2; α = 0.5 ± 0.08 for
d = 3; and α = 0.3 ± 0.03 for d = 4. For fixed values of µ > µ∗ , the same scalings apply.
These results suggest (at least approximative) the following the scalings in the modified
voter model:
• d = 1: Tκ ∝ N 2
• d = 2: Tκ ∝ N
• d = 3: Tκ ∝ N 1/2
• d = 4: Tκ ∝ N 1/3 .
In order to cope with the network topology, in Fig. 5.8(b) we plot the dependence of the
consensus times Tκ for small-world networks built with different rewiring probabilities.
Again, the degree of each node is kept constant by randomly selecting a pair of edges and
exchanging their ends with probability p (Maslov et al., 2003). It can be seen that the
effect of reduced consensus times for intermediate values of µ still exists and is amplified by
increasing the randomness of the network. This result implies that the spatial extension
of the system, e.g. in regular lattices, does not play a crucial role in the emergence of
this phenomenon. This can be confirmed by investigating the case shown in Fig. 5.8(c),
in which the neighborhood network is a fully-connected one (the solid lines correspond to
a theoretical approximation introduced below). The inset shows the results of a scaling
analysis, exhibiting the collapse of all the curves by applying the scaling relations µ′ =
|µ ln(η N ) − µ1 |, and Tκ′ = Tκ / ln(N/ξ)µ′ , with η = 1.8(1), µ1 = 1.5(1), ξ = 7.5(1). This
shows that the location of the minimum, as well as Tκ , scales logarithmically with N .
For the analytical approach to these results, we need more than the four quantities used
in the previous Section. Now, the global frequencies aτ (t) (bτ (t)) stand for voters with
opinion +1 (−1) and various persistence times τ (compare Figs. 5.6 and 5.9).
In analogy to Eq. (5.9), the frequencies satisfy
A(t) =
X
τ
aτ (t), B(t) =
X
τ
bτ (t).
72
Chapter 5. Slower is faster
a0
a1
a2
aT
b0
b1
b2
bT
Figure 5.9: General illustration of the fractions aτ and bτ and the possible transitions of
a voter. The fractions aT and bT contain all voters with a persistence time τi ≥ τs , i.e.
voters with maximal inertia.
Since the dynamics satisfy the Markov assumption, the rate equations for the evolution of
these subpopulations in the mean-field limit are given by the master equation
Xh
ȧτ (t) =
Ω(aτ |aτ ′ )aτ ′ + Ω(aτ |bτ ′ )bτ ′
τ′
−
Xh
τ′
i
Ω(aτ ′ |aτ ) + Ω(bτ ′ |aτ ) aτ .
i
(5.17)
Due to symmetry, the expressions for ḃτ (t) are obtained by consistently exchanging aτ ↔
bτ , i.e.
Xh
ḃτ (t) =
Ω(bτ |bτ ′ )bτ ′ + Ω(bτ |aτ ′ )aτ ′
τ′
−
Xh
τ′
i
Ω(bτ ′ |bτ ) + Ω(aτ ′ |bτ ) bτ .
i
These equations are very general and we will identify their constituents in detail below.
Note that most of the terms in Eq. (5.17) vanish because only two transitions are possible
for a voter: (i) it changes its state, thereby resetting its τ to zero, or (ii) it keeps its current
state and increases its persistence time by one. Case (i) is associated with the transition
rate Ω(b0 |aτ ), that in the mean-field limit reads
Ω(b0 |aτ ) = (1 − ν(τ )) B(t).
5.4. Results of multiple inertia states
73
B(t) is the frequency of voters with the opposite state that trigger this transition, while
the prefactor (1 − ν(τ )) is due to the inertia of voters of class aτ to change their state. For
case (ii),
Ω(aτ +1 |aτ ) = 1 − Ω(b0 |aτ ),
since no voter can remain in the same subpopulation. I.e., in the mean-field limit, the
corresponding transition rates are
Ω(aτ +1 |aτ ) = A(t) + ν(τ )B(t).
Therefore, if τ > 0, Eq. (5.17) reduces to
ȧτ (t) = Ω(aτ |aτ −1 ) aτ −1 (t) − aτ (t)
h
i
= A(t) + ν(τ − 1)B(t) aτ −1 (t) − aτ (t).
(5.18)
On the other hand, the fraction of voters with τ = 0 evolves as
ȧ0 (t) =
X
Ω(a0 |bτ )bτ (t) − a0 (t)
τ
h
i
= A(t) B(t) − IB (t) − a0 (t).
(5.19)
Due to the linear dependence of the transition rates on inertia, the terms involving ν can
be comprised into IB (t) and IA (t). The latter expressions stand for the average inertia of
voters with state −1 and +1, respectively, i.e.
IA (t) =
X
ν(τ )aτ (t)
τ
IB (t) =
X
ν(τ )bτ (t).
(5.20)
τ
Expressions (5.18), (5.19), (5.20), and the corresponding ones for subpopulations bτ can
be used to give an estimate of the time to reach consensus in the mean-field limit. Let us
consider an initial state a0 (t) = A(0) = 1/2 + N −1 and b0 (t) = B(0) = 1/2 − N −1 , i.e.
voters with state +1 are in slight majority. By neglecting fluctuations in the frequencies
74
Chapter 5. Slower is faster
(which drive the dynamics in the standard voter model), these equations are iterated until
B(t′ ) < N −1 (i.e. for a system size N , if the frequency of the minority state falls below
N −1 , the absorbing state is reached). Then, we assume Tκ = t′ . The full lines in Fig. 5.8(c)
show the results of this theoretical approach, exhibiting the minimum and displaying good
agreement with the simulation results for large values of µ. For low values of µ, fluctuations
drive the system faster into consensus compared to the deterministic approach.
Inserting Eqs. (5.18) and (5.19) into the time-derivative of Eq. (5.17) yields, after some
straight-forward algebra and in analogy with Eq. 5.15, the time evolution of the global
frequencies
Ȧ(t) = IA (t) B(t) − IB (t) A(t).
(5.21)
Compared to the standard voter model, the magnetization conservation is now broken
because of the influence of the evolving inertia in the two possible states. For ν(τ ) = ν•
(i.e. a time-independent inertia, that includes the standard voter model, ν• = 0), we
regain the magnetization conservation. This can be shown by reconsidering Eqs. (5.9) and
(5.20):
IA (t) =
X
ν• aτ (t)
τ
= ν•
X
aτ (t)
τ
= ν• A(t).
(5.22)
Then, considering the analogous considerations for IB (t), Eq. (5.21) reads
Ȧ(t) = IA (t) B(t) − IB (t) A(t)
= ν• A(t) B(t) − ν• B(t) A(t)
= ν• [A(t) B(t) − B(t) A(t)]
= 0,
(5.23)
i.e., independent of the state of the system, there is no change in the global frequency of
opinion A and, consequently, also not in the frequency of opinion B. The same amount of
voters changing from opinion +1 to −1, change from −1 to +1.
5.4. Results of multiple inertia states
75
However, let us return to the case of a persistence-time dependent inertia. Interestingly
enough, Eq. (5.21) implies that the frequency A(t) grows iff.
IA (t)/A(t) > IB (t)/B(t).
When the time dependence of the inertia on the persistence time is a linear one, as assumed
in this Section, inserting Eqs. (5.18, 5.19) into Eq. (5.20), we obtain an equation for the
time evolution of IA (t) up to first order in µ:
I˙A (t) = A(t) IA (t) + µA2 (t) − IA (t) + O(µ2 , aT ).
(5.24)
P
Here, aT = τ ≥τs aτ contains all subpopulations with maximum inertia. The last term,
O(µ2 , aT ), stands for the omitted terms that contain µ2 and aT . This simplification is
viable if these variables are so small that they can be neglected without changing the
results, which will be taken care of below. Eqs. (5.21) and (5.24) correspond to the
complete macroscopic level description of this model. Hence, we can analyze the system
of equations by identifying its fixed points. We find the saddle point,
A = B = 1/2 ;
IA = IB = µ/2 + O(µ2 ),
and two stable fixed points, one at
A = 1;
IA = νs
B = 1;
IB = νs .
and another at
Note that the saddle point is close to the initial condition of the simulations. Neglecting
fluctuations, the time to reach consensus has two main contributions: (i) the time to
escape from the saddle point, Ts ; and (ii) the time to reach the stable fixed point, Tf ;
namely
Tκ ∼ Ts + Tf .
We then linearize the system around the fixed points and calculate the largest eigenvalues
λs and λf (for the saddle and the stable fixed points, respectively) as a function of µ. In
76
Chapter 5. Slower is faster
Section 1.3, we already explained that the sign of the eigenvalues disclose the direction of
the dynamical flow along each dimension of the system. Furthermore, the magnitude of
the eigenvalues indicate the speed of the respective dynamical flows. At the saddle point,
we find
λs (µ) =
p
1 + 20 µ + 4 µ2 − 2 µ − 1 + O(µ2 ),
which equals 0 at µ = 0 and monotonously increases with µ. For larger values of µ,
where the first order term expansion is no longer valid, numerical computations show
that λs continues to increase monotonously with µ. This means that, for larger inertia
growth rates µ, the system will escape faster from the saddle point, thereby reducing the
contribution Ts to the consensus time Tκ . On the other hand, for µ → 0, λs vanishes and
the system leaves the saddle point only due to fluctuations.
Near the stable fixed points the contribution of aT to Eq. (5.24) cannot be neglected
anymore. Solving the complete expressions with the help of the technical computing
software “Mathematica”, we obtain λf,1 = −νs for µ < 1 − νs , whilst λf,2 = µ − 1 for
µ ≥ 1 − νs . Interestingly, both reflect different processes: the eigenvalue λf,1 is connected
to voters sharing the majority state which are, at the level of νs , inertial to adopt the
minority one (signalled by λf,1 being constant). For µ ≥ 1 − νs , the largest eigenvalue
λf,2 is related to voters with the minority state that are, for increasing µ, more inertial to
adopt the majority state (apparent by the decrease in |λf,2 |).
The contributions Ts and Tf are two competing factors in the dynamics towards consensus.
Qualitatively, they can be understood as follows: in the beginning of the dynamics, the
inertia mechanism amplifies any small asymmetry in the initial conditions. While this
causes faster time to consensus for (small) increasing values of µ, for sufficiently large
values of inertia growth, another process outweighs the former: the rate of minority voters
converting to the final consensus state is considerably reduced, too. It is worth mentioning
that the phenomenon described here is robust against changes in the initial condition:
starting from IA = IB < νs , it holds for any initial frequencies of opinions. Conversely,
starting from A = B = 1/2, it holds for any IA 6= IB .
5.5
Conclusions
The time for reaching a fully ordered state in a two-state system such as the voter model is
a problem that attracted attention from different fields in the last years. In this Chapter,
we studied the effect of social inertia in the voter model based on the assumption that
5.5. Conclusions
77
social inertia grows with the time the voter has been keeping its current opinion. We
focus our study on how the times to consensus vary depending on the level of inertia in
the population. In the two scenarios considered, this level of inertia was steered by the
value of the only non-zero inertia parameter ν and the rate of inertia growth µ, respectively.
In contradiction with the expectation that increasing inertia may lead to increasing times
to reach consensus (which has been underlined by benchmark investigations), we find
that, for intermediate values of ν (µ), this inertia mechanism causes the system to reach
consensus faster than in the standard voter model. Interestingly, this phenomenon implies
that individuals reluctant to change their opinion can have a counter-intuitive effect on
the consensus process, which was studied for some particular cases before (Galam, 2005).
Furthermore, an inertial minority can overcome a less inertial majority in a similar fashion
as previously discussed by Galam (2002) and Tessone et al. (2004).
We show that the phenomenon discussed here is robust against the exact topology of the
neighborhood network as we find it in regular lattices and small-world networks. In the
former it holds that the higher the dimension, the more noticeable the effect. Furthermore,
we found that the phenomenon also appears in random and fully-connected networks.
In simple words, this intriguing “slower-is-faster” effect can be understood as follows: Due
to fluctuations, one of the opinions is able to acquire a slight majority of voters. Therefore,
voters of this opinion change less likely and, hence, the average inertia of this opinion will
be higher than the other. Since inertia reduces emigration, but not immigration, the
majority will become even larger. This development is enforced by higher values of ν (µ)
and constitutes a clear direction of the ordering dynamics (as opposed to the standard
voter model, where the magnetization is conserved), which intuitively can lead to a faster
reaching of consensus. However, for high values of ν (µ), this development is outweighed
by the high level of average inertia in the complete system, i.e. also within the minority
population of voters, which slows down the overall time scale of the ordering dynamics
(shown in Fig. 5.1).
For both approaches (the reduced model based on only two levels of inertia (Stark et al.,
2008c) and the one with slowly increasing inertia (Stark et al., 2008b)) we observed the
described “slower-is-faster” phenomenon. While the results for both model specifications
are qualitatively very similar, they still differ in some details. Comparing Figs. 5.3, 5.4,
and 5.5 with Fig. 5.8, we recognize for example differences in the typical shapes of the
curves. Furthermore, the interior global maximum of consensus times for spatially extended systems, which get apparent in Fig. 5.8(a), we can only observe in the model with
multiple inertia states, but not in the binary setup.
We provided and analyzed the dynamical equations that unveiled the phenomenon’s origin,
78
Chapter 5. Slower is faster
namely the described aging-mechanism that breaks the magnetization conservation. This
is different from the standard voter model, where magnetization is always conserved. We
showed that the break of magnetization conservation only holds when the voters build
up a heterogeneity with respect to their inertia to change opinion. Therefore, once the
symmetry between (a) the global frequencies of the two possible states and/or (b) the
proportions of inertial voters is broken, the favored state (opinion) achieves both (i) a
reinforcement of its average inertia and (ii) a fast recruitment of the less inertial state.
Both effects contribute to a faster deviation from the symmetric state. For some parameter
ranges, these mechanisms outweigh the increase in the time to reach consensus generated
by the high inertia of the state that disappeared in equilibrium.
The heterogeneity of inertia values plays an important role for the appearance of the
described effect. However, it is important to note that it is not the heterogeneity in itself,
which triggers the effect. This is particularly underlined by the “benchmark” investigations
around Fig. 5.2, where we see that heterogeneously distributed inertia alone does not
produce surprising results. As already explained, only the co-evolutionary dependency
between heterogeneous inertia and the opinions, where higher inertia has a positive effect
on the frequency of an opinion and vice versa, leads to the interesting results reported
above.
Due to the general analytical approach taken in our analysis, we emphasize that this phenomenon is not restricted to the voter model, but is expected to appear in any spin system,
whenever the inertia mechanism is present. This conjecture is supported by preliminary
results of another dynamical system, which are presented as an outlook on further research
in the next subsection.
When introducing the implementation of inertia into the voter model, we have put some
analytical evidence on the intuitively expected effect of such an extension – increasing
consensus times. Since these results of a small system with N = 3 voters are contradictory
to our findings for larger systems sizes, we conclude an emergent phenomenon out of the
interactions in a multi agent system.
A more in depth investigation of the model in a 2-dimensional regular lattice would be
valuable. There, one could find out about the reason for the local maximum in the
consensus times shown in the left panel of Fig. 5.8. In the course of our investigations, we
also obtained some evidence that one would find the existence of surface tension in the
spatial dynamics of this model (see Fig. 5.10 and compare to Dall’Asta and Castellano
(2007)). This would be an important change compared to the fluctuation driven dynamics
of the standard voter model.
5.6. Outlook: inertia-effects in another nonlinear voter model
(a) t = 0
(b) t = 100
(c) t = 200
(d) t = 1000
(e) t = 10, 000
(f) t = 20, 000
(g) t = 30, 000
(h) t = 35, 800
(i) t = 35, 900
79
Figure 5.10: Exemplary time evolution of the inertial voter model with multiple inertia
states and µ = 0.1, νs = 0.9. Voters are positioned on a 2-dimensional, regular lattice,
where every voter interacts with its 4 nearest neighbors. The system consists of N =
10, 000 voters and the evolution time t is measured in generations, i.e. 10,000 single
updates correspond to 1 generation.
5.6
Outlook: inertia-effects in another nonlinear voter
model
In the following, we provide an outlook on a valuable follow-up investigation of the inertia
mechanism in the voter model. At the same time, the considerations below provide some
80
Chapter 5. Slower is faster
more evidence for the generality of the previous results. Let us consider another dynamical
update rule. Here, instead of the linear function in the voter model, the probability
to adopt opinion σ depends on its frequency following a logistic function (also called
multinomial logit function or Fermi function, see Fig. 5.11).
1
L
probability to choose σ, W (σ)
voter model rule
0.8
logistic rule, β = 4
0.6
0.4
0.2
0
0
0.2
0.4
0.6
0.8
1
frequency of opinion σ
Figure 5.11: Comparison of the logistic decision function with the one of the linear voter
model.
Hence, the transition probabilities are
WiL (σ) ≡ WiL (σ|σi , t) =
1
,
exp(β [1 − 2 WiV (σ|σi , t)])
(5.25)
where β is a parameter of inverse (social) temperature. Higher temperature means more
randomness in the decision of the voters and, in particular, even if no neighbor (all neighbors) choose a certain opinion, the focal voter might decide in favor (against) this opinion.
Again, the frequencies of opinions in the neighborhood of the updating voter are decisive 6 ,
but this time the relationship is an exponential one. It is well-known that the dynamics undergo a phase transition at a critical temperature Tc = 1/βc (compare Glauber-Metropolis
dynamics in the Ising model (Landau and Binder, 2005)). For β-values resulting in a
lower “social” temperature β > βc , one of the opinions will reach a stable majority and
the magnetization of the system (Eq. (2.2)) approaches ±1 with increasing β-values. If
Because of equality, we used the transition probability of the voter model WiV (σ) to account for the
frequency of opinion σ in the neighborhood of individual i.
6
5.6. Outlook: inertia-effects in another nonlinear voter model
81
the social temperature is high (β < βc ), the system reaches or remains in a completely
disordered state of zero magnetization.
Assuming an infinite, fully connected network, i.e. an updating agent does not decide
based on a local neighborhood, but takes the global frequencies of opinions into account
(the mean-field limit), leads to the master equation for the probability of finding an agent
with opinion σ
Ṗσ =
X
[ω(σ|σ ′ )Pσ′ − ω(σ ′ |σ)Pσ ] ,
(5.26)
σ′
where, according to Eq. (5.25), the transition rates are
ω(σ|σ ′ ) = exp β WiV (σ|σ ′ , t) .
Again representing the frequency of the +1 opinion by A and the frequency of the −1
opinion by B = 1 − A, we have
ω(A|B) = exp(β A),
ω(B|A) = exp [β (1 − A)] ,
(5.27)
i.e.
Ȧ = exp(β A) B − exp [β (1 − A)] A.
Setting Ȧ = 0, the stationary solution fulfills
A
ln 1−A
.
β=
2A − 1
(5.28)
For this mean-field considerations, Fig. 5.12 visualizes the above mentioned “order-disorder”
phase transition. The so-called magnetization is simply M = 2 A − 1, i.e. the average
opinion in the system. Numerical simulations in finite, fully connected networks agree
with this analytical result (not shown here, but indicated in Fig. 5.13).
Note that implementing a homogeneous, fixed inertia into this mean-field system does
not change the stationary solution. The transition rates here are the ones in Eq. (5.27),
multiplied by (1 − ν) each. Hence, the dynamics reads
Ȧ = (1 − ν) {exp(β A) B − exp[β (1 − A) A]} ,
82
Chapter 5. Slower is faster
1
0.8
0.6
"Magnetization"
0.4
0.2
T
c
0
−0.2
−0.4
−0.6
−0.8
−1
0
0.1
0.2
0.3
0.4
"Temperature", 1/β
0.5
0.6
Figure 5.12: Order-disorder phase transition for transition probabilities following a Fermi
function.
1
µ = 10−5
µ = 0.1
µ = 0.9
0.8
abs(Magnetization)
stat. solution
0.6
0.4
0.2
0
0
0.5
1
1.5
Temperature
Figure 5.13: Numerical simulations of an N = 900, fully connected network.
where the prefactor (1 − ν) has no influence on the stationary solution.
However, we performed numerical simulations in finite, fully connected networks, where
we implemented our persistence-time dependent inertia mechanism. Fig. 5.13 includes the
stationary solution shown in Fig. 5.12 and simulation results for 3 different µ-values. The
result for µ = 10−5 is basically identical to a system without inertia at all and shows perfect
agreement with the stationary solution (except for not reaching zero magnetization due to
5.6. Outlook: inertia-effects in another nonlinear voter model
83
finite size). For larger values of inertia growth rates µ, we see a pronounced difference to the
stationary solution: remarkably, the inertia mechanism may lead to considerably higher
(absolute) magnetization in a system of a given temperature and an increased critical
point of the temperature. Interestingly, this represents another form of the positive effect
on ordering dynamics than the one discussed before. Another intriguing parallel is that
we again find a non-monotonous behavior dependent on µ. Note that the curve of the
highest value µ = 0.9 (leading to binary inertia in the system since, like before, we applied
a saturation at νs = 0.9) is below the one for an intermediate value µ = 0.1.
Chapter 6
Social influence in social dilemmas
It is the aim of this Chapter to implement decision behavior solely dependent on social
influence into the fitness dependent environment modeled by evolutionary game theory. In
the main part of this Chapter, we present a simple and straight-forward implementation
of imitation into the evolutionary dynamics of cooperators and defectors (Section 6.4).
Before we do so, we present the motivation and results of two other investigations, which
were supposed to provide more insight into evolutionary dynamics: (i) we were interested
in the effects of assuming a population that is well-mixed (interactions can occur between
any two individuals), but an individual does only interact with a certain sample of the
population, instead of all other individuals (Section 6.1). We investigate this interaction
scheme for the reference case described in Section 3.3, but also for the dynamics of: (ii)
instead of cooperators and defectors, we assume social influence deciders only, but two
different kinds: some of the individuals are imitators (positive social influence) and the
others are contrarians (negative social influence). Contrarians are the counterpart of
imitators in that they decide for the opposite of what others do (as discussed in the
literature (De Martino et al., 1999; Galam, 2004; Selten et al., 2004)).
Furthermore, we apply a similar co-evolutionary dynamics as the inertia mechanism for
the voter model into a standard model in evolutionary game theory - the spatial Prisoner’s Dilemma. The approach and the results presented there illustrate the close relation
between the fields in Chapter 5 and Chapter 6.
84
6.1. Homogeneous, dynamic random networks
6.1
85
Evolutionary dynamics in homogeneous, dynamic
random networks
Complementing the results of Section 3.3, we want to consider an interaction scheme that
could be called “homogeneous, dynamic random networks”. In each update step, a focal
individual interacts with a fixed number k of other individuals that are randomly chosen
from the whole network. This means we have a well mixed population, but the number of
interactions is limited (instead of interacting with all other individuals, which is equivalent
to interacting with one “representative” individual). Although our implementation is
different, the concept is similar to the one of accelerating the selection rate compared
to the rate of interaction, as described by Roca et al. (2006). These authors report on
dramatical changes in the dynamics when increasing the selection rate as compared to
the usual mean field approach. However, here we show that these results are restricted to
their implementation where only a part of the individuals interact at all and others have
the lowest fitness, because they did not interact. In our implementation, all individuals
interact, but only with a limited sample of the population. The aim of this approach is
to find out whether one finds other or more fixed points of the dynamics that can lead to
different equilibrium states than the ones known from the mean-field approach.
As a simple example, consider k = 2. In the limit of an infinite population, the probability
to interact with two cooperators equals x2 . The probability to interact with two defectors
equals (1−x)2 and, finally, the probability to interact with one cooperator and one defector
equals x (1 − x). Hence, the expected payoff of a cooperator when interacting with two
randomly chosen individuals reads
πC,2 = 2 R x2 + 2(R + S) x (1 − x) + 2 S (1 − x)2 .
(6.1)
Simplifying this expression, we find that
πC,2 = 2(x(R − S) + S),
(6.2)
which is exactly twice the payoff of a cooperator in the usual mean field approach (Eq. (3.6)).
Similarly, πD,2 = 2 πD and, therefore, the fixed points of the replicator equation are not
affected. In general, the expected payoff of a cooperator when interacting with k randomly
chosen individuals reads
86
Chapter 6. Social influence in social dilemmas
πC,k
k X
k k−i
=
x (1 − x)i (i R + (k − i) S)
i
i=0
k k X
X
k
k k−i
k−i
i
= (R − S)
i x (1 − x) + k S
x (1 − x)i
i
i
i=0
i=0
= (R − S)k x + k S
= k (x(R − S) + S)
= k πC ,
(6.3)
leading to the same result as for the k = 2 example: the fixed points of the evolutionary
dynamics remain unchanged compared to the usual mean field approach. It would be
interesting to combine the implementations of an accelerated selection time scale of Roca
et al. (2006) and the one considered here. However, considering a homogeneous, dynamic,
and random network structure as introduced here bears no new insight into the respective,
underlying dynamics.
6.2
Evolutionary dynamics with imitators and contrarians
The idea here is to investigate the evolutionary dynamics between imitators and contrarians. This means we assume that individuals are not capable of consciously deciding for
one or the other alternative (C or D), but they only can either imitate the decisions of
others, or do the opposite.
The questions are: in which scenarios will contrarians dominate the system, thereby leading to persistent, partial cooperation in the system? Provided imitators dominate the
system, will they find consensus in cooperation or defection – of course dependent on the
payoff structure of the underlying game? In fact: what will be the equilibrium fraction of
cooperators in all the games compared to the result depicted in Fig. 3.3?
In order to answer these questions, we replace cooperators and defectors by imitators and
contrarians. For this purpose, we have to write the expected payoffs of imitators and
contrarians (πim , πcon ), because we assume replicator dynamics with the usual form
i̇ = i (1 − i) (πim − πcon ),
where i is the frequency of imitators (compare to Section 3.3).
(6.4)
6.2. Evolutionary dynamics with imitators and contrarians
87
Let us first assume the voter model dynamics as decision rule for the individuals, i.e.
imitators cooperate with a probability equal to the level of cooperation in the system
(pC,im (t) = x(t)) and defect with the with a probability equal to the level of defection
(pD,im (t) = 1 − x(t)). Contrarians decide in the opposite way, i.e. pC,con (t) = 1 − x(t) and
pD,con (t) = x(t). Note that the change in x, i.e. in the level of cooperation in the system,
is only a consequence of the frequency dynamics between imitators and contrarians.
An individual, imitator or contrarian, receives πC = x (R − S) + S if it cooperates and
πD = x (T − P ) + P if it defects (in accordance with Eq. (3.6)). Hence, the payoffs of the
individuals in the system read
πim = pC,im πC + pD,im πD
= x πC + (1 − x) πD ,
πcon = (1 − x) πC + x πD .
(6.5)
Applying these payoffs to Eq. (6.4), we can compute the fixed points of the dynamics
by setting i̇ = 0 and transform the result in order to obtain the equilibrium level of
cooperation in the system x. We have solved this with the technical computing software
“Maple”, and the results are quite interesting: we get the trivial fixed points at i = 0
(leading to x = 1/2) and i = 1 (with any initial x as equilibrium level of cooperation,
according to the voter model dynamics) and another fixed point, namely
x=
P −S
.
R+P −S−T
The fixed points i = 0 is unstable and the fixed point i = 1, where stable, leads to
the same equilibrium level of cooperation as in the reference case. The third fixed point
is interesting in that we already know it from the dynamics between cooperators and
defectors (see Eq. (3.8) and the text immediately below it). Since this fixed point is
independent of i, also its stability analysis does not change and we can conclude that the
outcome of the evolutionary dynamics between imitators and contrarians is identical to
the the reference case depicted in Fig. 3.3. In the regions where defectors or cooperators
dominate the reference model, imitators dominate the system and find consensus in the
respective decision (full cooperation or full defection). In the coexistence region of the
reference model, also imitators and contrarians coexist with frequencies that lead to an
identical frequency of cooperators compared to the reference case. Finally, the bi-stability
region has turned into a dominance region of imitators, but in terms of the final frequency
88
Chapter 6. Social influence in social dilemmas
of cooperators, it is still a bi-stable region and the interior, unstable fixed points correspond
to those from the reference case.
Although this result is entirely determined by the model definition, we did not expect it
due to the completely different decision process assumed in this model. However, so far it
seems like there are no additional dynamical properties to observe in this model setup.
In order to account for a higher level of stochasticity, let us also assume that individuals
decide according to a Fermi function (as we discussed for opinion dynamics in Section 5.6),
i.e. in a well mixed population, the probability to cooperate equals
pC (t) =
1
.
1 + exp(β [1 − 2x(t)])
(6.6)
To define our sub-populations, we will assume β ∈ {b, −b}, where imitators apply a β of
b > 0 and contrarians have β = −b. Note that we do not expect qualitative changes by
varying the value of b as long as it is well above the critical temperature known from the
Glauber dynamics (which lies around 2, see Section 5.6). Moreover, the results should be
similar for any decision rule where imitators tend to follow the majority and contrarians
tend to join the minority in the system. Using Eqs. (6.6) and (3.6), the expected payoff
of an individual reads
πβ =
(πC − πD )
+ πD ,
1 + exp{β [1 − 2x(t)]}
(6.7)
i.e.
(πC − πD )
+ πD
1 + exp{b [1 − 2x(t)]}
(πC − πD )
+ πD .
=
1 + exp{−b [1 − 2x(t)]}
πim =
πcon
(6.8)
What we still need is the global number of cooperators x(t) – not only to determine the
equilibrium level of cooperation, but also for the expected payoffs in Eqs. (6.8). The
respective equation reads
x(i, t) =
i
1−i
+
,
1 + exp{b [1 − 2x(i, t)]} 1 + exp{−b [1 − 2x(i, t)]}
(6.9)
which can be solved numerically. Doing so and computing the fixed points of this dynamics,
we again find that the equilibrium fraction of cooperators is identical to the dynamics
between cooperators and defectors.
6.3. Co-evolution in the spatial Prisoner’s Dilemma
89
Furthermore, the same result holds for the homogeneous, dynamic random networks, where
we assumed that a focal individual interacts with k other individuals that are randomly
chosen from the whole system. All we have to do is applying Eq. (6.3) to Eq. (6.7), which
yields
k (πC − πD )
+ k πD
1 + exp{β [1 − 2x(t)]}
= k πβ .
πβ,k =
(6.10)
In conclusion, in this Section we investigated the evolutionary consequences of assuming
that individuals in a symmetrical 2×2 game do not have sufficient information or capabilities in order to decide for one of the two decision alternatives, but only decide dependent
on the decision-frequencies in their environment. We allowed for two opposing behaviors:
imitation and contrarian behavior. In general, we found that this approach does not lead to
new results regarding the evolution of cooperative behavior in competitive environments.
6.3
Co-evolution of strategies and teaching abilities
in the spatial Prisoner’s Dilemma
In this Section, we bring together the idea and motivation of the persistence time dependent inertia mechanism in opinion dynamics (see Chapter 5) and the fitness dependent
dynamics of evolutionary game theory. In particular, we base our investigations on the
idea of heterogeneous “teaching abilities” within the player population, as it has been
introduced by Szolnoki and Szabó (2007). In their model of an evolutionary, spatial Prisoner’s Dilemma (i.e. where players repeatedly play the Prisoner’s Dilemma game only with
their fixed set of nearest neighbors and successful players are more likely to spread their
own strategy to a neighboring player, compare to (Frean and Abraham, 2001a; Nowak
and May, 1992; Perc and Szolnoki, 2008; Roca et al., 2008; Schweitzer et al., 2002)), they
assume that some individuals have (fixed) teaching abilities that allow them to disperse
their own decision (cooperate or defect) with an increased probability to their interaction
partners. This means that some individuals are more influential than others and their
behavior has better chances to spread in the population. By doing so, cooperators can
improve their payoff by converting defecting interaction partners into cooperative ones
and, contrarily, defectors detoriate their success by teaching neighboring cooperators to
defect. Thereby, the level of cooperation in the system can be enhanced compared to the
evolutionary, spatial Prisoner’s Dilemma game without teaching abilities.
These teaching abilities can have the same source as the inertia that we investigated
90
Chapter 6. Social influence in social dilemmas
in the opinion dynamics part of this thesis, namely conviction. The only difference is
that inertia leads to an increased probability to keep the current decision, while teaching
increases the probability to transfer the own current decision to another individual. Both
mechanisms, however, provide an evolutionary advantage to the respective decision. Due
to this equivalence of approaches, we further assume evolving instead of fixed teaching
abilities. As for the opinion dynamics investigations, we connect the teaching abilities of
an individual to the persistence time of their current decision (to cooperate or defect).
One could also interpret this as gaining experience in the course of time, so aging, as we
already used this term for the opinion dynamics investigations, is a good example for such
kind of processes 1 .
For the model setup, we consider an evolutionary Prisoner’s Dilemma game, that is characterized by the following payoff matrix
C
D
!
C
D
1
b
0
0
%
.
(6.11)
With the relation 1 < b ≤ 2, this payoff matrix bears the essential properties of a Prisoner’s
Dilemma while reducing the model complexity to one parameter b (Nowak and May, 1992).
This parameter adjusts the strength of the dilemma: is the temptation to defect large
(b >> 1), it is more difficult to cooperate for the individuals. If b is close to 1, the
dilemma is weak. Each player i on the regular L × L square lattice is initially designated
either as a cooperator (si = C) or defector (si = D) with equal probability, and the game
is iterated forward in accordance with the Monte Carlo simulation procedure comprising
the following elementary steps: first, a randomly selected player i acquires its payoff pi
by playing the game with its nearest neighbors. Next, one randomly chosen neighbor,
denoted by j, also acquires its payoff pj by playing the game with its four neighbors.
Finally, player i enforces its strategy si on player j with the fitness dependent probability
W (si → sj ) = wi
1
1
,
1 + exp[(pj − pi )/K]
(6.12)
Note that although the investigations of this approach have been designed in consequence of the
argumentation built within this thesis and with the aim to support the main argument of this thesis
(models of social influence and strategic decision making are only different perspectives to look at social
interactions and combining them leads to new and relevant insight to the intrinsic properties of social
interaction), the major part of the research was performed outside this thesis’ project. Moreover, the
results crucially rely on the spatial network structure, while the main focus of this thesis is more general.
Hence, we will here only summarize the main findings and discuss their relevance, while pointing the
interested reader to the respective publication for all details (Szolnoki et al., 2009).
6.3. Co-evolution in the spatial Prisoner’s Dilemma
91
where K denotes the amplitude of noise (Szabó and Tőke, 1998) or the intensity of selection (Altrock and Traulsen, 2009; Traulsen et al., 2007). Note that this again denotes
a Fermi function as we used it in the previous Section and for opinion dynamics (see
Section 5.6 on Glauber dynamics). wi characterizes the enforcing strength (or “teaching
ability” (Szolnoki and Szabó, 2007)) of player i. One full Monte Carlo step involves all
players having a chance to pass their strategies to their neighbors once on average. The
teaching ability wi is related to the integer age ei = 0, 1, . . . , emax in accordance with the
function
wi = (ei /emax )α ,
(6.13)
where emax = 99 denotes the maximal possible age of a player. Thereby, wi is kept within
the unit interval and α determines the level of heterogeneity in the ei → wi mapping.
Evidently, α = 0 corresponds to the classical spatial model, where wi = 1 for all players.
α = 1 leads to a model where wi and ei have the same distribution, whereas α ≥ 2 impose
a power law relation between the teaching ability and age.
Initially, each player’s age ei is randomly selected from a uniform distribution within the
interval [0, emax ]. In order to separate the influences of different parameters, three scenarios
have been investigated:
I A players “age” does not evolve in time and α determines the static level of artificially
introduced heterogeneity.
II The age of all players is increased by 1 every full Monte Carlo step. Furthermore, we
reset ei = 0 for all players i whose age exceeded emax (a newborn follows the dead
player).
III Additionally to the previous setup, we also reset ei = 0 for all players which changed
their decision (to cooperate or to defect). From a biological aspect, the more successful player replaces its neighbor with its own descendant. From a social viewpoint, a
player who has not changed its strategy seems to be more reliable than the one who
has just changed its strategy. Thus, the latter is considered to have less teaching
ability.
Note that setup III represents the implementation of our inertia mechanism. The resulting
dynamics is co-evolutionary as both the level of cooperation and the heterogeneity of
teaching abilities endogenously evolve in time.
92
Chapter 6. Social influence in social dilemmas
1
0.8
xs
0.6
0.4
0.2
0
1
1.02 1.04
1.06 1.08
b
1.1
1.12
1.14 1.16
Figure 6.1: Promotion of cooperation due to the increasing heterogeneity in the ei → wi
mapping via α. Stationary fraction of cooperators xs is plotted in dependence on b for
α = 0 (solid red line), α = 1 (dashed red line) and α = 2 (dotted blue line). In all three
cases K = 1.
Results of Monte Carlo simulations presented below were obtained on populations comprising 100 × 100 to 800 × 800 individuals, where the stationary fraction of cooperators
xs was determined within 105 to 106 full Monte Carlo steps. Moreover, since the coevolutionary aging process may yield highly heterogeneous distribution of ei , which may
be additionally amplified during the ei → wi mapping, final results were averaged over up
to 300 independent runs for each set of parameter values in order to take into account the
fluctuating output and assure accuracy.
In agreement with previous results (Perc and Szolnoki, 2008), model I yields an increased
promotion of cooperation with increasing heterogeneity in the system, as is illustrated in
Fig. 6.1. There, the results for different variants of the Prisoner’s Dilemma are shown
by varying the temptation parameter b – the larger the temptation to defect is, the more
difficult it is to maintain cooperation within the evolutionary Prisoner’s Dilemma. The
threshold of b, up to which a certain level of cooperation xs can be found in the final
population, increases with the parameter α, i.e. with the level of heterogeneity in teaching
abilities.
This preliminary result is more comprehensively illustrated in Fig. 6.2, where the whole
phase diagram of the model parameters b and K is presented for the values α = 0 and
6.3. Co-evolution in the spatial Prisoner’s Dilemma
93
1.4
(b)
(a)
1.3
1.2
b
D
1.1
D
C
1.0
C
0.9
0
0.5
1
K
1.5
2 0
0.5
1
K
1.5
2
Figure 6.2: Full b − K phase diagrams for the Prisoner’s Dilemma game with quenched
uniform distribution of ei , obtained by setting α = 0 [panel (a)] and α = 2 [panel (b)] in
the ei → wi mapping. Solid green and red lines mark the borders of pure C and D phases,
respectively, whereas the region in-between the lines characterizes a mixed distribution
of strategies on the spatial grid. The dashed, blue line at b = 1 denotes the boarder
between the essential Prisoner’s Dilemma payoff parameterization (above the line) and a
dilemma-free situation (below the line).
α = 2. Compared to the homogeneous scenario (panel (a)), the dominance region of
cooperators (below the solid, green line) is shifted towards higher values of b and the
coexistence region (between the green and the red line) enlarged in the heterogeneous
scenario.
Turning to models II and III, Fig. 6.3 shows the effects of the dynamical heterogeneity
(model II, panel (a)) and the co-evolution between heterogeneity and the level of cooperation (model III, panel (b)). Most interestingly, we find opposite results compared to
the case of static heterogeneity (Fig. 6.2, panel (b)): if the heterogeneity is dynamically
changing due to the deterministic aging of individuals, the positive effect for cooperation is there, but less pronounced as for the static case. In contrast, the co-evolutionary
model III clearly enlarges the promotion of cooperation. In the plain model III, smallest
values of the noise parameter K already lead to coexistence in the final population and
for K > 0.25, cooperators are even able to dominate the population for weak Prisoner’s
Dilemmas (i.e. for low values of b). Fig. 6.3(b) also shows that the observed effect is even
94
Chapter 6. Social influence in social dilemmas
1.4
(a)
(b)
1.3
b
1.2
D
1.1
C
1.0
0.9
0
0.5
1
K
1.5
2 0
0.5
1
K
1.5
2
Figure 6.3: Full b − K phase diagrams for the Prisoner’s Dilemma game incorporating
aging as a dynamical process. In both panels, α = 2 (to be compared with panel (b) of
Fig. 6.2). (a) results for model II, where aging is dynamical, but follows a deterministic
protocol; (b) results for co-evolutionary model III, where the individual teaching values
wi evolve corresponding to the persistence time of strategies, i.e. individuals who change
their strategy are considered newborn (ei = 0) in the next timestep. Additionally, the
dashed lines indicate the results for a slightly changed scenario, where only 10 % of the
individuals increase their age per time step. To avoid ambiguity, C and D symbols are not
given in panel (b), but the respective regions can be inferred by the line colors according
to panel (a).
more pronounced when the time scales of aging and strategy adaption are separated, i.e.
if the evolution of strategies (or frequency of interactions) is faster than the evolution of
age.
The comparison between models I, II, and III also shows that it is the persistent time
dependent aging mechanism that produces the most remarkable results. This has to be
seen in connection with the results of Chapter 5, where a similar co-evolutionary process
has led to counter-intuitive results within the framework of opinion dynamics. It does
not only show that the aging mechanism developed in Chapter 5 is of even more general
interest as described there, but it is also an indication for the close connection between
the two scientific approaches to social interaction. It denotes a good example for the
usefulness of exchanging and mixing methods of opinion dynamics and game theory in
6.4. The role of imitation in social dilemmas
95
order to obtain general results regarding the dynamics of social interaction.
However, we have to note that the observed results can be explained by understanding
the local dynamics in the spatial system (as done by Szolnoki et al. (2009)). If teaching
occurs between local neighbors, newborn cooperators are guaranteed to have protection
by the profitable interaction with (an)other cooperator(s), namely at least the individual
that taught to cooperate. Due to this protection, the newborns themselves have the
chance to gain teaching abilities and to further spread cooperation. On the other hand,
the same argument decreases the potential success of a newborn defector, also supporting
the effect. Therefore, we have to restrict the conclusion of a promotion of cooperation to
systems incorporating spatial structure.
In the following Section, we will investigate another approach to combine ideas of social
influence models with those of evolutionary game theory, that overcome the aforementioned
restrictions.
6.4
6.4.1
The role of imitation in social dilemmas
Motivation and literature
In our main approach to combine social influence with (evolutionary) game theory, we
assume a population which consists of three subpopulations: (unconditional) cooperators,
(unconditional) defectors, and imitators. It is important to note (and we will see this
below) that the imitators assumed here are not following the so-called “proportional imitation rule” or any other fitness-dependent imitation rule. Our imitators, representing
social influence in the model, are imitators in the plainest form: they deterministically
and with probability one imitate decisions from their interaction partners, irrespective
of the fitness of the imitated individual. This is in contrast to the formerly mentioned
rules, where individuals do only imitate the decisions of successful other individuals. We
denote the frequency of cooperators by x and the frequency of imitators by i. Hence,
the frequency of defectors equals 1 − x − i. Further, we assume a well-mixed population
where two random individuals meet per update step. Within one update step, the two
individuals play m iterations of the game defined by the payoff matrix in Eq. (3.5).
Cooperators apply C and defectors apply D in every iteration of the game. Imitators
take a random choice in the first iteration, and imitate their interaction partner in the
subsequent iterations. If two imitators meet, they will apply the same decision at latest
from iteration 2 on. Here, we assume that it is the nature of imitators to try to apply the
same decision and they will achieve this goal somehow. Such behavior is consistent with
96
Chapter 6. Social influence in social dilemmas
inequality aversion (Fehr and Schmidt, 1999) and egalitarian motives in humans (Dawes
et al., 2007; Fehr et al., 2008). In consequence, only one of the imitators switches its
decision in case they initially applied different decisions. This can also be seen as a
(fast) consensus process taking place in case of different initial decisions. In fact, such a
consensus process happens in every update step, where consensus might be found through
imitational decision update, or might not be found due to the interaction between different,
fully “inertial” individuals (a cooperator and a defector). Here, the connection between
the different perspectives on decision making taken in Chapters 2 and 3) becomes evident.
In a work on “a voter model of the spatial Prisoner’s Dilemma” (Frean and Abraham,
2001a), a very similar setup of imitators is investigated. The similarity between the approaches can be best illustrated by mentioning the few, but important differences. In their
model, they do not consider pure cooperators and defectors, but the respective individuals
cooperate (defect) with a probability of 0.8 and apply the opposite decision with probability 0.2. Besides this, the authors focus on the effects of spatial structure and, therefore, do
not investigate the general dynamical characteristics and their interesting results. Finally,
they restrict their investigations to the Prisoner’s Dilemma, whereas we are interested in
all the different scenarios modeled by symmetrical 2×2 games.
Let us mention more connections to previous work on the evolution of cooperation. On the
one hand, our imitators are similar to tit-for-tat (TFT) players that play a finite number
of prisoner’s dilemma games per interaction with another individual. To illustrate the
mechanism of direct reciprocity, (Taylor and Nowak, 2007) assumed the evolution of a
TFT population together with unconditional defectors. This leads to the payoff matrix
TFT
D
!
TFT
D
mR
T + (m − 1) P
S + (m − 1) P
mP
%
,
(6.14)
where m is the number of games played per interaction and T > R > P > S of the
prisoner’s dilemma. TFT becomes evolutionarily stable against defectors if m R > T +
(m − 1) P , i.e. if
m>
T −P
.
R−P
These TFT players correspond to imitators with the additional trait that they initially
choose the cooperative move C. The imitators in our setup do not have this specific trait.
This is consequent since we assume that these individuals cannot consciously decide for
one of the alternatives. Note that our imitators are therefore less advantaged than TFT
6.4. The role of imitation in social dilemmas
97
players since an encounter of two such individuals may lead to mutual cooperation or
mutual defection equally likely. Furthermore, our mechanism differs from this one in that
it also considers unconditional cooperators. We are interested in the effect of the presence
of simple imitators (herding behavior) on the evolution between cooperators and defectors.
For the TFT strategy, this was done by Imhof et al. (2005). Their work can be summarized
as follows: there are the three subpopulations of cooperators, defectors, and TFT players.
One interaction consists of m rounds of the game and, if m is large enough, domination of
TFT is the only stable fixed point. However, they assume a complexity cost for TFT, which
destabilizes the equilibrium and, thereby, leads to domination of defectors. Based on this,
they introduce mutation (with a small probability, reproduction of one individual leads to
offspring of another type) into the dynamics. Basically, this leads to oscillations between
domination of cooperators, defectors, and TFT players, where the dynamics spends most
of the time in the state where TFT dominates. This shows that even when considering
a complexity cost, the reciprocal behavior of TFT can have evolutionary advantages.
However, such results always come with some open problems: TFT is an artificial strategy
that always cooperates in the beginning of an interaction and then deterministically repeats
the previous action of the interaction partner. It also needs to be deterministic as otherwise
it only receives the average of all four payoff values when interacting with itself.
Taking up this argument, the similar strategy “win-stay-lose-shift” was suggested as more
robust strategy representing reciprocal behavior (Imhof et al., 2007; Nowak and Sigmund,
1993). This strategy repeats its last move if the last payoff was above an aspiration level
(win-stay) and switches to the other move if the payoff was lower (lose-shift). In the
Prisoner’s Dilemma, this usually means that an individual switches after payoff S or P
and repeats its move after payoff R or T . But this only resolves the problem of correcting
erroneous moves when playing against alike. At the same time, it brings along a new
problem: without further assumptions it is dominated by defectors because it cooperates
every second game iteration. Only if selection is weak and the temptation to defect in
the Prisoner’s Dilemma is low (b/c > 3 for the standard cooperation model in Eq. 3.10),
win-stay-lose-shift can be favored by selection.
Note that our imitators, on average, receive the same payoff with or without erroneous
moves, namely (R + P )/2. Therefore, such considerations are pointless in our model.
This fact, together with the results presented in the remainder of this Section, lead us to
the supposition that our imitators might be a more suitable concept for investigations on
reciprocal behavior in evolutionary game theory. In a sense, we would answer the question:
“tit-for-tat or win-stay-lose-shift?” (Imhof et al., 2007) by: imitation.
On the other hand, our mechanism can also be compared to the idea of teaching abilities
98
Chapter 6. Social influence in social dilemmas
of some individuals as considered by Szolnoki and Szabó (2007), which we explained and
extended in the previous Section. There, the teaching abilities are exogenously assumed
and one could argue that this is a quite strong assumption. Furthermore, these results
could only be obtained in spatially structured systems, implying a further assumption. The
analogy between our approach and the “teaching model” by Szolnoki and Szabó is that our
imitation could be framed as consequence of teaching. Then, all individuals in our setting
are teachers, but some are resistant against being taught (unconditional cooperators and
defectors). Imitators are not resistant and learn the decision of the interaction partner.
Therefore, when elaborating on the effect of imitation on evolutionary dynamics below,
we will obtain a natural motivation for a similar mechanism, and more general results at
the same time.
6.4.2
The evolution of cooperation through imitation
In the previous Section, we explained, motivated, and embedded our concept of including
imitators into the evolutionary dynamics between cooperators and defectors. Imitators are
zero confident individuals who copy the decisions of their interaction partners. This might
be due to missing information regarding the payoff structure, due to egalitarian considerations (in the symmetrical games considered here, identical decisions lead to equal payoffs),
or simply due to herding behavior. Therefore, we consider a population consisting of three
species: cooperators, defectors, and imitators. Cooperators and defectors unconditionally
repeat the same decision all the time and with whom ever they interact. Imitators apply
at every interaction the same decision as the interaction partner. If two imitators meet,
they mutually cooperate or defect with equal probability.2 All three species are subject
to fitness dependent selection, which is incorporated by the replicator dynamics in an infinite, well-mixed population. Hence, there is no noise in the system and we can write the
evolution equations for the frequencies of cooperators (x) and imitators (i) by
ẋ = x (πc − π̄),
(6.15)
i̇ = i (πi − π̄).
(6.16)
The frequency of defectors equals 1 − x − i and evolves as −(ẋ + i̇). The average payoff in
the system π̄ is obtained by
2
While we motivated especially the specification of imitators by an infinitely repeated game per interaction in the introduction, we now do not need to assume repeated games any more. Within the model,
we assume a one-shot game per interaction between cooperators, defectors, and imitators as specified in
the text.
6.4. The role of imitation in social dilemmas
99
π̄ = x πc + i πi + (1 − x − i) πd .
As usual, the model is described by the specification of the expected payoffs for an individual of each subpopulation. Assuming m iterations of the game per interaction event,
the payoff matrix is
C
D

C
mR

D
mT
T +R
+ (m − 1) R
I
2
S+P
2
I
R+S
+ (m − 1) R
2
T +P
+ (m − 1) P
2
R+S+T +P
+ (m − 1) R+P
4
2
mS
mP
+ (m − 1) P


.
(6.17)
To keep the analysis as simple as possible, we assume that the number of iterations is
large enough such that the payoff of the first iteration is negligible (m → ∞). Hence, we
can leave m out of the equations by considering the average expected payoff per played
iteration. The resulting payoff matrix
C
D
I
C R

D T
I R
S
P
P

R

P 

(6.18)
R+P
2
leads to the expected payoffs for individuals
πd = x T + (1 − x) P,
(6.19)
πc = (x + i) R + (1 − x − i) S,
i
i
R+ 1−x−
P,
πi =
x+
2
2
(6.20)
(6.21)
dependent on their species. The considered evolutionary dynamics leads to the five distinct
fixed points:
(i) x = 0, i = 0
(ii) x = 1, i = 0
(iii) x = 0, i = 1
(iv) x =
P −S
,
P +R−S−T
i=0
100
Chapter 6. Social influence in social dilemmas
(v) x =
(P −R) (P −S)
,
P 2 +R2 +2 S T −P (S+T )−R (S+T )
2 (P −S) (R−T )
,
P 2 +R2 +2 S T −P (S+T )−R (S+T )
i=
where in all cases the frequency of defectors is d = 1 − x − i.
S = 0, T = 1
S = 0, T = 1
1
0.8
0.8
0.6
0.6
x
x
1
0.4
0.4
0.2
0.2
0
0
2
2
1
1
1
1
0
P
0
−1
(a)
−1
0
R
0
P
R
(b)
Figure 6.4: Equilibrium fraction of cooperators x in the two non-trivial fixed points.
The x-axis and the y-axis are the same as in the two-dimensional Figs. 3.2 and 3.3, i.e.,
together with our general definition of S = 0 and T = 1, they define the respective
game. (a) Fraction of cooperators in the non-trivial fixed point of the evolution between
cooperators and defectors only [fixed point (iv) in the text]. (b) Fraction of cooperators in
the additional, non-trivial fixed point through the inclusion of imitators [fixed point (v) in
the text]. This one only exists for games with 0 < P < R < 1, i.e. only in the Prisoner’s
Dilemma. In this Figure, stability of fixed points is not illustrated.
The first three fixed points are the trivial ones that result from the non-innovative nature
of the replicator equations. The fourth fixed point (iv) we already know from the evolution
between cooperators and defectors, discussed in Section 3.3. Here, if stable, cooperators
and defectors coexist and imitators go extinct. In Fig. 6.4a, we show this fixed point
depending on the game in terms of the equilibrium fraction of cooperators x. This fixed
point exists in two regions of the game space, namely the two regions of bi-stability and
coexistence explained for the reference case around Fig. 3.3. There, the plane of fixed
points in the region P < 0 is stable and leads to coexistence, whereas the plane in the region
R > 1 is unstable and separates the attractor regions of the trivial fixed points. However,
6.4. The role of imitation in social dilemmas
101
the inclusion of imitators into this evolutionary scenario leads to another, non-trivial
fixed point (v), where cooperators, imitators, and defectors may coexist (see Fig. 6.4b).
Interestingly, note that this fixed point does only exist in the Prisoner’s Dilemma region
(0 < P < R < 1), which might be difficult to exactly see in Fig. 6.4b.
Of course, the most interesting question is now: which of the fixed points are stable and,
hence, what are the stable outcomes of this evolutionary dynamics? In general, we apply
the stability analysis introduced in Section 1.3.2, i.e. we build the Jacobian
!
%
∂ ẋ/∂x ∂ ẋ/∂i
J=
∂ i̇/∂x ∂ i̇/∂i
(6.22)
and take the eigenvalues of J at a fixed point as indicator for stability of the particular
fixed point. We computed the two eigenvalues for each fixed point with the help of the
technical computing software “Mathematica”, and found them to be:
(i) λ1 = −P, λ2 = 0
(ii) λ1,2 = 12 (1 − R ± |R − 1|)
(iii) λ1,2 = ± 12 |P − R|
(iv) λ1,2 =
P (R−1)
P +R−1
q
2
−R)2 (R−1)2
(v) λ1,2 = ± − [PP (P(P−1)+R
,
(R−1)]2
where our general assumption S = 0, T = 1 is implemented. As discussed in Section 1.3.1,
the applied linearization method can be problematic for fixed points at the boundary of
the state-space. This becomes apparent in the example of fixed point (i), where λ2 = 0
would imply a set of “neighboring” fixed points, which is not the case here. However, we
can apply a slightly different approach for these three points (i-iii): the concept behind
this eigenvalue-approach is to find the direction of dynamical flows along each dimension
of the system. The dimensions are specified by the eigenvectors that, however, might
only touch the limited state space (see eigenvector 2 in Fig. 6.5 – the illustration of the
state space and eigenvectors). From the known direction of dynamical flows along these
two eigenvectors, we can infer the resulting flow-direction at any other point, which is
sufficiently close to the fixed point, by superpositions of the eigenvectors as described in
Section 1.3.2. Since fixed points (i-iii) are at the corners of the triangular state space,
alternatively we can find out the dynamical flows exactly on the respective edges and
102
Chapter 6. Social influence in social dilemmas
Figure 6.5: Illustration of the state space (triangular simplex) for the dynamics on the
relative frequencies of three species: cooperators x, imitators i, and defectors 1 − x − i.
The dynamical system can only be in a state within the triangle or exactly at its border.
The black circle denotes fixed point (i), i.e. the domination of defectors. Dashed lines are
illustrations of eigenvectors, aimed at supporting the verbal explanation in the text.
apply the same argumentation of superposition vectors for any point between the edges.
The edges replace eigenvectors and the direction of flows replace the signs of eigenvalues.
Finding the direction of flows on the edges is straight-forward: one species (or genotype) is
not there so that we can consider a 2×2 game between the other species. If one species is
ESS against the other (see Section 3.3), the direction of flow close to its domination-corner
(the fixed point where the species under consideration is the only one present) is towards
that corner, i.e. along this dimension of the dynamical system the dynamics flows into
that fixed point. Hence, the particular domination fixed point is stable if the respective
species is bilaterally ESS against both other species. For a better orientation, let us write
the 3 bilateral games:
C
C
D
!
R
T
D
S
P
C
%
C
I
!
R
R
I
R
R+P
2
%
D
I
!
D
I
P
P
P
R+P
2
%
With these guidelines, we are ready to investigate the stability of our fixed points. Since
for fixed point (i) λ2 = 0, we use our ESS argumentation. Defectors are ESS against
cooperators if P > S (applying the first of Maynard Smith’s conditions (Maynard Smith,
1982)). This finding corresponds to λ1 since S = 0, suggesting that this eigenvector indeed
6.4. The role of imitation in social dilemmas
103
runs along this edge. Defectors are ESS against imitators if P > (R + P )/2, i.e. if P > R
(the second of Maynard Smith’s conditions). Therefore, fixed point (i) is a stable fixed
point of our dynamics in and only in games with P > S, R. This first result is already
quite remarkable: one of the fundamental characteristics of social dilemmas is that mutual
cooperation is more profitable than mutual defection, i.e. R > P . The conclusion is that
defectors cannot dominate in any of the social dilemmas. This is a dramatic change as
compared to the reference dynamics between cooperators and defectors (Fig. 3.3). There,
domination of defectors is the unique stable fixed point in the Prisoner’s Dilemma and
part of the bistability in Stag Hunt and Pure Coordination I.
2
1.5
λ1, λ2
1
0.5
0
−0.5
−1
−1
−0.5
0
0.5
1
1.5
2
R
Figure 6.6: Eigenvalues for the Jacobian at fixed point (ii), i.e. at the state where cooperators dominate. The value of λ1 is illustrated by the black line and λ2 by the red one.
They only depend on the payoff value R, provided that S = 0, T = 1.
Fig. 6.6 illustrates the eigenvalues at fixed point (ii), where only cooperators are present in
the population. The immediate conclusion is that dominance of cooperators is not stable
if R < T = 1. This fact is identical to the reference case and also corresponds to the fact
that cooperators are ESS against defectors if R > T . On the other hand, cooperators are
ESS against imitators if R > P . The latter fact is the only difference to the reference
case and only effects Pure Coordination II. Like in the reference case, cooperators can
dominate in Stag Hunt and Pure Coordination I (both are social dilemmas). Moreover,
we will see that domination of cooperators is the unique stable fixed point in these games,
thereby resolving the social dilemmas.
Imitators can never dominate the system, which gets apparent by the eigenvalues of fixed
point (iii). As long as R 6= P , it invariably is a saddle point. This is an interesting feature
104
Chapter 6. Social influence in social dilemmas
of our model: we assume the existence of a species that considerably effects the dynamics,
but itself is under no circumstances able to dominate the system.
Regarding fixed point (iv), the analysis is simple: since this fixed point exists on the
state-space boundary i = 0 and both eigenvalues are identical, we only have to find out
whether the fixed point is stable in the bilateral game between cooperators and defectors.
This means, the stability of this fixed point is identical to the reference case. It is stable
in Chicken, Battle of the Sexes, and Leader, and not stable in Stag Hunt and the Pure
Coordination games. Evaluating the eigenvalues (iv) in the regions where this fixed point
exists (R < 1, P < 0 and R > 1, P > 0, see Fig. 6.4a) yields, of course, the same result:
they are negative in the first case and positive in the latter one.
It remains fixed point (v) in the interior of the triangular simplex. This one does only
exist in the Prisoner’s Dilemma, as is shown in Fig. 6.4b. Let us firstly remark that none
of the other fixed points exhibited any form of stability in the Prisoner’s Dilemma – a fact
that already indicates stability of this last fixed point. Since the eigenvalues are square
roots of a negative number, they are purely imaginary, i.e. they are complex numbers with
a zero real part. As explained in Section 1.3.2, this leads to oscillations in the dynamics
with constant amplitude. For fixed point (v) we can conclude marginal stability, but not
attractiveness. In the triangular simplex of the state space, we can visualize the dynamics
as a trajectory of global states beginning from an initial condition. This is done in Fig. 6.7
for several initial conditions.
The trajectories are computed by numerically integrating the dynamical equations in
Eqs. (6.15) and (6.16).3 As expected, constant oscillations can be seen by the closed orbits
around the interior fixed point. A trajectory from an initial condition circles around the
fixed point and exactly passes by the initial condition. However, this is not true for some
initial conditions where imitators are relatively rare. In such cases (refer to the grey and
red trajectories of Fig. 6.7), the dynamics lead to a common, constant orbit that considerably keeps the system off the boundary i = 0. This fact has important implications for
the dynamics in finite systems, as we will see below.
The dynamics resulting from the evolutionary competition between cooperators, defectors,
and imitators in the Prisoner’s Dilemma are equivalent to the one described for the rockpaper-scissors game by Frean and Abraham (2001b) and complemented by Berr et al.
(2009). Therefore, this quite unexpected result can be understood by the “survival of
the weakest”, stemming from the fact that a species’ frequency does not depend on its
own invasion rate, but on the invasion rate of the species it invades. Complementing this
analysis, let us discuss the reasons for this phenomenon by calculating two quantities:
3
We used the Runge-Kutta method for numerical integration.
6.4. The role of imitation in social dilemmas
105
D
C
I
(a)
D
D
I
C
(b)
I
C
(c)
Figure 6.7: Oscillations of species’ frequencies in three Prisoner’s Dilemma games: (a)
R = 0.8, P = 0.4, (b) R = 0.95, P = 0.05, (c) R = 0.65, P = 0.6. The corners of a
simplex indicate dominance of cooperators (C), defectors (C), and imitators (I), respectively. Circles denote fixed points of the dynamics – a black filling denotes stability of the
fixed point. Shown are dynamical trajectories of different initial conditions. Generally,
the trajectories form constant orbits around the stable fixed point that include the initial
condition. Exceptional cases (red and grey trajectories) are discussed in the text.
Approximately, we can quantify this evolutionary advantage of imitators by computing
the dynamical forces at the boundaries of the simplex. On one hand, we compute the
“invasion speed” along the boundary. For example: at the boundary x = 0, imitators
dominate defectors and hence invade any subpopulation of defectors. The speed of this
invasion is proportional to the payoff difference πi −π̄, where π̄ = i πi +(1−i) πd . Therefore,
it differs for different states of the system i. On the other hand, we compute the “invasion
potential” of the (almost) non-existing species in a dimorph system consisting of the other
two species. For example, the invasion potential of imitators is proportional to the payoff
106
Chapter 6. Social influence in social dilemmas
0.4
cooperators at x + i = 1
defectors at i = 0
imitators at x = 0
0.15
Invasion potential
Invasion speed
0.2
0.1
0.05
0
0
0.2
0.4
0.6
x, i, 1−x
(a)
0.8
1
0.2
cooperators at x = 0
defectors at x + i = 1
imitators at i = 0
0
−0.2
−0.4
0
0.2
0.4
0.6
x, i, 1−x
0.8
1
(b)
Figure 6.8: Illustrations of dynamical forces at the simplex boundary for a Prisoner’s
Dilemma with R = 0.8, P = 0.4. (a): positive difference between the expected payoff
of the bilaterally dominant species and the average payoff in the system – the quantity
that reflects the dynamical speed of invasion. (b): payoff difference between the (potential)
expected payoff of the non-existent species and the average payoff in the system – reflecting
the invasion potential of the non-existent species. In both panels, the abscissa denotes the
frequency of the pairwise dominant (invading) species.
difference πi − π̄, where π̄ = x πc + (1 − x) πd . These quantities are illustrated in Fig. 6.8.
First, notice that cooperators invade imitators at a low rate when the frequency of imitators
is low (solid line in the left panel for x > 0.8). In this region of x, the invasion potential of
defectors turns positive and increases further for increasing x (dash-dot line in the right
panel). Hence, the very small frequency of defectors increases relatively fast as x increases
very slowly. In result, defectors are able to start invading the population considerably
before x = 1. Second, the invasion potential of imitators at the boundary i = 0 is always
close to zero (in the payoff example chosen for Fig. 6.8, it is even strictly non-negative),
i.e. the part of the trajectory that mainly reflects the invasion of defectors does not
approach the i = 0 boundary further, but is rather repelled from it. To summarize: (i)
the invasion of cooperators (that diminishes imitators) is interrupted by the invasion of
defectors considerably before imitators are close to extinction, (ii) in any population where
imitators are rare, the remaining imitators receive enough payoff to at least approximately
keep their share. These two processes prevent imitators from being close to extinction.
We can compare this with the same considerations for the other two invasion processes:
cooperators get closer to extinction because the invasion speed of defectors (dash-dot in
the left panel of Fig. 6.8) is always higher than that of cooperators and the invasion potential of imitators (solid line in the right panel) is relatively low for high values of 1 − x.
Furthermore, the invasion potential of cooperators is negative until i ≈ 0.6, leading to
6.4. The role of imitation in social dilemmas
107
a further decrease of a small rest frequency of cooperators. Defectors also get closer to
extinction because the invasion potential of cooperators is comparable to that of defectors
for high values of i and x, respectively, but the invasion speed of imitators is higher than
that of cooperators in that region. Like the cooperators, defectors also have negative invasion potential for lower values of x, which brings them even closer to extinction.
Let us summarize the stability analysis of our dynamics for any symmetrical 2×2 game.
Since, compared to the reference case depicted in Fig. 3.3, dominance of defectors lost
stability in some games, we do not have a region with bistability anymore. This makes the
color encoded representation (Fig. 6.9) more convenient, because we only have one stable
fixed point per payoff ranking. However, we have to keep in mind that the fixed point for
the Prisoner’s Dilemma (the triangle with 0 < P < R < 1) is not attractive, but we have
oscillating coexistence of all three species there.
Most remarkably, we can summarize the effect of including an imitating species into the
evolutionary dynamics by concluding an immense increase in the level of cooperation in
social dilemmas. Whereas the Chicken game is unaffected by this change, the dilemma
is completely resolved in Stag Hunt and Pure Coordination I. These are the two social
dilemmas where there is no temptation to defect, but there are two Nash equilibria and
the inefficient one (mutual defection) is risk dominant against the efficient equilibrium
(mutual cooperation). For this games, the dynamics possess one unique stable fixed point
in the domination of cooperators, i.e. irrespective of the initial condition, evolutionary
dynamics will exclusively favor cooperators. Reversely, for Pure Coordination II (not a
social dilemma), defectors become the unique stable fixed point, thereby also increasing
the average payoff in the population compared to the reference case.
In the Prisoner’s Dilemma, where there is both a temptation to defect and the risk dominance of the inefficient equilibrium, we find persistent, oscillatory coexistence between
all three species. Therefore, a considerable level of cooperation is preserved both by the
survival of cooperators, but also by the behavior of imitators in the system (compare the
respective part of Fig. 6.9(d)).
Departing from the analytical results for infinite populations, let us note some remarkable
results for finite populations. For the numerical simulations, we initialize a network with N
nodes and assign an individual of a randomly chosen species to each node of the network.
For some simulations, the probabilities for each species to be chosen are equal (1/3), but we
will also investigate differing initial conditions. Depending on the respective population
structure, individuals are connected to other individuals with whom they are able to
interact – let us call them “neighbors”. An individual receives payoff (fitness) from an
108
Chapter 6. Social influence in social dilemmas
S = 0, T = 1
2
1.5
1.5
1
1
P
P
S = 0, T = 1
2
0.5
0.5
0
0
−0.5
−0.5
−1
−1
−0.5
0
0.5
1
1.5
−1
−1
2
−0.5
0
0.5
1
1.5
2
R
R
(a) (unconditional) cooperators
(b) imitators
S = 0, T = 1
S = 0, T = 1
2
2
1.5
1.5
1
1
1
0.9
0.8
0.7
P
P
0.6
0.5
0.5
0.5
0.4
0
0
−0.5
−0.5
0.3
0.2
0.1
−1
−1
−0.5
0
0.5
1
1.5
R
(c) (unconditional) defectors
2
−1
−1
−0.5
0
0.5
1
1.5
2
0
R
(d) cooperative decisions
Figure 6.9: Color encoded equilibrium frequencies in the stable fixed points: (a) cooperators, (b) imitators, (c) defectors, and (d) individuals that apply the cooperative decision
at a time, i.e. cooperators plus imitators that meet a cooperator plus half of the imitators
that meet another imitator (x + x i + i2 /2). Red color indicates a frequency of 1, blue a
frequency of 0. The colorbar of (d) translates colors into frequencies for all panels (a-d).
For the Prisoner’s Dilemma (0 < P < R < 1), the displayed fixed point is not attractive,
but produces stable oscillations around it.
interaction with a randomly chosen neighbor according to the payoff matrix in Eq. (6.18).4
4
If two imitators meet, both receive either R or P , each with probability 1/2. The results essentially
remain unchanged when assuming that interactions occur with every neighbor of the focal individual
6.4. The role of imitation in social dilemmas
109
An evolutionary update step consists of N selection steps that proceed as follows: a random
node of the network is chosen for selection. The resident individual a can remain on this
node or it is replaced by a randomly chosen neighbor b. The probability for replacement
is proportional to the relative fitness difference between a and b:
P (b|a) = max(0,
πb − πa
),
πmax − πmin
(6.23)
where the fitness difference is normalized by the maximal possible difference of fitness.
This selection rule is known as “replicator rule”, because it is the finite size equivalent of
the replicator equation.
At first, let us remain in a well mixed population, i.e. we only switch to a finite number of
individuals, but still every individual can interact with all other individuals (unstructured
or fully connected network). In general, the results depicted in Fig. 6.10 confirm the
analytical results discussed above. The only difference regards the Prisoner’s Dilemma.
There, the finiteness of the system penalizes defectors even more than in infinite systems
– defectors almost always die out. The reason lies in the fact that the amplitude of
oscillations grows over time and the evolution of a single run always ends in domination
of one species.5 This happens by one species eventually dying out, leading to a bilateral,
evolutionary game between the other two species (see matrices in Eq. (6.4.2)). Depending
on which species died out first, one of the other two will dominate the system: imitators if
cooperators vanish first, cooperators if defectors vanish first, and defectors in case imitators
die out first. In agreement with the above discussion of the shape of oscillations in Fig. 6.7,
the latter scenario is not possible in most cases – imitators are of the species that is best
protected against extinction in the dynamics between all three species.
In Fig. 6.11, we show the trajectories of single simulation runs, corroborating this considerations. Each simplex displays one simulation with equal initial conditions – one in
which imitators dominate in the end and one in which only cooperators survived. For this
particular Prisoner’s Dilemma (R = 0.8, P = 0.4), 68.5% of equally initialized simulation
runs ended up in dominance of cooperators and 31.5% in dominance of imitators. Defectors always died out.6 The left simplex additionally displays two extreme initial conditions
in which imitators are very rare. Even in these cases, defectors do not survive. Whereas
imitators approximately keep their share initially, cooperators constantly diminish and
eventually die out first. Therefore, testing initial conditions with x = i = 0.02, i.e. where
(instead of one interaction with a random neighbor).
5
The shown simulation results consider that individuals receive payoff from a random interaction with
one other individual. However, the results basically remain unchanged when considering that an individual
interacts with all other individuals, which avoids stochasticity in the payoff calculation.
6
Percentages from a sample of 200 simulation runs.
110
Chapter 6. Social influence in social dilemmas
S = 0, T = 1
S = 0, T = 1
2
2
1.5
1
P
P
1
0
0.5
0
−0.5
−1
−1
0
1
−1
−1
2
R
0
1
2
R
(a) cooperators
(b) imitators
S = 0, T = 1
2
P
1
0
−1
−1
0
1
2
R
(c) defectors
Figure 6.10: Equilibrium fraction of cooperators x (up left), imitators i (up right), and
defectors, respectively. Values are averaged from 200 sample runs. The system size is
N = 900 individuals. In the Prisoner’s Dilemma (0 < P < R < 1), defectors almost never
survive.
96% of the initial population are defectors, all simulation runs (out of 100 sample runs)
ended in dominance of imitators.
As stated above, the dynamics corresponds to the one investigated by Frean and Abraham
(2001b). Therefore, these results are also in agreement with the findings there, where it
was shown that the finiteness of the system leads to “overshooting” and, eventually, to
the extinction of two species.
However, coexistence of species is preserved in structured populations. In contrast to fully
connected networks, spatially structured and even random networks lead to shrinking os-
6.5. Summary
111
D
D
I
C
(a)
I
C
(b)
Figure 6.11: Growing oscillations in single simulation runs that lead to extinction of 2
species. The networks of individuals are fully connected (well mixed) with system sizes
of N = 104 . The game is a Prisoner’s Dilemma with R = 0.8, P = 0.4 (S = 0, T = 1).
Initial conditions are marked by crosses: besides the two equal initializations, the other
two in the left panel are x = i = 0.02 and x = 0.39, i = 0.01, respectively.
cillations and relaxed, yet fluctuating, frequencies of species. Fig. 6.12 shows an exemplary
time evolution of frequencies in a 2-dimensional, regular lattice. The inset amplifies the
first 500 timesteps, showing the decrease of amplitude in the oscillations. After this short
period, the average frequencies relax to a fix value where they only fluctuate around.
In Fig. 6.13, we compare representative trajectories in random networks and 2-dimensional,
regular lattices.
6.5
Summary
In this Chapter, we have investigated the effects of assuming decision behavior that is
exclusively based on social influence in the dynamics of evolutionary games. In the first two
Sections, we presented two research approaches with which we started our investigations.
In Section 6.1, we modified the network of social interaction: assuming that interaction
can possibly occur between all individuals in the system, at each interaction one individual
is influenced by a fixed number of randomly chosen individuals. Due to the randomness
in choosing interaction partners, this procedure controls the level of stochasticity in the
decisions and, related to the findings of Roca et al. (2006), one could expect that the
evolutionary dynamics might change. However, in the example of replicator dynamics
112
Chapter 6. Social influence in social dilemmas
0.8
0.5
0.7
0.4
0.3
0.6
frequencies
0.2
0.5
0
100
200
300
400
500
0.4
0.3
0.2
0.1
0
1
2
3
time
4
5
4
x 10
Figure 6.12: Representative time evolution of initially equal frequencies of cooperators (red), imitators (magenta), and defectors (blue) in a numerical simulation of a 2dimensional, regular lattice with interactions between 8 nearest neighbors. In contrast
to fully connected networks, oscillations do not grow. In fact, they shrink and lead to a
fluctuating, but qualitatively stable coexistence between cooperators (mostly in majority),
imitators, and defectors (always in minority after a short, initial stage that is amplified in
the inset of the figure). The system size is N = 104 individuals.
between cooperators and defectors in all symmetrical 2×2 games, we showed that the
dynamics are independent of this form of stochasticity and, therefore, identical to assuming
that every individual is influenced by all other individuals (the usual mean-field approach).
In conclusion, we did not use this kind of interaction structure for any other investigation.
In a second approach (Section 6.2), we were interested in the dynamics between individuals
that react to social influence only, but in opposing ways: imitators and contrarians. We
modeled both behaviors by means of the linear voter model and a Fermi function, which
makes the system comparable to the Glauber dynamics. An individual was characterized
by its behavior (being an imitator or a contrarian) and the resulting decision (to cooperate
or to defect). The latter was only the consequence of its behavior and the global frequencies
of decisions. Interestingly, we found that, in all the games, the frequency of decisions to
cooperate in equilibrium is identical to the reference case, i.e. identical to the fraction of
cooperators in the fixed points of the replicator dynamics with cooperators and defectors.
Although remarkable and worth being recorded, such a result implies no new insight into
6.5. Summary
113
D
D
I
C
I
C
(a) random, k = 4
(b) random, k = 8
D
D
I
C
(c) 2-d, k = 4
I
C
(d) 2-d, k = 8
Figure 6.13: Evolutionary trajectories in homogeneous, random networks and 2dimensional, regular lattices. k denotes the number of links every individual has in the network. Every simplex shows one representative simulation run with equal initial conditions.
Panel (a) additionally displays a trajectory for the initial condition x = 0.9, i = 0.02.
the scientific questions addressed in Section 3.3.2: which are the evolutionary advantages
of cooperation that cannot straight-forwardly be explained by evolutionary principles?
Therefore, the focus of this Chapter lies on the results presented in the previous two
Sections. In Section 6.3, we reported and discussed the results of a model, where the
persistence time dependent aging mechanism of Chapter 5 has been implemented into
the fitness dependent “teaching” model of Szolnoki and Szabó (2007). As for the voter
model considerations, individuals are heterogeneous in their decision behavior and this
heterogeneity is endogenously co-evolving in the system. The underlying model here was
an evolutionary, spatial Prisoner’s Dilemma and the individual heterogeneity resulted from
an ability to transfer the own decision to an interaction partner (inside-out influence),
instead of an inertia to copy the decision of the interaction partner (outside-in influence).
However, the motivation of both specifications is still the same: individuals develop a kind
of conviction over time and this conviction tends to increase the abundance of the current
114
Chapter 6. Social influence in social dilemmas
decision in the system. We found that this mechanism leads to an additional positive effect
for the evolution of cooperation as compared to the original teaching model. It could be
shown that this effect profits from the heterogeneity in general, but the important factor
is the specific aging mechanism, which already led to remarkable results in the social
influence models of Chapter 5.
Particular attention should be paid to Section 6.4, where we analyzed another approach
in which the assumptions are intriguingly simple: in the classical evolutionary dynamics
between cooperators and defectors (the reference case of our studies), what is the effect
of the mere presence of imitators, i.e. individuals that are zero-confident and completely
susceptible to positive social influence? The results of this approach are striking:
• In three of the four social dilemmas, cooperation is considerably enhanced. Whereas
the Chicken (or Snowdrift) game is unaffected, this general statement holds for the
prominent Prisoner’s Dilemma game, Stag Hunt, and Pure Coordination I.
• In the games Stag Hunt and Pure Coordination, cooperators invariantly are the
winners of the evolutionary race – the only equilibrium state is characterized by the
dominance of cooperators. The temporary presence of imitators in the dynamics
paved the way for cooperation, thereby resolving the dilemma.
• The Prisoner’s Dilemma:
– In infinite, well mixed populations, the dynamical characteristics assume the
one of the rock-paper-scissors game described by Frean and Abraham (2001b)
and Berr et al. (2009): cooperators, defectors, and imitators coexist persistently. The frequencies in the population oscillate around the marginally stable
fixed point in the interior of the state-space simplex. During these oscillations,
defectors and cooperators get considerably closer to the point of their extinction
than imitators.
– Modifying the replicator dynamics in well mixed populations to finite system
sizes, these oscillations grow in the course of the dynamics, which lead to the
eventual extinction of two species, i.e. a final state with only one species left
in the system. Most interestingly, cooperators have the highest probability to
dominate the population, whereas dominance of defectors is very unlikely.
– In contrast, structured populations lead to damped oscillations, where the frequencies relax at a point close to the interior fixed point. Here, all three subpopulations coexist persistently with a fixed frequency (despite of some fluctuations). Only for extreme initial conditions, finite size fluctuations can lead to
the dominance of one species.
6.5. Summary
115
Here, we assumed the existence of imitational behavior in evolutionary dynamics. In the
implementation of imitation, we avoid further assumptions: an imitator defects with a defector, cooperates with a cooperator, and finds consensus in either cooperation or defection
with another imitator. It is important to note that we did not assume a given existence of
such imitators (in order to support cooperators against defectors), but imitators were subject to the same evolutionary rules as cooperators and defectors. Even though imitators
were only able to dominate under specific circumstances (well mixed, but finite population
in the Prisoner’s Dilemma), in many situations they survived long enough in order to affect
the dynamics considerably. More specifically, they turned the tide in favor of cooperation
in three social dilemmas. With these findings, we conclude a natural explanation for the
evolution of cooperation: the existence of pure (fitness-independent) imitation behavior.
Chapter 7
Dilemmas of partial cooperation
The main focus of this Chapter lies on the problem of partial cooperation that deals
with situations where seemingly unselfish behavior is required by some, but not all involved agents. Using a simple class of game theoretic models – symmetrical 2×2 games
– the subclass of partial cooperation dilemmas is defined and accurately related to the
broad literature on social dilemmas. Moreover, the 3 games belonging to this subclass are
shown to be instances of the standard cooperation model (e.g. in evolutionary biology)
when considering discounting effects of cooperation (according to Hauert et al. (2006),
see also Gardner et al. (2007); Queller (1985)). Consequently, according to Taylor and
Nowak (2007), we apply the evolutionary mechanisms of kin selection, group selection,
and network reciprocity in order to derive the respective conditions under which partial
cooperation can be a stable outcome of evolutionary dynamics in these scenarios.
7.1
Introduction
The emergence and maintenance of cooperative behavior in competitive environments is a
withstanding question in biology, economics, and social sciences, but it also attracts much
attention from mathematicians and physicists. Game theory, founded by Von Neumann
and Morgenstern (1944), has proven to be a powerful tool for describing and investigating
such real-life conflicts. Certainly, one of the most important solution concepts of such conflicts (represented in games) is that of the Nash equilibrium (Nash, 1950) where no player
has an incentive to unilaterally deviate from this state. If there is such an equilibrium solution that is not Pareto efficient, i.e. another solution is better for at least one player of the
game and not worse for any other player, one speaks of a social dilemma situation (Dawes,
1980; Hauert et al., 2006; Macy and Flache, 2002). If more than one equilibrium exists,
116
7.1. Introduction
117
the question is if the players are able to (anti-) coordinate 1 their actions in order to achieve
one of the equilibria and which equilibrium will be selected (Harsanyi and Selten, 1988;
Huyck et al., 1990; Samuelson, 1997). Maynard Smith (1982) extended the framework
by considering evolutionary scenarios and provided the concept of evolutionarily stable
strategies that is closely related to Nash’s equilibrium. The field of evolutionary game
theory (see e.g. Friedman (1991); Hofbauer and Sigmund (1998); Nowak and May (1992);
Szabó and Fáth (2007); Taylor and Jonker (1978)) has since been applied to innumerous
investigations regarding the evolution of cooperation in biology (Maynard Smith, 1982;
Nowak, 2006a; Nowak and Sigmund, 2004), social sciences (Fehr and Fischbacher, 2003;
Henrich et al., 2003), and economics (Gintis, 2005; Kreps et al., 2001). The advantage
of evolutionary considerations is that one can relax the requirements regarding players’
rationality that often is required in “classical” game theory. Note that in this Chapter
we successively apply both classical and evolutionary game theory in order to show the
existence and implications of partial cooperation dilemmas.
Although game theory provides an extensive framework for studying a variety of interdependent decision situations, the simplest class of games is mostly applied to investigations,
namely games with only 2 players and 2 strategies each, where both players have identical
roles (symmetrical 2 × 2 games). In 1967, Rapoport concluded four “archetypes” of such
games (Rapoport, 1967) whereas the others are evaluated to be not of theoretical interest
– a statement that, in this generality, will be subject to critical evaluation in this article.
Among these four games, the Prisoner’s Dilemma attracted most attention in various scientific fields since it incorporates the conflict between individually rational decisions and
collectively desired outcomes. Players have to decide whether to act cooperatively (C) or
to defect (D). If they both cooperate, each receives a payoff R (“reward”) that is better
than what they would receive if both did not cooperate, P (“punishment”, R > P ). If
only one player defects, he receives T (“temptation”), whereas the other player receives S
(“sucker”). The dilemma here is that T > R and P > S, i.e. whatever decision the other
one took, a player is better off if he did not cooperate. Therefore, the only equilibrium
strategy is to defect, yielding the Pareto-deficient outcome P for both. This can be best
1
Although both solutions require a form of coordinated acting of players, game theoretic literature
distinguishes between equilibria with equivalent actions of players (“coordination games”) and equilibria
where players take opposing actions (“anti-coordination games”). In some works (e.g. Browning and
Colman (2004); Neill (2003)), both situations are comprised in the term “coordination” and especially
“coordinated turn-taking” is commonly used to refer to alternating anti-coordination in repeated games.
118
Chapter 7. Dilemmas of partial cooperation
illustrated by the left of the matrices
C
C
D
!
R
T
D
S
P
C
%
C
D
!
D
b − c −c
b
0
%
(7.1)
where, due to symmetry, payoffs are given only for the row-player. A specific configuration
of the Prisoner’s Dilemma is often used as standard cooperation model in evolutionary biology (right matrix in Eq. (7.1)). Compared are 2 species or genetically encoded behaviors
that are subject to evolutionary selection. A behavior is called cooperative if individuals
endow a reproductive fitness benefit b (the evolutionary equivalent of payoff) to other
individuals at a certain fitness cost c to themselves (b > c > 0). The non-cooperative
counter-part is called defective as individuals will receive b from cooperators, but do not
act cooperatively. This parameterization (with or without the Prisoner’s Dilemma condition b > c > 0) is special as it incorporates the “equal-gains-from-switching” property
(R − T = S − P , (Nowak and Sigmund, 1990)), i.e. the individual cost of cooperation
is independent of the other individual’s behavior. Likewise, the individual benefit of (received) cooperation is constant, i.e. an individual, independent of its own behavior, gets
b more payoff from a cooperator than from a defector. Therefore, a cooperative act can
sufficiently be defined as the action that provides a fitness benefit b > 0 to another individual. Evolution of such behavior can easily be explained if cooperation induces a direct
fitness benefit also to the cooperator, i.e. if c < 0. If this is not the case, there must be
an indirect fitness benefit for the cooperator and it is a challenging research task to find
the relevant mechanisms that provide indirect fitness benefits.
In the present Chapter, we consider a more general universe of situations by applying the
left parameterization of Eq. (7.1), i.e. games with equal-gains-from-switching are only a
subgroup of all considered situations. This implies that, in most cases, the fitness effect
caused by a behavior is not static, but depends on the frequencies of behaviors in its
surrounding (see e.g. Nowak and Sigmund (2004)). In consequence, a general definition of
the cooperative act is more difficult, because one and the same act might have opposing
fitness consequences. Mainly for illustrative reasons, the choice in this work (which will
be further discussed below) is to impose T > S. However, it is important to note that
this is a naming convention for the binary choices in the context of this Chapter and not
meant to semantically redefine the notion of cooperation.
We use the different symmetrical 2×2 games as basis for repeated and evolutionary games.
In our argumentation on a specific subclass of games, we show the existence of a cooperation dilemma both in “classical” game theory (with a fixed set of players) as well as in
7.1. Introduction
119
evolutionary game theory (population dynamics). The specific kind of dilemma discussed
here has to be distinguished from the one of the Prisoner’s Dilemma (and other well known
social dilemmas). In particular, we focus on scenarios where partial cooperation plays an
important role. Partial cooperation means that, at a given time, only a part of the individuals apply the cooperative action, whereas the rest of the individuals do not, i.e.
the population of players partially consists of cooperators, and partially of defectors (1
cooperator and 1 defector in the two-person games).
Partial cooperation is, of course, relevant for games where the Nash equilibria are (C, D)
and (D, C). However, in this article, we will focus on whether max(2R, 2P ) > S +
T , or if this inequality is reversed. Verifying this inequality yields the system-optimal
outcome in the sense of Schelling’s collective total (Schelling, 1978), i.e. the solution of
the game that yields the highest possible overall payoff. In a wide range of the literature,
where game theory is applied to research on cooperation, the importance of this system
optimum is neglected and mostly the cases where S + T is system optimal excluded from
the investigations. In our view, thereby, a specific subclass of symmetrical 2×2 games
received too little scientific attention, although these games are shown to exhibit interesting
strategic conflicts in repeated and evolutionary games.
Within this Chapter, we will explicitly derive this subclass of games and introduce it as
“partial cooperation dilemmas” (PCD). The three members of this class are the games
“Route Choice”, “Deadlock”, and “Prisoner’s Dilemma”, but all three games exclusively
with the specification S + T > max(2R, 2P ). In repeated setups of these games, a suitable, Pareto-efficient solution is turn-taking (Browning and Colman, 2004; Duncan, 1972;
Helbing et al., 2005b; Neill, 2003; Stark et al., 2008a; Tanimoto and Sagara, 2007), i.e.
an anti-coordination of the players (where one player takes the opposite decision of the
other) and a permanent flipping between decision alternatives of both players. Despite its
efficiency, this alternating behavior is not an equilibrium state in finitely repeated games,
i.e. players are permanently tempted to leave this solution. Hence, we find a cooperation
dilemma in repeated games. In evolutionary terms, the most successful population of individuals would consist of cooperators (that provide help to others) and defectors (that only
receive help), but such a coexistence is not a fixed point of the evolutionary dynamics.
This constitutes the evolutionary dilemma of cooperation.
In order to clarify the connection to the standard cooperation model (right matrix in
Eq. (7.1)), we show that all 3 partial cooperation dilemmas can be derived out of this
payoff matrix by considering discounting effects of cooperation (Hauert et al., 2006). Furthermore, we investigate known evolutionary concepts (Nowak, 2006b; Taylor and Nowak,
2007) with respect to their ability to sustain partial cooperation. Analytical conditions
120
Chapter 7. Dilemmas of partial cooperation
under which partial cooperation can be a stable outcome of evolutionary dynamics are
presented.
Note that we are not the first to deal with the particular games. However, the games
Deadlock and Route Choice are, with rare exceptions (Helbing et al., 2005b; Kaplan and
Ruffle, 2007; Stark et al., 2008a), almost completely neglected by literature so far. One
reason might be the choice of payoff values and the respective conclusion of Rapoport that
was, for example, applied to investigations regarding the evolution of turn-taking behavior
only in the archetype games (Browning and Colman, 2004), but not to one of the partial
cooperation dilemmas. Most surprisingly, even the third member of this class, the prominent Prisoner’s Dilemma with the specification S + T > 2R, is, despite some exceptions
(e.g. Kreps et al. (2001); Neill (2003); Schüßler (1986)), often explicitly excluded from the
scientific investigations, although it is recognized as equivalent cooperation problem (see
also May (1987)). We conjecture that partial cooperation dilemmas are able to serve as
distinct and relevant models in the different scientific fields applying (evolutionary) game
theory.
The Chapter is organized as follows: in the following Section, we shortly repeat the introduction to the framework of symmetrical 2×2 games and all its ordinally distinct instances
(as more extensively done in Section 3). These games serve us as basis for the argumentation in this work which we start by considering repeated games in Section 7.3. First,
we illuminate a specific form of partial cooperation in repeated games: “turn-taking”.
Then, we discuss what constitutes a social dilemma and name four games as instances of
social dilemmas (largely in agreement with previous literature). Having defined all needed
terminology, we finally name three games that exhibit a “partial cooperation dilemma”
when they are repeated at least once. This (so far not yet concisely specified) kind of
dilemmas can be resolved by partial instead of full cooperation, which clearly separates
them from the well-known and often discussed class of social dilemmas. In this context,
partial cooperation can be realized by turn-taking or by applying an intermediate mixed
strategy.
Turning to evolutionary game theory, Section 7.4 starts by showing how partial cooperation
dilemmas are related to the standard cooperation model in evolutionary biology: they
have in common that the presence of cooperators can improve the fitness of a species
compared to the case of prevailing defectors (constituting a social dilemma). The difference
lies in the fact that, above a certain threshold, the benefit induced by an additional
cooperator decreases with an increasing number of cooperators in the system (“discounting
effects of cooperation” (Hauert et al., 2006)). Considering an evolutionary scenario, a
dilemma even arises in one-shot games. A population that preserves an intermediate level
7.2. Symmetrical 2×2 games
121
of cooperation, e.g. through coexistence between cooperators and defectors, would be
optimal. However, how can cooperators survive at all when their reproductive fitness is
below defectors? In order to shed some light on this question, in Section 7.5 we analytically
derive the conditions under which coexistence can be maintained by the three evolutionary
mechanisms of kin selection, group selection, and network reciprocity (Nowak, 2006b;
Taylor and Nowak, 2007).
Finally, Section 7.6 summarizes and discusses the role of partial cooperation dilemmas
and their relevance. We argue that such situations are likely to occur in reality, but many
solutions to cooperation problems are derived from “classical” social dilemmas and do not
work for partial cooperation dilemmas. Whereas similar issues have been addressed by a
number of publications for specific instances of such games, we here provide a holistic conceptualization of partial cooperation dilemmas. Although we restrict our argumentation
to symmetrical 2×2 games, it can be applied to other classes of games as well.
7.2
Symmetrical 2×2 games
Let us briefly repeat the main facts about the framework of symmetrical 2×2 games as
introduced in Section 3. We have two players with two strategies each and, hence, 4
distinct outcomes from one-shot game. They are represented by the payoff matrix
C
C
D
!
R
T
D
S
P
%
,
(7.2)
where the strategies are named cooperation (C) and defection (D), respectively. In order to
apply these names to all different game situations, we postulated the following definition:
Let us only consider an encounter of different strategies leading to the payoffs S and T
(“partial cooperation”). In such a situation, the strategy which yields the lower payoff is
regarded the cooperative strategy (C), and the other one the defective strategy (D).
We assume that the absolute payoff values are not decisive for the strategic situation, but
only the ranking of them (we will qualify this point later on). Since we defined T > S,
which eliminates equivalent rankings, one can discern 12 ordinally distinct games. Fig. 7.1
conveniently visualizes the phase space of symmetrical 2×2 games in a coordinate system
and includes exemplary payoff matrices. This figure extends the basic representation of
Fig. 3.2 by focusing on system-optimal solutions and the respective differences between
variants of the same (ordinally distinct) game. The whole relevance of the included ex-
122
Chapter 7. Dilemmas of partial cooperation
tensions will become clear below, where Fig. 7.1 will support the discussion on partial
cooperation dilemmas.
Figure 7.1: Classification of symmetrical 2×2 games according to payoff ranking and
system-optimal solutions. Two-dimensionality is achieved by fixing T > S and classifying
ordinal differences only. Parcels separated by solid lines denote different rankings of the
payoff values. The dashed lines divide the whole space in 2 regions according to whether
partial cooperation is system optimal (dark-grey background) or not. Social dilemma
games have a red background color. For each area a respective payoff matrix (in form of
the left matrix in Eq. (7.1), with T > S) is given. Red payoff matrices denote “partial
cooperation dilemmas”.
Each of the 12 rectangular or triangular parcels of the coordinate system (separated by
full lines) host one ordinal payoff ranking. There, we find the prominent games like
the Prisoner’s Dilemma and the Chicken game, which is also often called “Hawk-Dove”
game or “Snowdrift” game. Complemented by Leader and Battle-of-the sexes, these are
7.3. Derivation of PCD in repeated games
123
the four archetypes of Rapoport (1967) (“Martyr”, “Exploiter”, “Hero”, and “Leader”).
Harmony is also referred to as “By-Product Mutualism”. The game of Route Choice
reflects important characteristics of (vehicular- or data-) traffic systems and was named
and experimentally investigated by Helbing et al. (2005b) and Stark et al. (2008a). The
name Own Goal was chosen arbitrarily in order to not leave one parcel empty. The game
is trivial as any deviation from the dominant strategy hurts the deviant most. The names
of the other games are taken from the literature (see e.g. Skyrms (2004) and Szabó and
Fáth (2007)).
7.3
7.3.1
Derivation of PCD in repeated games
Partial cooperation in repeated games
Taking turns, originally describing the sequential form of human conversation (Duncan,
1972), has a considerable impact on repeated games, too. Here, it means that players anticoordinate their actions over time such that both take different decisions, but switch their
decisions in an alternating manner. This is also called “alternating cooperation” (Helbing
et al., 2005b; Stark et al., 2008a), “alternating reciprocity” (Browning and Colman, 2004),
or “ST-reciprocity” (Tanimoto and Sagara, 2007). The games in the dark-grey area of
Fig. 7.1 are the ones with the system-optimal solutions in partial cooperation, i.e. one
of the players profits more than the other. We call this area “turn-taking phase” as, in
repeated games, taking turns would strengthen the relevance of this solutions because of
the fairness with respect to the equal average payoffs (see also Bornstein et al. (1997);
Browning and Colman (2004); Helbing et al. (2005b); Neill (2003); Stark et al. (2008a)).
In games outside the dark-grey region or exactly on the dashed lines, an equal distribution
of payoffs is provided by the system-optimal solution, i.e. a unique strategy leads to
equal and system-optimal payoffs both in one-shot and repeated games. Since there is a
significant difference between games with the same payoff ranking depending on whether
they are within or outside the turn-taking phase, it is important to address them precisely.
For the Prisoner’s Dilemma game with S + T > 2R, the name Turn-Taking Dilemma was
already proposed (Neill, 2003). Accordingly, we will speak of the TT-Chicken, TT-Route
Choice, and the TT-Deadlock for the respective games in the turn-taking phase.
Partial cooperation in repeated games can also mean to apply an interior mixed strategy.
That means a player randomizes its decisions and applies a probability to cooperate.
This allows for any individual level of cooperation, but does not bear the possibility to
anti-coordinate with other players over time. We will refer back to this form of partial
124
Chapter 7. Dilemmas of partial cooperation
cooperation in an example later on.
7.3.2
From social dilemmas to PCD
Strictly relying on Dawes (1980) (“[...] (a) the social payoff to each individual for defecting
behavior is higher than the payoff for cooperating behavior, regardless of what the other
society members do, yet (b) all individuals in the society receive a lower payoff if all
defect than if all cooperate.”), only the Prisoner’s Dilemma constitutes a social dilemma.
However, according to a more recent definition by Macy and Flache (2002), we face a
social dilemma situation if players prefer
• mutual cooperation over mutual defection (R > P ),
• mutual cooperation over unilateral cooperation (R > S),
• mutual cooperation over partial cooperation, even if players would receive the average
payoff (2R > T + S), and
• the defection outcome over the cooperation outcome for at least one given strategy
of the other player, i.e. T > R and/or P > S.
To put it in one sentence: if choosing defection can lead to a Pareto-deficient Nash equilibrium, the game is called a social dilemma since players are either tempted to unilaterally
defect, are afraid of being unilaterally defected on, or even both. In our opinion, condition
2R > T + S is less important for this definition since the fact that players can get stuck
in a Pareto-deficient solution might be sufficient to constitute a dilemma. Let us remark
that in addition to the three games nominated by Macy and Flache (2002) – Prisoner’s
Dilemma, Chicken, and Stag Hunt – also Pure Coordination I belongs into this subclass.
Social dilemmas, i.e. games that fulfill all the above mentioned conditions, are indicated
by a red background color in Fig. 7.1.
In this Chapter, we investigate another type of dilemma that is there only in repeated
games, but not in the one-shot game. In repeated versions of a symmetrical 2×2 game,
the sufficient condition for a dilemma is that, in the underlying one-shot game, there is
a Nash equilibrium that is not system optimal. In addition to the social dilemmas, this
is true for the games TT-Dilemma, TT-Route Choice, and TT-Deadlock, i.e. each game
with S + T > max(2R, 2P ). Whereas this variant of the Prisoner’s Dilemma can also be
seen as a social dilemma, the other two one-shot games do not hold a dilemma since their
Nash equilibria are strict and Pareto efficient. Therefore, in the classification of Rapoport,
the payoff rankings of these two games are assessed “almost trivial”.
7.3. Derivation of PCD in repeated games
125
However, in repeated setups of all the three games, players might take turns in order
to persistently exploit the system optimum while sharing the payoffs evenly among each
other. Of course, such a solution would imply a Pareto-improvement compared to the
persistently played one-shot Nash equilibrium (itself the only Nash equilibrium of the
definitely repeated game), hence the dilemma. This can be best illustrated by the payoff
matrix for a twice played symmetrical 2×2 game:
CC

CD
CC
2R
R+S

CD  R + T R + P

DC  T + R T + S
DD
2T
T +P
DC
DD

S+R
2S
S+T S+P 

.
P +R P +S
P +T
2P
(7.3)
The three games (TT-Dilemma, TT-Route Choice, and TT-Deadlock) have in common
that T > R and P > S. It follows that the solution (DD, DD) is the unique and strict
Nash equilibrium. In contrast to the one-shot game, this strict equilibrium is Paretodominated by the solutions (CD, DC) and (DC, CD), because there both players receive
S + T > 2 P .2
We call this dilemma situation “partial cooperation dilemma” (PCD), because an efficient
solution requires partial cooperation instead of full cooperation. Whereas here we only
use the pure possibility of taking-turns as argument to illustrate the existence of a social
dilemma in repeated games, we refer to other works that investigate how turn-taking can
emerge and be maintained (Browning and Colman, 2004; Helbing et al., 2005b; Kaplan and
Ruffle, 2007; Neill, 2003; Stark et al., 2008a; Tanimoto, 2008). Interestingly enough, turntaking in repeated games of PCD combines the game theoretical problems of cooperation
and anti-coordination (see also Neill (2003)). A similar argument holds for another form
of partial cooperation, namely interior mixed strategies. Although they are not as efficient
as coordinated turn-taking, a Pareto-improvement can still be achieved (the next Section
contains a corresponding quantification). Partial cooperation dilemmas are indicated by
red payoff matrices in Fig. 7.1.
It is worth noticing that, in his analysis of binary choices, Schelling already pointed to
the relevance of the Route Choice game (which he, with respect to the “Tragedy of the
Commons”, named maybe somewhat misleading “The Commons”) and other underrepresented conflicts (Schelling, 1978). However, the relevance of the TT-Route Choice game
2
It is worth noting that the repeated Turn-Taking Dilemma possesses a particularly interesting feature:
it’s Nash equilibrium is twice Pareto-dominated. Hence, there are three solutions of interest: (i) the strict
equilibrium, (ii) mutual cooperation without anti-coordination efforts required, but still Pareto-dominated
by (iii) turn-taking, which is Pareto-efficient, but requires temporal anti-coordination.
126
Chapter 7. Dilemmas of partial cooperation
for scientific investigations has only recently been pointed out by two independent, but
related papers (Helbing et al., 2005b; Kaplan and Ruffle, 2007). Furthermore, there are
only a few publications that explicitly have dealt with the Turn-Taking Dilemma; see
particularly Neill (2003); Schüßler (1986), and, for example, Kreps et al. (2001).
7.4
Derivation of PCD in evolutionary games
So far, we have derived the notion of partial cooperation dilemma games with respect to
their relevance for repeated interactions. In the following, we will argue that this classification is also meaningful in evolutionary game theory. Particularly in evolutionary biology,
the evolution of cooperation under natural selection remains a not fully understood, scientific topic. Here, cooperation means that an individual has a genetic trait that makes
it help another individual at a certain cost (in terms of reproductive fitness) to itself.
In the standard model (see right matrix in Eq. (7.1)), every cooperator induces exactly
the same benefit b, independent of the number of cooperators in the population. Queller
(1985) argued that this strict additivity of benefits is not necessarily realistic and proposed
to implement a synergistic term in the inclusive fitness model of Hamilton (1964), which
is closely related to the model depicted right in Eq. (7.1). This synergistic term accounts
for the possibility that one cooperator induces a benefit of b, but two cooperators together
induce a benefit larger than 2b. In response, Grafen (1984) showed that this synergyterm has no effect if selection is weak. However, this result is restricted to the specific
model applied there. If synergistic effects lead to a change in the payoff ranking of the
underlying game (investigated below), evolutionary dynamics will be affected also in the
limit of weak selection. More recently, the possibility of synergistic or discounting effects
in N -person social dilemmas has been discussed (Hauert et al., 2006), where discounting
is the consequent opposite to synergy. If one cooperator induces a benefit of b, discounting
effects lead to a joint benefit of less than 2b induced by two cooperators. Implementing
this concept into the framework of symmetrical 2×2 games, we obtain
C
D
!
C
D
(1 + w)β − γ
β
β−γ
0
%
.
(7.4)
The parameter w determines whether cooperation has synergistic effects (w > 1), discounting effects (w < 1), or none of both (w = 1). By specifying γ > β = b > 0 and
β − γ = −c, we find the according implementation of the synergy/discounting-concept
7.4. Derivation of PCD in evolutionary games
127
into the standard cooperation model:
C
C
D
!
D
%
w b − c −c
.
b
0
(7.5)
The question is now: what are the different possible scenarios when considering synergistic
and discounting effects of cooperation based on the standard model? For this purpose, let
us systematically vary the parameter w: For w ∈ [(b + c)/2b, (b + c)/b], which includes
w = 1, we regain the traditional Prisoner’s Dilemma game with 2R > S + T . Hence,
to a certain extent, this model covers both synergistic and discounting effects. However,
increasing w above (b + c)/b, the game effectively transforms into a Stag Hunt game.
That means if synergistic effects are strong enough, defection is not anymore a dominant
strategy. In evolutionary terms, we derive a bistable system where both strategies are
evolutionarily stable against each other. Most interestingly, the remaining three possible
games, generated by a discounting factor w < (b + c)/2b, are exclusively the partial
cooperation dilemmas. For w ∈ [c/b, (b + c)/2b], we find a TT-Dilemma. For w ∈ [0, c/b],
the payoff ranking is the one of TT-Deadlock. By the same reasoning, we obtain that
values w < 0 result in the TT-Route Choice game (see arrow 2 in Fig. 7.2).
Fig. 7.2 gives examples of payoff transformations when considering different relations between β and γ. In all scenarios, γ > 0 is respected such that synergistic and discounting
effects of cooperation are considered. One could likewise investigate synergistic and discounting effects of defection, and the according transformations would lead to vertical
arrows in Fig. 7.2. The possibility to draw such straight arrows also show that the scheme
of Fig. 7.1 is integrative in the sense of visualizing relations between the games by spatial
arrangement. For example, the different mutualistic scenarios studied by Bergstrom and
Lachmann (2003) can be found in the parcels below the P = S line (arrow 1).
Let us return to arrow 2 in Fig. 7.2. Note that in all the different cases along this arrow,
the evolutionary problem of cooperation is addressed (helping behavior that induces a
fitness-benefit to the recipient and a fitness-loss to the helping individual), but in different environmental scenarios. Note further that the three partial cooperation dilemmas
possess a dominant strategy, just like the standard cooperation model. When, for example, considering replicator dynamics (strategies that are more successful than average
increase their share in the population, see e.g. Hofbauer and Sigmund (1998) and Taylor
and Jonker (1978)) in infinite, well mixed populations (interactions occur between random
individuals), a stable population would consist of defectors only. However, what makes
these games worth considering besides the standard cooperation model is that too many
128
Chapter 7. Dilemmas of partial cooperation
Figure 7.2: Scheme of symmetrical 2 × 2 games according to Fig. 7.1. A possible position
of the standard cooperation model (right matrix in Eq. (7.1)) is indicated by the black
circle on arrow 2. The dotted arrows 1-4 indicate transformations of the payoff matrices
when increasing the parameter w, i.e. the synergistic/discounting effects of cooperation
in the framework of matrix 7.4. In line with the argumentation of this article, T > S is
always respected.
cooperators may reduce overall fitness. For example, a group (be it in the sense of group
selection, spatial clusters, or similar) consisting of cooperators only is not the most successful group, but one in which cooperators and defectors coexist persistently would have
the highest group fitness. Let us quantify the fitness of a group as the expected payoff π
of a random interaction between two group members:
π = 2 x2 R + 2 x (1 − x) (S + T ) + 2 (1 − x)2 P.
(7.6)
In this equation, x is the frequency of cooperators in the group. If two cooperators
(defectors) meet (which randomly happens with probability x2 ((1 − x)2 )), both receive
7.5. Evolution of partial cooperation
129
payoff R (P ). If a cooperator and a defector meet (probability 2x (1 − x)), one receives S
and the other T . π has its maximum at
x∗ =
2P − (S + T )
,
2[(R + P ) − (S + T )]
(7.7)
where 0 < x∗ < 1, because S + T > max [2R, 2P, (R + P )] by the definition of partial
cooperation dilemmas. Let us remark that the value of x∗ also corresponds to a mutually
optimal mixed strategy, i.e. if every player cooperates with probability to the amount of
x∗ , the outcome is system-optimal and characterized by equal expected payoffs. This is in
contrast to the individually optimal strategy, which is x = 0. For the Prisoner’s Dilemma,
x∗ = 1, i.e. only full cooperation would be mutually optimal. Due to the fact that x∗
is intermediate in partial cooperation dilemmas, the conceptual difference to “classical”
social dilemmas becomes obvious: we do not ask the question how cooperation can achieve
evolutionary stability, but how an “efficient” coexistence of strategies can stabilize.3 In
fact, similar questions are addressed by many researchers seeking for explanations regarding the huge biodiversity (Doebeli et al., 2004; Kerr et al., 2002; Reichenbach et al., 2007)
and variation in cooperation (Kurzban and Houser, 2005) and helping efforts (Field et al.,
2006). The game “rock-paper-scissors”, where three strategies dominate each other in a
cyclic fashion, is then most often used as paradigmatic modeling approach (Czaran et al.,
2002; Kerr et al., 2002; Reichenbach et al., 2007). However, this game requires at least
three strategies and may straight-forwardly promote coexistence (see also Claussen and
Traulsen (2008)). Contrarily, partial cooperation dilemmas could be used to investigate
the emergence of coexistence states where evolutionary dynamics is expected to drive
the system into dominance of only one specific behavior (in line with the considerations
by Imhof et al. (2005)).
7.5
Evolution of partial cooperation
As a first step to investigate the possibility of stable coexistence in partial cooperation
dilemmas, let us consider the “Five Rules for the Evolution of Cooperation” (Nowak,
2006b), i.e. five evolutionary concepts that, under certain circumstances, can effectively
change the strategic situation (the game) compared to a single, binary interaction. The
five concepts are direct and indirect reciprocity (direct: if I help you, you might help me;
indirect: if I help you, someone else might acknowledge it and help me), kin selection (if
I help a genetic relative, I actually help my own genotype to survive), group selection (if
3
Compare to general results in finite systems (Antal and Scheuring, 2006).
130
Chapter 7. Dilemmas of partial cooperation
I help a group member, I strengthen my own group which might protect myself in the
competition with other groups), and network reciprocity (similar to group selection, if I
help my neighbors and we build up a cooperative, spatial community, our neighborhood
(including myself) might be protected against defective invaders).4
In the work of Nowak (2006b), these concepts are valuably simplified by implementing
them into the standard cooperation model. Using the scheme of Fig. 7.1, the specific
transformations for this Prisoner’s Dilemma game have been constructed in the Appendix.
In a subsequent work, these mechanisms have been applied to an arbitrary Prisoner’s
Dilemma game and, among others, the conditions for stability of coexistence between
cooperators and defectors within this payoff ranking derived (Taylor and Nowak, 2007).
It was found that the concepts of kin selection, group selection, and network reciprocity
can lead to stable coexistence if S + T > R + P , i.e. in the discounting region (w < 1).
Direct and indirect reciprocity cannot lead to stable coexistence. But do these results also
hold for discounting factors beyond the Turn-Taking Dilemma?
As derived by Nowak (2006b), we can illustrate the effects of kin selection, group selection,
and network reciprocity on a symmetrical 2×2 game:
C
C
D
!
D
C
(1 + r)R S + rT
T + rS (1 + r)P
C
D
!
%
C
D
!
D
(n + m)R nS + mR
nT + mP (n + m)P
C
D
R
T −H
S+H
P
%
%
,
(7.8)
where r is the relatedness coefficient (mostly defined as probability of two individuals
sharing a gene, i.e. r ∈ [0, 1]), n, m are group size and number of groups, respectively,
and H = [(k + 1)(R − P ) + S − T ]/[(k + 1)(k − 2)], with the degree of the network
k > 2.5 Differently from the other two mechanisms, the role of relatedness can vividly
be understood: inclusive fitness theory (Hamilton, 1964) states that the fate (in terms of
reproductive fitness) of a genetically related species also influences the fate of the species
itself, because the probability of sharing a gene is higher than between unrelated species.
The strength of this influence depends on the level of relatedness r. For calculating the
overall fitness effects for one species, one therefore has to add the fitness of the related
species, scaled by r. This is done in the upper left matrix of Eq. (7.8). In the same spirit,
4
5
In the Discussion of this Chapter, we will comment on criticisms related with these concepts.
Note that these results are obtained in the limit of weak selection.
7.5. Evolution of partial cooperation
131
albeit not likewise comprehensive, the other two concepts are implemented (see Nowak
(2006b) and references therein for details).
Evolutionary dynamics lead to stable coexistence of species if none of the species is evolutionarily stable. The conditions, under which this is fulfilled in the payoff rankings of the
three partial cooperation dilemmas, can be found in Table 7.1, repeating and extending
the results of Taylor and Nowak (2007). If a condition cannot be met in the respective
payoff matrix, this impossibility of stable coexistence is indicated by dashes. For group
selection, this happens because m/(m + n) can only vary between 0 and 1, thereby violating the according (not shown) conditions. At a first glance, this result seems surprising
as a mixed group of cooperators and defectors performs better than a group of defectors.
However, this mechanism bases on individual reproduction and not on the reproduction
of groups. Since, in contrast to the Turn-Taking Dilemma, a cooperator in any group is
less fit than a defector in any group, also higher-level selection favors defection (compare
to Traulsen and Nowak (2006)). For network reciprocity, the condition to be hold is the
same for all three games, i.e. the one displayed for the TT-Dilemma. In networks with
k > 2, FD is always positive in the games TT-Deadlock and TT-Route Choice, because
S + T > R + P and P > max(R, S), thereby violating the condition. This result is rather
intuitive as, in contrast to the TT-Dilemma, a cluster of cooperators performs worse than
a cluster of defectors.
Only in kin selection, there is a range of r leading to stable coexistence in all three games of
partial cooperation dilemmas (direct and indirect reciprocity are left out of the discussion
because in none of the scenarios stable coexistence of strategies can emerge).
TT-Dilemma
P −S
T −P
KS
GS
NR
P −S
R−S
TT-Deadlock
−R
< r < TR−S
m
−R
< m+n
< TT −P
FD < 0 < FC
TT-Route Choice
r>
P −S
T −P
–
–
–
–
Table 7.1: Conditions for stable coexistence of strategies in the three partial cooperation
dilemma games for kin selection (KS, with relatedness r), group selection (GS, with group
size n and number of groups m), and network reciprocity (NR, where FD = k 2 (P − S) −
k (R − S) + S + T − R − P, FC = k 2 (P − S) − k (R − S) + S + T − R − P and k > 2 denotes
the number of neighbors per individual in the network). Direct and indirect reciprocity
cannot lead to stable coexistence.
132
7.6
Chapter 7. Dilemmas of partial cooperation
Discussion
Although symmetrical 2×2 games are very simplistic interaction models, some of them
attracted considerable attention in various fields of science. The reason lies in the fact
that it is possible to model paradigmatic scenarios of interaction between social individuals,
economic actors, biological species, etc.. In the classification of such games, we explicitly
distinguish between games that have their system optimal solution in the main-diagonal
of the payoff matrix and games where a solution in the off-diagonal, i.e. an outcome of
partial cooperation, leads to the highest overall payoff. In the latter case (“turn-taking
phase”), payoffs are different for the involved individuals and turn-taking would be an
advantageous behavior in repeated games. In four of the 12 ordinally distinct games, both
cases are possible.
Related to the well known subclass of social dilemmas, we speak of a dilemma in repeated
games if equilibrium play might lead to a solution that is not system optimal. Among the
symmetrical 2 × 2 games, this is additionally true for variants of the Prisoner’s Dilemma,
the Route-Choice game, and Deadlock that lie within the “turn-taking phase”. We call
them “partial cooperation dilemmas”, because these games bear a dilemma situation both
in repeated and evolutionary games that can only be resolved by partial cooperation.
In repeated games, partial cooperation might be realized by coordinated turn-taking or
the application of intermediate mixed strategies. Both variants are forms of (partial)
cooperation that yield a payoff improvement for both players (compared to the strict
Nash equilibrium in definitely repeated games).
In many works on symmetrical 2×2 games, the condition R > P is applied to name
one of the binary choices “cooperation” and the other one “defection”. We used the
differing condition T > S, because it allows to illustrate the relations between neighboring
games in Fig. 7.2 by different strengths of synergistic or discounting effects of cooperation.
However, it is important to note that both approaches can only fulfill the task to apply
the commonly accepted Prisoner’s Dilemma notation to the whole of symmetrical 2×2
games. The scientific term “cooperation” is defined as helping act, i.e. an individual
behavior that induces a fitness benefit to another individual. This definition is clear and
unproblematic when considering frequency independent fitness effects like in the example
of the right matrix in Eq. (7.1): if the fitness effect of my strategy on another individual
(b) is positive, I am regarded as cooperator. If, however, the fitness effect of my strategy
is frequency dependent, one and the same action can lead to a positive or a negative
fitness effect for the other. Hence, a clearly stated adaptation of the term cooperation is
useful because it avoids a confusing naming of strategies conditional on frequencies in the
population. In the games below the diagonal of Fig. 7.1, which include all social dilemma
7.6. Discussion
133
games, the conditions R > P and T > S coincide. In the other games, including the
partial cooperation dilemmas Route Choice and Deadlock, the strategy naming is inverted.
Especially with respect to the PCD, we also prefer using T > S: in view of the system
optimal, polymorphic population, we rather would evaluate the behavior as “cooperative”
that ignores its potential gain from switching for the benefit of the community (instead of
the one which actually is the receiver of a cooperative act).
In evolutionary biology, where individual payoff gains contribute to the reproductive fitness of a species, another form of partial cooperation plays an important role: stable
coexistence of cooperative and non-cooperative strategies. A species that maintains such
a coexistence (think of different roles in ant colonies or the differentiation in eukaryotic
microorganisms and similar forms of cooperation, see e.g. Wingreen and Levin (2006))
might be advantageous, but it remains a challenge for evolutionary biologists to completely understand how such forms are protected against “cheating”, i.e. other organisms
that profit from cooperation but contribute less cooperation themselves. Therefore, in
evolutionary game theory, partial cooperation dilemmas are even relevant when considering one-shot games, i.e. interactions without the possibility of turn-taking or similar,
memory dependent strategies.
Whereas instances of partial cooperation dilemmas have been discussed in previous works
(Helbing et al., 2005b; Kaplan and Ruffle, 2007; Kreps et al., 2001; Neill, 2003; Schüßler,
1986; Stark et al., 2008a), we here provide a concise conceptualization of the general kind of
dilemma. For symmetrical 2×2 games, we show that partial cooperation dilemma games
translate to the standard model of biological cooperation when considering discounting
effects of helping efforts. Consequently, we derive the conditions for stable coexistence
dependent on the strength of evolutionary mechanisms at work, thereby complementing
recent findings (Taylor and Nowak, 2007). We are convinced that there is room for new
thoughts on realistic mechanisms that are able to explain diversity in a wide range of
evolutionary scenarios, especially in partial cooperation dilemmas.
Some concepts that are applied in this work have been and still are subject to a scientific
dispute. In particular, the multi-level selection approach (Traulsen and Nowak (2006),
leading to the group selection concept) and and the role of spatial structure (leading to
the network reciprocity concept) is sometimes proposed to be identical to kin selection (see
particularly Lehmann and Keller (2006), West et al. (2007) and references therein). In our
view, it is semantically rather productive than misleading to distinguish between different
sources of indirect fitness benefits. In situations where differences in genetical relatedness
can be cancelled out, the mechanism to explain why cooperation is selected for should
not be “kin selection”. Apart from this semantic argument, in this work it quantitatively
134
Chapter 7. Dilemmas of partial cooperation
proved valid to distinguish between the concepts: as shown above and in the Appendix,
they transform the dilemma in different ways and lead to differing results. As stated above,
a major source of discrepancy seems to be the choice of the underlying model: inclusive
fitness theory assumes the costs end benefits of cooperation to be frequency independent,
whereas other approaches, like the present one, emphasize frequency dependence.6
Although widely neglected by the literature so far, the games exhibiting a partial cooperation dilemma could widen the range of models describing complex scenarios of reality
without increasing the complexity of the model (they base on a simple symmetrical 2×2
game). Our conjecture is that applying the idea of partial cooperation dilemmas into
respective models and experimental setups will lead to new and relevant insight regarding the evolution of cooperation in biological systems and human society. In particular,
we reckon advances in investigations on the huge biodiversity and heterogeneity of social
behaviors in nature.
6
Although the considerations in the Appendix base on a frequency independent model, the investigated
transformations explicitly depart from this characteristics.
Part III
Conclusion
In the present thesis, we have investigated different models of social interaction. These
models are either based on the field of opinion dynamics, the field of (evolutionary) game
theory, or they explicitly use approaches from both fields. Besides presenting the results
of particular models, we constantly discuss the differences, but more importantly the large
common ground of decisions based on social influence (opinion dynamics) and in strategic
decision situations (game theory). The main difference can be determined in the focus
of both approaches: in evolutionary game theory, the most important question is which
of the qualitatively different behaviors (strategies) is present in a stable population of
individuals and what are the consequences for the payoff or fitness of this population?
Especially in investigations on the emergence and maintenance of cooperation, one seeks
for explanations of successful cooperation in real social and biological scenarios. Without further extensions, evolutionary dynamics favors defectors and cooperation cannot be
found in stable populations. Hence, the focus in these kind of models lies on evolutionary
mechanisms that can explain the evolutionary advantage of a particular behavior, namely
being cooperative. In contrast, models of opinion dynamics do not have a measure of individual payoff. The focus here lies on whether and why a system reaches an ordered state
(consensus) and how the dynamics in the system can be understood. The absence of payoff considerations shifts the focus to the intrinsic dynamical properties of the interaction
system, which are often non-linear and sometimes lead to unexpected phenomena. Apart
from this difference, both research fields contribute to the understanding of social interaction systems. It is one aim of this thesis to combine both fields in an explicit manner,
i.e. to apply model constituents that are well motivated in opinion dynamics to models
of evolutionary game theory. After introducing models of social influence in Chapter 2
and models of strategic decision situations in Chapter 3, we discussed this ambition of the
present work in Chapter 4.
Starting with opinion dynamics, we have investigated an extension of the voter model in
Chapter 5. Instead of fully symmetrical decisions (opinions), we assumed that individuals
distinguish between their own current decision and the other one. Dependent on the
persistence time of their current decision (the time since they changed their decision for the
136
last time), individuals have an inertia to change their decision – the longer an individual
has applied its current decision, the less likely it will change. Such an inertia can for
example be explained by a conviction that builds up through discussions. If an individual
has discussed a certain topic many times without changing its view on this topic, the
more it gets convinced of its own view. Of course, this invariably leads to decreased
probabilities of individuals to change their decision and we expected this mechanism to
hinder the consensus process in the observed systems. However, we found the effect to be
non-monotonous and partially counter-intuitive. If the individual inertia grows slowly with
persistence time, the average time to reach consensus in the systems gets lower compared to
the original voter model, i.e. the ordered state is reached faster. This means that systems
with a higher level of inertia exhibit faster instead of hindered ordering dynamics. Only for
a faster growth of inertia with persistence time, this effect is reversed and the average time
to consensus is larger in systems with a higher level of inertia. We show that the origin
of this phenomenon lies in the coevolution between inertia and decisions in the system, as
consensus times invariably increase when assuming homogeneous, or static, heterogeneous
inertia. These results are particularly important as they are neither restricted to the voter
model nor to the particular definition of the persistence time dependence. Therefore,
we have found a general example of an emergent phenomenon, resulting from non-linear
dynamics.
In one approach to combine opinion dynamics and game theory, we applied this persistencetime dependent mechanism to the dynamics of a spatial Prisoner’s Dilemma with heterogeneous teaching abilities of individuals. Teaching means that certain individuals have a
probability to spread their own decision which lies above the probability stemming from
the original game dynamics. They are more influential than others and initially these
teaching abilities are equally distributed over cooperators and defectors. If this heterogeneity of teaching abilities is constant, it has been shown that this promotes the evolution
of cooperation in the spatial Prisoner’s Dilemma. We here let these teaching abilities, in
accordance to inertia in the voter model, grow with persistence time of decisions. As in evolutionary dynamics the persistence time of a decision often is translated into the life-time
of an individual (which has a certain decision hard-coded in its genes), we speak of teaching abilities depending on the age of individuals. We find that this mechanism induces
a considerable, additional promotion of cooperation in the spatial Prisoner’s Dilemma.
Although we found the reasons for this effect to stem from the spatial structure of the
system, and we try to have a much more general scope within this thesis, these results still
fit very well into the argumentation of this work: they provide a straight-forward example
of applying a mechanism explicitly derived in one framework (opinion dynamics) to the
methodology of the other (evolutionary game theory). This was possible without much
137
difficulty in motivating the approach and has led to relevant new results.
In a second approach, we investigated a model which is far more general. The idea is
to apply a fundamental mechanism of opinion dynamics to one of the most fundamental
frameworks of evolutionary game theory: we implement pure (i.e. payoff independent)
imitation behavior into the replicator dynamics between cooperators and defectors. In the
main part of these investigations, the only further assumption is that interactions occur repeatedly in order to allow for imitation, but notably our analytical results hold for infinite,
well-mixed populations, i.e. without assuming spatial structure and without finite-size effects. The population consists of pure cooperators, pure defectors, and pure imitators.
An imitator defects with a defector, cooperates with a cooperator, and finds consensus in
either cooperation or defection with another imitator. In our view, this setup implements
imitational behavior with the lowest possible level of assumptions. Computing the stable
populations that result from this dynamics for the whole of symmetrical 2×2 games, we
find striking consequences for the evolutionary dynamics in 3 of 4 social dilemmas. Imitators act as catalyst for cooperators, thereby completely resolving the dilemma in the
games Stag Hunt and Pure Coordination I. In the Prisoner’s Dilemma, where the temptation to defect is particularly strong, dominance of defectors is not possible if imitators
are present in the system. Imitators, although they are themselves not able to dominate
in any of the scenarios, are successful enough to not face extinction, thereby preventing
the victory of defectors. Due to the imitators, cooperators also do not die out and the
final population is characterized by coexistence of all three types. Simulations in finite-size
systems and structured populations emphasize these general results: comparing the evolutionary success of all three types in all scenarios, defectors did not only lose dominance,
but they turned into the weakest sub-population if imitation was present. Especially due
to the absence of strong assumptions in our model setup, we conclude that imitation is
a very natural and powerful mechanism to support the evolution of cooperation in social
dilemmas.
In addition to the relevance of these findings for research on the evolution of cooperation,
we want to emphasize their importance for our discussion of different perspectives on social
interaction. The simple combination of fundamental mechanisms from the fields of opinion
dynamics and evolutionary game theory proved to be relevant for a highly interdisciplinary
research topic. We are convinced that both social influence and strategic decision making
determine the dynamics of social interaction and that, therefore, further attempts to unify
knowledge from both fields into one framework denotes a promising field of future research.
Such further research, especially when trying to model real strategic decision situations
by symmetrical 2×2 games, should also carefully identify the best suited model, instead
138
of just applying the most prominent ones (e.g. the Prisoner’s Dilemma). Throughout the
respective parts of this thesis, we always investigated the whole range of models comprised
in the class of symmetrical 2×2 games, which is however not common practice. In the
final Chapter 7, we conceptualize the framework of “partial cooperation dilemmas” as a
distinct subclass of symmetrical 2×2 games. These games exhibit a cooperation dilemma in
repeated and evolutionary games, but not in a once played (one-shot) game. We separately
derive the dilemma situation for repeated and evolutionary games and argue, that this kind
of dilemma may exist in reality as likely as the well recognized social dilemmas (it is in most
cases not possible to exactly determine the payoff structure of real situations). However,
scientific applications of such models are very rare and especially a precise embedding of
this concept into the literature has been lacking so far. This argumentation also highlights
many specific connections between the mainly economically motivated field of game theory
and social and biological evolutionary dynamics.
The present thesis contains two dimensions of content: (i) contribution to the understanding of particular dynamic models and their results in the fields of opinion dynamics
and evolutionary game theory and (ii) a more conceptual argumentation on the modeling aspects themselves and contributing to bridging the gap between the two different
fields. Both by means of efforts dedicated and concision of results, dimension (i) is surely
more pronounced and deserves particular attention (achievements and their conlusions
have been summarized above). For dimension (ii), it was the aim to not only comment
on commonalities and differences between the two research fields, but to explain why we
consider them as being two perspectives on one and the same matter: models of social
interaction. In conclusion we can say that the perspectives differ in their focus – while
in game theory the focus lies on the rationality or the evolutionary success of different
behaviors, in opinion dynamics one does not assume an evolutionary competition between
different behaviors, but one tries to fully understand the dynamical properties of one or
more particular interaction behaviors under certain circumstances. Although focused on
“language competition”, the work of Castellò et al. (2006) constitutes a good example
to distinguish between the perspectives: while in game theory one would be interested in
which language survives due to the evolutionary advantages or disadvantages a specific
language causes for its speaker, the authors in this work assume symmetric languages,
i.e. every language causes the same effects for their speakers. They are interested in the
resulting dominance or coexistence of languages under the assumption of a certain decision
behavior of the individuals, which itself is not subject to competition. The same differentiation can be illustrated by comparing the voter model with the evolutionary dynamics in
a simple coordination game, as has been done around Fig. 4.1. In the voter model, pure,
success independent imitation behavior is investigated without questioning the rational-
139
ity of this behavior. In evolutionary game theory, one could assume that individuals get
rewarded for beeing alike with their interaction partner(s), and one would find out that
pure imitation is less successful than proportional imitation, i.e. conditional imitation
depending on the success of the observed behavior. Despite these differences, the modeling frameworks of both fields are actually identical, which allows to easily recombine
approaches and findings between both fields. Within this thesis, we explicitly followed
this idea of recombination by applying the co-evolutionary rule of Chapter 5 to the spatial
Prisoner’s Dilemma (Section 6.3), exclusively investigated behavior through social influence in evolutionary games (Section 6.2), and exposed the purely imitational behavior of
the voter model to evolutionary dynamics in social dilemmas (Section 6.4). The most
remarkable conclusion of these approaches is surely the survivability of purely imitational
behavior in social dilemmas and their striking effect on the evolution of cooperation. Due
to the discussions and results in this thesis, we would expect further attempts to explicitly
recombine modeling approaches from both fields to yield further interesting and relevant
results.
Appendix: Transformations of the
standard cooperation model
Referring to Nowak (2006b), there are (at least) “Five rules for the Evolution of Cooperation”. Assuming the standard cooperation model of matrix (3.10), Nowak and colleagues
implemented 5 evolutionary mechanisms in the Prisoner’s Dilemma payoff matrix and
derived the conditions under which cooperation can become evolutionarily stable against
defection. It is important to note that this is only possible if the effective payoff matrix
gets transformed. In the Prisoner’s Dilemma game, cooperation is under no circumstances
evolutionaily stable against defection. When implementing the differrent mechanisms to
the underlying payoff matrix of the Prisoner’s Dilemma, the resulting payoff matrix does
not necessarily reflect a Prisoner’s Dilemma game anymore. If it does (or in the parameter
regions where it does), the mechanism has no effect on the dynamics.
In order to obtain a better conceivability of the mechanisms and how they work, let
us explicitly name the transformations induced by applying the five mechanisms to the
standard cooperation model. If the necessary conditions on the parameters are fulfilled,
the PD game is transformed into:
• Harmony I for kin selection
• Stag Hunt for direct and indirect reciprocity
• network reciprocity:
– Deadlock if b < H
– Harmony I if b > H and (b + c)/2 > H
– Harmony II, if b > H and (b + c)/2 < H
• group selection:
140
(H = [(b − c)k − 2c]/[(k + 1)(k − 2)])
141
– Harmony I if 2b/(b + c) < 1 + n/m < b/c
– Harmony II if 1 + n/m < 2b/(b + c),
where k is the average number of neighbors per individual in the network, n is the maximum group size and m is the number of groups (see Nowak (2006b) for details). With
these transitions, the evolutionary dynamics changes from a system with one stable fixed
point at 100% defectors (PD) to a bistable system with two fixed points at 100% of either
cooperators or defectors (Stag Hunt) or a system with 100% cooperators in the only fixed
point (Harmony I).
List of Tables
7.1
Conditions for stable coexistence of strategies in the three partial cooperation dilemma games for kin selection (KS, with relatedness r), group
selection (GS, with group size n and number of groups m), and network
reciprocity (NR, where FD = k 2 (P − S) − k (R − S) + S + T − R − P, FC =
k 2 (P − S) − k (R − S) + S + T − R − P and k > 2 denotes the number
of neighbors per individual in the network). Direct and indirect reciprocity
cannot lead to stable coexistence. . . . . . . . . . . . . . . . . . . . . . . . 131
142
List of Figures
1.1
2.1
3.1
3.2
The dynamical function of Eq. (1.1) dependent on the state x. Arrows
indicate the direction of the dynamics in the vicinity of the three fixed points.
9
Exemplary time evolution of the voter model on a 2-dimensional, regular
lattice, where every voter interacts with its 4 nearest neighbors. The system
consists of N = 10, 000 voters and the evolution time t is measured in
generations, i.e. 10,000 single updates correspond to 1 generation. . . . . .
21
Presentation of the rock-paper-scissors game in extensive form (top) and
in normal form (bottom). In the upper picture, big, black circles represent
“decision nodes” where the respective player has to decide for one of its
alternatives. The dashed ellipse indicates an “information set”: a player
does not know which of the decision nodes within one information set is
its current one. Therefore, the example game is a simultaneous one. The
small, black circles at the ends of branches indicate the possible outcomes
of the game and the payoffs are given in brackets: the first one for player 1
and the second one for player two. The lower picture contains exactly the
same information about the game in form of a payoff bimatrix. . . . . . . .
26
Classification of symmetrical 2×2 games according to payoff rankings. For
each area a respective payoff matrix (in the form of matrix (3.5), with T >
S) is given. Nash equilibria are marked by bold payoff numbers. Parcels
separated by solid lines denote different rankings of the payoff values. Twodimensionality is achieved by fixing T > S and classifying ordinal differences
only. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
34
143
144
3.3
3.4
4.1
5.1
List of Figures
Analytically computed equilibrium fraction of cooperators according to
replicator dynamics between cooperators and defectors in an infinite, wellmixed population. In case of bistability (R > T ∩ P > S), the stable
fixed point with the bigger attractor region is displayed. Every R, P coordinate constitutes one specific payoff matrix (a game) with fixed values
of S = 0, T = 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
43
Simulation results: equilibrium fraction of cooperators after the evolution
of cooperators and defectors in a system of 400 agents. Considered are
different population structures: (a) fully connected networks; (b) random
networks with 8 interactions per agent; (c) 2-dimensional, regular lattices
with 4 neighbors and (d) 8 neighbors, respectively. Compared to Fig. 3.3,
the most interesting region around P = S = 0, R = T = 1 is magnified. . .
44
Simulation results of two systems with N = 100 individuals, where every
individual is connected to all other individuals (fully connected network).
The magenta colored symbols display the results of the voter model dynamics (as explained in the text, without selection pressure). The blue symbols
represent results from the replicator dynamics in the coordination problem,
i.e. in a game with R = P = 1, S = T = 0. Circles show the development
of the absolute magnetization in the system, a measure for the homogeneity
of opinions in the system, where the value zero indicates equal frequencies
and the value one consensus (M = |1 − 2x|, if x is the relative frequency
of one opinion). Crosses denote the “wealth” in each system in terms of
the expected payoff of one individual when playing the coordination problem game with a random other individual. Values are averaged over 10,000
simulation runs for each system. . . . . . . . . . . . . . . . . . . . . . . . .
49
Left: Average time to consensus Tκ in the voter model with a fixed and
homogeneous inertia value ν0 = ν. The line corresponds to the theoretical
prediction Tκ (ν) = Tκ (ν = 0)/(1 − ν). Details are given in the text. Right:
comparison of the development of the average interface density ρ in the
voter model and the model with fixed inertia. Right, inset: collapse of
the curves when the time scale is rescaled according to t → t/(1 − ν). In
both panels, the simulations results stem from averaging over 500 sample
runs, where the system size is N = 30 × 30 and the voters are placed on a
two-dimensional, regular lattice. . . . . . . . . . . . . . . . . . . . . . . .
61
List of Figures
5.2
5.3
5.4
5.5
5.6
5.7
145
Average consensus times in simulations of the voter model embedded into
fully connected networks. (a) Surface of average consensus times when
varying both the non-zero inertia value ν and the frequency of voters holding
the non-zero inertia value. The system size is N = 200 and averages are
obtained from 4000 simulation runs. (b) Here, the inertia values are not
binary, but continuous between 0 and the maximum inertia value and all the
voters have a non-zero inertia. Results are presented for different system
sizes N and the number of simulation runs varies between 10000 for the
smallest system size and 2000 for the largest system size. . . . . . . . . . .
62
Average time to reach consensus Tκ as a function of the maximum inertia
value ν. Panels (a)-(d) show the results for different system sizes in one-,
two-, three-, and four-dimensional regular lattices, respectively. The results
are averaged over 104 realizations. The system sizes for the different panels
are the following: (a) N = 50 (◦), N = 100 (△), N = 500 (); (b) N = 302
(◦), N = 502 (△), N = 702 (); (c) N = 103 (◦), N = 153 (△), N = 183
(); (d) N = 44 (◦), N = 54 (△), N = 74 (). . . . . . . . . . . . . . . . .
63
Average time to reach consensus Tκ as a function of the maximum inertia
value ν in small world networks (see text for details). The symbols represent
different rewiring probabilities ω, starting with a 2-dimensional, regular
lattice (ω = 0). The curves corresponds to ω = 0 (◦), ω = 0.03 (△),
ω = 0.1 (), and ω = 0.9 (♦). System size is N = 302 and results are
averaged over 104 realizations. . . . . . . . . . . . . . . . . . . . . . . . . .
64
Average time to reach consensus Tκ as a function of the maximum inertia
value ν in fully connected networks of different size. Results are averaged
over 104 realizations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
65
Illustration of the four fractions aτ and bτ and the possible transitions of a
voter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
66
Average time to reach consensus Tκ as a function of the maximum inertia
value ν in fully connected networks of different size. Symbols show the
simulation results for different system sizes (averaged over 104 realizations),
lines the results of the theoretical estimation (explained in the text). . . . .
69
146
5.8
List of Figures
Average consensus times Tκ for varying values of the inertia slope µ and fixed
saturation value νs = 0.9. Sample sizes vary between 103 − 104 simulation
runs. Filled, black symbols always indicate the values of Tκ at µ = 0. (a)
2d regular lattices (ki = 4) with system sizes N = 100 (◦), N = 400 (△),
N = 900 (). The inset shows how consensus time scales with system size
in regular lattices at µ = µ∗ for d = 1 (⋄), d = 2 (×), d = 3 (⊳), d = 4
(⋆). (b) Small-world networks obtained by randomly rewiring a 2d regular
lattice with probability: (◦) pr = 0, (△) pr = 0.001, () pr = 0.01, (⋄)
pr = 0.1, (⋆) pr = 1. The system size is N = 900. (c) Fully connected
networks (mean field case, ki = N − 1) with system sizes N = 100 (◦),
N = 900 (), N = 2500 (⋄), N = 104 (⋆). Lines represent the numerical
solutions of Eqs. (5.18), (5.19), (5.20) with the specifications in the text.
The inset shows the collapse of the simulation curves by scaling µ and Tκ
as explained in the text. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
70
General illustration of the fractions aτ and bτ and the possible transitions
of a voter. The fractions aT and bT contain all voters with a persistence
time τi ≥ τs , i.e. voters with maximal inertia. . . . . . . . . . . . . . . . .
72
5.10 Exemplary time evolution of the inertial voter model with multiple inertia
states and µ = 0.1, νs = 0.9. Voters are positioned on a 2-dimensional,
regular lattice, where every voter interacts with its 4 nearest neighbors. The
system consists of N = 10, 000 voters and the evolution time t is measured
in generations, i.e. 10,000 single updates correspond to 1 generation. . . . .
79
5.11 Comparison of the logistic decision function with the one of the linear voter
model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
80
5.12 Order-disorder phase transition for transition probabilities following a Fermi
function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
82
5.13 Numerical simulations of an N = 900, fully connected network. . . . . . . .
82
Promotion of cooperation due to the increasing heterogeneity in the ei → wi
mapping via α. Stationary fraction of cooperators xs is plotted in dependence on b for α = 0 (solid red line), α = 1 (dashed red line) and α = 2
(dotted blue line). In all three cases K = 1. . . . . . . . . . . . . . . . . .
92
5.9
6.1
List of Figures
6.2
6.3
147
Full b − K phase diagrams for the Prisoner’s Dilemma game with quenched
uniform distribution of ei , obtained by setting α = 0 [panel (a)] and α = 2
[panel (b)] in the ei → wi mapping. Solid green and red lines mark the
borders of pure C and D phases, respectively, whereas the region in-between
the lines characterizes a mixed distribution of strategies on the spatial grid.
The dashed, blue line at b = 1 denotes the boarder between the essential
Prisoner’s Dilemma payoff parameterization (above the line) and a dilemmafree situation (below the line). . . . . . . . . . . . . . . . . . . . . . . . . .
93
Full b − K phase diagrams for the Prisoner’s Dilemma game incorporating
aging as a dynamical process. In both panels, α = 2 (to be compared with
panel (b) of Fig. 6.2). (a) results for model II, where aging is dynamical,
but follows a deterministic protocol; (b) results for co-evolutionary model
III, where the individual teaching values wi evolve corresponding to the
persistence time of strategies, i.e. individuals who change their strategy are
considered newborn (ei = 0) in the next timestep. Additionally, the dashed
lines indicate the results for a slightly changed scenario, where only 10 %
of the individuals increase their age per time step. To avoid ambiguity, C
and D symbols are not given in panel (b), but the respective regions can
be inferred by the line colors according to panel (a). . . . . . . . . . . . . .
94
6.4
Equilibrium fraction of cooperators x in the two non-trivial fixed points.
The x-axis and the y-axis are the same as in the two-dimensional Figs. 3.2
and 3.3, i.e., together with our general definition of S = 0 and T = 1, they
define the respective game. (a) Fraction of cooperators in the non-trivial
fixed point of the evolution between cooperators and defectors only [fixed
point (iv) in the text]. (b) Fraction of cooperators in the additional, nontrivial fixed point through the inclusion of imitators [fixed point (v) in the
text]. This one only exists for games with 0 < P < R < 1, i.e. only in the
Prisoner’s Dilemma. In this Figure, stability of fixed points is not illustrated.100
6.5
Illustration of the state space (triangular simplex) for the dynamics on
the relative frequencies of three species: cooperators x, imitators i, and
defectors 1 − x − i. The dynamical system can only be in a state within the
triangle or exactly at its border. The black circle denotes fixed point (i), i.e.
the domination of defectors. Dashed lines are illustrations of eigenvectors,
aimed at supporting the verbal explanation in the text. . . . . . . . . . . . 102
148
List of Figures
6.6
Eigenvalues for the Jacobian at fixed point (ii), i.e. at the state where
cooperators dominate. The value of λ1 is illustrated by the black line and
λ2 by the red one. They only depend on the payoff value R, provided that
S = 0, T = 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
6.7
Oscillations of species’ frequencies in three Prisoner’s Dilemma games: (a)
R = 0.8, P = 0.4, (b) R = 0.95, P = 0.05, (c) R = 0.65, P = 0.6. The
corners of a simplex indicate dominance of cooperators (C), defectors (C),
and imitators (I), respectively. Circles denote fixed points of the dynamics
– a black filling denotes stability of the fixed point. Shown are dynamical
trajectories of different initial conditions. Generally, the trajectories form
constant orbits around the stable fixed point that include the initial condition. Exceptional cases (red and grey trajectories) are discussed in the
text. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
6.8
Illustrations of dynamical forces at the simplex boundary for a Prisoner’s
Dilemma with R = 0.8, P = 0.4. (a): positive difference between the
expected payoff of the bilaterally dominant species and the average payoff
in the system – the quantity that reflects the dynamical speed of invasion.
(b): payoff difference between the (potential) expected payoff of the nonexistent species and the average payoff in the system – reflecting the invasion
potential of the non-existent species. In both panels, the abscissa denotes
the frequency of the pairwise dominant (invading) species. . . . . . . . . . 106
6.9
Color encoded equilibrium frequencies in the stable fixed points: (a) cooperators, (b) imitators, (c) defectors, and (d) individuals that apply the cooperative decision at a time, i.e. cooperators plus imitators that meet a cooperator plus half of the imitators that meet another imitator (x + x i + i2 /2).
Red color indicates a frequency of 1, blue a frequency of 0. The colorbar of
(d) translates colors into frequencies for all panels (a-d). For the Prisoner’s
Dilemma (0 < P < R < 1), the displayed fixed point is not attractive, but
produces stable oscillations around it. . . . . . . . . . . . . . . . . . . . . . 108
6.10 Equilibrium fraction of cooperators x (up left), imitators i (up right), and
defectors, respectively. Values are averaged from 200 sample runs. The
system size is N = 900 individuals. In the Prisoner’s Dilemma (0 < P <
R < 1), defectors almost never survive. . . . . . . . . . . . . . . . . . . . . 110
List of Figures
149
6.11 Growing oscillations in single simulation runs that lead to extinction of
2 species. The networks of individuals are fully connected (well mixed)
with system sizes of N = 104 . The game is a Prisoner’s Dilemma with
R = 0.8, P = 0.4 (S = 0, T = 1). Initial conditions are marked by crosses:
besides the two equal initializations, the other two in the left panel are
x = i = 0.02 and x = 0.39, i = 0.01, respectively. . . . . . . . . . . . . . . . 111
6.12 Representative time evolution of initially equal frequencies of cooperators
(red), imitators (magenta), and defectors (blue) in a numerical simulation of
a 2-dimensional, regular lattice with interactions between 8 nearest neighbors. In contrast to fully connected networks, oscillations do not grow. In
fact, they shrink and lead to a fluctuating, but qualitatively stable coexistence between cooperators (mostly in majority), imitators, and defectors
(always in minority after a short, initial stage that is amplified in the inset
of the figure). The system size is N = 104 individuals. . . . . . . . . . . . . 112
6.13 Evolutionary trajectories in homogeneous, random networks and 2-dimensional,
regular lattices. k denotes the number of links every individual has in the
network. Every simplex shows one representative simulation run with equal
initial conditions. Panel (a) additionally displays a trajectory for the initial
condition x = 0.9, i = 0.02. . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
7.1
Classification of symmetrical 2×2 games according to payoff ranking and
system-optimal solutions. Two-dimensionality is achieved by fixing T > S
and classifying ordinal differences only. Parcels separated by solid lines
denote different rankings of the payoff values. The dashed lines divide the
whole space in 2 regions according to whether partial cooperation is system
optimal (dark-grey background) or not. Social dilemma games have a red
background color. For each area a respective payoff matrix (in form of the
left matrix in Eq. (7.1), with T > S) is given. Red payoff matrices denote
“partial cooperation dilemmas”. . . . . . . . . . . . . . . . . . . . . . . . . 122
7.2
Scheme of symmetrical 2 × 2 games according to Fig. 7.1. A possible
position of the standard cooperation model (right matrix in Eq. (7.1)) is
indicated by the black circle on arrow 2. The dotted arrows 1-4 indicate
transformations of the payoff matrices when increasing the parameter w,
i.e. the synergistic/discounting effects of cooperation in the framework of
matrix 7.4. In line with the argumentation of this article, T > S is always
respected. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
Bibliography
Abelson, R. (1964). Mathematical models of the distribution of attitudes under controversy. Contributions to Mathematical Psychology 14, 1–160.
Altrock, M.; Traulsen, A. (2009). Fixation times in evolutionary games under weak selection. New Journal of Physics 11, 013012.
Anderson, J. (1991). The place of cognitive architectures in a rational analysis. In: K. van
Lehn (ed.), Architectures for intelligence. Lawrence Erlbaum Associates, pp. 1–24.
Antal, T.; Scheuring, I. (2006). Fixation of Strategies for an Evolutionary Game in Finite
Populations. Bulletin of Mathematical Biology 68(8), 1923–1944.
Arrow, K. (1970). Social choice and individual values. Yale University Press.
Asch, S. (1951). Effects of group pressure upon the modification and distortion of judgments. In: Groups, Leadership, and Men. pp. 177–190.
Aubin, J. (1979). Mathematical Methods of Game and Economic Theory. New York:
North-Holland Publ. Co. .
Axelrod, R. (1984). The Evolution of Cooperation. New York: Basic Books.
Bandura, A. (1977). Social learning theory. New Jersey: Prentice Hall Englewood Cliffs.
Behera, L.; Schweitzer, F. (2003). On spatial consensus formation: is the Sznajd model
different from the voter model? International Journal of Modern Physics C 14, 1331–
1354.
Bergstrom, C.; Lachmann, M. (2003). The Red King effect: When the slowest runner
wins the coevolutionary race. Proceedings of the National Academy of Sciences, USA
100(2), 593.
Berr, M.; Reichenbach, T.; Schottenloher, M.; Frey, E. (2009). Zero-one survival behavior
of cyclically competing species. Physical Review Letters 102(4), 048102.
150
BIBLIOGRAPHY
151
Binney, J.; Dowrick, N.; Fisher, A.; Newman, M. (1993). The Theory of Critical Phenomena: An Introduction to the Renormalization Group. Oxfors: Clarendon Press.
Bonabeau, E.; Dorigo, M.; Theraulaz, G. (1999). Swarm intelligence: from natural to
artificial systems. Oxford University Press, USA.
Bornstein, G.; Budescu, D.; Zamir, S. (1997). Cooperation in Intergroup, N-Person, and
Two-Person Games of Chicken. The Journal of Conflict Resolution 41(3), 384–406.
Browning, L.; Colman, A. (2004). Evolution of coordinated alternating reciprocity in
repeated dyadic games. Journal of Theoretical Biology 229.
Cameron, L. (1999). Raising the stakes in the ultimatum game: Experimental evidence
from Indonesia. Economic Inquiry 37(1), 47–59.
Castellano, C.; Fortunato, S.; Loreto, V. (2007). Statistical physics of social dynamics.
Arxiv preprint, arXiv:0710.3256.
Castellano, C.; Vilone, D.; Vespignani, A. (2003). Incomplete ordering of the voter model
on small-world networks. Europhysics Letters 63, 153–158.
Castellò, X.; Eguı́luz, V. M.; San Miguel, M. (2006). Ordering dynamics with two nonexcluding options: bilingualism in language competition. New Journal of Physics 8,
308.
Chave, J. (2001). Spatial Patterns and Persistence of Woody Plant Species in Ecological
Communities. The American Naturalist 157, 51–65.
Choe, J.; Crespi, B. (1997). The evolution of social behavior in insects and arachnids.
Cambridge University Press.
Claussen, J.; Traulsen, A. (2008). Cyclic Dominance and Biodiversity in Well-Mixed
Populations. Physical Review Letters 100(5), 58104.
Clutton-Brock, T. (2002). Breeding Together: Kin Selection and Mutualism in Cooperative Vertebrates. Science 296(5565), 69–72.
Cox, J.; Durrett, R. (1991). Nonlinear voter models. In: R. Durrett; H. Kesten (eds.),
Random walks, Brownian motion, and interacting particle systems. Boston: Birkhäuser.
Czaran, T.; Hoekstra, R.; Pagie, L. (2002). Chemical warfare between microbes promotes
biodiversity. Proceedings of the National Academy of Sciences, USA 99(2), 786–790.
152
BIBLIOGRAPHY
Dall’Asta, L.; Castellano, C. (2007). Effective surface-tension in the noise-reduced voter
model. Europhysics Letters 77(6), 60005.
Darwin, C. (1859). On the origin of species by means of natural selection or the preservation of favoured races in the struggle for life. London: J. Murray .
Dawes, C.; Fowler, J.; Johnson, T.; McElreath, R.; Smirnov, O. (2007). Egalitarian
Motives in Humans. Nature 446, 794–796.
Dawes, R. (1980). Social Dilemmas. Annual Review of Psychology 31(1), 169–193.
Dawkins, R. (1989). The Selfish Gene. Oxford University Press.
De Martino, A.; Giardina, I.; Tedeschi, A.; Marsili, M. (1999). Generalized minority games
with adaptive trend-followers and contrarians. Physical Review E 70, 025104.
Deffuant, G.; Neau, D.; Amblard, F.; Weisbuch, G. (2000). Mixing beliefs among interacting agents. Advances in Complex Systems 3(1-4), 87–98.
Dittrich, P.; Liljeros, F.; Soulier, A.; Banzhaf, W. (2000). Spontaneous group formation
in the seceder model. Physical Review Letters 84(14), 3205–3208.
Doebeli, M.; Hauert, C. (2005). Models of cooperation based on the Prisoner’s Dilemma
and the Snowdrift game. Ecology Letters 8(7), 748–766.
Doebeli, M.; Hauert, C.; Killingback, T. (2004). The Evolutionary Origin of Cooperators
and Defectors. Science 306(5697), 859–862.
Dornic, I.; Chaté, H.; Chave, J.; Hinrichsen, H. (2001). Critical Coarsening without
Surface Tension: The Universality Class of the Voter Model. Physical Review Letters
87(4), 045701.
Duncan, S. (1972). Some Signals and Rules for Taking Speaker Turns in Conversations.
Journal of Personality and Social Psychology 23(2), 283–292.
Einstein, A. (1934). On the method of theoretical physics. Philosophy of science , 163–169.
Eisenführ, F.; Weber, M. (2002). Rationales Entscheiden. Springer.
Epstein, J.; Axtell, R. (1996). Growing artificial societies: Social science from the bottom
up. Cambridge: MIT Press.
Fehr, E.; Bernhard, H.; Rockenbach, B. (2008). Egalitarianism in young children. Nature
454, 1079.
BIBLIOGRAPHY
153
Fehr, E.; Fischbacher, U. (2003). The nature of human altruism. Nature 425, 785–791.
Fehr, E.; Gächter, S. (1998). Reciprocity and economics: the economic implications of
Homo Reciprocans. European Economic Review 42, 845–859.
Fehr, E.; Gächter, S. (2002). Altruistic punishment in humans. Nature 415, 137–140.
Fehr, E.; Schmidt, K. (1999). A Theory Of Fairness, Competition, and Cooperation.
Quarterly Journal of Economics 114(3), 817–868.
Field, J.; Cronin, A.; Bridge, C. (2006). Future fitness and helping in social queues. Nature
441, 214–217.
Fishbein, M.; Ajzen, I. (1975). Belief, attitude, intention, and behavior: an introduction
to theory and research. Addison Wesley Publishing Company.
Fontes, L. R.; Isopi, M.; Newman, C. M.; Stein, D. L. (2001). Aging in 1D discrete spin
models and equivalent systems. Physical Review Letters 87(11), 110201.
Frachebourg, L.; Krapivsky, P. L. (1996). Exact results for kinetics of catalytic reactions.
Physical Review E 53(4), 3009–3012.
Frean, M.; Abraham, E. (2001a). A voter model of the spatial prisoner’s dilemma. IEEE
Transactions on Evolutionary Computation 5(2), 117–121.
Frean, M.; Abraham, E. (2001b). Rock-scissors-paper and the survival of the weakest.
Proceedings of the Royal Society B: Biological Sciences 268(1474), 1323–1327.
French, J. (1956). Formal Theory of Social Power. Psychological Review 63, 181–194.
Friedman, D. (1991). Evolutionary Games in Economics. Econometrica 59(3), 637–666.
Fuchs-Heinritz et al., W. (1994). Lexikon zur Soziologie. Opladen: Westdeutscher Verlag.
Fudenberg, D.; Tirole, J. (1991). Game Theory. MIT Press.
Galam, S. (2002). Minority Opinion Spreading in Random Geometry. European Physical
Journal B 25, 403–406.
Galam, S. (2004). Contrarian deterministic effects on opinion dynamics: ”the hung elections scenario”. Physica A: Statistical Mechanics and its Applications 333, 453–460.
Galam, S. (2005). Local dynamics vs. social mechanisms: A unifying frame. Europhysics
Letters 20, 705.
154
BIBLIOGRAPHY
Galam, S.; Gefen, Y.; Shapir, Y. (1982). Sociophysics: a new approach of sociological
collective behavior. J. Math. Sociology 9(1).
Galeotti, A.; Goyal, S. (2007). Games of Social Influence. Economics Discussion Papers,
http://ideas.repec.org/p/esx/essedp/639.html, downloaded: 2008/09/12.
Gardner, A.; West, S.; Barton, N. (2007). The relation between multilocus population
genetics and social evolution theory. The American Naturalist 169(2), 207–226.
Gigerenzer, G.; Selten, R. (2001). Bounded rationality: The adaptive toolbox. MIT Press,
Cambridge, MA.
Gigerenzer, G.; Todd, P. (1999). Simple heuristics that make us smart. New York: Oxford
University Press.
Gintis, H. (2005). Moral Sentiments and Material Interests: The Foundations of Cooperation in Economic Life. MIT Press, Cambridge, MA.
Grafen, A. (1984). Hamilton’s rule OK. Nature 318, 310–311.
Granovetter, M. (1978). Threshold models of collective behavior. American Journal of
Sociology 83(6), 1420–1443.
Guare, J. (1990). Six degrees of separation: A play. New York: Vintage Books.
Gunton, J. D.; San Miguel, M.; Sahni, P. S. (1983). Phase Transitions and Critical
Phenomena. London: Academic Press. C. Domb and J. Lebowitz, Eds.
Güth, W. (1999). Spieltheorie und ökonomische (Bei) Spiele. Springer.
Hamilton, W. (1963). The Evolution of Altruistic Behavior. The American Naturalist
97(896), 354–356.
Hamilton, W. (1964). The genetical evolution of social behaviour. Journal of Theoretical
Biology 7(1), 17–52.
Harsanyi, J.; Selten, R. (1988). A General Theory of Equilibrium Selection in Games.
MIT Press, Cambridge, MA.
Hauert, C. (2002). Effects of space in 2× 2 games. Int. J. Bifurcat. Chaos 12, 1531–1548.
Hauert, C.; De Monte, S.; Hofbauer, J.; Sigmund, K. (2002). Volunteering as Red Queen
Mechanism for Cooperation in Public Goods Games. Science 296(5570), 1129–1132.
BIBLIOGRAPHY
155
Hauert, C.; Michor, F.; Nowak, M.; Doebeli, M. (2006). Synergy and discounting of
cooperation in social dilemmas. Journal of Theoretical Biology 239(2), 195–202.
Hauert, C.; Szabó, G. (2005). Game theory and physics. American Journal of Physics
73, 405.
Hegselmann, R.; Krause, U. (2002). Opinion Dynamics and Bounded Confidence Models,
Analysis, and Simulation. Journal of Artifical Societies and Social Simulation (JASSS)
5(3).
Helbing, D. (1992). A mathematical model for behavioral changes by pair interactions. In:
G. Haag; U. Müller; K. Troitzsch (eds.), Economic evolution and demographic change.
Formal models in social sciences. Berlin: Springer, pp. 330–348.
Helbing, D. (1993). Stochastische Methoden, nichtlineare Dynamik und quantitative Modelle sozialer Prozesse. Shaker.
Helbing, D. (1995). Quantitative Sociodynamics: Stochastic Methods and Models of Social
Interaction Processes. Springer.
Helbing, D.; Buzna, L.; Johansson, A.; Werner, T. (2005a). Self-organized pedestrian
crowd dynamics: Experiments, simulations, and design solutions. Transportation science
39(1), 1.
Helbing, D.; Farkas, I.; Vicsek, T. (2000). Simulating Dynamical Features of Escape Panic.
Nature 407, 487–490.
Helbing, D.; Schönhof, M.; Stark, H.-U.; Holyst, J. (2005b). How individuals learn to take
turns: Emergence of alternating cooperation in a congestion game and the prisoner’s
dilemma. Advances in Complex Systems 8, 87.
Henrich, J.; Boyd, R.; Bowles, S.; Camerer, C.; Fehr, E.; Gintis, H.; McElreath, R. (2001).
In search of homo economicus: Behavioral experiments in 15 small-scale societies. American Economic Review , 73–78.
Henrich et al., J. (2003). Group Report: The Cultural and Genetic Evolution of Human
Cooperation. In: P. Hammerstein (ed.), Genetic and Cultural Evolution of Cooperation.
MIT Press.
Hofbauer, J.; Sigmund, K. (1998). Evolutionary Games and Population Dynamics. Cambridge University Press.
156
BIBLIOGRAPHY
Holley, R. A.; Liggett, T. M. (1975). Ergodic Theorems for Weakly Interacting Infinite
Systems and the Voter Model. The Annals of Probability 3(4), 643–663.
Holt, C.; Roth, A. (2004). The Nash equilibrium: A perspective. Proceedings of the
National Academy of Sciences, USA 101(12), 3999–4002.
Holyst, J.; Kacperski, K.; Schweitzer, F. (2001). Social impact models of opinion dynamics. In: D. Stauffer (ed.), Annual Review of Computational Physics. Singapore: World
Scientific, vol. IX, pp. 253–273.
Huyck, J.; Battalio, R.; Beil, R. (1990). Tacit coordination games, strategic uncertainty,
and coordination failure. American Economic Review 80, 234–248.
Imhof, L.; Fudenberg, D.; Nowak, M. (2005). Evolutionary cycles of cooperation and
defection. Proceedings of the National Academy of Sciences, USA 102(31), 10797.
Imhof, L.; Fudenberg, D.; Nowak, M. (2007). Tit-for-tat or win-stay, lose-shift? Journal
of Theoretical Biology 247(3), 574–580.
Kacperski, K.; Holyst, J. (1997). A simple model of social group with a leader. Journal
of Technical Physics 38(2), 393–396.
Kagel, J.; Kim, C.; Moser, D. (1996). Fairness in ultimatum games with asymmetric
information and asymmetric payoffs. Games and Economic Behavior 13, 100–110.
Kagel, J.; Roth, A. (1995). The handbook of experimental economics. Princeton: Princeton
University Press.
Kaplan, T.; Ruffle, B. (2007). Which Way to Cooperate. Available at SSRN:
http://ssrn.com/abstract=580903, (downloaded March 10, 2009).
Kennedy, J.; Eberhart, R.; Shi, Y. (2001). Swarm intelligence. San Francisco: Morgan
Kauffmann and Academic Press.
Kerr, B.; Riley, M.; Feldman, M.; Bohannan, B. (2002). Local dispersal promotes biodiversity in a real-life game of rock-paper-scissors. Nature 418(6894), 171–4.
Kirman, A. (1993). Ants, rationality, and recruitment. The Quarterly Journal of Economics 108(1), 137–156.
Kreps, D. (1990). Game theory and economic modelling. Oxford: Clarendon Press.
Kreps, D.; Milgrom, P.; Roberts, J.; Wilson, R. (2001). Rational cooperation in the finitely
repeated prisoners’ dilemma. Journal of Economic Theory 27, 245–252.
BIBLIOGRAPHY
157
Kurzban, R.; Houser, D. (2005). Experiments investigating cooperative types in humans:
A complement to evolutionary theory and simulations. Proceedings of the National
Academy of Sciences, USA 102(5), 1803.
Landau, D.; Binder, K. (2005). A Guide to Monte Carlo Simulations in Statistical Physics.
Cambridge: Cambridge University Press.
Latané, B. (1981). The psychology of social impact. American Psychologist 36(4), 343–
356.
Latané, B. (1996). Dynamic social impact: The creation of culture by communication.
The Journal of Communication 46(4), 13–25.
Latané, B.; Wolf, S. (1981). The Social Impact of Majorities and Minorities. Psychological
Review 88(5), 438–53.
Lehmann, L.; Keller, L. (2006). The evolution of cooperation and altruism. A general
framework and classification of models. Journal of Evolutionary Biology 19, 1365–1376.
Liggett, T. M. (1995). Interacting Particle Systems. New York: Springer.
Lorenz, J. (2007). Continuous Opinion Dynamics under Bounded Confidence: A Survey.
International Journal of Modern Physics C 18(12), 1–20.
Luce, R.; Raiffa, H. (1957). Games and decisions. Wiley New York.
Macy, M.; Flache, A. (2002). Learning Dynamics in Social Dilemmas. Proceedings of the
National Academy of Sciences, USA 99(10), 7229–7236.
Maslov, S.; Sneppen, K.; Alon, U. (2003). Correlation profiles and motifs in complex
networks. In: S. Bornholdt; H. Schuster (eds.), Handbook of graphs and networks. From
the genoma to the internet. Wiley VCH and Co.
Mason, W.; Conrey, F.; Smith, E. (2007). Situating social influence processes: Dynamic,
multidirectional flows of influence within social networks. Personality and Social Psychology Review 11(3), 279.
May, R. (1987). More evolution of cooperation. Nature 327(6117), 15–17.
Maynard Smith, J. (1982). Evolution and the Theory of Games. Cambridge, MA: Cambridge University Press.
Maynard Smith, J.; Price, G. (1973). The logic of animal conflict. Nature 246, 15–18.
158
BIBLIOGRAPHY
Maynard Smith, J.; Szathmáry, E. (1995). The major transitions in evolution. Oxford:
W.H. Freeman.
Milgram, S. (1967). The small world problem. Psychology today 2(1), 60–67.
Milgram, S. (1974). Obedience to authority: An experimental view. Tavistock Publications
Ltd.
Miller, N.; Dollard, J. (1998). Social learning and imitation. Routledge.
Moran, P. (1962). The statistical processes of evolutionary theory. Oxford: Clarendon
Press.
Nash, J. (1950). Equilibrium point in n-person games. Proceedings of the National Academy
of Sciences, USA 36, 48–49.
Nash, J. (1951). Noncooperative Games. Annals of Mathematics 54, 286–295.
Neill, D. (2003). Cooperation and coordination in the turn-taking dilemma. In: Proceedings
of the 9th conference on Theoretical Aspects of Rationality and Knowledge. New York:
ACM Press, pp. 231–244.
Nowak, M. (2006a). Evolutionary dynamics: Exploring the equations of life. Harvard
University Press.
Nowak, M. (2006b). Five Rules for the Evolution of Cooperation. Science 314(5805),
1560–1563.
Nowak, M.; May, R. (1992). Evolutionary games and spatial chaos. Nature 359, 826–829.
Nowak, M.; Sigmund, K. (1990). The evolution of stochastic strategies in the Prisoner’s
Dilemma. Acta Applicandae Mathematicae 20(3), 247–265.
Nowak, M.; Sigmund, K. (1993). A strategy of win-stay, lose-shift that outperforms titfor-tat in the Prisoner’s Dilemma game. Nature 364, 56–58.
Nowak, M.; Sigmund, K. (1998). Evolution of indirect reciprocity by image scoring. Nature
393, 573.
Nowak, M.; Sigmund, K. (2004). Evolutionary Dynamics of Biological Games. Science
303, 793.
Osborne, M.; Rubinstein, A. (1994). A Course in Game Theory. MIT Press.
BIBLIOGRAPHY
159
Perc, M.; Szolnoki, A. (2008). Social diversity and promotion of cooperation in the spatial
prisoner’s dilemma game. Physical Review E 77, 11904.
Pingle, M. (1995). Imitation versus rationality: An experimental perspective on decision
making. Journal of Socio-Economics 24(2), 281–315.
Queller, D. (1985). Kinship, reciprocity, and synergism in the evolution of social behavior.
Nature 318, 366–367.
Rapoport, A. (1967). Exploiter, leader, hero, and martyr: the four archetypes of the 2
times 2 game. Behavioral Science 12(2), 81–4.
Rashotte, L. (2007). Social Influence. The Blackwell Encyclopedia of Social Psychology ,
562–563.
Rasmusen, E. (2001). Readings in Games and Information. Blackwell Publishers.
Ravasz, M.; Szabo, G.; Szolnoki, A. (2004). Spreading of families in cyclic predator-prey
models. Physical Review E 70(1), 012901.
Redner, S. (2001). A guide to first-passage processes. Cambridge: Cambridge University
Press.
Reichenbach, T.; Mobilia, M.; Frey, E. (2007). Mobility promotes and jeopardizes biodiversity in rock-paper-scissors games. Nature 448(7157), 1046–1049.
Riolo, R.; Cohen, M.; Axelrod, R. (2001). Evolution of cooperation without reciprocity.
Nature 414, 441–443.
Roca, C.; Cuesta, J.; Sánchez, A. (2006). Time Scales in Evolutionary Dynamics. Physical
Review Letters 97(15), 158701.
Roca, C.; Cuesta, J.; Sánchez, A. (2008). Sorting out the effect of spatial structure on the
emergence of cooperation. Arxiv preprint, arXiv:0806.1649.
Samuelson, L. (1997). Evolutionary Games and Equilibrium Selection. MIT Press, Cambridge, MA.
Sanfey, A.; Rilling, J.; Aronson, J.; Nystrom, L.; Cohen, J. (2003). The neural basis of
economic decision-making in the ultimatum game. Science 300(5626), 1755.
Santos, F.; Pacheco, J.; Lenaerts, T. (2006). Evolutionary dynamics of social dilemmas in
structured heterogeneous populations. Proceedings of the National Academy of Sciences,
USA 103(9), 3490–3494.
160
BIBLIOGRAPHY
Santos, F.; Santos, M.; Pacheco, J. (2008). Social diversity promotes the emergence of
cooperation in public goods games. Nature 454, 213–216.
Schelling, T. (1978). Micromotives and macrobehavior. New York: Norton.
Schlag, K. (1998). Why imitate, and if so, how? A boundedly rational approach to
multi-armed bandits. Journal of Economic Theory 78, 130–156.
Schüßler, R. (1986). The evolution of reciprocal cooperation. In: A. Diekmann; P. Mitter (eds.), Paradoxical effects of social behavior. Essays in honor of Anatol Rapoport.
Heidelberg, Wien: Physica-Verlag, pp. 105–121.
Schweitzer, F.; Behera, L. (2009). Nonlinear voter models: the transition from invasion to
coexistence. The European Physical Journal B 67(3), 301–318.
Schweitzer, F.; Behera, L.; Mühlenbein, H. (2002). Evolution of cooperation in a spatial
prisoner’s dilemma. Advances in Complex Systems 5, 269–299.
Selten, R.; Schreckenberg, M.; Pitz, T.; Chmura, T.; Wahle, J. (2004). Experimental
investigation of day-to-day route-choice behaviour and network simulations of autobahn
traffic in North Rhine-Westphalia. In: M. Schreckenberg; R. Selten (eds.), Human
Behaviour and Traffic Networks. Springer, pp. 7–12.
Shor, M. (2005). Strategy. Available at http://www.gametheory.net/dictionary/strategy.html,
(downloaded May 14, 2011).
Simon, H. (1955). A behavioral model of rational choice. The Quarterly Journal of
Economics , 99–118.
Simon, H. (1956). Rational choice and the structure of the environment. Psychological
Review 63(2), 129–38.
Simon, H. (1972). Theories of bounded rationality. In: Decision and organization. Amsterdam: North Holland, pp. 161–176.
Skyrms, B. (2004). The stag hunt and the evolution of social structure. Cambridge University Press.
Slanina, F.; Lavicka, H. (2003). Analytical results for the Sznajd model of opinion formation. European Physical Journal B 35(2), 279–288.
Smith, V. (1991). Rational choice: The contrast between economics and psychology.
Journal of Political Economy , 877–897.
BIBLIOGRAPHY
161
Sood, V.; Redner, S. (2005). Voter Model on Heterogeneous Graphs. Physical Review
Letters 94(17), 178701(4).
Stark, H.-U. (2010). Dilemmas of partial cooperation. Evolution 64(8), 2458–2465.
Stark, H.-U.; Helbing, D.; Schönhof, M.; Holyst, J. (2008a). Alternating cooperation
strategies in a route choice game: Theory, experiments, and effects of a learning scenario.
In: A. Innocenti; P. Sbriglia (eds.), Games, Rationality and Behavior. Houndmills and
New York: Palgrave MacMillan, pp. 256–273.
Stark, H.-U.; Tessone, C. (2010). Imitation catalyzes cooperation in three social dilemmas.
In preparation.
Stark, H.-U.; Tessone, C. J.; Schweitzer, F. (2008b). Decelerating microdynamics can accelerate macrodynamics in the voter model. Physical Review Letters 101(1), 018701(4).
Stark, H.-U.; Tessone, C. J.; Schweitzer, F. (2008c). Slower is Faster: Fostering Consensus
Formation by Heterogeneous Inertia. Advances in Complex Systems 11(4).
Stauffer, D.; Sousa, A.; de Oliveira, S. (2000). Generalization to Square Lattice of Sznajd
Sociophysics Model. International Journal of Modern Physics C 11(6), 1239–1246.
Strogatz, S. (1994). Nonlinear dynamics and chaos. Cambridge, MA: Perseus Publishing.
Suchecki, K.; Eguı́luz, V. M.; San Miguel, M. (2005a). Conservation laws for the voter
model in complex networks. Europhysics Letters 69, 228–234.
Suchecki, K.; Eguı́luz, V. M.; San Miguel, M. (2005b). Voter Model Dynamics in Complex
Networks: Role of Dimensionality, Disorder, and Degree Distribution. Physical Review
E 72, 036132.
Sugden, R. (2001). The evolutionary turn in game theory. Journal of Economic Methodology 8(1), 113–130.
Suzumura, K. (1983). Rational choice, collective decisions, and social welfare/Kotaro
Suzumura. Cambridge: Cambridge University Press.
Szabó, G.; Fáth, G. (2007). Evolutionary games on graphs. Physics Reports 446(4-6),
97–216.
Szabó, G.; Tőke, C. (1998). Evolutionary prisoner’s dilemma game on a square lattice.
Physical Review E 58(1), 69–73.
162
BIBLIOGRAPHY
Szolnoki, A.; Perc, M.; Szabó, G.; Stark, H.-U. (2009). Impact of aging on the evolution
of cooperation in the spatial prisoner’s dilemma game. Physical Review E 80, 021901.
Szolnoki, A.; Szabó, G. (2007). Cooperation enhanced by inhomogeneous activity of
teaching for evolutionary Prisoner’s Dilemma games. Europhysics Letters 77(3), 30004.
Tanimoto, J. (2008). What initially brought about communications? BioSystems 92(1),
82–90.
Tanimoto, J.; Sagara, H. (2007). A study on emergence of alternating reciprocity in a 2×
2 game with 2-length memory strategy. BioSystems 90(3), 728–737.
Taylor, C.; Nowak, M. (2007). Transforming the dilemma. Evolution 61(10), 2281–2292.
Taylor, P.; Jonker, L. (1978). Evolutionarily stable strategies and game dynamics. Mathematical Biosciences 40(2), 145–56.
Tessone, C. J.; Toral, R.; Amengual, P.; Wio, H. S.; San Miguel, M. (2004). Neighborhood
models of minority opinion spreading. European Physical Journal B 39, 535–544.
Traulsen, A.; Nowak, M. (2006). Evolution of cooperation by multilevel selection. Proceedings of the National Academy of Sciences, USA 103(29), 10952–10955.
Traulsen, A.; Nowak, M. (2007). Chromodynamics of Cooperation in Finite Populations.
PLoS ONE 2(3).
Traulsen, A.; Pacheco, J.; Nowak, M. (2007). Pairwise comparison and selection temperature in evolutionary game dynamics. Journal of Theoretical Biology 246(3), 522–529.
Trivers, R. (1971). The Evolution of Reciprocal Altruism. The Quarterly Review of Biology
46(1), 35–57.
Trotter, W. (2005). Instincts of the Herd in Peace and War. New York: Cosimo Inc.
Tversky, A.; Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases.
Science 185(4157), 1124–1131.
Tversky, A.; Kahneman, D. (1986). Rational choice and the framing of decisions. Journal
of Business 59(S4), 251.
Van Kampen, N. (2007). Stochastic processes in physics and chemistry. North Holland.
Von Neumann, J.; Morgenstern, O. (1944). Theory of Games and Economic Behaviour.
Princeton: Princeton University Press.
BIBLIOGRAPHY
163
Wasserman, S. (1994). Social network analysis: Methods and applications. Cambridge:
Cambridge University Press.
Watts, D.; Strogatz, S. (1998). Collective dynamics of small-world networks. Nature 393,
440.
Weidlich, W. (1971). The statistical description of polarization phenomena in society.
British Journal of Mathematical and Statistical Psychology 24(2), 251–266.
Weidlich, W.; Haag, G. (1983). Concepts and Models of a Quantitative Sociology: The
Dynamics of Interacting Populations. Berlin: Springer-Verlag.
West, S.; Griffin, S.; Gardner, A. (2007). Social semantics: altruism, cooperation, mutualism, strong reciprocity, and group selection. Journal of Evolutionary Biology 20,
415–432.
Wingreen, N.; Levin, S. (2006). Cooperation among Microorganisms. PLoS Biology 4(9),
e299.
Zimbardo, P. (2008). The Lucifer effect: Understanding how good people turn evil. Random
House Inc.