ARTICLE IN PRESS
Journal of Economic Dynamics & Control 32 (2008) 1569–1599
www.elsevier.com/locate/jedc
A numerical analysis of the evolutionary stability
of learning rules
Jens Josephson
Department of Economics and Business, Universitat Pompeu Fabra,
Ramon Trias Fargas 25-27, 08005 Barcelona, Spain
Received 10 November 2005; accepted 22 June 2007
Available online 4 July 2007
Abstract
In this paper, we define an evolutionary stability criterion for learning rules. Using
simulations, we then apply this criterion to three types of symmetric 2 2 games for a class of
learning rules that can be represented by the parametric model of Camerer and Ho [1999.
Experience-weighted attraction learning in normal form games. Econometrica 67, 827–874].
This class contains stochastic versions of reinforcement and fictitious play as extreme cases.
We find that only learning rules with high or intermediate levels of hypothetical reinforcement
are evolutionarily stable, but that the stable parameters depend on the game.
r 2007 Elsevier B.V. All rights reserved.
JEL classification: C72; C73; C15
Keywords: Monte Carlo simulation; Evolutionary stability; Learning in games; Fictitious play;
Reinforcement learning; EWA learning
1. Introduction
The bounded rationality paradigm is based on the assumption that people learn to
play games by using simple rules of adaptation, often referred to as learning rules. In
almost all of the literature, it is assumed that all players of the game employ the same
Tel.: +34 93 542 22 68; fax: +34 93 542 17 46.
E-mail address: [email protected]
0165-1889/$ - see front matter r 2007 Elsevier B.V. All rights reserved.
doi:10.1016/j.jedc.2007.06.008
ARTICLE IN PRESS
1570
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
type of learning rule. Moreover, the type of rule is generally motivated by its
asymptotic properties in such a homogeneous setting, such as convergence to Nash
equilibrium. This seems questionable from an evolutionary perspective. Evolutionary selection should favor the use of learning rules that perform well in a
heterogeneous setting with mutant learning rules present in the population.
Furthermore, since most individual interactions are of limited length, survival
should to a great extent be determined by the payoffs to using the learning rule in the
short or intermediate run in such a heterogeneous setting.
In this paper, we implement these ideas by defining an evolutionary stability
criterion for learning rules and applying this criterion to a set of well-known learning
rules using Monte Carlo simulations. More specifically, we ask if there is a rule such
that, if applied by a homogeneous population of individuals, it will survive any
sufficiently small invasion of mutants using a different rule. We call such an
uninvadable rule an evolutionarily stable learning rule (ESLR). This concept is an
extension of Taylor and Jonker’s (1978) definition of evolutionarily stable strategies
(ESSs) (a concept originally due to Maynard Smith and Price, 1973; Maynard Smith,
1974) to learning rules and behavioral strategies.
The setting is a world where the members of a large population, consisting of an
even number of individuals, in each of a finite number of periods are all randomly
matched in pairs to play a finite two-player game. Each individual uses a learning
rule, which is a function of his private history of past play, and fitness is measured in
terms of expected average payoff. This framework provides a rationale for the use of
learning rules and it is of particular interest since very little analysis of learning in
this ‘repeated rematching’ context has previously been done (see Hopkins, 1999, for
an exception).
Technically, learning rules are mappings from the history of past play to the set of
pure or mixed strategies. There are many models of learning and we therefore restrict
the numerical analysis to a class of learning rules that can be described by the general
parametric model of Camerer and Ho (1999), called experience-weighted attraction
(EWA) learning. The rules in this class have experimental support and perform well
in an environment where the game changes from time to time.1 Moreover, the class
contains rules which differ considerably in their use of information. Two of the most
well-known learning rules, reinforcement learning and fictitious play, are special cases
of this model for specific parameter values.
Reinforcement learning is an important model in the psychological literature on
individual learning. It was introduced by Bush and Mosteller (1951) and further
developed by Erev and Roth (1998), although the principle behind the model, that
choices which have led to good outcomes in the past are more likely to be repeated in
the future, originally is due to Thorndike (1898). Under reinforcement learning in
games, players assign probability distributions to their available pure strategies. If a
pure strategy is employed in a particular period, the probability of the same pure
strategy being used in the subsequent period increases as a function of the realized
payoff. The model has very low information and rationality requirements in the
1
See, for example, Camerer and Ho (1999) and Stahl (2003).
ARTICLE IN PRESS
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
1571
sense that individuals need not know the strategy realizations of the opponents or
the payoffs of the game; all that is necessary is knowledge of player-specific past
strategy and payoff realizations.
Fictitious play (Brown, 1951) is a model where the individuals in each of the player
roles of a game in every period play a pure strategy that is a best reply to the
accumulated empirical distribution of their opponents’ play. This means that
knowledge of the opponents’ strategy realizations and the player’s own payoff
function is required.
Several different models of both reinforcement and fictitious play have been
developed over the years. The ones that can be represented by Camerer and Ho’s
(1999) model correspond to stochastic versions with exponential probabilities.2 This
means that each pure strategy in each period is assigned an attraction, which is a
function of the attraction in the previous period and the payoff to the particular
strategy in the current period. The attractions are then exponentially weighted in
order to determine the mixed strategy to be employed in the next period.
In the case of reinforcement learning, the attractions only depend on the payoff to
the pure strategy actually chosen. In the case of fictitious play, the hypothetical
payoffs to the pure strategies that were not chosen are of equal importance (this is
sometimes referred to as hypothetical reinforcement). However, Camerer and Ho’s
(1999) model also permits intermediate cases where payoffs corresponding to pure
strategies that were not chosen are given a weight strictly between zero and one. The
weight of such hypothetical payoffs is given by a single parameter, d.
Two other important parameters of the EWA model are the discount factor, f,
which depreciates past attractions, and the attraction sensitivity, l, which determines
how sensitive mixed strategies are to attractions. We assume all initial attractions are
zero, such that the individuals have almost no prior knowledge of the game they are
drawn to play. This implies that the analysis in this paper boils down to testing if any
particular parameter vector ðd; l; fÞ of the EWA model corresponds to an ESLR.
We do this by simulating a large number of runs in which all members of a finite
population with a large share of incumbents and a small share of mutants are
randomly matched in pairs to play a two-player game in each of a finite number of
periods. We then calculate the average payoff for each share of the population. We
consider games from each of the three generic categories of 2 2 games: dominance
solvable, coordination, and hawk–dove.
We find that in all games, we cannot reject the ESLR hypothesis for a learning
rule with either high or intermediate level of hypothetical reinforcement, although
the stable values of the d and s parameters depend on the payoffs. We also find that
in all games, the ESLR hypothesis can be rejected for learning rules with low
attraction sensitivity and for reinforcement learners. These results appear consistent
with the positive levels of hypothetical reinforcement and differences in parameter
estimates between games found in most experiments.
2
Fudenberg and Levine (1998) show that stochastic fictitious play can be motivated by a setting where
payoffs are subject to noise and the player maximizes next period’s expected payoff given an empirical
distribution of the opponents’ past play.
ARTICLE IN PRESS
1572
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
The main intuition for our results is the difference in the speed of learning. The null
hypothesis of an ESLR cannot be rejected for the learning rule with the fastest
convergence to the equilibrium population share in a homogeneous population in any of
the games. Learning rules with low attraction sensitivity converge relatively slowly to the
equilibrium strategies. Moreover, learning rules with high attraction sensitivity and either
low levels of hypothetical reinforcement or high level of hypothetical reinforcement and
low discount factor, risk getting stuck at an initial population distribution of strategies.
This paper is organized as follows. Section 2 discusses related literature. Section 3
introduces the theoretical model underlying the simulations. Section 4 presents the
results of the Monte Carlo simulations. Section 5 contains a discussion of the results
and Section 6 concludes. Theoretical results can be found in Appendix A, and
diagrams and tables of simulation results for homogeneous populations in
Appendix B. Simulation results for heterogeneous populations can be found in the
supplementary archive of this journal using the link provided in Appendix C.
2. Related literature
The present paper is related to the theoretical literature on learning in games, but
also to experimental tests of different learning rules. An early theoretical reference,
asking similar questions, is Harley (1981). He analyzes the evolution of learning rules
in the context of games with a unique ESS. He assumes the existence of an ESLR
and then discusses the properties of such a rule. Harley finds that, given certain
assumptions, ‘‘. . .the evolutionarily stable learning rule is a rule for learning
evolutionarily stable strategies.’’ He also develops an approximation to such a rule
and simulates its behavior in a homogeneous population. The current paper differs
from that of Harley (1981) in that it explicitly formulates an evolutionary criterion
for learning rules and does not assume the existence of an ESLR. Moreover, the
analysis is not limited to games with a single ESS.
Anderlini and Sabourian (1995) develop a dynamic model of the evolution of
algorithmic learning rules. They find that under certain conditions, the frequencies of
different learning rules in the population are globally stable and that the limit points
of the distribution of strategies correspond to Nash equilibria. However, they do not
investigate the properties of the stable learning rules.
Heller (2004) explores the evolutionary fitness of learning to best reply (at a cost)
relative to playing a pure strategy in an environment where the game changes
stochastically over time. She shows asymptotic dominance of learning when it is
strictly better, averaging over different regimes, than each pure strategy against any
possible opponent.3
There are also a number of recent theoretical results on the asymptotic behavior of
reinforcement learning and fictitious play. Hopkins (2002) shows that the expected
motion of both stochastic fictitious play and perturbed reinforcement learning can be
3
For more on evolution of strategic sophistication see the surveys by Hommes (2006) and Robson
(2001).
ARTICLE IN PRESS
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
1573
written as a perturbed form of the replicator dynamics, and that, in many cases, they
will therefore have the same asymptotic behavior. In particular, he claims that they
have identical local stability properties at mixed equilibria. He also demonstrates
that the main difference between the two learning rules is that fictitious play gives
rise to faster learning. Hofbauer and Sandholm (2002) establish global convergence
results for stochastic fictitious play in games with an interior ESS, zero-sum games,
potential games, and supermodular games. Berger (2005) proves convergence of
fictitious play to Nash equilibria in nondegenerate 2 n games. Beggs (2005) gives
convergence result under Erev and Roth (1998) type of reinforcement learning for
constant-sum games and iteratively dominance solvable games. Hopkins and Posch
(2005) show that this type of learning gives convergence with probability zero to
points that are unstable under the replicator dynamics.
In the experimental literature, the objective is generally to find the learning rule
which gives the best fit of experimental data. Camerer and Ho (1999) give a concise
overview of the most important findings in earlier studies. They argue that the
overall picture is unclear, but that comparisons appear to favor reinforcement
learning in constant-sum games and fictitious play in coordination games. In their
own study of asymmetric constant-sum games, median-action games, and beautycontest games, they find support for a learning rule with parameter values in between
reinforcement learning and fictitious play. Ho et al. (2002), Camerer et al. (2002) and
Camerer (2003) consider extensions of the EWA model and report EWA parameter
estimates for a number of experiments by themselves and other authors.
Stahl (2003) compares the prediction performance of seven learning models,
including a restricted version of the EWA model. He pools data from a variety of
symmetric two-player games and finds a logit best-reply model with inertia and
adaptive expectations to perform best, closely followed by the EWA.
Hanaki et al. (2005) develop a model of learning where individuals in a first long-run
phase with rematching generate limit attractions for a complete set of repeated game
strategies that satisfy a complexity constraint, and in a second short-run phase, without
rematching, use the limit attractions as initial attractions. They find that relative to
existing models of learning, their model is better at explaining the behavior of subjects
in environments where fairness and reciprocity appear to play a significant role.
A pair of recent papers also demonstrate the difficulties in correctly estimating the
parameters of learning models. Salmon (2001) simulates experimental data for a
number of games and shows that the estimations of several well-known learning
models, including EWA learning, sometimes do not accurately distinguish between
the data generating processes. Wilcox (2006) shows that pooled tests of learning
models tend to be biased towards reinforcement learning if the heterogeneity of
subjects is not taken into account.
3. The model
Let G be a symmetric two-player game in normal form, where each player
has a finite pure-strategy set X ¼ f1; . . . ; Jg, with the mixed-strategy extension
ARTICLE IN PRESS
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
1574
P
DðX Þ ¼ fp 2 RJþ j Jj¼1 pj ¼ 1g. Each player’s payoff is represented by the function
p : X X ! Rþ , where pðx; yÞ denotes the payoff to playing the pure strategy x
when the opponent is playing the pure strategy y. In each of T periods, all
individuals of a finite population, consisting of an even number MX4 of individuals,
are drawn to play this game. The mixed strategy of individual k in period t 2
f1; 2; . . . ; Tg ¼ U is denoted by pk ðtÞ. The pure-strategy realization of individual k is
denoted by xk ðtÞ and that of his opponent (in this period) by yk ðtÞ. Let ðxð0Þ; yð0ÞÞ be
an initial pure-strategy profile, common to all individuals. The sequence
hk ðtÞ ¼ ððxð0Þ; yð0ÞÞ; ðxk ð1Þ; yk ð1ÞÞ; . . . ; ðxk ðt 1Þ; yk ðt 1ÞÞÞ
(1)
is referred to as individual k’s history in period t. Let HðtÞ be the finite set of such
possible histories at time t and let H ¼ [Tt¼1 HðtÞ. We define a learning rule as a
function f : H ! DðX Þ that maps histories to mixed strategies and denote the set of
possible learning rules by F.
The matching procedure can be described as follows. In each of T periods, all
members of the population are randomly matched in pairs to play the game G
against each other. This can be illustrated by an urn with M balls, from which
randomly selected pairs of balls (with equal probability) are drawn successively until
the urn is empty. This procedure is repeated for a total of T periods, and the draws in
each period are independent of the draws in all other periods. Each individual k
receives a payoff pðxk ðtÞ; yk ðtÞÞ in each period and has a private history of realized
strategy profiles.
Let the expected payoff for an individual employing learning rule f in a
heterogeneous population of size M, where the share of individuals employing rule f
is 1 e and the share of individuals employing rule g is e, be denoted by V M ðf ; g; eÞ.
Let F 0 be an arbitrary subset of F containing at least two elements. We define the
following evolutionary stability criterion for learning rules.4
Definition 1. A learning rule f 2 F is evolutionarily stable in the class F 0 for
0
population
size M if, for every g 2 F nf , there exists an e^ g 2 ð2=M; 1Þ such that for all
e 2 0; e^ g such that eM is integer,
V M ðf ; g; eÞ4V M ðg; f ; 1 eÞ.
(2)
3.1. EWA learning
In this paper, we focus on a set of learning rules that can be described by Camerer
and Ho’s (1999) model of EWA learning. These are learning rules such that
individual k’s probability of playing strategy j in period t 2 U can be written as
j
pjk ðtÞ
4
elAk ðt1Þ
¼ PJ
l¼1 e
lAlk ðt1Þ
,
(3)
In the definition, we require a fraction of at least 2=M mutants in order to allow for interaction between
them.
ARTICLE IN PRESS
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
1575
where l is a constant and Ajk ðt 1Þ is the attraction of strategy j.5 The latter is
updated according to the formula
Ajk ðtÞ ¼
fNðt 1ÞAjk ðt 1Þ þ ðd þ ð1 dÞIðj; xk ðtÞÞÞpðj; yk ðtÞÞ
NðtÞ
(4)
for t 2 U, and Ajk ð0Þ is a constant. NðtÞ is a function of time given by
NðtÞ ¼ fð1 kÞNðt 1Þ þ 1
(5)
for t 2 U, and Nð0Þ is a positive constant. Iðj; xk ðtÞÞ is an indicator function which
takes the value of one if xk ðtÞ ¼ j and zero otherwise. Finally, yk ðtÞ is the realized
pure strategy of the opponent in period t, and d, f, and k are constants in the unit
interval.
Note that this class of learning rules includes two of the most common learning
rules used in the literature. When d ¼ 0, k ¼ 0, and Nð0Þ ¼ 1=ð1 fÞ, EWA
reduces
P
to (average) reinforcement learning.6 When d ¼ 1, k ¼ 0 and Ajk ð0Þ ¼ Jl¼1 pðj; lÞqlk ,
where qlk is some initial relative frequency of strategy l, EWA becomes stochastic
fictitious play. Also note that the EWA probabilities (3) depend on cardinal payoffs.
However, multiplying all payoffs and the initial attractions by a positive constant has
the same effect as multiplying l by the same constant.
In order to make the analysis more tractable, we further restrict the set of rules to
EWA learning rules such that k ¼ 0, Nð0Þ ¼ 1=ð1 fÞ, and Ajk ð0Þ ¼ 0 for all k and
j.7 This means that NðtÞ N ð0Þ and that the initial attractions will not generally
correspond exactly to those of stochastic fictitious play. The assumption of k ¼ 0
implies that attractions are a weighted average of lagged attractions and past payoffs
and the assumption of Nð0Þ ¼ 1=ð1 fÞ that weights are fixed. The assumption of
equal initial attractions is motivated by a setting where the players have very limited
information about the game before the first period and where they cannot use
previous experience.8
We denote the set of rules with the above parameter values by F e . Substituting in
(4) gives
Ajk ðtÞ ¼ fAjk ðt 1Þ þ ð1 fÞðd þ ð1 dÞIðj; xk ðtÞÞÞpðj; yk ðtÞÞ
(6)
for t 2 U, and Ajk ð0Þ ¼ 0. The formula in (3) now corresponds to stochastic fictitious
play (with modified initial weights) for d ¼ 1 and to reinforcement learning for
d ¼ 0. The parameter d captures the extent to which the hypothetical payoffs of the
5
Camerer and Ho (1999) note that it is also possible to model probabilities as a power function of
attractors.
6
Camerer and Ho (1999) distinguish between average and cumulative reinforcement, which results if
d ¼ 0, k ¼ 1, and Nð0Þ ¼ 1. The analysis in this paper is based on average reinforcement.
7
Stahl (2003) finds that a time varying NðtÞ only improves the predictive power of the model marginally
and assumes NðtÞ ¼ 1 for all t. He also assumes all initial attractors are zero and uses the updating formula
to determine the attractors in period one.
8
Although the game is fixed in the analysis below, a rationale for the assumption of uniform initial
weight could be a setting where the game is drawn at random from some set of games before the first round
of play.
ARTICLE IN PRESS
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
1576
pure strategies not played in a period are taken into account and f is a constant
determining the relative weights of recent and historical payoffs in the updating of
mixed strategies.
4. Numerical analysis
The analysis is based on Monte Carlo simulations of repeated encounters between
individuals using different learning rules with parameter vectors in the finite grid
D ¼ fðl; f; dÞ : l 2 f1; 10g; f; d 2 f0; 0:25; 0:5; 0:75; 1gg, with representative element
d.9 We focus on games from the three generic categories of 2 2 games: dominance
solvable, coordination, and hawk–dove, with payoffs in the interval ½1; 5.
In the simulations, each member of a population of 20 individuals, among which 2
are mutants with a different learning rule, is randomly matched with another
member in each of T ¼ 10, 20, or 30 periods. The expected payoff to a learning rule
is estimated by computing the mean of the average payoff for all individuals with the
same learning rule in the population and by simulating 1000 such T-period
population interactions. Since the mean payoff difference in each simulation is
independently and identically distributed relative to the mean payoff difference in
another simulation with the same population mixture, the central limit theorem
applies and the mean payoff difference is approximately normally distributed. For
each value of d, the null hypothesis is that the corresponding learning rule is an
ESLR. This hypothesis is rejected if the mean payoff to any mutant rule is
statistically significantly higher than the mean payoff to the incumbent rule in the
class, in accordance with Definition 1 above. More specifically, the decision rule is as
follows. The null hypothesis,
Hd0 :
f d is an ESLR in the class F e ,
is rejected in favor of the alternative hypothesis,
Hd1 :
f d is not an ESLR in the class F e ,
if and only if, for some d 0 2 Dnd,
M
M
V^ ðf d ; f d 0 ; eÞ V^ ðf d 0 ; f d ; 1 eÞ
(7)
o za ,
SE M ðf d ; f d 0 ; eÞ
M
where V^ ð:; :; :Þ denotes the estimated mean of the average payoff, SE M ð:; :; :Þ
denotes the sample standard error of the difference in average payoffs, computed
over the 1000 simulations, and za is the critical value of the standard normal
distribution at the a% level of significance.10
9
f ¼ 1 implies that each strategy is played with equal probability in every period and it is included for
the sake of comparison.
10
For detailed simulation results with heterogeneous populations, see the supplementary archive of this
journal.
ARTICLE IN PRESS
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
2
1
1577
4
3
Fig. 1.
3
2
5
1
Fig. 2.
4.1. Dominance solvable games
In dominance solvable games, a learning rule that plays the dominant strategy will
fare better than one that plays the dominated strategy against any opponent. We
would thus expect the ESLR to be the rule that induces play of the dominant strategy
with the highest probability. The theoretical results in Appendix A help us find the
parameter values of this rule.
To start with, Lemma 1 says that for any individual and any history, the attraction
of the dominant strategy is strictly larger than that of the dominated strategy for
^
period tX1 if d is above d^ ¼ maxfpð2; 1Þ=pð1; 1Þ; pð2; 2Þ=pð1; 2Þg, where 0odo1
(for
3
2
^
^
the game in Fig. 1, d ¼ 4, and for the game in Fig. 2, d ¼ 3Þ. Lemma 2 says that if the
attraction of strategy one is larger than that of strategy two, then the conditional
probability of playing strategy one is increasing in l and converges to one as l tends
^ the share
to infinity. Proposition 1 uses these two results to prove that, if d is above d,
of incumbents playing the dominant strategy in period tX2 in an infinite population
tends to one as l tends to infinity. Hence, it appears, among the learning rules with
high d, that we should expect the rules with higher l to do better.
According to Proposition 2, the share of the population playing the dominant
strategy in period two is above 0.5, increasing in d and, for sufficiently high d,
decreasing in f. Simulations of a homogeneous population (see Figs. B1–B4 and
Tables B1 and B2 in Appendix B.1) show that the share is increasing in d also for
later periods, but for rules with small f, differences are small and the relationship
not necessarily monotonic. We would thus expect learning rules with l ¼ 10, d ¼ 1,
and small f to do particularly well.
Conversely, we would expect learning rules with high l and d ¼ 0 to perform
particularly bad in games with strictly positive payoffs, since they have a tendency to
lock in on a particular strategy in a first period. The reason is that no matter what an
individual plays in the first period, this strategy will be played with a probability of
approximately one in the second period. If the strategy is played in period two, then
it will be repeated with a probability of approximately one in the third period, etc.
This feature has previously been noted by Fudenberg and Levine (1998).
Recall that the null hypothesis that a learning rule with a particular d is an ESLR
can be rejected if the z-statistic of the simulated difference in average payoffs is
ARTICLE IN PRESS
1578
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
smaller than the critical value of the standard normal distribution, za , for some
mutant learning rule in the class, different from f d . For the game in Fig. 1 (where
payoffs correspond to the row player) and T ¼ 10; 20; 30, the null of an ESLR can be
rejected at the 1% level for all learning rules except the three rules with l ¼ 10,
f 2 f0; 0:25; 0:5g, and d ¼ 1.
For the game in Fig. 2, we obtain similar result: at the 5% level and for
T ¼ 10; 20; 30, the null of an ESLR can be rejected for all learning rules except the
three rules with l ¼ 10, f 2 f0; 0:25; 0:5g, and d ¼ 1. The only difference is that at the
1% level we also cannot reject the learning rules with l ¼ 10, f 2 f0; 0:25g, and
d ¼ 0:75.
4.2. Coordination games
In 2 2 coordination games, we conjecture that the learning rule that gives the
fastest convergence to one of the pure equilibria in a homogeneous population will
be an ESLR. When we simulate such a setting, we find that the parameters of the
learning rule with the fastest convergence depend on the payoffs of the game.
In the game in Fig. 3, the pure-strategy profile (1,1) is the unique risk and Pareto
dominant equilibrium. The learning rule with l ¼ 10, d ¼ 1 and f ¼ 0:75 tends to
this equilibrium and gives the fastest rate of convergence of the average strategy to
an equilibrium in a homogeneous population (see Figs. B5 and B6 and Table B3 in
Appendix B.3).
What is the intuition behind this result? It is clear that learning rules with high l
and d ¼ 0 lock in on initial strategies for the same reasons as in dominance solvable
games. However, whereas the fastest convergence is achieved for d ¼ 1 and f ¼ 0 in
such games, here there is a trade-off between a high d and a low f, and the optimal
combination of the two seems to depend on the payoffs of the game in a non-trivial
way.
A high f means that a lot of weight is put on previous period’s attractions when
the attractions are updated, creating inertia on an individual level. In the extreme
case when f ¼ 1, there is no updating at all and both strategies are played with equal
probabilities in every period.
On the other hand, if d and l are high, then a low f implies that the share of
the population playing a particular strategy may get stuck at the initial level (see
Table 5). Consider the extreme case when all individuals use learning rules with
l ¼ 10, f ¼ 0, and d ¼ 1 to play the game in Fig. 3. In this situation, only two
attraction vectors are possible in each period tX1: (A1k ðtÞ,A2k ðtÞ) is either ð5; 2Þ (if the
opponent plays strategy one) or ð1; 3Þ (if the opponent plays strategy two). The first
vector induces play of strategy one with a probability of approximately one and the
5
2
1
3
Fig. 3.
ARTICLE IN PRESS
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
4
2
1579
1
3
Fig. 4.
second vector induces play of strategy two with a probability of approximately one.
This implies that if two individuals with equal attractions are matched, they will
both play the same strategy in the next period with a probability of approximately
one. If two individuals with different attractions are matched, they will both switch
strategy with a probability of approximately one. Hence, with a high probability the
initial shares of the two strategies in the population will be constant in subsequent
periods.
The simulations with a heterogeneous population are consistent with the results on
the speed of convergence for a homogeneous population. For the game in Fig. 3 with
a heterogeneous population interacting for T ¼ 20 or 30 periods, the null hypothesis
of an ESLR can be rejected at the 5% level for all learning rules except for the one
with parameter vector l ¼ 10, f ¼ 0:75, d ¼ 1. If T ¼ 10 or the level of significance
is 1%, we also cannot reject the rule with l ¼ 10, f ¼ 0:5, d ¼ 0:75.
However, these results are sensitive to changes in payoffs. If we, for instance,
change the payoff of the strategy profile (1,1) to 4, such that both equilibria become
risk dominant (see Fig. 4), the learning rule with the fastest convergence of the
average strategy played to one of the equilibria tends to (2,2) and has parameters
l ¼ 10, f ¼ 0, and d ¼ 0:5 (see Table B4 and Figs. B7 and B8 in Appendix B.4). The
rule with l ¼ 10, f ¼ 0:75, and d ¼ 1 also gives fast convergence in this case, but a
histogram reveals that for this rule the population average converges to each
equilibria with a probability of approximately one-half.
When we simulate the population interaction for this game, the null hypothesis of
an ESLR can be rejected at the 5% level for all learning rules except for the ones with
parameter vectors such that l ¼ 10, f 2 f0; 0:25g, d ¼ 0:5 (for all time horizons),
l ¼ 10, f ¼ 0:75, d ¼ 1, and l ¼ 10, f ¼ 0:5, d ¼ 0, 75 (for T ¼ 20, 30). At the 1%
level we also cannot reject the rule with l ¼ 10, f ¼ 0:75, d ¼ 0:75 for T ¼ 20, 30.
4.3. Hawk– dove games
In the hawk–dove game, there is a unique and completely mixed Nash equilibrium
which is also evolutionarily stable. A conjecture is that a learning rule that converges
fast to a neighborhood of this equilibrium will be an ESLR.
In the hawk–dove game in Fig. 5, the unique Nash equilibrium is to play strategy
one with probability 0.6. When we simulate the interaction for a homogeneous
population, the learning rules with l ¼ 10, f ¼ 0:75, and d 2 f0:75; 1g appear to have
the fastest convergence of the population average to the equilibrium (see Figs. B9
and B10 and Table B5 in Appendix B.5).
Learning rules that will do particularly bad are those that lock in on a strategy
different from the equilibrium one at an early stage. This is the case for learning rules
with high d and l, and low f, for similar reasons as in the coordination games and
ARTICLE IN PRESS
1580
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
1
3
5
2
Fig. 5.
1
4
3
2
Fig. 6.
for learning rules with high l and d ¼ 0 for the same reason as explained above
under dominance solvable games.
Consistent with the findings for the homogeneous population, the ESLR
hypothesis can be rejected at the 5% level for all rules except the ones with
l ¼ 10, f ¼ 0:75, and d 2 f0:75; 1g when T ¼ 10; 20; 30. At the 1% level, the
hypothesis also cannot be rejected for the learning rule with l ¼ 10, f ¼ 0:5, and
d ¼ 0:75.
However, the evolutionarily stable value of d is affected by changes in payoffs. For
the game in Fig. 6, where the unique Nash equilibrium is to play strategy one with
probability 0:25, the learning rule with l ¼ 10, f ¼ 0:75, and d ¼ 0:5 appears to give
the fastest convergence of the average strategy to the equilibrium in a homogeneous
population (see Figs. B11 and B12 and Table B6 in Appendix B.6). The simulations
with a heterogeneous population confirm this finding: the ESLR hypothesis cannot
be rejected for this rule for T ¼ 20; 30.
4.4. Summary of simulation results
Tables 1 and 2 summarize the results of the simulations. In all games, the ESLR
hypothesis can be rejected for all learning rules with l ¼ 1 and all learning rules
with d ¼ 0. Learning rules with high attraction sensitivity and either low degrees
of hypothetical reinforcement or high degree of hypothetical reinforcement and
low discount factor risk getting stuck at an initial population distribution of
strategies. Some type of stochastic fictitious play (d ¼ 1) is an ESLR in all games
except one of the hawk–dove games, but there is no learning rule which is an ESLR
in all settings.
5. Discussion
The results in this paper indicate that an ESLR is indeed a rule for learning an
ESS, as predicted by Harley (1981) in a different framework. More precisely, it
appears to be the rule with the fastest convergence of the population’s strategy
distribution to such a strategy.
ARTICLE IN PRESS
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
1581
Table 1
Parameters for which the ESLR hypothesis cannot be rejected at the 5% level
Game
Dominance solvable (Fig. 1)
Dominance solvable (Fig. 2)
Coordination (Fig. 3)
Coordination (Fig. 4)
Hawk–dove (Fig. 5)
Hawk–dove (Fig. 6)
T
10; 20; 30
10; 20; 30
10; 20; 30
10
20; 30
10; 20; 30
20; 30
10; 20; 30
20; 30
10
ESLR(s) at the 5% level
l
f
d
10
10
10
10
10
10
10
10
10
No ESLR
0; 0:25; 0:5
0; 0:25; 0:5
0:75
0:5
0:75
0; 0:25
0:5
0:75
0:75
1
1
1
0:75
1
0:5
0:75
0:75; 1
0:5
Table 2
Parameters for which the ESLR hypothesis cannot be rejected at the 1% level
Game
Dominance solvable (Fig. 1)
Dominance solvable (Fig. 2)
Coordination (Fig. 3)
Coordination (Fig. 4)
Hawk–dove (Fig. 5)
Hawk–dove (Fig. 6)
T
10; 20; 30
10; 20; 30
10; 20; 30
10; 20; 30
10; 20; 30
10; 20; 30
10; 20; 30
10; 20; 30
20; 30
10; 20; 30
10; 20
20; 30
10
ESLR(s) at the 1% level
l
f
d
10
10
10
10
10
10
10
10
10
10
10
10
No ESLR
0; 0:25; 0:5
0; 0:25; 0:5
0; 0:25
0:75
0:5
0:75
0; 0:25
0:5
0:75
0:75
0:5
0:75
1
1
0:75
1
0:75
1
0:5
0:75
0:75
0:75; 1
0:75
0:5
The ESLR hypothesis can be rejected for reinforcement learning in all of the
games analyzed. Our analysis of homogeneous populations confirms Hopkins’
(2002) finding (in a different matching environment) that stochastic fictitious play
gives rise to faster learning. When attraction sensitivity is high, reinforcement
learners even risk getting stuck at an initial strategy, as previously noted by
Fudenberg and Levine (1998).
However, it is important to bear in mind that reinforcement learning has lower
information and computational requirements than other EWA rules. The perceived
evolutionary disadvantage of this learning rule is thus conditional on sufficiently low
costs of obtaining the additional information and performing the extra calculations
necessary for hypothetical reinforcement. On the other hand, the results in this paper
ARTICLE IN PRESS
1582
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
may be interpreted as an evolutionary rationale for learning rules with such low costs
(see Hommes, 2006; Robson, 2001 for more on evolutionary advantages to strategic
sophistication).
In all of the games studied, we cannot reject the ESLR hypothesis for at least one
learning rule with high or intermediate level of hypothetical reinforcement, but
there are differences in the stable values of the parameters between games. Our
analysis assumes the learning rules cannot condition on the type of game being
played, but the differences between games provide evolutionary support for such
conditioning.
This seems consistent with most experimental evidence. Many papers find that
EWA learning does a good job at explaining the observed behaviors in experiments,
but the parameter estimates vary widely from game to game. Camerer and Ho
(1999) estimate separate sets of parameters for asymmetric constant-sum games,
median-action games, and beauty-contest games. Their estimates of the degree of
hypothetical enforcement d are generally around 0.5, those of the discount factor f
in the range of 0.8–1.0, those of the attraction sensitivity l varies from 0:2 to 18, and
those of k are in the range 0–1. Stochastic reinforcement learning and fictitious play
are generally rejected in favor of an intermediate model.
Camerer et al. (2002) reports EWA parameter estimates of d and f by themselves
and other authors from 31 data sets. The estimated d ranges from 0 in 4 4 mixedstrategy games, patent race Games with symmetric players, and functional alliance
games to 0.95 in p-beauty-contest games, with an average of about 0.3 and a median
of about 0.2. The estimated f ranges from 0.11 in p-beauty games to 1.04 in 4 4
mixed-strategy games with an average and median of about 0:9.
Ho et al. (2002) reports EWA parameter estimates for seven games. The estimated
d ranges from 0.27 in mixed-strategy games to 0.89 in median-action games, with a
value of 0.76 when all games are pooled. The estimated f ranges from 0.36 in the
p-beauty contest to 0.98 in mixed-strategy games, with a pooled estimate of 0.78. The
estimated value of l ranges from 0.19 in pot games to 42.21 in patent race games,
with a pooled estimate of 2.95.
Stahl (2003) conducts experiments with finite symmetric two-player games with
and without symmetric pure-strategy equilibria. He pools data from several such
games and estimates a d of 0.67, a f of 0.33, and a l of 0.098 for games with payoffs
between 0 and 100. He assumes that the initial attractions of the EWA model are
zero and uses the updating formula to determine their values in period one.
Most experimental studies thus lend support to the hypothesis that people
take hypothetical payoffs into account, but the estimated d is generally smaller than
one. The estimated f is generally in the upper half of the unit interval, consistent
with our findings for the hawk–dove and coordination games. Finally, the estimated
l varies widely, which is not surprising given its aforementioned dependence on
payoffs.
One should, however, be cautious in making a direct comparison between the
results in this paper and the experimental findings. Apart from certain econometric
difficulties in estimating learning models (see Salmon, 2001; Wilcox, 2006), the games
played in most experiments differ considerably from the ones analyzed in this paper.
ARTICLE IN PRESS
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
1583
Moreover, most studies (with the exception of Stahl, 2003) do not restrict initial
attractions to zero or assume NðtÞ is constant as we do in this paper.
6. Conclusion
In this paper, we define an evolutionary stability criterion for learning rules. We
then apply this criterion to a class of rules which contains stochastic versions of two
of the most well-known learning rules, reinforcement learning and fictitious play, as
well as intermediate rules in terms of hypothetical reinforcement. We perform Monte
Carlo simulations for three types of 2 2 games of a matching scheme where all
members of a large population are randomly matched in pairs in every period. We
find that the learning rule with the fastest convergence to an ESS in a homogeneous
population is evolutionarily stable in all games. Reinforcement learning is never
evolutionarily stable, mainly because it either gets stuck at an initial mixed strategy
or converges more slowly than other learning rules. Some types of stochastic
fictitious play are evolutionarily stable in all but one game, but the stable value of the
discount factor for this rule depends on the type of game. For two types of games we
also find evidence of ESLRs with intermediate levels of hypothetical reinforcement.
The results appear consistent with the positive levels of hypothetical reinforcement
and differences in parameter estimates between games found in most experiments.
Acknowledgments
I am indebted to Jörgen W. Weibull for many inspiring conversations. I am also
grateful to Colin Camerer, Francisco Peñaranda, Ariel Rubinstein, Maria SaezMarti, the editor, and two anonymous referees, as well as seminar audiences at the
Cornell University, the Society for Economic Dynamics 2001 Annual Meeting in
Stockholm, and the 16th Annual Congress of the European Economic Association in
Lausanne, for helpful comments. Part of this work was completed at the Department
of Economics of Cornell University, which I thank for its hospitality. Financial
support from the Jan Wallander and Tom Hedelius Foundation, the BBVA
Foundation through the project ‘Aprender a jugar’, the Spanish Ministry of
Education and Science through Grants SEJ2004-03149 and SEJ2006-09993, and
Barcelona Economics (CREA) is also gratefully acknowledged.
Appendix A. Theoretical results
Let G be a symmetric dominance solvable 2 2 game where strategy one strictly
dominates strategy two and all payoffs are positive. Further, assume that all learning
rules belong to F e and that f 2 ½0; 1Þ.
Lemma 1. If d4d^ ¼ maxfpð2; 1Þ=pð1; 1Þ; pð2; 2Þ=pð1; 2Þg, then A1k ðtÞ4A2k ðtÞ for
all tX1.
ARTICLE IN PRESS
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
1584
Proof. This follows immediately from the definition of Ajk ðtÞ by observing that
^ and any history of strategy realizations,
A1k ð0Þ ¼ A2k ð0Þ ¼ 0, and that for d4d,
ðd þ ð1 dÞIð1; xk ðtÞÞÞpð1; yk ðtÞÞ4ðd þ ð1 dÞIð2; xk ðtÞÞÞpð2; yk ðtÞÞ.
(A.1)
Lemma 2.
1
2
qðp1k ðtÞ=p2k ðtÞÞ
¼ ðA1k ðt 1Þ A2k ðt 1ÞÞelðAk ðt1ÞAk ðt1ÞÞ
ql
and
8
>
0 if A1k ðt 1ÞoA2k ðt 1Þ;
>
<
1
2
1
lim p1k ðtÞ ¼ 2 if Ak ðt 1Þ ¼ Ak ðt 1Þ;
>
l!1
>
: 1 if A1 ðt 1Þ4A2 ðt 1Þ:
k
k
(A.2)
(A.3)
Proof. This follows immediately from the definition of pjk ðtÞ.
Now suppose that an infinite population of measure one, composed of a share
ð1 eÞ40 of incumbents and a share eX0 of mutants, is repeatedly matched
to play the dominance solvable game. Let the fraction of the incumbents
playing strategy one in period tX1 be denoted by x1 ðtÞ. Assume that the incumbents
have parameter vector ðl; f; dÞ and that the mutants have the parameter vector
ðl0 ; f0 ; d0 Þ.
^ then liml!1 x1 ðtÞ ¼ 1 for tX2.
Proposition 1. If d4d,
Proof. Consider the set of incumbents with minimal difference A1k ðt 1Þ A2k ðt 1Þ
at time tX2. The measure of this set is a function of l (the difference is not),
but it is bounded from above by 1 e and below by zero, and by Lemma 1,
A1k ðt 1Þ4A2k ðt 1Þ. Hence, by Lemma 2 the fraction of incumbents playing
strategy one in this set in period tX2 converges to one as l tends to infinity. It is
obvious that the same must hold for the set of all other incumbents, proving the
statement.
Proposition 2. The following holds for x1 ð2Þ:
(i) x1 ð2Þ is strictly increasing in d.
(ii) x1 ð2Þ40:5.
(iii) For d sufficiently close to 1, x1 ð2Þ is strictly decreasing in f.
Proof. Assume without loss of generality that strategy one is dominant. To prove (i),
note that
2
1
2
1
qp1k ðt þ 1Þ=qA1k ðtÞ ¼ lelðAk ðtÞAk ðtÞÞ =ð1 þ elðAk ðtÞAk ðtÞÞ Þ2 ,
qp1k ðt þ 1Þ=qA2k ðtÞ ¼ le
lðA2k ðtÞA1k ðtÞÞ
=ð1 þ e
lðA2k ðtÞA1k ðtÞÞ
Þ2 .
ðA:4Þ
ðA:5Þ
ARTICLE IN PRESS
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
1585
If player k uses strategy one and his opponent strategy j ¼ 1; 2 in period t, then
qA1k ðtÞ=qd ¼ 0 and qA2k ðtÞ=qd ¼ ð1 fÞpð2; jÞ. If he uses strategy two and his
opponent strategy j ¼ 1; 2, then qA1k ðtÞ=qd ¼ ð1 fÞpð1; jÞ and qA2k ðtÞ=qd ¼ 0.
Exactly one-fourth of the incumbents (and mutants if they have positive
measure) obtain the payoff pð1; 1Þ in period one, another fourth the payoff pð1; 2Þ,
a third fourth the payoff pð1; 2Þ, and the final fourth the payoff pð2; 2Þ. This means
that
4qx1 ð2Þ=qd ¼ ð1 fÞpð2; 1Þlelð1fÞðdpð2;1Þpð1;1ÞÞ =ð1 þ elð1fÞðdpð2;1Þpð1;1ÞÞ Þ2
ð1 fÞpð2; 2Þlelð1fÞðdpð2;2Þpð1;2ÞÞ =ð1 þ elð1fÞðdpð2;2Þpð1;2ÞÞ Þ2
þ ð1 fÞpð1; 1Þlelð1fÞðpð2;1Þdpð1;1ÞÞ =ð1 þ elð1fÞðpð2;1Þdpð1;1ÞÞ Þ2
þ ð1 fÞpð1; 2Þlelð1fÞðpð2;2Þdpð1;2ÞÞ =ð1 þ elð1fÞðpð2;2Þdpð1;2ÞÞ Þ2 .
ðA:6Þ
The function
ex =ð1 þ ex Þ2 ¼ 1=ðex þ 2 þ ex Þ
(A.7)
is positive, symmetric around x ¼ 0, and decreasing for x40. Moreover, since in the
dominance solvable game, 0opð2; jÞopð1; jÞ, for j ¼ 1; 2, it follows that
jpð2; jÞ dpð1; jÞjpjdpð2; jÞ pð1; jÞj
(A.8)
with equality only for d ¼ 1. These two observations together imply that qx1 ð2Þ=
qd40 for all 0pdp1.
To prove (ii), note that equal shares of the incumbents obtain the four possible
payoffs in period one. If d ¼ 0, then p1k ð2Þ is higher among the individuals
who played strategy one and were matched with an individual playing strategy j in
the first period, than p2k ð2Þ among the individuals who played strategy two
and were matched with an individual playing strategy j in the first period. Since
this holds for any j ¼ 1; 2, it is obvious that x1 ð2Þ40:5. By (i), this result extends to
any d 2 ½0; 1.
(iii) If player k uses strategy one and his opponent strategy j ¼ 1; 2 in period t, then
qA1k ðtÞ=qf ¼ pð1; jÞ and qA2k ðtÞ=qf ¼ dpð2; jÞ. If he uses strategy two and his
opponent strategy j ¼ 1; 2, then qA1k ðtÞ=qf ¼ dpð1; jÞ and qA2k ðtÞ=qf ¼ pð2; jÞ.
This gives the following partial:
4qx1 ð2Þ=qf ¼ ðdpð2; 1Þ pð1; 1ÞÞlelð1fÞðdpð2;1Þpð1;1ÞÞ =ð1 þ elð1fÞðdpð2;1Þpð1;1ÞÞ Þ2
þ ðdpð2; 2Þ pð1; 2ÞÞlelð1fÞðdpð2;2Þpð1;2ÞÞ =ð1 þ elð1fÞðdpð2;2Þpð1;2ÞÞ Þ2
þ ðpð2; 1Þ dpð1; 1ÞÞlelð1fÞðpð2;1Þdpð1;1ÞÞ =ð1 þ elð1fÞðpð2;1Þdpð1;1ÞÞ Þ2
þ ðpð2; 2Þ dpð1; 2ÞÞlelð1fÞðpð2;2Þdpð1;2ÞÞ =ð1 þ elð1fÞðpð2;2Þdpð1;2ÞÞ Þ2 .
ðA:9Þ
By continuity it immediately follows that the expression is negative for d
sufficiently close to 1.
ARTICLE IN PRESS
1586
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
Appendix B. Simulation results for homogeneous populations
The following diagrams and tables illustrate the average share of individuals
playing strategy one in each period in a homogeneous population of size 20 over
1000 simulations with l ¼ 10. In the tables, the value of d is given by the entry
in the first row, the value of f by the entry in the second row, and the time
period by the entry in the left-most column. The learning rule with the fastest
convergence to the equilibrium strategy is indicated in bold. The average population share playing strategy one for various cases are shown in Figs. B1–B12 and
Tables B1–B6.
B.1. Dominance solvable game in Fig. 1
For further details, see Table B1 and Figs. B1 and B2
B.2. Dominance solvable game in Fig. 2
For further details, see Table B2 and Figs. B3 and B4
B.3. Coordination game in Fig. 3
For further details, see Table B3 and Figs. B5 and B6
B.4. Coordination game in Fig. 4
For further details, see Table B4 and Figs. B7 and B8
B.5. Hawk– dove game in Fig. 5
For further details, see Table B5 and Figs. B9 and B10
B.6. Hawk– dove game in Fig. 6
For further details, see Table B6 and Figs. B11 and B12
Appendix C. Supplementary data
Supplementary data associated with this article can be found in the online version
at doi: 10.1016/j.jedc.2007.06.008.
Table B1
Average population share playing strategy one in the dominance solvable game in Fig. 1
0.00
0.25
0.00
0.50
0.00
0.75
0.00
1.00
0.25
0.00
0.25
0.25
0.25
0.50
0.25
0.75
0.25
1.00
0.50
0.00
0.50
0.25
0.50
0.50
0.50
0.75
0.50
1.00
0.75
0.00
0.75
0.25
0.75
0.50
0.75
0.75
0.75
1.00
1.00
0.00
1.00
0.25
1.00
0.50
1.00
0.75
1.00
1.00
t¼1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.508
0.508
0.508
0.508
0.508
0.508
0.508
0.508
0.508
0.508
0.508
0.508
0.508
0.508
0.508
0.508
0.508
0.508
0.508
0.508
0.508
0.508
0.508
0.505
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.507
0.502
0.521
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.523
0.497
0.503
0.502
0.502
0.496
0.498
0.500
0.497
0.499
0.491
0.495
0.505
0.498
0.506
0.497
0.494
0.492
0.497
0.499
0.500
0.500
0.502
0.497
0.502
0.501
0.496
0.506
0.502
0.500
0.500
0.502
0.504
0.506
0.507
0.509
0.511
0.512
0.514
0.516
0.517
0.520
0.521
0.523
0.524
0.526
0.528
0.529
0.531
0.532
0.533
0.535
0.537
0.538
0.540
0.542
0.544
0.545
0.546
0.548
0.549
0.500
0.505
0.506
0.507
0.508
0.509
0.509
0.510
0.510
0.511
0.512
0.512
0.513
0.514
0.514
0.515
0.515
0.516
0.517
0.517
0.518
0.518
0.519
0.519
0.520
0.520
0.521
0.522
0.523
0.523
0.504
0.523
0.526
0.527
0.528
0.528
0.528
0.529
0.529
0.530
0.530
0.530
0.531
0.531
0.531
0.532
0.532
0.532
0.532
0.533
0.533
0.533
0.534
0.534
0.535
0.535
0.535
0.535
0.536
0.536
0.502
0.555
0.568
0.572
0.574
0.574
0.575
0.575
0.575
0.576
0.576
0.576
0.576
0.576
0.577
0.577
0.577
0.577
0.577
0.577
0.577
0.577
0.577
0.578
0.578
0.578
0.578
0.578
0.578
0.578
0.500
0.499
0.499
0.506
0.503
0.504
0.493
0.500
0.502
0.505
0.501
0.503
0.506
0.499
0.500
0.501
0.503
0.498
0.501
0.501
0.498
0.502
0.501
0.499
0.496
0.504
0.508
0.497
0.498
0.499
0.501
0.624
0.736
0.830
0.896
0.938
0.966
0.981
0.989
0.994
0.997
0.998
0.999
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
0.503
0.628
0.687
0.742
0.795
0.841
0.881
0.914
0.938
0.958
0.971
0.981
0.988
0.992
0.994
0.996
0.997
0.998
0.999
0.999
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
0.496
0.625
0.676
0.716
0.750
0.782
0.814
0.844
0.869
0.892
0.913
0.929
0.942
0.953
0.961
0.968
0.974
0.979
0.983
0.986
0.989
0.991
0.992
0.994
0.995
0.997
0.997
0.997
0.998
0.998
0.499
0.638
0.705
0.746
0.775
0.799
0.821
0.841
0.858
0.874
0.888
0.901
0.913
0.924
0.933
0.941
0.948
0.955
0.960
0.965
0.970
0.973
0.977
0.980
0.982
0.984
0.986
0.988
0.989
0.990
0.502
0.495
0.495
0.496
0.497
0.502
0.496
0.500
0.504
0.501
0.497
0.501
0.497
0.508
0.498
0.499
0.498
0.501
0.499
0.503
0.498
0.501
0.504
0.494
0.493
0.502
0.500
0.499
0.499
0.496
0.499
0.877
0.991
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
0.501
0.870
0.988
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
0.499
0.850
0.979
0.999
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
0.501
0.806
0.949
0.990
0.999
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
0.500
0.503
0.505
0.506
0.506
0.502
0.499
0.501
0.504
0.500
0.498
0.497
0.502
0.499
0.504
0.499
0.495
0.501
0.507
0.501
0.495
0.501
0.502
0.498
0.498
0.499
0.501
0.494
0.497
0.494
0.503
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
0.504
0.999
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
0.499
0.993
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
0.501
0.921
0.988
0.997
0.999
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
0.498
0.498
0.500
0.503
0.502
0.502
0.497
0.505
0.500
0.503
0.502
0.501
0.495
0.497
0.498
0.501
0.500
0.501
0.496
0.501
0.496
0.500
0.498
0.501
0.505
0.504
0.501
0.501
0.496
0.497
ARTICLE IN PRESS
0.00
0.00
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
d
f
1587
ARTICLE IN PRESS
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
1588
1.1
Av. fraction playing strategy one
1.0
0.9
0.8
=0
= 0.25
= 0.5
= 0.75
=1
0.7
0.6
0.5
0.4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Period
Fig. B1. Average population share playing strategy one in the dominance solvable game in Fig. 1 for
different values of f, holding d ¼ 1 fixed.
1.1
Av. fraction playing strategy one
1.0
0.9
0.8
0.7
=0
= 0.25
= 0.5
= 0.75
=1
0.6
0.5
0.4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Period
Fig. B2. Average population share playing strategy one in the dominance solvable game in Fig. 1 for
different values of d, holding f ¼ 0 fixed.
Table B2
Average population share playing strategy one in the dominance solvable game in Fig. 2
0.00
0.25
0.00
0.50
0.00
0.75
0.00
1.00
0.25
0.00
0.25
0.25
0.25
0.50
0.25
0.75
0.25
1.00
0.50
0.00
0.50
0.25
0.50
0.50
0.50
0.75
0.50
1.00
0.75
0.00
0.75
0.25
0.75
0.50
0.75
0.75
0.75
1.00
1.00
0.00
1.00
0.25
1.00
0.50
1.00
0.75
1.00
1.00
t¼1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.499
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.502
0.523
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.525
0.495
0.503
0.496
0.496
0.493
0.501
0.500
0.501
0.494
0.499
0.500
0.500
0.498
0.506
0.501
0.498
0.504
0.499
0.503
0.500
0.493
0.494
0.500
0.495
0.502
0.496
0.495
0.500
0.501
0.498
0.496
0.732
0.794
0.831
0.854
0.871
0.884
0.895
0.904
0.912
0.918
0.921
0.926
0.930
0.933
0.936
0.939
0.941
0.944
0.945
0.947
0.948
0.950
0.952
0.953
0.954
0.956
0.957
0.957
0.959
0.500
0.714
0.748
0.771
0.788
0.802
0.813
0.824
0.831
0.839
0.847
0.854
0.861
0.866
0.870
0.874
0.878
0.882
0.885
0.888
0.891
0.894
0.896
0.898
0.899
0.902
0.903
0.905
0.907
0.908
0.497
0.690
0.716
0.732
0.741
0.749
0.756
0.761
0.765
0.771
0.775
0.780
0.784
0.787
0.791
0.795
0.797
0.800
0.802
0.805
0.807
0.809
0.812
0.814
0.816
0.817
0.820
0.821
0.822
0.824
0.494
0.673
0.709
0.722
0.730
0.735
0.738
0.740
0.741
0.743
0.744
0.745
0.747
0.748
0.749
0.749
0.750
0.751
0.752
0.753
0.754
0.755
0.755
0.756
0.757
0.757
0.758
0.759
0.759
0.760
0.500
0.499
0.499
0.502
0.500
0.497
0.501
0.501
0.499
0.492
0.503
0.497
0.500
0.503
0.504
0.495
0.496
0.506
0.493
0.502
0.503
0.494
0.500
0.503
0.501
0.509
0.497
0.500
0.495
0.507
0.494
0.753
0.810
0.845
0.870
0.885
0.898
0.908
0.914
0.921
0.928
0.932
0.936
0.940
0.943
0.946
0.947
0.949
0.952
0.954
0.956
0.957
0.958
0.959
0.960
0.961
0.962
0.963
0.964
0.965
0.495
0.753
0.816
0.851
0.870
0.884
0.898
0.906
0.914
0.922
0.929
0.933
0.937
0.939
0.941
0.945
0.948
0.951
0.952
0.955
0.956
0.959
0.959
0.961
0.962
0.963
0.964
0.965
0.966
0.966
0.504
0.770
0.825
0.855
0.875
0.891
0.901
0.909
0.917
0.923
0.929
0.934
0.938
0.941
0.943
0.945
0.947
0.950
0.952
0.954
0.956
0.958
0.959
0.960
0.961
0.962
0.963
0.964
0.965
0.965
0.500
0.793
0.853
0.879
0.895
0.906
0.914
0.919
0.923
0.928
0.931
0.934
0.937
0.939
0.941
0.943
0.945
0.947
0.948
0.950
0.951
0.952
0.954
0.955
0.956
0.957
0.958
0.959
0.959
0.960
0.500
0.500
0.499
0.496
0.496
0.502
0.502
0.499
0.494
0.498
0.494
0.505
0.500
0.496
0.497
0.497
0.491
0.497
0.506
0.498
0.502
0.498
0.504
0.501
0.497
0.501
0.500
0.497
0.501
0.489
0.499
0.980
0.998
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
0.502
0.965
0.997
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
0.497
0.939
0.993
0.999
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
0.501
0.908
0.979
0.996
0.999
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
0.495
0.504
0.501
0.497
0.507
0.498
0.499
0.501
0.501
0.501
0.501
0.505
0.500
0.500
0.505
0.495
0.501
0.498
0.501
0.494
0.508
0.504
0.498
0.499
0.497
0.503
0.497
0.496
0.505
0.494
0.495
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
0.506
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
0.504
0.996
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
0.498
0.963
0.992
0.999
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
0.501
0.495
0.497
0.498
0.499
0.498
0.498
0.505
0.498
0.507
0.502
0.505
0.499
0.504
0.498
0.504
0.505
0.505
0.500
0.500
0.499
0.510
0.498
0.501
0.494
0.502
0.501
0.503
0.501
0.501
ARTICLE IN PRESS
0.00
0.00
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
d
f
1589
ARTICLE IN PRESS
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
1590
1.1
Av. fraction playing strategy one
1.0
0.9
0.8
0.7
=0
= 0.25
= 0.5
= 0.75
=1
0.6
0.5
0.4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Period
Fig. B3. Average population share playing strategy one in the dominance solvable game in Fig. 2 for
different values of f, holding d ¼ 1 fixed.
1.1
Av. fraction playing strategy one
1.0
0.9
0.8
0.7
=0
= 0.25
= 0.5
= 0.75
=1
0.6
0.5
0.4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Period
Fig. B4. Average population share playing strategy one in the dominance solvable game in Fig. 2 for
different values of d, holding f ¼ 0 fixed.
Table B3
Average population share playing strategy one in the coordination game in Fig. 3
0.00
0.25
0.00
0.50
0.00
0.75
0.00
1.00
0.25
0.00
0.25
0.25
0.25
0.50
0.25
0.75
0.25
1.00
0.50
0.00
0.50
0.25
0.50
0.50
0.50
0.75
0.50
1.00
0.75
0.00
0.75
0.25
0.75
0.50
0.75
0.75
0.75
1.00
1.00
0.00
1.00
0.25
1.00
0.50
1.00
0.75
1.00
1.00
t¼1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.504
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.510
0.501
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.503
0.486
0.485
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.484
0.496
0.491
0.506
0.496
0.501
0.503
0.500
0.496
0.505
0.503
0.494
0.498
0.493
0.500
0.500
0.509
0.498
0.502
0.503
0.501
0.500
0.502
0.497
0.505
0.504
0.500
0.502
0.506
0.502
0.492
0.501
0.483
0.463
0.444
0.425
0.407
0.392
0.375
0.358
0.341
0.324
0.309
0.294
0.280
0.264
0.250
0.237
0.223
0.212
0.201
0.190
0.179
0.167
0.156
0.147
0.140
0.130
0.122
0.114
0.106
0.504
0.470
0.459
0.452
0.447
0.442
0.436
0.431
0.426
0.420
0.414
0.410
0.404
0.398
0.392
0.386
0.381
0.375
0.369
0.363
0.357
0.351
0.345
0.340
0.334
0.328
0.323
0.317
0.312
0.306
0.503
0.451
0.436
0.429
0.426
0.423
0.421
0.418
0.416
0.414
0.411
0.409
0.407
0.405
0.403
0.401
0.398
0.396
0.394
0.391
0.389
0.387
0.384
0.382
0.380
0.378
0.375
0.372
0.370
0.368
0.500
0.443
0.418
0.410
0.405
0.403
0.401
0.400
0.399
0.399
0.398
0.398
0.397
0.396
0.396
0.395
0.395
0.394
0.393
0.393
0.392
0.392
0.392
0.391
0.391
0.390
0.389
0.389
0.388
0.387
0.498
0.493
0.505
0.500
0.502
0.504
0.504
0.501
0.505
0.505
0.503
0.504
0.500
0.502
0.501
0.499
0.503
0.498
0.508
0.498
0.499
0.498
0.499
0.498
0.494
0.498
0.502
0.501
0.500
0.504
0.503
0.504
0.504
0.504
0.504
0.503
0.503
0.502
0.503
0.503
0.503
0.503
0.504
0.504
0.505
0.505
0.505
0.506
0.505
0.505
0.506
0.506
0.506
0.506
0.505
0.505
0.504
0.505
0.505
0.505
0.504
0.504
0.532
0.543
0.567
0.588
0.608
0.626
0.640
0.655
0.666
0.671
0.676
0.680
0.681
0.683
0.685
0.685
0.686
0.687
0.688
0.688
0.688
0.688
0.688
0.688
0.687
0.688
0.688
0.688
0.504
0.504
0.508
0.532
0.538
0.556
0.570
0.582
0.595
0.606
0.612
0.620
0.625
0.631
0.636
0.637
0.640
0.641
0.643
0.644
0.646
0.647
0.648
0.648
0.649
0.650
0.650
0.650
0.651
0.651
0.501
0.504
0.507
0.519
0.527
0.533
0.542
0.552
0.558
0.565
0.574
0.580
0.585
0.592
0.597
0.600
0.605
0.607
0.609
0.614
0.616
0.617
0.619
0.618
0.619
0.621
0.621
0.622
0.622
0.623
0.503
0.498
0.502
0.502
0.499
0.496
0.499
0.502
0.493
0.494
0.500
0.500
0.498
0.500
0.505
0.510
0.501
0.503
0.495
0.498
0.505
0.501
0.496
0.504
0.502
0.508
0.502
0.500
0.499
0.495
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.501
0.507
0.527
0.549
0.573
0.597
0.621
0.647
0.673
0.697
0.722
0.746
0.769
0.792
0.813
0.833
0.849
0.867
0.881
0.895
0.907
0.918
0.928
0.937
0.945
0.951
0.958
0.962
0.966
0.495
0.495
0.616
0.663
0.731
0.791
0.840
0.876
0.899
0.916
0.927
0.934
0.939
0.944
0.946
0.949
0.950
0.951
0.952
0.952
0.953
0.953
0.954
0.954
0.955
0.955
0.955
0.955
0.955
0.955
0.505
0.512
0.626
0.612
0.722
0.740
0.794
0.826
0.852
0.871
0.882
0.892
0.899
0.903
0.907
0.909
0.911
0.912
0.913
0.914
0.915
0.915
0.915
0.915
0.915
0.915
0.915
0.915
0.916
0.916
0.495
0.497
0.505
0.501
0.497
0.498
0.500
0.504
0.497
0.496
0.500
0.498
0.498
0.497
0.500
0.498
0.498
0.494
0.499
0.506
0.504
0.496
0.498
0.498
0.500
0.499
0.501
0.504
0.496
0.503
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.501
0.501
0.501
0.501
0.501
0.501
0.497
0.497
0.515
0.603
0.673
0.741
0.807
0.862
0.908
0.939
0.960
0.973
0.981
0.987
0.990
0.992
0.994
0.995
0.996
0.997
0.997
0.998
0.998
0.998
0.998
0.998
0.999
0.999
0.999
0.999
0.498
0.502
0.656
0.703
0.781
0.849
0.902
0.937
0.958
0.972
0.980
0.984
0.987
0.989
0.990
0.991
0.991
0.991
0.991
0.991
0.991
0.991
0.992
0.992
0.992
0.992
0.992
0.992
0.992
0.992
0.500
0.498
0.504
0.500
0.500
0.504
0.495
0.503
0.500
0.502
0.502
0.500
0.504
0.498
0.504
0.503
0.500
0.503
0.497
0.501
0.499
0.493
0.498
0.501
0.496
0.495
0.501
0.501
0.497
0.500
ARTICLE IN PRESS
0.00
0.00
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
d
f
1591
ARTICLE IN PRESS
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
1592
1.1
Av. fraction playing strategy one
1.0
0.9
0.8
0.7
=0
= 0.25
= 0.5
= 0.75
=1
0.6
0.5
0.4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Period
Fig. B5. Average population share playing strategy one in the coordination game in Fig. 3 for different
values of f, holding d ¼ 1 fixed.
1.1
1.0
Av. fraction playing strategy one
0.9
=0
= 0.25
= 0.5
= 0.75
=1
0.8
0.7
0.6
0.5
0.4
0.3
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Period
Fig. B6. Average population share playing strategy one in the coordination game in Fig. 3 for different
values of d, holding f ¼ 0:75 fixed.
Table B4
Average population share playing strategy one in the coordination game in Fig. 4
0.00
0.25
0.00
0.50
0.00
0.75
0.00
1.00
0.25
0.00
0.25
0.25
0.25
0.50
0.25
0.75
0.25
1.00
0.50
0.00
0.50
0.25
0.50
0.50
0.50
0.75
0.50
1.00
0.75
0.00
0.75
0.25
0.75
0.50
0.75
0.75
0.75
1.00
1.00
0.00
1.00
0.25
1.00
0.50
1.00
0.75
1.00
1.00
t¼1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.497
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.499
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.498
0.502
0.484
0.482
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.481
0.495
0.503
0.496
0.496
0.493
0.501
0.500
0.501
0.494
0.499
0.500
0.500
0.498
0.506
0.501
0.498
0.504
0.499
0.503
0.500
0.493
0.494
0.500
0.495
0.502
0.496
0.495
0.500
0.501
0.498
0.496
0.478
0.460
0.442
0.422
0.404
0.387
0.368
0.351
0.333
0.318
0.303
0.287
0.274
0.258
0.244
0.232
0.218
0.205
0.193
0.182
0.172
0.161
0.151
0.142
0.133
0.126
0.118
0.111
0.105
0.500
0.467
0.455
0.449
0.444
0.439
0.433
0.426
0.420
0.414
0.407
0.402
0.395
0.390
0.384
0.378
0.372
0.365
0.359
0.353
0.346
0.340
0.333
0.328
0.321
0.313
0.307
0.302
0.295
0.290
0.497
0.444
0.428
0.422
0.418
0.415
0.411
0.408
0.406
0.403
0.400
0.397
0.394
0.391
0.388
0.386
0.383
0.380
0.378
0.375
0.372
0.369
0.365
0.362
0.359
0.356
0.354
0.350
0.347
0.344
0.494
0.428
0.400
0.390
0.385
0.383
0.381
0.380
0.379
0.378
0.377
0.376
0.375
0.374
0.374
0.373
0.372
0.371
0.370
0.369
0.369
0.368
0.367
0.366
0.365
0.365
0.363
0.363
0.362
0.361
0.500
0.499
0.499
0.502
0.500
0.497
0.501
0.501
0.499
0.492
0.503
0.497
0.500
0.503
0.504
0.495
0.496
0.506
0.493
0.502
0.503
0.494
0.500
0.503
0.501
0.509
0.497
0.500
0.495
0.507
0.494
0.374
0.260
0.171
0.104
0.060
0.034
0.018
0.010
0.005
0.002
0.001
0.001
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.495
0.373
0.319
0.235
0.170
0.119
0.079
0.058
0.041
0.031
0.024
0.019
0.017
0.015
0.014
0.013
0.013
0.013
0.013
0.013
0.013
0.013
0.013
0.013
0.013
0.013
0.013
0.013
0.013
0.013
0.504
0.398
0.359
0.308
0.262
0.222
0.185
0.155
0.129
0.107
0.092
0.078
0.069
0.060
0.051
0.045
0.041
0.039
0.036
0.034
0.034
0.032
0.032
0.031
0.030
0.030
0.029
0.029
0.029
0.028
0.500
0.428
0.393
0.371
0.346
0.327
0.308
0.291
0.274
0.260
0.247
0.236
0.225
0.214
0.206
0.197
0.188
0.180
0.172
0.167
0.162
0.158
0.155
0.151
0.148
0.146
0.143
0.141
0.139
0.136
0.500
0.500
0.499
0.496
0.496
0.502
0.502
0.499
0.494
0.498
0.494
0.505
0.500
0.496
0.497
0.497
0.491
0.497
0.506
0.498
0.502
0.498
0.504
0.501
0.497
0.501
0.500
0.497
0.501
0.489
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.502
0.502
0.497
0.489
0.484
0.476
0.468
0.461
0.453
0.445
0.435
0.427
0.417
0.408
0.400
0.390
0.381
0.373
0.365
0.356
0.347
0.338
0.329
0.321
0.313
0.305
0.297
0.288
0.280
0.271
0.497
0.495
0.462
0.465
0.439
0.426
0.409
0.396
0.385
0.374
0.372
0.365
0.361
0.360
0.359
0.358
0.358
0.358
0.358
0.358
0.358
0.357
0.357
0.356
0.356
0.356
0.356
0.356
0.356
0.356
0.501
0.494
0.469
0.469
0.444
0.436
0.419
0.413
0.401
0.395
0.386
0.383
0.380
0.380
0.377
0.377
0.377
0.377
0.376
0.376
0.375
0.375
0.374
0.374
0.373
0.373
0.373
0.373
0.373
0.373
0.495
0.504
0.501
0.497
0.507
0.498
0.499
0.501
0.501
0.501
0.501
0.505
0.500
0.500
0.505
0.495
0.501
0.498
0.501
0.494
0.508
0.504
0.498
0.499
0.497
0.503
0.497
0.496
0.505
0.494
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.495
0.506
0.506
0.506
0.506
0.506
0.506
0.506
0.506
0.506
0.506
0.506
0.506
0.505
0.505
0.506
0.506
0.506
0.506
0.506
0.506
0.506
0.506
0.506
0.506
0.506
0.506
0.506
0.506
0.506
0.506
0.504
0.504
0.504
0.502
0.502
0.502
0.503
0.500
0.500
0.499
0.500
0.500
0.503
0.503
0.501
0.502
0.503
0.500
0.502
0.502
0.502
0.502
0.502
0.502
0.503
0.504
0.504
0.504
0.505
0.504
0.498
0.498
0.500
0.498
0.503
0.501
0.504
0.504
0.505
0.507
0.508
0.511
0.514
0.516
0.517
0.518
0.519
0.519
0.520
0.520
0.520
0.520
0.520
0.520
0.520
0.520
0.520
0.520
0.519
0.519
0.501
0.495
0.497
0.498
0.499
0.498
0.498
0.505
0.498
0.507
0.502
0.505
0.499
0.504
0.498
0.504
0.505
0.505
0.500
0.500
0.499
0.510
0.498
0.501
0.494
0.502
0.501
0.503
0.501
0.501
ARTICLE IN PRESS
0.00
0.00
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
d
f
1593
ARTICLE IN PRESS
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
1594
0.6
Av. fraction playing strategy one
0.5
0.4
0.3
=0
= 0.25
= 0.5
= 0.75
=1
0.2
0.1
0.0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Period
Fig. B7. Average population share playing strategy one in the coordination game in Fig. 4 for different
values of f, holding d ¼ 0:5 fixed.
0.6
Av. fraction playing strategy one
0.5
0.4
0.3
=0
= 0.25
= 0.5
= 0.75
=1
0.2
0.1
0.0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Period
Fig. B8. Average population share playing strategy one in the coordination game in Fig. 4 for different
values of d, holding f ¼ 0 fixed.
Table B5
Average population share playing strategy one in the hawk–dove game in Fig. 5
0.00
0.25
0.00
0.50
0.00
0.75
0.00
1.00
0.25
0.00
0.25
0.25
0.25
0.50
0.25
0.75
0.25
1.00
0.50
0.00
0.50
0.25
0.50
0.50
0.50
0.75
0.50
1.00
0.75
0.00
0.75
0.25
0.75
0.50
0.75
0.75
0.75
1.00
1.00
0.00
1.00
0.25
1.00
0.50
1.00
0.75
1.00
1.00
t¼1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.499
0.502
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.494
0.479
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.477
0.496
0.490
0.501
0.498
0.502
0.496
0.501
0.499
0.493
0.505
0.506
0.502
0.502
0.499
0.502
0.504
0.501
0.497
0.506
0.501
0.499
0.502
0.490
0.499
0.500
0.497
0.499
0.504
0.500
0.498
0.500
0.482
0.466
0.450
0.436
0.421
0.408
0.396
0.384
0.372
0.363
0.354
0.345
0.338
0.329
0.321
0.313
0.306
0.299
0.293
0.287
0.282
0.276
0.271
0.266
0.262
0.258
0.254
0.250
0.247
0.496
0.466
0.456
0.452
0.449
0.445
0.442
0.440
0.437
0.434
0.432
0.429
0.427
0.425
0.423
0.421
0.420
0.418
0.416
0.413
0.411
0.410
0.408
0.407
0.405
0.403
0.402
0.400
0.398
0.397
0.495
0.446
0.435
0.433
0.431
0.430
0.429
0.429
0.428
0.428
0.427
0.427
0.426
0.426
0.425
0.425
0.425
0.424
0.424
0.424
0.423
0.423
0.423
0.422
0.422
0.422
0.421
0.421
0.421
0.420
0.500
0.451
0.437
0.432
0.430
0.430
0.430
0.430
0.430
0.430
0.430
0.430
0.430
0.430
0.430
0.430
0.430
0.430
0.430
0.430
0.430
0.430
0.430
0.430
0.429
0.429
0.429
0.429
0.429
0.429
0.494
0.504
0.502
0.509
0.497
0.500
0.499
0.497
0.510
0.496
0.492
0.500
0.496
0.506
0.502
0.503
0.500
0.499
0.492
0.497
0.497
0.497
0.505
0.501
0.501
0.502
0.509
0.497
0.500
0.500
0.495
0.504
0.496
0.504
0.496
0.503
0.497
0.503
0.497
0.503
0.497
0.502
0.498
0.502
0.499
0.501
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.499
0.501
0.499
0.501
0.499
0.504
0.496
0.534
0.503
0.526
0.507
0.517
0.513
0.513
0.515
0.516
0.513
0.514
0.516
0.515
0.516
0.513
0.515
0.517
0.513
0.517
0.514
0.519
0.514
0.514
0.518
0.510
0.518
0.515
0.516
0.500
0.500
0.502
0.525
0.518
0.523
0.523
0.524
0.522
0.521
0.522
0.521
0.525
0.525
0.525
0.522
0.521
0.522
0.522
0.526
0.523
0.525
0.524
0.520
0.520
0.521
0.522
0.523
0.521
0.524
0.500
0.498
0.504
0.511
0.516
0.516
0.522
0.521
0.521
0.523
0.523
0.526
0.525
0.527
0.527
0.527
0.528
0.528
0.529
0.528
0.528
0.528
0.532
0.529
0.530
0.530
0.531
0.530
0.531
0.531
0.499
0.499
0.498
0.502
0.499
0.499
0.501
0.498
0.500
0.498
0.499
0.499
0.501
0.501
0.499
0.500
0.502
0.503
0.497
0.500
0.498
0.498
0.501
0.498
0.500
0.490
0.496
0.504
0.496
0.506
0.501
0.499
0.501
0.499
0.501
0.499
0.501
0.499
0.501
0.499
0.501
0.499
0.501
0.499
0.501
0.499
0.501
0.499
0.501
0.499
0.501
0.499
0.501
0.499
0.501
0.499
0.501
0.499
0.501
0.499
0.497
0.503
0.505
0.516
0.510
0.515
0.511
0.515
0.509
0.515
0.509
0.517
0.509
0.516
0.509
0.516
0.509
0.517
0.510
0.515
0.510
0.517
0.509
0.513
0.512
0.514
0.511
0.514
0.512
0.514
0.501
0.499
0.624
0.506
0.559
0.536
0.551
0.545
0.548
0.542
0.546
0.546
0.542
0.550
0.545
0.546
0.545
0.546
0.544
0.547
0.547
0.548
0.546
0.543
0.547
0.544
0.548
0.546
0.545
0.548
0.495
0.512
0.609
0.493
0.596
0.533
0.564
0.558
0.558
0.565
0.557
0.564
0.562
0.560
0.562
0.559
0.563
0.558
0.562
0.557
0.561
0.562
0.564
0.558
0.558
0.563
0.557
0.564
0.561
0.560
0.495
0.496
0.496
0.506
0.502
0.496
0.500
0.496
0.498
0.491
0.501
0.502
0.498
0.495
0.511
0.494
0.496
0.500
0.501
0.500
0.491
0.494
0.500
0.502
0.497
0.497
0.502
0.501
0.502
0.499
0.505
0.495
0.505
0.495
0.505
0.495
0.505
0.495
0.505
0.495
0.505
0.495
0.505
0.495
0.505
0.495
0.505
0.495
0.505
0.495
0.505
0.495
0.505
0.495
0.505
0.495
0.505
0.495
0.505
0.495
0.504
0.496
0.504
0.496
0.504
0.496
0.504
0.497
0.504
0.497
0.504
0.497
0.504
0.497
0.503
0.497
0.503
0.497
0.503
0.497
0.503
0.497
0.503
0.497
0.503
0.497
0.503
0.497
0.503
0.497
0.498
0.502
0.519
0.577
0.528
0.556
0.532
0.553
0.533
0.549
0.538
0.548
0.537
0.549
0.537
0.549
0.538
0.545
0.541
0.542
0.547
0.538
0.544
0.543
0.543
0.541
0.544
0.540
0.549
0.535
0.501
0.502
0.650
0.499
0.568
0.561
0.562
0.560
0.563
0.563
0.559
0.561
0.562
0.560
0.559
0.562
0.567
0.559
0.558
0.565
0.557
0.567
0.557
0.561
0.564
0.561
0.558
0.564
0.560
0.560
0.505
0.501
0.502
0.501
0.503
0.499
0.494
0.504
0.497
0.491
0.498
0.497
0.499
0.496
0.490
0.496
0.504
0.501
0.498
0.499
0.499
0.499
0.502
0.497
0.501
0.503
0.503
0.505
0.501
0.503
ARTICLE IN PRESS
0.00
0.00
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
d
f
1595
ARTICLE IN PRESS
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
1596
0.65
Av. fraction playing strategy one
0.60
=0
= 0.25
= 0.5
= 0.75
=1
0.55
0.50
0.45
0.40
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Period
Fig. B9. Average population share playing strategy one in the hawk–dove game in Fig. 5 for different
values of f, holding d ¼ 0:75 fixed.
0.70
Av. fraction playing strategy one
0.65
=0
= 0.25
= 0.5
= 0.75
=1
0.60
0.55
0.50
0.45
0.40
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Period
Fig. B10. Average population share playing strategy one in the hawk–dove game in Fig. 5 for different
values of d, holding f ¼ 0:75 fixed.
Table B6
Average population share playing strategy one in the hawk–dove game in Fig. 6
0.00
0.25
0.00
0.50
0.00
0.75
0.00
1.00
0.25
0.00
0.25
0.25
0.25
0.50
0.25
0.75
0.25
1.00
0.50
0.00
0.50
0.25
0.50
0.50
0.50
0.75
0.50
1.00
0.75
0.00
0.75
0.25
0.75
0.50
0.75
0.75
0.75
1.00
1.00
0.00
1.00
0.25
1.00
0.50
1.00
0.75
1.00
1.00
t¼1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.502
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.500
0.507
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.505
0.496
0.480
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.478
0.493
0.497
0.499
0.506
0.499
0.496
0.497
0.505
0.499
0.501
0.498
0.504
0.497
0.498
0.500
0.499
0.499
0.501
0.503
0.496
0.505
0.492
0.497
0.498
0.496
0.506
0.497
0.500
0.501
0.494
0.508
0.380
0.308
0.262
0.232
0.206
0.187
0.172
0.160
0.149
0.141
0.131
0.125
0.117
0.111
0.106
0.102
0.098
0.094
0.091
0.088
0.085
0.082
0.080
0.078
0.076
0.075
0.073
0.072
0.070
0.499
0.375
0.350
0.337
0.327
0.319
0.311
0.304
0.297
0.291
0.285
0.279
0.274
0.269
0.266
0.262
0.257
0.254
0.251
0.247
0.244
0.241
0.238
0.236
0.234
0.232
0.229
0.227
0.225
0.223
0.498
0.371
0.346
0.341
0.338
0.335
0.333
0.331
0.329
0.328
0.326
0.324
0.323
0.321
0.320
0.318
0.317
0.316
0.315
0.313
0.312
0.311
0.310
0.309
0.308
0.306
0.306
0.305
0.304
0.303
0.502
0.384
0.358
0.353
0.351
0.350
0.349
0.349
0.348
0.348
0.348
0.348
0.347
0.347
0.347
0.347
0.347
0.347
0.347
0.346
0.346
0.346
0.346
0.346
0.346
0.346
0.346
0.346
0.346
0.346
0.502
0.503
0.506
0.499
0.497
0.499
0.500
0.501
0.497
0.502
0.498
0.497
0.497
0.494
0.497
0.501
0.497
0.500
0.505
0.501
0.499
0.497
0.501
0.501
0.496
0.506
0.496
0.506
0.498
0.503
0.501
0.252
0.196
0.167
0.146
0.134
0.124
0.115
0.110
0.107
0.102
0.098
0.096
0.094
0.093
0.091
0.089
0.089
0.088
0.087
0.086
0.084
0.085
0.084
0.084
0.084
0.083
0.084
0.085
0.085
0.502
0.253
0.196
0.164
0.145
0.131
0.120
0.113
0.108
0.102
0.097
0.096
0.094
0.092
0.090
0.088
0.086
0.084
0.083
0.081
0.082
0.081
0.082
0.082
0.082
0.082
0.081
0.080
0.081
0.081
0.501
0.266
0.235
0.225
0.215
0.208
0.200
0.195
0.190
0.186
0.183
0.179
0.176
0.173
0.170
0.167
0.166
0.165
0.163
0.160
0.158
0.157
0.156
0.155
0.154
0.153
0.152
0.151
0.149
0.149
0.506
0.327
0.302
0.288
0.278
0.272
0.270
0.268
0.264
0.262
0.259
0.257
0.255
0.253
0.252
0.250
0.248
0.247
0.245
0.244
0.242
0.241
0.241
0.239
0.239
0.238
0.236
0.235
0.234
0.234
0.495
0.494
0.498
0.497
0.499
0.494
0.500
0.500
0.495
0.501
0.493
0.497
0.502
0.497
0.501
0.500
0.499
0.499
0.500
0.500
0.502
0.498
0.494
0.501
0.506
0.498
0.504
0.495
0.497
0.499
0.505
0.477
0.501
0.479
0.501
0.481
0.500
0.481
0.501
0.481
0.498
0.484
0.496
0.486
0.495
0.487
0.493
0.488
0.494
0.487
0.493
0.488
0.493
0.488
0.492
0.489
0.493
0.489
0.490
0.492
0.498
0.467
0.284
0.374
0.412
0.356
0.370
0.386
0.370
0.371
0.376
0.376
0.373
0.367
0.376
0.371
0.377
0.372
0.374
0.377
0.371
0.372
0.370
0.378
0.374
0.374
0.373
0.373
0.374
0.372
0.501
0.443
0.276
0.265
0.336
0.339
0.322
0.317
0.322
0.322
0.322
0.323
0.320
0.321
0.321
0.320
0.322
0.320
0.316
0.322
0.321
0.322
0.320
0.323
0.322
0.325
0.322
0.320
0.327
0.319
0.507
0.402
0.302
0.285
0.316
0.323
0.308
0.300
0.301
0.306
0.303
0.300
0.300
0.303
0.301
0.299
0.300
0.305
0.303
0.301
0.302
0.298
0.299
0.300
0.297
0.301
0.303
0.300
0.302
0.297
0.506
0.505
0.496
0.505
0.495
0.508
0.501
0.500
0.504
0.498
0.497
0.507
0.500
0.503
0.496
0.496
0.499
0.502
0.506
0.500
0.496
0.508
0.499
0.498
0.500
0.501
0.503
0.499
0.501
0.499
0.506
0.494
0.506
0.494
0.506
0.494
0.506
0.494
0.506
0.494
0.506
0.494
0.506
0.494
0.506
0.494
0.506
0.494
0.506
0.494
0.506
0.494
0.506
0.495
0.506
0.495
0.505
0.495
0.506
0.494
0.501
0.499
0.468
0.471
0.467
0.469
0.473
0.465
0.471
0.467
0.471
0.469
0.466
0.473
0.467
0.472
0.467
0.473
0.466
0.473
0.468
0.471
0.466
0.472
0.469
0.472
0.465
0.475
0.466
0.473
0.501
0.496
0.266
0.411
0.433
0.359
0.393
0.405
0.381
0.390
0.397
0.388
0.394
0.389
0.390
0.392
0.391
0.389
0.394
0.392
0.385
0.392
0.392
0.389
0.395
0.385
0.396
0.389
0.386
0.393
0.505
0.458
0.277
0.336
0.366
0.341
0.337
0.345
0.348
0.342
0.338
0.342
0.348
0.340
0.347
0.341
0.341
0.343
0.343
0.345
0.345
0.343
0.338
0.348
0.338
0.343
0.343
0.348
0.346
0.342
0.503
0.502
0.502
0.502
0.503
0.502
0.500
0.502
0.502
0.497
0.497
0.502
0.500
0.504
0.499
0.498
0.502
0.499
0.500
0.502
0.502
0.497
0.501
0.496
0.501
0.496
0.502
0.500
0.503
0.499
ARTICLE IN PRESS
0.00
0.00
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
d
f
1597
ARTICLE IN PRESS
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
1598
0.6
Av. fraction playing strategy one
0.5
0.4
0.3
=0
= 0.25
= 0.5
= 0.75
=1
0.2
0.1
0.0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Period
Fig. B11. Average population share playing strategy one in the hawk–dove game in Fig. 6 for different
values of f, holding d ¼ 0:5 fixed.
0.55
Av. fraction playing strategy one
0.50
0.45
0.40
=0
= 0.25
= 0.5
= 0.75
=1
0.35
0.30
0.25
0.20
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Period
Fig. B12. Average population share playing strategy one in the hawk–dove game in Fig. 6 for different
values of d, holding f ¼ 0:75 fixed.
ARTICLE IN PRESS
J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599
1599
References
Anderlini, L., Sabourian, H., 1995. The evolution of algorithmic learning rules: a global stability result.
Mimeo, Cambridge University.
Beggs, A.W., 2005. On the convergence of reinforcement learning. Journal of Economic Theory 122, 1– 36.
Berger, U., 2005. Fictitious play in 2 n games. Journal of Economic Theory 120, 139– 154.
Brown, G.W., 1951. Iterative solutions of games by fictitious play. In: Koopmans, T.C. (Ed.), Activity
Analysis of Production and Allocation. Wiley, New York.
Bush, R., Mosteller, F., 1951. A mathematical model for simple learning. Psychological Review 58,
313– 323.
Camerer, C., 2003. Behavioral Game Theory: Experiments in Strategic Interaction. Princeton University
Press, Princeton, NJ.
Camerer, C., Ho, T.-H., 1999. Experience-weighted attraction learning in normal form games.
Econometrica 67, 827– 874.
Camerer, C., Ho, T.-H., Chong, J.-K., 2002. Sophisticated experience-weighted attraction learning and
strategic teaching in repeated games. Journal of Economic Theory 104, 137– 188.
Erev, I., Roth, A., 1998. Predicting how people play games: reinforcement learning in experimental games
with unique, mixed strategy equilibria. American Economic Review 88, 848– 881.
Fudenberg, D., Levine, D., 1998. The Theory of Learning in Games. The MIT Press, Cambridge, MA.
Hanaki, N., Sethi, R., Erev, I., Peterhansl, A., 2005. Learning strategies. Journal of Economic Behavior
and Organization 56, 523– 542.
Harley, C.B., 1981. Learning the evolutionarily stable strategy. Journal of Theoretical Biology 89,
611– 633.
Heller, D., 2004. An evolutionary approach to learning in a changing environment. Journal of Economic
Theory 114, 31– 55.
Ho, T.-H., Camerer, C., Chong, J.-K., 2002. Functional EWA: a one-parameter theory of learning in
games. Mimeo, Caltech.
Hofbauer, J., Sandholm, W.H., 2002. On the global convergence of stochastic fictitious play.
Econometrica 70, 2265– 2294.
Hommes, C., 2006. Heterogeneous agent models in economics and finance. In: Tesfatsion, L., Judd,
K.L. (Eds.), Handbook of Computational Economics, volume 2. Agent-Based Computational
Economics. Elsevier, Amsterdam, pp. 1109– 1186.
Hopkins, E., 1999. Learning, matching, and aggregation. Games and Economic Behavior 26, 79– 110.
Hopkins, E., 2002. Two competing models of how people learn in games. Econometrica 70, 2141– 2166.
Hopkins, E., Posch, M., 2005. Attainability of boundary points under reinforcement learning. Games and
Economic Behavior 53, 110– 125.
Maynard Smith, J., 1974. The theory of games and the evolution of animal conflicts. Journal of
Theoretical Biology 47, 209– 221.
Maynard Smith, J., Price, G.R., 1973. The logic of animal conflict. Nature 246, 15– 18.
Robson, A., 2001. The biological basis of economic behavior. Journal of Economic Literature 39, 11– 33.
Salmon, T.C., 2001. An evaluation of econometric models of adaptive learning. Econometrica 69,
1597– 1628.
Stahl, D.O., 2003. Action-reinforcement learning versus rule learning. Mimeo, University of Texas.
Taylor, P.D., Jonker, L.B., 1978. Evolutionary stable strategies and game dynamics. Mathematical
Biosciences 40, 145– 156.
Thorndike, E.L., 1898. Animal intelligence: an experimental study of the associative process in animals.
Psychological Monographs 2.
Wilcox, N.T., 2006. Theories of learning in games and heterogeneity bias. Econometrica 74, 1271– 1292.