Evolution & Economics
No. 4
1
Evolutionary Stability in Repeated
Games Played by Finite Automata
K. Binmore & L. Samuelson
J.E.T. 1991
2
Finite Automata playing the Prisoners’ Dilemma
transitions
C
D
C
C
C,D
D
D
D
C
D
C
Grim
Tit For Tat (TFT)
states
(& actions)
C
D
D
C
C
C
D
Tat For Tit (TAFT)
D
D
C
C
D
Tweedledum
3
Automata playing the Prisoners’ Dilemma
C
C
D
C,D
C
D
C,D
D
C
D
Tweedledee
CA
C,D
C,D
C
D
D
C
4
•
•
•
•
Two Automata playing together, eventually follow a cycle
(handshake)
The payoff is the limit of the means.
The cost of an automaton is the number of his states.
The cost enters the payoffs lexicographically.


Let a,b be automata, and let pi a,b be the payoff at stage i
of the repeated game.
The payoff in the repeated game is the limit of the means:
1 T -1
π  a,b  = lim  pi  a,b 
T T j=0
The no.of states of an automaton a is denoted by: a .
5
The Structure of Nash Equilibrium in Repeated Games with Finite Automata
Dilip Abreu & Ariel Rubinstein
Econometrica,1988
C
D
2,2
-1 , 3
D 3 , -1
0,0
C
In Abreu Rubinstein, The fitness
U  a,c  >U  b,c  iff:
i) π  a,c   π  b,c
U  a,b  satisfies

and
ii) if π  a,c  = π  b,c  then a  b .
6
The Structure of Nash Equilibrium in Repeated Games with Finite Automata
Dilip Abreu & Ariel Rubinstein
Econometrica,1988
C
D
2,2
-1 , 3
D 3 , -1
0,0
C
(-1,3)
(2,2)
N.E. of repeated Game
N.E in Repeated Games with Finite Automata
(Abreu Rubinstein)
(0,0)
(3,-1)
7
Binmore Samuelson:
An automaton a is an ESS, if for all automata b  a :
i) π  a,a   π  b,a

and
ii) if π  a,a  = π  b,a  then π  a,b   π  b,b

and
iii) if π  a,a  = π  b,a  and π  a,b  = π  b,b  then a  b .
8
Lemma : For any game G, if automaton a is an ESS,
it has a single state.
Proof : If G is 1x1, then any ESS must have a single state.
If G has more than one action, let it have an ESS
a with more than one state.
Let a begin with the action x.
x
Let b be identical to a, except that
a
x
?
y
it acts differently when it
observes y at the start.
If
x
x
y
a
then:
x
?
x
y
b
?
9
Lemma : For any game G, if automaton a is an ESS,
it has a single state.
Proof : If G is 1x1, then any ESS must have a single state.
If G has more than one action, let it have an ESS
a with more than one state.
Let a begin with the action x.
x
Let b be identical to a, except that
a
x
?
y
it acts differently when it
observes y at the start.
If
x
a
x
y
then:
x
?
x
y
b
?
10
Lemma : For any game G, if automaton a is an ESS,
it has a single state.
Proof : If G is 1x1, then any ESS must have a single state.
If G has more than one action, let it have an ESS
a with more than one state.
Let a begin with the action x.
x
Let b be identical to a, except that
a
x
?
y
it acts differently when it
observes y at the start.
b can invade the population a,
hence a cannot be an ESS.
Q.E.D.
11
Lemma : For any game G, if automaton a is an ESS,
it has a single state.
For the
P.D. the two singletons are:
C,D
C,D
C
D
D
C
C
C
D
D
D
C
Tit For Tat (TFT)
C is not an ESS, it can be invaded by D.
D is not an ESS, it can be invaded by Tit For Tat.
The P.D. has no ESS.
12
Definition : A Modified ESS. MESS.
An automaton a is an MESS
ESS,, if for all automata b  a :
i) π  a,a   π  b,a

and
ii) if π  a,a  = π  b,a  then π  a,b   π  b,b 
and
i) if π  a,a  = π  b,a  and π  a,b  = π  b,b  then
then a 
 b.
iii)
13
Lemma : If a is a MESS then it uses all its states
when it plays against itself.
Proof : Assume that a does not use its state s when playing
against itself. Construct an automaton b, identical to a
except that state s has been deleted.
There is no difference between a,b when playing a or b.
But
a < b.
Hence such a cannot be an ESS.
Q.E.D.
14
Lemma : If a is a MESS then it uses all its states
when it plays against itself.
Proof : Assume that a does not use its state s when playing
against itself. Construct an automaton b, identical to a
except that state s has been deleted.
There is no difference between a,b when playing a or b.
a < b.
But
Hence such a cannot be an ESS.
Q.E.D.
In the P.D. Tit For Tat and Grim are not MESS
(they do not use one state against themselves)
C
C
D
Grim
C
C,D
D
C
D
C
D
D
Tit For Tat (TFT)
15
For a general, possibly non symmetric game
Define the symmetrized version of G:
G.
G # #.
A player is player 1 with probability 0.5 and player 2 with probability 0.5

1
π  a,b  = π  a1 ,b2  + π  a2 ,b1  .
2

An automaton for this game is a pair: a = a1,a2 .
The complexity of a :
a = a1 + a 2 .
The previous lemmas apply to
(a1,a2)
1. An ESS has a single state │a1│=│a2│=1
2. If (a1,a2) is a MESS it uses all its states when playing against itself,
i.e. a1,a2 use all their states when playing against the other.
16
Consider the highest payoff that can be attained in G.
1

max
G
s
,s
+
G
s
,s




1
1
2
2
1
2

s1 ,s2 2 
s1 ,s2 actions of G.
This can be achieved by the automaton a* =
a ,a , playing against itself.
*
1
*
2
a* - a utilitarian automaton.
Lemma : If an automaton in G# # .is a MESS then it is utilitarian.
Proof : Let a be a non utilitarian MESS.
There exists a mutant b which earns a higher payoff.
The mutant b has the following properties:
1.
bi starts with an action different to that of ai .
2. If the initial action of the opponent is not that of a-i then it must
be b-i and bi continues by imitating a*i the utilitarian automaton.
3. If the initial action of the opponent is that of a-i then bi fools it to
17
'believe' that it plays itself, and so obtains the payoff Gi  ai , a-i  .
1. bi starts with an action different to that of ai .
2. If the initial action of the opponent is not that of a-i then it must
be b-i and bi continues by imitating a*i the utilitarian automaton.
3. If the initial action of the opponent is that of a-i then bi fools it to
'believe' that it plays itself, and so obtains the payoff Gi  ai , a-i  .
Properties 1,2 are easy to obtain.
Property 3 is less obvious. It cannot be done with Grim.
But a-i goes through all its states when playing ai .
If bi 's initial action made a-i move to the state q. If q is reached when a-i
plays ai who is then in state q*, then bi mimics ai in state q.
  b,a     a,a     a,b 
a cannot be a MESS.
  b,b     a,b  b mimics a, so a gets the same against b as against a.
Q.E.D.
but   b,b     a*,a*     a,a  =   a,b 
18
To prove existence of a MESS is like in repeated game (Folk Theorem)
If m is the minimax of G, then any payoff above it can be supported a MESS.
In the Prisoners' Dilemma :
No 'nice' automaton can be a MESS: It can be invaded by C , which is shorter.
19
Tat For Tit is a MESS :
C
D
It can be invaded by:
C
D
C
D
Tat For Tit (TAFT)
C
D
C,D
C
C
C,D
D
D
C
C,D
D
AC
CA
C
C
D
D
CC
C
D
C,D
C
AA
C
C
D
D
D
C,D
CD
20
Tat For Tit is a MESS :
C
D
It can be invaded by:
D
C
C
D
Tat For Tit (TAFT)
C
D
C,D
C
D
AC
21
Tat For Tit is a MESS :
C
D
It can be invaded by:
D
C
C
D
Tat For Tit (TAFT)
C
D
D
C,D
C
CA
22
Tat For Tit is a MESS :
C
D
It can be invaded by:
D
C
C
D
Tat For Tit (TAFT)
C
C
D
C
D
D
CC
23
Tat For Tit is a MESS :
C
D
D
C
C
D
Tat For Tit (TAFT)
No other (longer and more sophisticated) automaton can invade.
D against his C) makes TAFT play D,
so the average of these two periods is (3+0)/2 = 1.5 < 2, the average of cooperating.
Any exploitation of TAFT (playing
C
D
2,2
-1 , 3
D 3 , -1
0,0
C
24
Polymorphic MESS
C
D
A population consisting of:
C
D
D
C,D
Tat For Tit (TAFT)
C
C
CA
AC
invaded, it does not do well against
CD
C
D
AC
C
D
CC
C
C
D
D
C
D
D C D C …….
C D C D …….
C,D
C
D
can be invaded only by:
If
D
C
C
D
D
C,D
CD
25
Polymorphic MESS
C
D
A population consisting of:
can be invaded only by
CA
C
D
C,D
Tat For Tit (TAFT)
C
C
CA
AA
invaded, it does not do well against
CC
C
C
D
CC
C
C,D
D
D
C
D
D C C C C…….
C D D D D…….
C,D
C
D
D
If
D
C
C
D
D
C,D
AA
CD
26
Polymorphic MESS
C
D
A population consisting of:
can be invaded only by
CA
C
D
C,D
Tat For Tit (TAFT)
C
C
CA
but if
invaded then a sophisticated
starts with C. if it saw C it continues with C
automaton
can invade and exploit
forever (the opponent must be
or
).
S
C
D
D
S CA
D
C
C
CD CA
CC.
D
C
D
D
CC
If it saw D, it plays D again, if the other then
plays D it must be TAFT.
S plays another D
and then C forever.
If, however, after 2x D, the other played C, then it
must be
CA,
and
S should play D forever.
C
C
D
D
C,D
CD
27
Polymorphic MESS
C
D
A population consisting of:
can be invaded only by
D
C
C
D
CA
C
D
D
C,D
Tat For Tit (TAFT)
C
C
CA
C
D
C
D
D
When S invades, CA will vanish, and
CC
then S which is a complex automaton
will die out.
C
C
D
D
C,D
CD
28