Completely Uncoupled
Dynamics and Nash Equilibria
by
Yakov Babichenko
Center for the Study of Rationality
The Hebrew University of Jerusalem
Under the supervision of prof. Sergiu Hart
Uncoupled dynamics
An Uncoupled dynamic is a repeated
play of a game, where the strategy of
every player does not depend on the
payoff functions of the other players.
Uncoupled dynamics
An Uncoupled dynamic is a repeated
play of a game, where the strategy of
every player does not depend on the
payoff functions of the other players.
The knowledge of each player before the game starts is his
payoff function only.
Uncoupled dynamics
An Uncoupled dynamic is a repeated
play of a game, where the strategy of
every player does not depend on the
payoff functions of the other players.
The knowledge of each player before the game starts is his
payoff function only.
The information of each player after every step is the played
actions of all players.
Completely uncoupled dynamics
A Completely uncoupled dynamic is a
repeated play of a game, where the
strategy of each player depends only on
his past payoffs.
Completely uncoupled dynamics
A Completely uncoupled dynamic is a
repeated play of a game, where the
strategy of each player depends only on
his past payoffs.
The knowledge of each player before the game starts is his
own set of actions only.
Completely uncoupled dynamics
A Completely uncoupled dynamic is a
repeated play of a game, where the
strategy of each player depends only on
his past payoffs.
The knowledge of each player before the game starts is his
own set of actions only.
The information of each player after every step is his own
payoff only (actually received).
Definitions
Type of convergence
Definitions
Type of convergence
A combination of strategies leads to [pure Nash/
Nash/ ε-Nash] equilibrium if the frequency of time
periods where the equilibrium is played converges to 1
(a.s).
Definitions
Type of convergence
A combination of strategies leads to [pure Nash/
Nash/ ε-Nash] equilibrium if the frequency of time
periods where the equilibrium is played converges to 1
(a.s).
A combination of strategies leads to [pure Nash/
Nash/ ε-Nash] equilibrium 1 − ε of the time if the
frequency of time periods where the equilibrium is
played ≥ 1 − ε from some time on (a.s).
Definitions
Genericity
Definitions
Genericity
An n-player game with action combinations set A, can be
thought of as an element in Rn|A| .
Definitions
Genericity
An n-player game with action combinations set A, can be
thought of as an element in Rn|A| .
⇒ The Lebesgue measure induces a measure on the set of
games with constant action combinations set A.
Definitions
Genericity
An n-player game with action combinations set A, can be
thought of as an element in Rn|A| .
⇒ The Lebesgue measure induces a measure on the set of
games with constant action combinations set A.
A set of games Ψ consists of almost all games, if for every
A, the set of games with action combinations set A, that are
not in Ψ, is of measure 0.
Definitions
Genericity
An n-player game with action combinations set A, can be
thought of as an element in Rn|A| .
⇒ The Lebesgue measure induces a measure on the set of
games with constant action combinations set A.
A set of games Ψ consists of almost all games, if for every
A, the set of games with action combinations set A, that are
not in Ψ, is of measure 0.
Type of Strategies
Definitions
Genericity
An n-player game with action combinations set A, can be
thought of as an element in Rn|A| .
⇒ The Lebesgue measure induces a measure on the set of
games with constant action combinations set A.
A set of games Ψ consists of almost all games, if for every
A, the set of games with action combinations set A, that are
not in Ψ, is of measure 0.
Type of Strategies
Strategy is finite memory strategy if it can be
implemented by a finite automaton
Uncoupledness and PNE
Hart and Mas Colell (2006): There exist finite-memory
uncoupled strategies that lead to a pure Nash equilibrium,
in every game where such an equilibrium exists.
Uncoupledness and PNE
Hart and Mas Colell (2006): There exist finite-memory
uncoupled strategies that lead to a pure Nash equilibrium,
in every game where such an equilibrium exists.
Pure Nash equilibrium is played with
limit frequency 1.
Uncoupledness and PNE
Hart and Mas Colell (2006): There exist finite-memory
uncoupled strategies that lead to a pure Nash equilibrium,
in every game where such an equilibrium exists.
Pure Nash equilibrium is played with
limit frequency 1.
Generality of the solution: For every
game.
Completely uncoupledness and PNE
Young’s (2009) "Interactive Trial and Error Learning": For
every ε > 0, there exist finite-memory completely
uncoupled strategies that lead to pure Nash equilibria
1 − ε of the time, in almost every game where such an
equilibrium exists.
Completely uncoupledness and PNE
Young’s (2009) "Interactive Trial and Error Learning": For
every ε > 0, there exist finite-memory completely
uncoupled strategies that lead to pure Nash equilibria
1 − ε of the time, in almost every game where such an
equilibrium exists.
Pure Nash equilibria are played with
limited frequency 1 − ε.
Completely uncoupledness and PNE
Young’s (2009) "Interactive Trial and Error Learning": For
every ε > 0, there exist finite-memory completely
uncoupled strategies that lead to pure Nash equilibria
1 − ε of the time, in almost every game where such an
equilibrium exists.
Pure Nash equilibria are played with
limited frequency 1 − ε.
Generality of the solution: For almost
every game.
Uncoupledness and ε-NE
Hart and Mas Colell (2006): There exist finite-memory
uncoupled strategies that lead to a Nash ε-equilibrium in
every game.
Uncoupledness and ε-NE
Hart and Mas Colell (2006): There exist finite-memory
uncoupled strategies that lead to a Nash ε-equilibrium in
every game.
A Nash ε-equilibrium is played with limit
frequency 1.
Uncoupledness and ε-NE
Hart and Mas Colell (2006): There exist finite-memory
uncoupled strategies that lead to a Nash ε-equilibrium in
every game.
A Nash ε-equilibrium is played with limit
frequency 1.
Generality of the solution: For every
game.
Completely uncoupledness and ε-NE
"Regret Testing" (Foster&Young (2006) for two-player
games, Germano&Lugosi (2007) for n-player games): For
every ε > 0, there exist finite-memory completely
uncoupled strategies that lead to Nash ε-equilibrium in
1 − ε of the time, in almost every game.
Completely uncoupledness and ε-NE
"Regret Testing" (Foster&Young (2006) for two-player
games, Germano&Lugosi (2007) for n-player games): For
every ε > 0, there exist finite-memory completely
uncoupled strategies that lead to Nash ε-equilibrium in
1 − ε of the time, in almost every game.
A Nash ε-equilibrium is played with
limited frequency 1 − ε
Completely uncoupledness and ε-NE
"Regret Testing" (Foster&Young (2006) for two-player
games, Germano&Lugosi (2007) for n-player games): For
every ε > 0, there exist finite-memory completely
uncoupled strategies that lead to Nash ε-equilibrium in
1 − ε of the time, in almost every game.
A Nash ε-equilibrium is played with
limited frequency 1 − ε
Generality of the solution: For almost
every game.
Established results
PNE
ε-NE
uncoupled
completely uncoupled
frequency 1
frequency 1-ε
all games
generic games
frequency 1
frequency 1-ε
all games
generic games
Questions
For completely uncoupled dynamic:
Questions
For completely uncoupled dynamic:
Is it possible to guarantee convergence
to Nash equilibrium in every game?
Questions
For completely uncoupled dynamic:
Is it possible to guarantee convergence
to Nash equilibrium in every game?
Is it possible to guarantee a limited
frequency of 1 of playing Nash
equilibrium?
Model- Game
N = {1, 2, ..., n} - the players.
Model- Game
N = {1, 2, ..., n} - the players.
C is a countable set of all possible actions of the
players.
Model- Game
N = {1, 2, ..., n} - the players.
C is a countable set of all possible actions of the
players.
Ai = {ai1 , ai2 , ..., aimi } ⊂ C - action set of player i
(|Ai | ≥ 2).
Model- Game
N = {1, 2, ..., n} - the players.
C is a countable set of all possible actions of the
players.
Ai = {ai1 , ai2 , ..., aimi } ⊂ C - action set of player i
(|Ai | ≥ 2).
B - the set of all possible action sets of single player.
Model- Game
N = {1, 2, ..., n} - the players.
C is a countable set of all possible actions of the
players.
Ai = {ai1 , ai2 , ..., aimi } ⊂ C - action set of player i
(|Ai | ≥ 2).
B - the set of all possible action sets of single player.
A := A1 × A2 × ... × An - action combinations set.
Model- Game
N = {1, 2, ..., n} - the players.
C is a countable set of all possible actions of the
players.
Ai = {ai1 , ai2 , ..., aimi } ⊂ C - action set of player i
(|Ai | ≥ 2).
B - the set of all possible action sets of single player.
A := A1 × A2 × ... × An - action combinations set.
ui : A → R - payoff function of player i.
Model- Dynamic setup
ai (t) ∈ Ai - the action of player i at time t.
Model- Dynamic setup
ai (t) ∈ Ai - the action of player i at time t.
a(t) = (a1 (t), a2 (t), ..., an (t)) ∈ A - the combination of
actions played at the time t.
Model- Dynamic setup
ai (t) ∈ Ai - the action of player i at time t.
a(t) = (a1 (t), a2 (t), ..., an (t)) ∈ A - the combination of
actions played at the time t.
At the end of each period player i observes his own
action ai (t) and his own payoff ui (a(t)).
Model- Dynamic setup
ai (t) ∈ Ai - the action of player i at time t.
a(t) = (a1 (t), a2 (t), ..., an (t)) ∈ A - the combination of
actions played at the time t.
At the end of each period player i observes his own
action ai (t) and his own payoff ui (a(t)).
At time t player i knows only his past acts and payoffs
which will be denoted by
hi (t) = ((ai (t0 ))tt0 =1 , (ui (a(t0 ))tt0 =1 ).
Model- Dynamic setup
ai (t) ∈ Ai - the action of player i at time t.
a(t) = (a1 (t), a2 (t), ..., an (t)) ∈ A - the combination of
actions played at the time t.
At the end of each period player i observes his own
action ai (t) and his own payoff ui (a(t)).
At time t player i knows only his past acts and payoffs
which will be denoted by
hi (t) = ((ai (t0 ))tt0 =1 , (ui (a(t0 ))tt0 =1 ).
Ht,B - the set of all the histories of length t of a player
with action set B .
Model- Dynamic setup
ai (t) ∈ Ai - the action of player i at time t.
a(t) = (a1 (t), a2 (t), ..., an (t)) ∈ A - the combination of
actions played at the time t.
At the end of each period player i observes his own
action ai (t) and his own payoff ui (a(t)).
At time t player i knows only his past acts and payoffs
which will be denoted by
hi (t) = ((ai (t0 ))tt0 =1 , (ui (a(t0 ))tt0 =1 ).
Ht,B - the set of all the histories of length t of a player
with action set B .
?
HB
∞
:= ∪ Ht,B
t=0
Model- Strategy mapping
A completely uncoupled strategy of a
player with action set B, fB , is a Borel
measurable mapping fB : HB? → ∆(B).
Model- Strategy mapping
A completely uncoupled strategy of a
player with action set B, fB , is a Borel
measurable mapping fB : HB? → ∆(B).
A completely uncoupled strategy
mapping is a function ϕ that assigns a
completely uncoupled strategy
ϕ(B) = fB to every action set B ∈ B.
Dynamics for all games
Question: Is it possible to guarantee
convergence to pure Nash equilibria in
every game, by completely uncoupled
strategies?
Dynamics for all games
Question: Is it possible to guarantee
convergence to pure Nash equilibria in
every game, by completely uncoupled
strategies?
Answer: No.
Impossibility Claim for general games
There is no completely uncoupled strategy
mapping that leads to a pure Nash
equilibrium 1 − ε of the time, in every game
with more than 2 players where such an
equilibrium exists.
Idea of the proof
Γ1 :
a21
1, 1, 1?
1, 1, 1?
a11
a12
|
Γ2 :
a22
1, 1, 1?
1, 1, 1?
{z
a31
a21
1, 0, 1
0, 1, 1
a11
a12
|
{z
a31
a11
a12
}
a22
0, 1, 1
1, 0, 1
|
a22
0, 1, 1
1, 0, 1
{z
a31
|
}
a22
1, 1, 1?
1, 1, 1?
a21
1, 1, 1?
1, 1, 1?
a11
a12
}
a21
1, 0, 1
0, 1, 1
{z
a31
}
Pure Nash equilibria
Question: Is it possible to guarantee
playing of pure Nash equilibria with limit
frequency 1, in almost every game, by
completely uncoupled finite memory
strategies?
Pure Nash equilibria
Question: Is it possible to guarantee
playing of pure Nash equilibria with limit
frequency 1, in almost every game, by
completely uncoupled finite memory
strategies?
Answer: No.
Impossibility Theorem for PNE
Let A = A1 × A2 × ... × An be an actions
1
2
combinations set such that A = A . Then
there is no completely uncoupled mapping
into finite memory strategies leading to a
pure Nash equilibrium in almost every
game with action combination set A, and
also in almost every game with action
−1
combination set A
2
3
n
:= A × A × ... × A .
Sharpness of the Theorem
Possibility in each one of the following cases:
Sharpness of the Theorem
Possibility in each one of the following cases:
Type of solution: If we allow leading to pure Nash
equilibria just 1 − ε of the time.
Sharpness of the Theorem
Possibility in each one of the following cases:
Type of solution: If we allow leading to pure Nash
equilibria just 1 − ε of the time.
The structure of the actions combinations set: If Ai 6=Aj
for all i 6= j .
Sharpness of the Theorem
Possibility in each one of the following cases:
Type of solution: If we allow leading to pure Nash
equilibria just 1 − ε of the time.
The structure of the actions combinations set: If Ai 6=Aj
for all i 6= j .
Strategies’ type: If we drop the finite memory
requirement
Sharpness of the Theorem
Possibility in each one of the following cases:
Type of solution: If we allow leading to pure Nash
equilibria just 1 − ε of the time.
The structure of the actions combinations set: If Ai 6=Aj
for all i 6= j .
Strategies’ type: If we drop the finite memory
requirement
Knowledge of the players: If we assume that the players
have some addition knowledge:
Sharpness of the Theorem
Possibility in each one of the following cases:
Type of solution: If we allow leading to pure Nash
equilibria just 1 − ε of the time.
The structure of the actions combinations set: If Ai 6=Aj
for all i 6= j .
Strategies’ type: If we drop the finite memory
requirement
Knowledge of the players: If we assume that the players
have some addition knowledge:
-Each player knows his index.
Sharpness of the Theorem
Possibility in each one of the following cases:
Type of solution: If we allow leading to pure Nash
equilibria just 1 − ε of the time.
The structure of the actions combinations set: If Ai 6=Aj
for all i 6= j .
Strategies’ type: If we drop the finite memory
requirement
Knowledge of the players: If we assume that the players
have some addition knowledge:
-Each player knows his index.
-Each player knows the total number of players.
Idea of the proof
a1
Consider the following one player game:
a2
1
2?
Idea of the proof
a1
Consider the following one player game:
a2
1
2?
The strategy of the player is f := ϕ({a1 , a2 }), f is a finite
memory strategy, so there exists a realizable history of play
h = (a(s))ts=1 such that after h happens then the player
plays a2 (as long as his payoff is 2).
Idea of the proof- Theorem 1
Now consider the following two player game:
a1
a2
a1 1, 1 4, 4?
a2 3, 3? 2, 2
Idea of the proof- Theorem 1
Now consider the following two player game:
a1
a2
a1 1, 1 4, 4?
a2 3, 3? 2, 2
The strategy of each one of the two players will be f =
ϕ({a1 , a2 }).
Idea of the proof- Theorem 1
Now consider the following two player game:
a1
a2
a1 1, 1 4, 4?
a2 3, 3? 2, 2
The strategy of each one of the two players will be f =
ϕ({a1 , a2 }).
The history hh := (a(s), a(s))ts=1 is realizable.
Idea of the proof- Theorem 1
Now consider the following two player game:
a1
a2
a1 1, 1 4, 4?
a2 3, 3? 2, 2
The strategy of each one of the two players will be f =
ϕ({a1 , a2 }).
The history hh := (a(s), a(s))ts=1 is realizable.
So if hh will happen, the players will play (a2 , a2 ) forever.
Idea of the proof- Theorem 1
Now consider the following two player game:
a1
a2
a1 1, 1 4, 4?
a2 3, 3? 2, 2
The strategy of each one of the two players will be f =
ϕ({a1 , a2 }).
The history hh := (a(s), a(s))ts=1 is realizable.
So if hh will happen, the players will play (a2 , a2 ) forever.
But (a2 , a2 ) is not a Nash equilibrium.
Idea of the proof- Theorem 1
Now consider the following two player game:
a1
a2
a1 1, 1 4, 4?
a2 3, 3? 2, 2
The strategy of each one of the two players will be f =
ϕ({a1 , a2 }).
The history hh := (a(s), a(s))ts=1 is realizable.
So if hh will happen, the players will play (a2 , a2 ) forever.
But (a2 , a2 ) is not a Nash equilibrium.
Note: These considerations will not work for games in the
neighborhood of Γ2 .
Idea of the proof- Theorem 1
How we generalize this idea to a set of games with positive
measure?
Idea of the proof- Theorem 1
How we generalize this idea to a set of games with positive
measure?
)
(
a1 u1
| 0 < u1 < u2 < 1
- S :=
?
a2 u2
Idea of the proof- Theorem 1
How we generalize this idea to a set of games with positive
measure?
)
(
a1 u1
| 0 < u1 < u2 < 1
- S :=
?
a2 u2
-S has a positive measure.
Idea of the proof- Theorem 1
How we generalize this idea to a set of games with positive
measure?
)
(
a1 u1
| 0 < u1 < u2 < 1
- S :=
?
a2 u2
-S has a positive measure.
- For almost every game in S there exists a realizable
history, that leads to a constant play of a2 .
Idea of the proof- Theorem 1
How we generalize this idea to a set of games with positive
measure?
)
(
a1 u1
| 0 < u1 < u2 < 1
- S :=
?
a2 u2
-S has a positive measure.
- For almost every game in S there exists a realizable
history, that leads to a constant play of a2 .
- Let Bh be the set of the one-player games such that the
history h is realizable and it leads to constant play of a2 .
Idea of the proof- Theorem 1
How we generalize this idea to a set of games with positive
measure?
)
(
a1 u1
| 0 < u1 < u2 < 1
- S :=
?
a2 u2
-S has a positive measure.
- For almost every game in S there exists a realizable
history, that leads to a constant play of a2 .
- Let Bh be the set of the one-player games such that the
history h is realizable and it leads to constant play of a2 .
- ∪B h ⊃ S .
h
Idea of the proof- Theorem 1
- There are countable many histories, so there exists h such
that R := Bh has positive measure; i.e. the history h leads
to a constant play of a2 in a set of games R that has positive
measure.
Idea of the proof- Theorem 1
- There are countable many histories, so there exists h such
that R := Bh has positive measure; i.e. the history h leads
to a constant play of a2 in a set of games R that has positive
measure.

a1 u11



∈
R




a2 u22



a
a
1
2


- P :=
a1 u11 , v11 u12 , v12 

a1 v11

 a u ,v

∈
R
u
,
v

2
21 21
22 22

a2 v22



u12 , u21 , v12 , v21 > 2 
Idea of the proof- Theorem 1
- There are countable many histories, so there exists h such
that R := Bh has positive measure; i.e. the history h leads
to a constant play of a2 in a set of games R that has positive
measure.

a1 u11



∈
R




a2 u22



a
a
1
2


- P :=
a1 u11 , v11 u12 , v12 

a1 v11

 a u ,v

∈
R
u
,
v

2
21 21
22 22

a2 v22



u12 , u21 , v12 , v21 > 2 
- In every game in P , there is a positive probability that hh
will happen and then the players will play (a2 , a2 ) forever. Nash ε-equilibrium
Question: Is it possible to guarantee
playing Nash ε-equilibria with limited
frequency 1, in almost every game, by
completely uncoupled strategies?
Nash ε-equilibrium
Question: Is it possible to guarantee
playing Nash ε-equilibria with limited
frequency 1, in almost every game, by
completely uncoupled strategies?
Answer: Yes! (under some mild
assumptions(?))
Assumptions (?) about the game:
(a) Number of players is bounded by P .
Assumptions (?) about the game:
(a) Number of players is bounded by P .
(b) the number of actions of every player is bounded by T .
Assumptions (?) about the game:
(a) Number of players is bounded by P .
(b) the number of actions of every player is bounded by T .
(c) The payoffs in the game is bounded by M .
Assumptions (?) about the game:
(a) Number of players is bounded by P .
(b) the number of actions of every player is bounded by T .
(c) The payoffs in the game is bounded by M .
(d) Every two payoffs of a player are different.
Assumptions (?) about the game:
(a) Number of players is bounded by P .
(b) the number of actions of every player is bounded by T .
(c) The payoffs in the game is bounded by M .
(d) Every two payoffs of a player are different.
(e) A player can encode his payoffs in finite memory, such
that every two different payoffs have different encodings.
Possibility Theorem for ε-NE
Fix P, T, M . For every ε > 0, there exists a
completely uncoupled mapping into finite
memory strategies that leads to a Nash
ε-equilibrium in every game that satisfies
(a)-(e).
Idea of the proof
Remark The following strategy in not a
reasonable one, in the sense of simplicity,
adaptiveness, rationality, etc...
Idea of the proof
Remark The following strategy in not a
reasonable one, in the sense of simplicity,
adaptiveness, rationality, etc...
- If every player knows his index then
completely uncoupled dynamic can lead to
Nash ε-equilibrium.
Idea of the proof
Step 1: All the players play their first action, and remember
the payoff: ui (1).
Idea of the proof
Step 1: All the players play their first action, and remember
the payoff: ui (1).
Step 2, the players identify the actions combinations set A:
Player 1 plays his actions by the following order:
a12 , a13 , ..., a1m1 , a11 , whereas all the other players play ai1 until
they get the payoff ui (1).
The players do the same with player 2. Etc... Until they
find player that has one action.
Idea of the proof
Step 1: All the players play their first action, and remember
the payoff: ui (1).
Step 2, the players identify the actions combinations set A:
Player 1 plays his actions by the following order:
a12 , a13 , ..., a1m1 , a11 , whereas all the other players play ai1 until
they get the payoff ui (1).
The players do the same with player 2. Etc... Until they
find player that has one action.
Step 3, the players identify their payoff function:
The players play all the actions by the lexicographic
order.
Idea of the proof
Step 4, the players find Nash ε-equilibrium and play it:
The players make an ε-discretization of ∆(A). They
consider all the possible mixed actions by the lexicographic
order. If an action of player is ε-best reply to the actions of
the other players he plays ai1 , otherwise he play ai2 . When a
player gets ui (1), he plays this mixed action forever.
Idea of the proof
- If every player knows the number of players then
completely uncoupled dynamic can lead to Nash
ε-equilibrium.
Idea of the proof
- If every player knows the number of players then
completely uncoupled dynamic can lead to Nash
ε-equilibrium.
Each player choose an index j from {1, 2, ..., n}
randomly.
Idea of the proof
- If every player knows the number of players then
completely uncoupled dynamic can lead to Nash
ε-equilibrium.
Each player choose an index j from {1, 2, ..., n}
randomly.
The players try to identify the actions combinations set
A. If they find out that there are n players, they go on to
the next steps. Otherwise they choose an index again.
Idea of the proof
- How to find Nash ε-equilibrium if the number of players is
unknown?
Idea of the proof
- How to find Nash ε-equilibrium if the number of players is
unknown?
Each player starts form belief that there is one player
(himself).
Idea of the proof
- How to find Nash ε-equilibrium if the number of players is
unknown?
Each player starts form belief that there is one player
(himself).
When player believe that there are k players, he finds a
Nash 2ε -equilibrium, and play it with probability 1 − 2ε .
With probability 2ε he chooses uniformly an action from
Ai . If a player gets some payoff that does not exists in
the k -player game, he change the belief to k + 1
players.
Summary
established results
P- pure Nash equilibrium.
A- approximate Nash equilibrium (ε-equilibrium).
×
√- impossible.
- possible.
- there exists a "nice" dynamic that guarantees this result.
general games
frequency 1
frequency 1 − ε
P: × A: ×
P: × A:
√
generic games
√
P: × A:
P:
√
A:
√
Summary
In the pure Nash equilibrium case, the
known strategies, achieve the optimal
results.
Summary
In the pure Nash equilibrium case, the
known strategies, achieve the optimal
results.
In the Nash ε-equilibrium case, the
known strategies playing Nash
ε-equilibrium with frequency 1 − ε,
whereas it is possible to play it with
frequency 1.