Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Second Order Game Dynamics
Rida Laraki
Panayotis Mertikopoulos
ADGO – November 15, 2013
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Continuous Time Games Dynamics
A fundamental question in game theory:
▶
R Laraki
which solution concept is the result of a dynamic learning process where
the participants
“accumulate empirical information on the relative advantages of the
various pure strategies at their disposal”
(Nash’s PhD thesis).
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Continuous Time Games Dynamics
A fundamental question in game theory:
▶
which solution concept is the result of a dynamic learning process where
the participants
“accumulate empirical information on the relative advantages of the
various pure strategies at their disposal”
(Nash’s PhD thesis).
To that end:
▶
R Laraki
numerous classes of dynamical systems have been proposed from both a
learning and a population perspective
(Hofbauer and Sigmund 1988, Weibull 1995, Sandholm 2010).
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Differences between dynamics
Many differences among game dynamics:
R Laraki
▶
they can be imitative (replicator) or innovative (Smith);
▶
rest points might properly contain the game’s Nash set or coincide with it;
▶
strictly dominated strategies might extinct or survive.
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Differences between dynamics
Many differences among game dynamics:
▶
they can be imitative (replicator) or innovative (Smith);
▶
rest points might properly contain the game’s Nash set or coincide with it;
▶
strictly dominated strategies might extinct or survive.
Many negative results:
R Laraki
▶
there is no class of uncoupled game dynamics that always converges to a
Nash equilibrium (Hart and Mas-Collel 2003).
▶
weakly dominated strategies may survive (Samuelson 1993).
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
▶
R Laraki
Rationality Analysis
. . . . . . .
Concluding Remarks
The main unifying feature of game dynamics is that they are
first order dynamical systems over the game’s mixed strategy space.
CNRS - LAMSADE, Ecole Polytechnique
Introduction
R Laraki
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
▶
The main unifying feature of game dynamics is that they are
first order dynamical systems over the game’s mixed strategy space.
▶
Can second order dynamics be introduced naturally in games?
CNRS - LAMSADE, Ecole Polytechnique
Introduction
R Laraki
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
▶
The main unifying feature of game dynamics is that they are
first order dynamical systems over the game’s mixed strategy space.
▶
Can second order dynamics be introduced naturally in games?
▶
Do second order systems allow to have better convergence properties?
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Contributions
▶
R Laraki
We derive a wide class of higher order imitative dynamics;
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Contributions
R Laraki
▶
We derive a wide class of higher order imitative dynamics;
▶
We show that strictly dominated strategies become extinct faster than the
corresponding first order dynamics;
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Contributions
R Laraki
▶
We derive a wide class of higher order imitative dynamics;
▶
We show that strictly dominated strategies become extinct faster than the
corresponding first order dynamics;
▶
For Nash equilibria, an analogue of the folk theorem of evolutionary game
theory holds;
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Contributions
R Laraki
▶
We derive a wide class of higher order imitative dynamics;
▶
We show that strictly dominated strategies become extinct faster than the
corresponding first order dynamics;
▶
For Nash equilibria, an analogue of the folk theorem of evolutionary game
theory holds;
▶
In contrast to first order, weakly dominated strategies become extinct.
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Model and Notation
Basic ingredient: a multi-player game in normal form G ≡ G(N , A, u):
▶
N = {1, . . . , N }: players of the game
▶
Ak = {α0 , α1 , . . . }: actions of player k
▶
Xk ≡ ∆(Ak ): mixed strategies (or population distributions) of player k
∏
uk : X ≡ k Xk → R: the players’ (multilinear) payoff functions
▶
uk (x) =
R Laraki
∑k
α
xkα ukα (x)
where
ukα (x) = uk (α; x−k )
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Model and Notation
Basic ingredient: a multi-player game in normal form G ≡ G(N , A, u):
▶
N = {1, . . . , N }: players of the game
▶
Ak = {α0 , α1 , . . . }: actions of player k
▶
Xk ≡ ∆(Ak ): mixed strategies (or population distributions) of player k
∏
uk : X ≡ k Xk → R: the players’ (multilinear) payoff functions
▶
uk (x) =
∑k
α
xkα ukα (x)
where
ukα (x) = uk (α; x−k )
Objective: write a well-behaved dynamical system of the form ẍ = F (x, ẋ).
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Towards Higher Order I: the Direct Approach
The most direct route to second order dynamics: add a dot!
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Towards Higher Order I: the Direct Approach
The most direct route to second order dynamics: add a dot!
For instance, take the standard (multi-population) replicator dynamics:
dxkα
= xkα (ukα (x) − uk (x))
dt
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Towards Higher Order I: the Direct Approach
The most direct route to second order dynamics: add a dot!
For instance, take the standard (multi-population) replicator dynamics:
dxkα
= xkα (ukα (x) − uk (x))
dt
and write:
d2 xkα
= xkα (ukα (x) − uk (x))
dt2
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Towards Higher Order I: the Direct Approach
The most direct route to second order dynamics: add a dot!
For instance, take the standard (multi-population) replicator dynamics:
dxkα
= xkα (ukα (x) − uk (x))
dt
and write:
d2 xkα
= xkα (ukα (x) − uk (x))
dt2
The problem:
These dynamics are not well-defined: solutions may escape the strategy space.
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Towards Higher Order II: Morgan and Flåm (2004)
A solution: project this velocity on the corresponding tangent cone
ẋ = v
v̇ = projTx X V (x)
where Tx X is the tangent cone to X at x:
∏ A ∑k
Tx X = {z ∈
R k :
zkα = 0 and zkα ≥ 0 whenever xkα = 0}
k
R Laraki
α
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Towards Higher Order II: Morgan and Flåm (2004)
A solution: project this velocity on the corresponding tangent cone
ẋ = v
v̇ = projTx X V (x)
where Tx X is the tangent cone to X at x:
∏ A ∑k
Tx X = {z ∈
R k :
zkα = 0 and zkα ≥ 0 whenever xkα = 0}
k
α
Drawbacks: no justification, discontinuous dynamics, problem of existence...
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Towards Higher Order III - Infinite Potential Walls
Another solution: Erect infinite walls at the boundary of X:
(
)
∑k
ẍkα = xkα ukα (x) −
xkβ ukβ (x) + Wkα (xk , ẋk )
β
where:
∑
1.
α Wkα = 0
2. Wkα → ∞ as xkα → 0
3. Wkα → 0 as ẋkα → 0
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Towards Higher Order III - Infinite Potential Walls
Another solution: Erect infinite walls at the boundary of X:
(
)
∑k
ẍkα = xkα ukα (x) −
xkβ ukβ (x) + Wkα (xk , ẋk )
β
where:
∑
1.
α Wkα = 0
2. Wkα → ∞ as xkα → 0
3. Wkα → 0 as ẋkα → 0
Drawback: How to fix W ? Which learning or population justification?
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Towards Higher Order IV: Reinforcement Learning loop
.
R Laraki
.
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Towards Higher Order IV: Reinforcement Learning loop
.
Pick strategies
R Laraki
x(t) .∈ X
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Towards Higher Order IV: Reinforcement Learning loop
.
Pick strategies
x(t) .∈ X
Play
Record payoffs
R Laraki
ukα (x(t))
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Towards Higher Order IV: Reinforcement Learning loop
.
Pick strategies
x(t) .∈ X
Play
Record payoffs
ukα (x(t))
Score
Score performances
R Laraki
ykα (t) 7→ ykα (t + 1)
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Towards Higher Order IV: Reinforcement Learning loop
.
Pick strategies
x(t) .∈ X
Play
Record payoffs
ukα (x(t))
Score
Score performances
ykα (t) 7→ ykα (t + 1)
Update
Map performance
scores to strategies
R Laraki
y 7→ x
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Which mapping from scores to strategies?
Key Step 1: mapping scores y ∈ RA to strategies x ∈ ∆(A).
Desired properties of the evaluation map G : RA → ∆(A):
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Which mapping from scores to strategies?
Key Step 1: mapping scores y ∈ RA to strategies x ∈ ∆(A).
Desired properties of the evaluation map G : RA → ∆(A):
1. Monotonicity: Gα (y) increases in yα .
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Which mapping from scores to strategies?
Key Step 1: mapping scores y ∈ RA to strategies x ∈ ∆(A).
Desired properties of the evaluation map G : RA → ∆(A):
1. Monotonicity: Gα (y) increases in yα .
2. Symmetry: yα ↔ yβ ⇐⇒ Gα ↔ Gβ
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Which mapping from scores to strategies?
Key Step 1: mapping scores y ∈ RA to strategies x ∈ ∆(A).
Desired properties of the evaluation map G : RA → ∆(A):
1. Monotonicity: Gα (y) increases in yα .
2. Symmetry: yα ↔ yβ ⇐⇒ Gα ↔ Gβ
/
3. Independence of Irrelevant Alternatives: Gα Gβ only depends on yα , yβ
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Which mapping from scores to strategies?
Key Step 1: mapping scores y ∈ RA to strategies x ∈ ∆(A).
Desired properties of the evaluation map G : RA → ∆(A):
1. Monotonicity: Gα (y) increases in yα .
2. Symmetry: yα ↔ yβ ⇐⇒ Gα ↔ Gβ
/
3. Independence of Irrelevant Alternatives: Gα Gβ only depends on yα , yβ
4. Invariance: G(y0 , . . . , yn ) = G(y0 + c, . . . , yn + c) for any c ∈ R.
(Relative score differences is all that matters, just like adding a constant to the
game’s payoffs does not change the game.)
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Which mapping from scores to strategies?
Key Step 1: mapping scores y ∈ RA to strategies x ∈ ∆(A).
Desired properties of the evaluation map G : RA → ∆(A):
1. Monotonicity: Gα (y) increases in yα .
2. Symmetry: yα ↔ yβ ⇐⇒ Gα ↔ Gβ
/
3. Independence of Irrelevant Alternatives: Gα Gβ only depends on yα , yβ
4. Invariance: G(y0 , . . . , yn ) = G(y0 + c, . . . , yn + c) for any c ∈ R.
(Relative score differences is all that matters, just like adding a constant to the
game’s payoffs does not change the game.)
Proposition
If G : RS → ∆(S) satisfies the above properties, then G is a Gibbs map:
exp(λyα )
Gα (y) = ∑
β exp(λyβ )
R Laraki
for some λ > 0.
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Reinforcement Learning: Updating Scores
Key Step 2: how to measure a strategy’s performance?
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Reinforcement Learning: Updating Scores
Key Step 2: how to measure a strategy’s performance?
A simple updating rule: score’s rate of change is the instantaneous payoff:
ẏkα (t) = ukα (x(t))
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Reinforcement Learning: Updating Scores
Key Step 2: how to measure a strategy’s performance?
A simple updating rule: score’s rate of change is the instantaneous payoff:
ẏkα (t) = ukα (x(t))
/∑
k
Coupled with the Gibbs map xkα = exp(ykα )
β exp(ykβ ), this updating
rule yields the (first-order) replicator dynamics:
(
)
∑k
ẋkα = xkα ukα (x) −
xkβ ukβ (x)
β
(Hofbauer et al. 09; Mertikopoulos-Moustakas 10; Rustichini 99; Sorin, 09)
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Reinforcement Learning: Second Order Effects
What if the rate of change corresponds to the cumulative payoff ?
ẏkα = Ukα
where Ukα (t) =
R Laraki
∫t
0
ukα (x(s))ds is the cumulative payoff of strategy α.
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Reinforcement Learning: Second Order Effects
What if the rate of change corresponds to the cumulative payoff ?
ẏkα = Ukα
where Ukα (t) =
∫t
0
ukα (x(s))ds is the cumulative payoff of strategy α.
A second derivation yields:
ÿkα (t) = ukα (x(t))
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Reinforcement Learning: Second Order Effects
What if the rate of change corresponds to the cumulative payoff ?
ẏkα = Ukα
where Ukα (t) =
∫t
0
ukα (x(s))ds is the cumulative payoff of strategy α.
A second derivation yields:
ÿkα (t) = ukα (x(t))
Coupled with the Gibbs map, we obtain the second order replicator dynamics:
(
)
/
/
∑
. kα ẋ2ka x2kα − k ẋ2kβ xkβ
ẍkα = xkα (ukα (x) − uk (x)) + x
β
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Reinforcement Learning: Second Order Effects
What if the rate of change corresponds to the cumulative payoff ?
ẏkα = Ukα
where Ukα (t) =
∫t
0
ukα (x(s))ds is the cumulative payoff of strategy α.
A second derivation yields:
ÿkα (t) = ukα (x(t))
Coupled with the Gibbs map, we obtain the second order replicator dynamics:
(
)
/
/
∑
. kα ẋ2ka x2kα − k ẋ2kβ xkβ
ẍkα = xkα (ukα (x) − uk (x)) + x
β
Important observations: the initial velocity ẏ(0) is 0.
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Population justification
ẋkα = xkα (Ukα (t) − Uk (t)), where Ukα (t) =
R Laraki
∫t
0
ukα (x(s))ds
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Population justification
ẋkα = xkα (Ukα (t) − Uk (t)), where Ukα (t) =
▶
0
ukα (x(s))ds
By differentiating with respect to time we obtain:
(
)
(
)
∑k
∑k
∑k
ẍkα = ẋkα Ukα −
xkβ Ukβ +xkα ukα −
xkβ ukβ −xkα
ẋkβ Ukβ
β
R Laraki
∫t
β
β
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Population justification
ẋkα = xkα (Ukα (t) − Uk (t)), where Ukα (t) =
▶
∫t
0
ukα (x(s))ds
By differentiating with respect to time we obtain:
(
)
(
)
∑k
∑k
∑k
ẍkα = ẋkα Ukα −
xkβ Ukβ +xkα ukα −
xkβ ukβ −xkα
ẋkβ Ukβ
β
▶
R Laraki
Some easy algebra yields
β
∑k
β
ẋkβ Ukβ =
β
/
∑
β
ẋ2kβ xkβ
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Population justification
ẋkα = xkα (Ukα (t) − Uk (t)), where Ukα (t) =
▶
∫t
0
ukα (x(s))ds
By differentiating with respect to time we obtain:
(
)
(
)
∑k
∑k
∑k
ẍkα = ẋkα Ukα −
xkβ Ukβ +xkα ukα −
xkβ ukβ −xkα
ẋkβ Ukβ
β
▶
Some easy algebra yields
▶
Consequently:
β
∑k
β
ẋkβ Ukβ =
β
/
∑
β
ẋ2kβ xkβ
(
)
∑k 2 /
/
ẍkα = xkα (ukα (x) − uk (x)) + xkα ẋ2kα x2kα −
ẋkβ xkβ
β
This is the second order replicator equation !
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Higher Order Imitative Dynamics
More generally, players may use other ”payoff observables” wkα : X → R to
update their scores:
(n)
ykα = wkα (x)
leading to the higher order imitative dynamics:
(
)
(
)
∑
∑
(n)
n−1
n−1
.
xkα = xkα wkα (x) − kβ xkβ wkβ (x)
+ xkα Rkα
− kβ xkβ Rkβ
n−1
where Rkα
is the higher order adjustment term of the replicator dynamics.
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Examples I
First and second order replicator dynamics in a dominance solvable game.
First Order Replicator Dynamics
Second Order Replicator Dynamics
1.0
1.0
H0, 0L
H1, 1L
H0, 0L
0.6
0.6
y
0.8
y
0.8
0.4
0.4
0.2
0.2
H0, 0L
H-1, -1L
0.0
H0, 0L
H-1, -1L
0.0
0.0
0.2
0.4
0.6
x
R Laraki
H1, 1L
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
x
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Examples II
First and second order replicator dynamics in a coordination game.
First Order Replicator Dynamics
Second Order Replicator Dynamics
1.0
1.0
H0, 0L
H1, 1L
H0, 0L
0.6
0.6
y
0.8
y
0.8
0.4
0.4
0.2
0.2
H1, 1L
H0, 0L
0.0
H1, 1L
H0, 0L
0.0
0.0
0.2
0.4
0.6
x
R Laraki
H1, 1L
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
x
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Examples III
First and second order replicator dynamics in Matching Pennies.
First Order Replicator Dynamics
Second Order Replicator Dynamics
1.0
1.0
H-1, 1L
0.6
0.6
H1, -1L
y
0.8
y
0.8
0.4
0.4
0.2
0.2
H-1, 1L
H1, -1L
0.0
H-1, 1L
H1, -1L
0.0
0.0
0.2
0.4
0.6
x
R Laraki
H-1, 1L
H1, -1L
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
x
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Extinction of Dominated Strategies I
Theorem
Let x(t) be an interior solution path of the n-th order replicator dynamics. If
qk ∈ Xk is iteratively strictly dominated, then:
/
DKL (qk ∥ xk (t)) ≥ λk ctn n! + O(tn−1 ),
where c > 0 and ∑
DKL is the(K-L /divergence
)
DKL (qk ∥ xk ) = kα qkα log qkα xkα .
In particular, for pure strategies α ≺ β, we have:
/
(
/
)
xkα (t) xkβ (t) ≤ exp −λk ∆uβα tn n! + O(tn−1 ) ,
where ∆uβα = minx∈X {ukβ (x) − ukα (x)} > 0.
In other words, iteratively strictly dominated strategies become extinct in the
n-th order replicator dynamics n orders as fast as in the first order replicator
dynamics.
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Extinction of Dominated Strategies I
Sketch of Proof.
▶
R Laraki
(n)
ykα = ukα .
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Extinction of Dominated Strategies I
Sketch of Proof.
R Laraki
(n)
▶
ykα = ukα .
▶
If q ≺ q ′ , then ukα (qk′ ; x−k ) − ukα (qk ; x−k ) ≥ δ > 0 for all x−k ∈ X−k .
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Extinction of Dominated Strategies I
Sketch of Proof.
(n)
▶
ykα = ukα .
▶
If q ≺ q ′ , then ukα (qk′ ; x−k ) − ukα (qk ; x−k ) ≥ δ > 0 for all x−k ∈ X−k .
▶
The entropic difference V (x) = DKL (qk ∥ xk ) − DKL (qk′ ∥ xk ) then gives:
dn
V (x(t)) ≥ δ > 0,
dtn
and the theorem follows by an n-fold application of the mean value theorem
and induction on the rounds of elimination of dominated strategies.
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Weakly Dominated Strategies
What about weakly dominated strategies?
In first order dynamics, weakly dominated strategies survive, even in simple
2 × 2 games.
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Weakly Dominated Strategies
What about weakly dominated strategies?
In first order dynamics, weakly dominated strategies survive, even in simple
2 × 2 games.
Theorem
Let x(t) be an interior solution orbit of the n-th order (n ≥ 2) replicator
dynamics that starts at rest: ẋ(0) = . . . = x(n−1) (0) = 0.
If qk ∈ Xk is weakly dominated, then it becomes extinct along x(t) at a rate
/
DKL (qk ∥ xk (t)) ≥ λk ctn−1 (n − 1)!
where λk is the learning rate of player k and c > 0 is a positive constant.
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Weakly Dominated Strategies
What about weakly dominated strategies?
In first order dynamics, weakly dominated strategies survive, even in simple
2 × 2 games.
Theorem
Let x(t) be an interior solution orbit of the n-th order (n ≥ 2) replicator
dynamics that starts at rest: ẋ(0) = . . . = x(n−1) (0) = 0.
If qk ∈ Xk is weakly dominated, then it becomes extinct along x(t) at a rate
/
DKL (qk ∥ xk (t)) ≥ λk ctn−1 (n − 1)!
where λk is the learning rate of player k and c > 0 is a positive constant.
R Laraki
▶
In higher order, weakly dominated strategies extinct if players start unbiased.
▶
No extension for iteratively weakly dominated strategies.
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Weakly Dominated Strategies
Weakly dominated strategies in the second order replicator dynamics:
Entry Deterrence
Outside Option
Second Order Portrait of an Entry Deterrence Game
81, 3<
82, 2<
1.0
80, 0<
H0, 0L
H2, 2L
Player 2: 'fight' H0L vs. 'yield' H1L
0.8
0.6
First Order Portrait
1.0
0.4
0.8
0.6
80, 0<
0.2
0.4
82, 2<
0.2
H2, 0L
H0, 0L
0.0
0.0
0.0
0.0
0.2
0.4
0.2
0.4
0.6
Player 1: 'stay out' H0L vs. 'enter' H1L
R Laraki
0.6
0.8
0.8
1.0
1.0
83, 1<
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Strengthening rationalizability
Kohlberg and Mertens 1986:
A good concept of “strategically stable equilibrium” should satisfy
[...] iterated dominance rationality of the normal form.
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Strengthening rationalizability
Kohlberg and Mertens 1986:
A good concept of “strategically stable equilibrium” should satisfy
[...] iterated dominance rationality of the normal form.
Hillas and Kohlberg 2001:
There seems to us one slight strengthening of rationalizability that is
well motivated. It is one round of elimination of weakly dominated
strategies followed by an arbitrarily large number of rounds of
elimination of strictly dominated strategies. This solution is obtained
by Dekel and Fudenberg, under the assumption that there is some
small uncertainty about the payoffs...
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Strengthening rationalizability
Kohlberg and Mertens 1986:
A good concept of “strategically stable equilibrium” should satisfy
[...] iterated dominance rationality of the normal form.
Hillas and Kohlberg 2001:
There seems to us one slight strengthening of rationalizability that is
well motivated. It is one round of elimination of weakly dominated
strategies followed by an arbitrarily large number of rounds of
elimination of strictly dominated strategies. This solution is obtained
by Dekel and Fudenberg, under the assumption that there is some
small uncertainty about the payoffs...
Theorem
Higher order dynamics perform one round of elimination of weakly dominated
strategies followed by repeated elimination of strictly dominated strategies.
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Nash Play and the Folk Theorem
What about stability and convergence to Nash equilibrium?
In first order, the golden standard is the folk theorem:
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Nash Play and the Folk Theorem
What about stability and convergence to Nash equilibrium?
In first order, the golden standard is the folk theorem:
I. Nash equilibria are stationary.
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Nash Play and the Folk Theorem
What about stability and convergence to Nash equilibrium?
In first order, the golden standard is the folk theorem:
I. Nash equilibria are stationary.
II. If an interior solution orbit converges, its limit is Nash.
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Nash Play and the Folk Theorem
What about stability and convergence to Nash equilibrium?
In first order, the golden standard is the folk theorem:
I. Nash equilibria are stationary.
II. If an interior solution orbit converges, its limit is Nash.
III. If a point is (Lyapunov) stable, then it is also Nash.
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Nash Play and the Folk Theorem
What about stability and convergence to Nash equilibrium?
In first order, the golden standard is the folk theorem:
I. Nash equilibria are stationary.
II. If an interior solution orbit converges, its limit is Nash.
III. If a point is (Lyapunov) stable, then it is also Nash.
IV. A point is asymptotically stable if and only if it is a strict equilibrium.
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Nash Play and the Folk Theorem
What about stability and convergence to Nash equilibrium?
In first order, the golden standard is the folk theorem:
I. Nash equilibria are stationary.
II. If an interior solution orbit converges, its limit is Nash.
III. If a point is (Lyapunov) stable, then it is also Nash.
IV. A point is asymptotically stable if and only if it is a strict equilibrium.
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Nash Play and the Folk Theorem
What about stability and convergence to Nash equilibrium?
In first order, the golden standard is the folk theorem:
I. Nash equilibria are stationary.
II. If an interior solution orbit converges, its limit is Nash.
III. If a point is (Lyapunov) stable, then it is also Nash.
IV. A point is asymptotically stable if and only if it is a strict equilibrium.
(Hofbauer and Sigmund 1988, Weibull 1995, Sandholm 2010)
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
The Higher Order Folk Theorem
Theorem
Let x(t) be a solution orbit of the n-th order replicator dynamics. Then:
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
The Higher Order Folk Theorem
Theorem
Let x(t) be a solution orbit of the n-th order replicator dynamics. Then:
I. x(t) = q for all t ≥ 0 if and only if q is a restricted equilibrium of G.
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
The Higher Order Folk Theorem
Theorem
Let x(t) be a solution orbit of the n-th order replicator dynamics. Then:
I. x(t) = q for all t ≥ 0 if and only if q is a restricted equilibrium of G.
II. If x(0) ∈ int(X) and lim x(t) = q, then q is a Nash equilibrium of G.
t→∞
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
The Higher Order Folk Theorem
Theorem
Let x(t) be a solution orbit of the n-th order replicator dynamics. Then:
I. x(t) = q for all t ≥ 0 if and only if q is a restricted equilibrium of G.
II. If x(0) ∈ int(X) and lim x(t) = q, then q is a Nash equilibrium of G.
t→∞
III. If every neighborhood U of q in X admits an interior orbit xU (t) such that
xU (t) ∈ U for all t ≥ 0, then q is a Nash equilibrium of G.
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
The Higher Order Folk Theorem
Theorem
Let x(t) be a solution orbit of the n-th order replicator dynamics. Then:
I. x(t) = q for all t ≥ 0 if and only if q is a restricted equilibrium of G.
II. If x(0) ∈ int(X) and lim x(t) = q, then q is a Nash equilibrium of G.
t→∞
III. If every neighborhood U of q in X admits an interior orbit xU (t) such that
xU (t) ∈ U for all t ≥ 0, then q is a Nash equilibrium of G.
IV. Let q be a strict equilibrium. Then, for every neighborhood U of q in X
one has limt→∞ x(t) = q for all trajectories that starts at rest whenever
x(0) ∈ U . Conversely, only strict equilibria have this property.
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
The Higher Order Folk Theorem
Theorem
Let x(t) be a solution orbit of the n-th order replicator dynamics. Then:
I. x(t) = q for all t ≥ 0 if and only if q is a restricted equilibrium of G.
II. If x(0) ∈ int(X) and lim x(t) = q, then q is a Nash equilibrium of G.
t→∞
III. If every neighborhood U of q in X admits an interior orbit xU (t) such that
xU (t) ∈ U for all t ≥ 0, then q is a Nash equilibrium of G.
IV. Let q be a strict equilibrium. Then, for every neighborhood U of q in X
one has limt→∞ x(t) = q for all trajectories that starts at rest whenever
x(0) ∈ U . Conversely, only strict equilibria have this property.
▶
R Laraki
Convergence rates to strict equilibria are n orders as fast as in first order.
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Concluding Remarks and Future Directions
▶
R Laraki
Non convergence to interior points in all imitative dynamics.
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Concluding Remarks and Future Directions
R Laraki
▶
Non convergence to interior points in all imitative dynamics.
▶
Higher order innovative dynamics.
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Concluding Remarks and Future Directions
R Laraki
▶
Non convergence to interior points in all imitative dynamics.
▶
Higher order innovative dynamics.
▶
One player against nature in continuous time (exponential weight
algorithm induces a non-autonomous replicator dynamic on the simplex,
Sorin 2009).
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Concluding Remarks and Future Directions
R Laraki
▶
Non convergence to interior points in all imitative dynamics.
▶
Higher order innovative dynamics.
▶
One player against nature in continuous time (exponential weight
algorithm induces a non-autonomous replicator dynamic on the simplex,
Sorin 2009).
▶
What happens
∫ t with more general scoring schemes
ykα (t) = 0 ϕ(t − s)ukα (s)ds? ϕ(s) = sn gives n-th order dynamics
Other kernels give more general integral dynamics.
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Concluding Remarks and Future Directions
▶
Non convergence to interior points in all imitative dynamics.
▶
Higher order innovative dynamics.
▶
One player against nature in continuous time (exponential weight
algorithm induces a non-autonomous replicator dynamic on the simplex,
Sorin 2009).
▶
What happens
∫ t with more general scoring schemes
ykα (t) = 0 ϕ(t − s)ukα (s)ds? ϕ(s) = sn gives n-th order dynamics
Other kernels give more general integral dynamics.
How behave the dynamics ?
(
)
/
/
∑
ẍkα = xkα (ukα (x) − uk (x)) + 12. xkα ẋ2ka x2kα − kβ ẋ2kβ xkβ
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Concluding Remarks and Future Directions
▶
Non convergence to interior points in all imitative dynamics.
▶
Higher order innovative dynamics.
▶
One player against nature in continuous time (exponential weight
algorithm induces a non-autonomous replicator dynamic on the simplex,
Sorin 2009).
▶
What happens
∫ t with more general scoring schemes
ykα (t) = 0 ϕ(t − s)ukα (s)ds? ϕ(s) = sn gives n-th order dynamics
Other kernels give more general integral dynamics.
How behave the dynamics ?
(
)
/
/
∑
ẍkα = xkα (ukα (x) − uk (x)) + 12. xkα ẋ2ka x2kα − kβ ẋ2kβ xkβ
It is not well defined!... (Laraki Mertikopoulos 2013)
R Laraki
CNRS - LAMSADE, Ecole Polytechnique
Introduction
Towards Higher Order
Rationality Analysis
. . . . . . .
Concluding Remarks
Thank you Amir, Hofbauer, Sorin.
Thanks to the audience.
R Laraki
CNRS - LAMSADE, Ecole Polytechnique