Continuous Mixed Strategy Equilibria
in Static and Dynamic Tournaments
Vladimir Petkov
VUW
October 2013
Vladimir Petkov (VUW)
Continuous Mixed Strategy Equilibria
October 2013
1 / 25
Introduction
In this project I study several static and dynamic games that do not
have Nash equilibria in pure strategies.
I characterize the symmetric equilibria in which the players use
continuous mixed strategies.
The main novelty in this paper is the analysis of a setting with state
dynamics.
I derive a closed-form solution for the Markovian mixed equilibrium
strategies.
It enables us to make predictions about players’behavior in the short
run and in the long run.
This analysis can be used to study the bonus structure of salesmen
within a company.
Another application: to construct prizes for sport events.
Vladimir Petkov (VUW)
Continuous Mixed Strategy Equilibria
October 2013
2 / 25
Literature Review
Probably the most famous game with a continuous mixed strategy
equilibrium is Varian’s model of sales.
Two …rms play a Bertrand game.
Firms sell to both captive customers (who do not migrate) and
opportunistic customers (who buy at the lowest price).
There are no pure strategy equilibria:
each producer wants to undercut the opponent’s price in order to take
advantage of the opportunistic buyers;
however, charging low prices will cause …rms to forfeit pro…ts from the
captive customers.
There is little work on mixed strategies in dynamic games.
I am aware of two exceptions (neither exhibits state dynamics).
War of attrition games: waiting is costly, but the player who waits the
longest gets the prize.
Preemption games: if one player chooses to act he gets the prize, but if
multiple players act the prize gets destroyed.
Vladimir Petkov (VUW)
Continuous Mixed Strategy Equilibria
October 2013
3 / 25
Setup of the Stage Game
First we look at a one-shot game of e¤ort choice.
There are two players: 1 and 2.
They choose their e¤ort levels, x1 and x2 , simultaneously.
Each of them is required to exert e¤ort at least a > 0.
If xi > x j , player i wins.
Both players get positive prizes!
i gets the …rst prize, αP.
j gets the second prize, (1
Assume that 1 > α > 0.5.
α) P.
The prize fund is increasing in the joint e¤ort of the players:
P = β ( x1 + x2 ),
where 2 > β > 0.
To recap, if xi > x j , then the payo¤s of i and j are
αβ( x1 + x2 )
Vladimir Petkov (VUW)
xi ,
(1
α ) β ( x1 + x2 )
Continuous Mixed Strategy Equilibria
xj.
October 2013
4 / 25
No Nash Equilibrium in Pure Strategies
This game does not have an equilibrium in pure strategies.
If j exerts the minimum e¤ort x j = a, player i will want to work
slightly harder than j (i.e. choose xi = a + ε).
Obviously, j will then trump i’s e¤ort by choosing x j = xi + ε, etc.
But if i’s e¤ort is above some critical y, j may settle for the 2nd prize.
if xi > y, j would be better o¤ free-riding on i’s e¤ort.
he will choose the minimum e¤ort a, and get (1 α) β( xi + a)
The critical value y solves αβ(2y)
y=
If 1 + β
y = (1
α) β(y + a)
a.
a, or
[1 β + αβ] a
.
1 + β 3αβ
3αβ > 0, then we get a < y, so we will have a “cycle”.
If this condition is violated, both players will exert in…nite e¤ort.
Vladimir Petkov (VUW)
Continuous Mixed Strategy Equilibria
October 2013
5 / 25
No NE in Pure Strategies (Continued)
For example, suppose that a = 5, α = 0.6, β = 1.1.
Note that 1 + β 3αβ = 0.12 > 0.
Vladimir Petkov (VUW)
Continuous Mixed Strategy Equilibria
October 2013
6 / 25
Symmetric Mixed Strategy Equilibrium of the Stage Game
Assume that 1 + β 3αβ > 0 and focus on the symmetric mixed
strategy Nash equilibrium of this game.
Both players randomize according to a cdf F ( x ) with support [l, h].
If j uses this strategy and i chooses xi , i’s expected payo¤ will be
ui =
Zxi
βα( xi + x j )dF ( x j ) +
Zh
β (1
α)( xi + x j )dF ( x j )
xi .
xi
l
To randomize, i must be indi¤erent between any xi in [l, h].
Thus, F ( x ) must be such that ∂ui /∂xi = 0 for any xi !
Di¤erentiating w.r.t. xi and using the Leibnitz rule yields
F ( xi ) + 2xi F 0 ( xi )
where
R=
Vladimir Petkov (VUW)
1
R = 0,
β (1 α )
> 1.
β(2α 1)
Continuous Mixed Strategy Equilibria
October 2013
7 / 25
Solving the Di¤erential Equation
p
The solution to this di¤erential equation is F ( xi ) = R + C/ xi ,
where C is the constant of integration.
We also need to determine the support of F.
p
The constant C must satisfy F (l ) = R + C/ l = 0.
p
Solving for C yields C = R l.
The upper bound of the support satis…es F (h) = R(1
Solving for h gives us h = l
R 2
R 1 .
So the cdf F of each player’s strategy is F ( x ) = R 1
The corresponding pdf is f ( x ) =
Vladimir Petkov (VUW)
F0 (x)
=
p
Continuous Mixed Strategy Equilibria
lRx
2
p p
l/ h) = 1.
p !
l
p .
x
1.5
.
October 2013
8 / 25
Solving the Di¤erential Equation (Continued)
l ( RR 1 )
The expected e¤ort of player i is x̄i =
Z
2
xi
p
lRxi
2
1.5
dxi = l
R
R 1
.
l
If j sticks to this strategy, i will be indi¤erent between any xi in [l, h].
To get i’s expected payo¤, we can set xi = l and substitute this in ui :
Zh
β (1
α)(l + x j )dF ( x j )
l = β (1
α)l
l
R
R
1
l [1
β (1
α)].
But if i chooses a and a < l, he will get a higher expected payo¤:
Zh
β (1
α)( a + x j )dF ( x j )
a = β (1
α)l
l
R
R
1
a [1
β (1
α)].
October 2013
9 / 25
Therefore, it must be the case that l = a!
Vladimir Petkov (VUW)
Continuous Mixed Strategy Equilibria
Equilibrium Mixed Strategy and Payo¤s
To recap, the equilibrium mixed strategy cdf is
p
a
F(x) = R 1 p
,
x
where R =
1
β (1 α )
.
β(2α 1)
The strategy support is [ a, a
Each player’s expected e¤ort
R 2
R 1 ].
is a RR 1
Each gets an expected payo¤ β(1
.
α) a
R
R 1
a [1
β (1
α)].
Note that if β = 1, we have a zero-sum game.
In a symmetric equilibrium, each player must expect a zero payo¤.
Indeed, when β = 1 we get ui = u j = 0.
Vladimir Petkov (VUW)
Continuous Mixed Strategy Equilibria
October 2013
10 / 25
Illustration
Let a = 5, α = 0.6, β = 1.1. Expected e¤ort is x̄ = 8.24.
The pdf is shown below.
x
Vladimir Petkov (VUW)
Continuous Mixed Strategy Equilibria
October 2013
11 / 25
Illustration (Continued)
Game value for β = 1.1.
Game value for α = 0.6.
The …gures illustrate how the game value changes with α and β.
If β < 1, the game value is negative and decreasing in α.
Vladimir Petkov (VUW)
Continuous Mixed Strategy Equilibria
October 2013
12 / 25
Dynamic Game Setting
Next I consider a dynamic version of this game.
In each period, the players play the stage game described above.
Their period-t objective is to maximize the discounted sum of
expected instantaneous payo¤s
U1t =
∞
∑ δ(τ
t)
Et (u1τ ),
U2t =
∞
∑ δ(τ
t)
Et (u2τ ).
τ =t
τ =t
The state variable is the minimum e¤ort level at (“reputation”).
Assume that it evolves according to at+1 = µ + η ( xit + x tj ).
Again, using arguments similar to those in the one-shot game, we can
show that a pure strategy equilibrium does not exist.
Focus on the symmetric equilibrium in Markovian mixed strategies the cdf is conditional on the state:
F t = F ( j a t ).
Note that such an equilibrium will be subgame-perfect.
Vladimir Petkov (VUW)
Continuous Mixed Strategy Equilibria
October 2013
13 / 25
Analysis of the Dynamic Game
Now players have two types of incentive to exert e¤ort: static (to win
the current stage game) and dynamic (to a¤ect future e¤ort).
Let Ui ( at+1 ) be the continuation value of the game for player i.
That is, Ui ( at+1 ) is sum of i’s expected discounted payo¤s from t + 1
onward, where the expectation is taken at the beginning of period t + 1
(i.e., once at+1 has been realized).
We conjecture that Ui ( at+1 ) is linear in at+1 : Ui ( at+1 ) = r + sat+1 .
Therefore, i’s expected lifetime payo¤ at time t is
t
Zxi
lt
βα( xit + x tj )dF ( x tj j at ) +
Zh
xit
t
β (1
α)( xit + x tj )dF ( x tj j at )
xit
+δEt (Ui ( at+1 ))
where Et (Ui ( at+1 )) =
Vladimir Petkov (VUW)
Zh
t
[r + s(µ + η ( xit + x tj )]dF ( x tj j at ).
lt
Continuous Mixed Strategy Equilibria
October 2013
14 / 25
Analysis of the Dynamic Game (Continued)
Again, i must be indi¤erent between any xit in [l t , ht ].
Therefore, F ( j at ) must be such that the derivative of the expression
for the expected lifetime payo¤ is 0 for any xit .
Di¤erentiation w.r.t. xit yields the following di¤erential equation:
F ( x t j at ) + 2xit F 0 ( x t j at )
R̃ = 0,
where R̃ is now given by
R̃ =
1
β(1 α) δsη
.
β(2α 1)
Using similar arguments to those in the static game, it can be shown
2
that the period-t strategy support is [ at , at R̃R̃ 1 ].
As before, solving the di¤erential equation yields
p !
at
F ( x t j at ) = R̃ 1 p
.
xt
Vladimir Petkov (VUW)
Continuous Mixed Strategy Equilibria
October 2013
15 / 25
Analysis of the Dynamic Game (Continued)
Now we pin down the coe¢ cients r, s of the continuation payo¤.
We can de…ne Ui ( a) recursively.
Let ht = at
R̃
R̃ 1
r + sat =
Zxi
2
. For any xit in [ at , ht ], we have
t
at
βα( xit + x tj )dF ( x tj j at ) +
Zh
Zh
t
β (1
xit
α)( xit + x tj )dF ( x tj j at )
t
xit + δ [r + s(µ + η ( xit + x tj )]dF ( x tj j at ).
at
Setting xit = at gives us
r + sat = β(1
α) at + at R̃/( R̃
1)
+δ r + s µ + η at + at R̃/( R̃
Vladimir Petkov (VUW)
Continuous Mixed Strategy Equilibria
at
1)
.
October 2013
16 / 25
Analysis of the Dynamic Game (Continued)
The method of undetermined coe¢ cients yields equations for r and s.
The parameter s must satisfy
s = β (1
α) 1 +
R̃
R̃
1 + δsη 1 +
1
R̃
R̃
.
1
De…ne A = 1 β(1 α), B = β(2α 1), D = β(1 α).
Also, let Z be the discriminant of the quadratic equation for s:
Z = [A
4δη (2δη
B + δη (2D
2A + B
1)]2
1)[ A
D (2A
B)].
B
There are 2 solutions for s, but we focus on this one:
s=
Vladimir Petkov (VUW)
[A
B + δη (2D 2A + B
2δη ( 1 + 2δη )
Continuous Mixed Strategy Equilibria
1)] +
p
Z
.
October 2013
17 / 25
Analysis of the Dynamic Game (Continued)
Also, the coe¢ cient r must solve r = δr + sµ. Therefore,
r=
δsµ
.
1 δ
Again, this is a zero-sum game when β = 1, so we must get
Ui ( at ) = 0.
With the above expression for s, we do obtain r = s = 0.
The other solution of the equation for s does not satisfy this property.
A necessary condition for the existence a mixed strategy equilibrium is
Z > 0.
This condition is satis…ed if δ and β are not too big.
Otherwise, players will want to exert in…nite e¤ort.
Vladimir Petkov (VUW)
Continuous Mixed Strategy Equilibria
October 2013
18 / 25
The Markovian Mixed Strategy Equilibrium
To recap, the Markovian symmetric mixed strategy is characterized by
p !
1 β(1 α) δsη
at
t t
F(x ja ) =
1 p
,
β(2α 1)
xt
The support of the strategy is [ at , at
1 β(1 α) δsη
1 β(1 α) δsη β(2α 1)
2
].
Each player’s expected e¤ort is
x̄ t = at
1
1
β (1
β (1 α )
α) δsη
δsη
β(2α
.
1)
Given at , the expected value of the game is
t
U (a ) =
[A
B + δη (2D 2A + B
2δη ( 1 + 2δη )
1)] +
p
Z
δµ
1
δ
+ at .
The variables r, s, A, B, D, Z are as de…ned above.
Vladimir Petkov (VUW)
Continuous Mixed Strategy Equilibria
October 2013
19 / 25
Long-Run Forecasting
Suppose that we want to make a forecast about the e¤ort levels and
the value of the game many periods from now.
Such a forecast should be independent of the current state, and so
can be used as a criterion to compare di¤erent games.
The period-t expected value of the state in period t + τ, at+τ , is
Et a
t+τ
=
2η R̃
R̃ 1
Suppose that 2η R̃/( R̃
τ 1
t
a +
∑µ
ζ =0
2η R̃
R̃ 1
ζ
.
1) < 1. Then we have
lim
τ !∞
2η R̃
R̃ 1
Thus, for τ ! ∞, we get
Et at+τ = ā
Vladimir Petkov (VUW)
τ
τ
at = 0.
µ( R̃ 1)
.
R̃(1 2η ) 1
Continuous Mixed Strategy Equilibria
October 2013
20 / 25
Long-Run Forecasting (Continued)
In the long run, a player’s e¤ort will be somewhere in the interval
µ
1
µ( R̃ 1)2
2η ( R̃ 1)2 2η R̃2
,
The long-run forecast for each player’s e¤ort is
µ( R̃ 1)
R̃(1 2η ) 1
1
1
β (1
β (1 α )
α) δsη
δsη
β(2α
1)
.
The long-term forecast for the value of this game to a player is
p
Z [ A B + δη (2D 2A + B 1)]
δµ
µ( R̃ 1)
+
2δη ( 1 + 2δη )
1 δ R̃(1 2η ) 1
.
Note that when δ ! 0, our long-term forecasts converge to the
predictions for a static game in which a = ā.
Vladimir Petkov (VUW)
Continuous Mixed Strategy Equilibria
October 2013
21 / 25
Numerical Example: Dynamic Incentives
We set parameter values η = 0.2, µ = 5, δ = 0.9, β = 1.1.
The long-term forecast for the value of the game is increasing in α.
If β < 1, the game value is negative and decreasing in α.
Vladimir Petkov (VUW)
Continuous Mixed Strategy Equilibria
October 2013
22 / 25
Extension: The Winner Determines the State
Suppose that the winner of the stage game determines the state:
at+1 = µ + η maxf x1t , x2t g.
Conjecture that Uit+1 = r + sat+1 . Mr. i’s expected lifetime payo¤ is
t
Zxi
lt
βα( xit + x tj )dF ( x tj j at ) +
Zh
t
α)( xit + x tj )dF ( x tj j at )
β (1
xit
xit
Z
+ δ [r + s ( µ
lt
+ ηxit )]dF ( x tj j at ) + δ
Zh
xit
t
[r + s(µ + ηx tj )]dF ( x tj j at ).
Di¤erentiation yields F ( x t j at ) + Gx t F 0 ( x t j at )
G=
Vladimir Petkov (VUW)
2β(2α 1)
,
β(2α 1) + δsη
R̂ =
xit
R̂ = 0, where
1 β (1 α )
.
β(2α 1) + δsη
Continuous Mixed Strategy Equilibria
October 2013
23 / 25
Extension (Continued)
The solution to this di¤erential equation is
t
t
F ( x j a ) = R̂ 1
Now the strategy support is [ at , at
1
G
at
xt
!
.
G
R
].
R 1
The expected e¤ort is
x̄it =
R̂at
( G 1)
R̂G 1
( R̂ 1)G
1
1 .
Next we determine the coe¢ cient’s r, s of the continuation payo¤.
Again, we can apply the method of undetermined coe¢ cients.
Vladimir Petkov (VUW)
Continuous Mixed Strategy Equilibria
October 2013
24 / 25
Extension (Continued)
Setting x t = at allows us to write the following recursive equation:
r + sat =
R̂G 1
( R̂ 1)G
β(1 α) R̂at
( G 1)
1
1
a t [1
R̂at
( G 1)
R̂G 1
( R̂ 1)G
1 [ β (1
α) + δsη ]
+δ r + s µ + η
1
β (1
α)]
1
.
Thus, s must satisfy
s=
β (1
(G
α) R̂
1)
R̂G 1
( R̂ 1)G
1
[1
β (1
α)].
The equation for s can be solved numerically.
Note that s appears in R̂ and G.
Also, r is given by
r=
Vladimir Petkov (VUW)
δsµ
.
1 δ
Continuous Mixed Strategy Equilibria
October 2013
25 / 25