Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
Microeconomics
2. Game Theory
Alex Gershkov
http://www.econ2.uni-bonn.de/gershkov/gershkov.htm
18. November 2008
1 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
Dynamic games
Time permitting we will cover
2.a Describing a game in extensive form (efg)
2.b Imperfect information
2.c Mixed and behavioural strategies; Kuhn’s Theorem
2.d Nash equilibrium
2.e Credible strategies and Subgame Perfection
2.f Backward induction
2.g Applications
2 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.a Describing a game in Extensive form
We want to capture dynamic aspects of a game where timing is
important. Consider the following game
1. 2 parties {P1, P2} are trying to share two indivisible units of
a good yielding positive utility (say one util each)
2. suppose that P1 makes a take-it-or-leave-it offer to P2
3. P2—after having observed this offer—decides whether to
accept or reject this offer
4. if no agreement is reached (ie. if P2 rejects the offer), both
P1 and P2 get nothing.
A game’s Extensive form (efg) is a description of the sequential
structure of dynamic games such as above.
3 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.a Describing a game in Extensive form
Features:
A1 moves occur in sequence,
A2 all previous moves are observed before a move is chosen, and
A3 payoffs and structure of the game are common knowledge.
Definition
Games satisfying (A1–A3) are called finite efg of
perfect information.
4 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.a Describing a game in Extensive form
The efg of a perfect information game consists of
E1. the set of players N
E2. the set of sequences (finite or infinite) H, that satisfies the
following three properties:
0.1 ∅ ∈ H
k L
0.2 if {ak }K
k=1 ∈ H and K > L, then {a }k=1 ∈ H for every L
0.3 if an infinite sequence {ak }∞
satisfies
{ak }Lk=1 ∈ H for all
k=1
finite L, then {ak }∞
∈
H.
k=1
5 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.a Describing a game in Extensive form
Each member of H is a history; each component of a history ak is
an action taken by a player. A history {al }kl=1 is terminal if it is
+1
infinite or if there is no aK +1 such that {al }K
l=1 ∈ H. The set of
terminal histories is denoted Z .
E3. a player fn P : H − Z → N where P(h) is the label of the
player who is supposed to choose an action after history h ∈ H
E4. for each player i ∈ N, a payoff fn ui : Z → R.
6 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.a Describing a game in Extensive form
Definition
A game is called finite if the number of stages is finite and the
number of feasible actions at any stage is finite.
We denote, for every h ∈ H − Z such that P(h) = i , the set of
actions available to i by Ai (h) = {si |(h, si ) ∈ H}.
Definition
A perfect information efg Γ consists of Γ = {N, H, P, u}.
7 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.a Describing a game in Extensive form
Typically, an efg of perfect info can be described by the ”game
tree”
The game tree T consisting of the set of nodes (including decision
and terminal nodes) and the branches which are directed
connections between nodes; T must satisfy the ‘tree conditions’
◮ there is one node without incoming branches called the ‘initial
node’ (the open circle)
◮ for any given node, there is a unique path connecting it to the
initial node.
◮ any non-terminal node corresponds to the player that chooses
the action there.
◮ Each branch from a node corresponds to an action a available
to the player at this info set.
◮ Payoffs are given for all i ∈ N at the terminal nodes
The tree captures the temporal structure of how events unfold over
time.
8 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.a Describing a game in Extensive form
The key definition of the efg is the strategy. It is a complete,
contingent plan of action specifying a choice for the concerned
player at every possible history.
Definition
A strategy for player i ∈ N is a sequence si = {si (h)}h∈Hi , where
Hi = {h|h ∈ H − Z and P(h) = i } and si (h) ∈ Ai (h).
9 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.g Backward induction
Finite games satisfying (A1–A3) can be solved by
Backward Induction. Roughly speaking this is
◮
replacing each choice set which only leads to terminal nodes
in Z by the corresponding NE’qm outcome
◮
notice that this creates a new set of terminal nodes Z ′ in the
‘shortened’ game
◮
now again replace all choice sets which only lead to terminal
nodes in Z ′ by the corresponding NE’qm outcome
◮
repeat the above until the game is reduced to the initial node
and a set of choices leading to terminal nodes only; the
NE’qm outcome of this game is the backward induction
outcome of the original game.
10 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.g Backward induction
Theorem
Any perfect information game with a finite number of strategies
and players can be solved backwards and therefore has a pure
strategy eq’m.
11 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.b Imperfect information
To introduce information imperfections we require the following
additions to the perfect information requirements (E1–E4)
E3.’ the opportunity for chance moves by the additional player
Nature (N)
E3.” a players fn P : H − Z → 2N assigns to each nonterminal
history a set of players
E4.’ we assume that the ui : Z → R satisfy the axioms of vN-M
expected utility theory
E4.” for every h ∈ H − Z such that N∈ P(h), we assume that
there exists a probability measure fN (·|h) defined on AN (h),
where each such probability measure is independent of every
other such measure
12 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.b Imperfect information
E5. for every i ∈ N, Di is a partition of {h ∈ H|i ∈ P(h)} with
the property that Ai (h) = Ai (h′ ) whenever h and h′ are in the
same element di ∈ Di
Definition
A set di ∈ Di is called information set of player i .
Interpretation: any given member of di is indistinguishable to
player i .
Definition
A game is of perfect recall if no player ever forgets any information
he once knew and all players know the actions they have chosen
previously.
In perfect information games of perfect recall all information sets
are singletons.
13 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.b Imperfect information
Definition
An imperfect information efg Γ consists of Γ = {N, H, P, f , D, u}.
Definition
The probability measures fN (·|h) over Nature’s moves AN (h) at
h ∈ H are called prior probabilities over A(h).
14 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.c Mixed & behavior strategies
Convexity considerations lead us to allow for mixed actions.
Definition
Player i ’s mixed strategy σi is a probability distribution over a set
of (pure) strategies.
Definition
A behavior strategy for player i , βi is an element of the Cartesian
product ×di ∈Di △ (A(di )).
So the difference is that
◮ a mixed strategy is a mixture over complete, contingent plans:
a pure strategy is selected randomly before play starts
◮ a behavioral strategy specifies a probability distribution over
actions at each di and the probability distribution at different
info sets are independent.
15 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.c Mixed & behavioural strategies
These two objects are different
◮
the mixed strategy selects one pure complete, contingent plan
randomly at the beginning of the game
◮
the behavioural strategy specifies a randomisation over the
available actions for each point of choice
but the difference only matters in games of imperfect recall.
Definition
Two strategies σi and σi′ are equivalent if they lead to the same
prob distribution over outcomes for all σ−i .
Theorem
(Kuhn 1953) In a game of perfect recall, mixed and behavior
strategies are equivalent.
16 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.d Nash equilibrium
Definition
We denote the terminal histories when each player i follows
si , i ∈ N by o(s) ∈ Z.
Definition
A NE’qm of an efg is a strategy profile σ ∗ such that for every
player i ∈ N and all σi ∈ ∆(Si )
∗
ui (o(σ ∗ )) ≥ ui (o(σi , σ−i
)).
We can solve an efg for NE’qa by transforming the game into a
reduced strategic form game (rsfg) and solving this game in the
usual way.
17 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.d Nash equilibrium
Consider the following efg
1
u
(2,1)
2
r1
(0,0)
A1 = {u, d}
◮
S1 = {u, d}
◮
A2 = {l1 , r1 , l2 , r2 }
◮
S2 = {(l1 , l2 ), (l1 , r2 ),
(r1 , l2 ), (r1 , r2 )}
d
2
l1
◮
l2
(-1,1)


 [u, (l1 , l2 )] , 
N=
[d, (l1 , r2 )] ,


[d, (r1 , r2 )]
r2
The rsfg for this game is
{N, S, u(o(s))}
(3,2)
u
d
l1 , l2
2,1
-1,1
l1 , r2
2,1
3,2
r1 , l2
0,0
-1,1
r1 , r2
0,0
3,2
18 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.d Nash equilibrium
Notice that
◮
there are several efg’s for the same sfg
◮
the set of NE’qa of the efg can be found by looking at the set
of NE’qa of the sfg
◮
there is a serious problem with NE’qa in efg: there may be
actions in a strategy which do not affect an (e’qm) outcome
which are inconsistent with what the associated player would
choose if moving at that node.
19 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.e Subgame Perfection
Definition: A proper subgame G of an efg Γ consists of a single
node and all its successors in Γ, with the following properties:
1. if node x ′ ∈ G and for some i ∈ N x ′′ ∈ di (x ′ ), then x ′′ ∈ G .
That is x ′ and x ′′ are in the same information set in the
subgame if and only if they are in the same information set in
the original game
2. the payoff function of the subgame is just the restriction of the
original payoff function to the terminal nodes of the subgame
20 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.e Subgame Perfection
Less formally, part of an efg is called a subgame if
◮
it starts from a singleton information set d
◮
it contains all successors to d (until the end of the game)
◮
no node outside the set of successors to d is contained in any
of the subgame’s information sets.
Notice that
◮
by definition, the entire game is a subgame of itself
◮
after different histories follow different subgames
◮
players know that they are in the same subgame.
21 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.e Subgame Perfection
Definition (Selten): A NE’qm s ∗ of an efg Γ is called
subgame perfect e’qm (SGPE’qm) iff it induces a NE’qm for every
(proper) subgame Γ(h) for every h ∈ H − Z.
22 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.e Subgame Perfection
Consider the games such that
1. in each stage k, every player knows all the actions, including
those by Nature, that were taken at any previous stage
2. each player moves at most once within a given stage
3. no information set contained in stage k provides any
knowledge of play in that stage
these games are called multi-stage games with observed actions.
Notice that multi-stage games with observed actions can be both
finite and infinite horizon games.
23 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.e Subgame Perfection
Definition: A strategy profile s satisfies the one-stage-deviation
condition if no player i can gain by deviating from s in a single
stage and conforming to s thereafter.
Theorem(one-stage-deviation principle): In a finite multi-stage
game with observed actions, strategy profile s is subgame perfect if
and only if it satisfies the one-stage-deviation condition. More
precisely, profile s is subgame perfect iif there is no player i and
strategy b
si that agrees with si except at a single stage t and
history ht , and such that b
si is a better response to s−i than si
t
conditional on h being realized.
24 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.e Subgame Perfection
Definition: A game is continuous at infinity if for each player i the
utility function ui satisfies
sup
h,e
h s.t. ht =e
ht
|ui (h) − ui (e
h)| → 0 as t → ∞
Theorem(one-stage-deviation principle): In an infinite multi-stage
game with observed actions that is continuous at infinity, strategy
profile s is subgame perfect if and only if it satisfies the
one-stage-deviation condition.
25 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.e Subgame Perfection
Example: Bargaining model
◮
two players must agree on how to share a pie of size 1
◮
in periods 0,2,4,... P1 proposes sharing rule (x, 1 − x) and P2
can accept or reject. If P2 accepts, the game ends
◮
if P2 rejects in period 2k, then in period 2k + 1 P2 proposes
sharing rule (x, 1 − x) and P1 can accept or reject.
◮
if (x, 1 − x) is accepted at date t, the payoffs are
(δt x, δt (1 − x)), where δ < 1 is the players’ discount factor
26 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.e Subgame Perfection
The SGPE’qm of this game is:
◮
Pi always demands a share (1 − δ)/(1 − δ2 ) when it is his
turn to make an offer.
◮
He accepts any share greater or equal to δ(1 − δ)/(1 − δ2 ).
27 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.f The value of commitment
Tournament (Dixit 87)
◮
2 players choose how much effort to exert (e1 , e2 ) ≥ 0
◮
player i wins the tournament with probability
ei
pri (ei , ej ) = ei +e
where i ∈ {1, 2} and j = 3 − i if (e1 , ej ) 6= 0
j
and
1
2
otherwise
◮
the winner gets the prize of K > 0
◮
the costs of exerting effort e ≥ 0 is c1 (e) = e for player 1 and
c2 (e) = 2e for player 2
◮
agent i ’s expected utility is given by
Kpri (ei , ej ) − ci (ei )
28 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.f The value of commitment
Without commitment:
Nash equilibrium (e1∗ , e2∗ ) should satisfy
e1∗ = argmaxe1 Kpr1 (e1 , e2∗ ) − c1 (e1 )
e2∗ = argmaxe2 Kpr2 (e2 , e1∗ ) − c2 (e2 )
Which is given by (e1∗ , e2∗ ) = ( 29 K , 19 K )
while equilibrium utilities are ( 29 K , 19 K )
29 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.f The value of commitment
With commitment: Assume that player 1 is able to commit to
some effort level e1 .
P2 will choose effort
e2 = argmaxe2 Kpr2 (e1 , e2 ) − c2 (e2 )
q
The solution is given by e2 (e1 ) = K2 e1 − e1
Hence, P1 should commit to the effort level e1
e1 = argmaxe1 Kpr1 (e1 , e2 (e1 )) − c1 (e1 )
The solution is given by e1 = K2 .
Hence, the vector of the chosen actions is ( K2 , 0) and of utilities
( K2 , 0) ⇒ the ability to commit increases the utility.
30 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.f Repeated games
Repeated ”prisoner’s dilemma”. Current payoffs gi (a∗ ) are given by
cooperate
defect
cooperate
1,1
2,-1
defect
-1,2
0,0
◮
players’ discount factor is δ
◮
the utility of a sequence (a0 , ..., aT ) is
T
1−δ X t
δ gi (at )
1 − δT +1
t=0
31 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.f Repeated games
Stage game
◮ finite game (just for simplicity)
◮ N players
◮ simultaneous move (just for simplicity) game
◮ Ai action space for i
◮ stage-game payoff function gi : A → R with A = ×i Ai
Repeated game
◮ history h t = (a0 , ..., at−1 ) is the realized choice of actions
τ )
before t with aτ = (a1τ , ..., aN
◮ a mixed (behavior) strategy σi : H t −→ ∆(Ai ) where
H t = (A)t is the space of all possible period t histories
◮ payoff function, for instance
T
T
1−δ X t
1 X
t
δ
g
(a
)
or
gi (at )
i
1 − δT +1
T
t=0
t=0
32 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.f Repeated games
Equilibria of repeated ”prisoner’s dilemma”
◮
The game is played only once, (defect, defect) is the only
equilibrium.
◮
The game is repeated finite number of times. The unique
subgame perfect eq. is for both players to defect every period.
◮
The game is played infinitely often. The profile (defect, defect)
in all periods remains a subgame perfect equilibrium.
33 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.f Repeated games
For infinitely repeated game with δ > 1/2 the following strategy
profile is a subgame-perfect eq as well:
◮
cooperate in the first period and continue to cooperate as
long as no player has ever defected.
◮
if any player has ever defected, then defect for the rest of the
game
34 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.f Repeated games
Two repetition of the stage game:
U
M
D
L
0,0
4,3
0,6
M
3,4
0,0
0,0
R
6,0
0,0
5,5
If played once, there are three equilibria: (M, L), (U, M), and
mixed equilibrium ((3/7U, 4/7M), (3/7L, 4/7M)) with
corresponding payoffs (4, 3), (3, 4) and (12/7, 12/7).
35 / 36
Microeconomics
-
2.2 Extensive form games
Description Backward Induction Imperfect info Mixed and behav. str Nash Subgame Applications
2.f Repeated games
Assume that δ > 7/9 Consider the following strategy profile:
◮
P1: Play D in the first stage. If the first stage outcome is
(D, R), then play M in the second stage, otherwise, play
(3/7U, 4/7M)
◮
P2: Play R in the first stage. If the first stage outcome is
(D, R), then play L in the second stage, otherwise, play
(3/7L, 4/7M)
It constitutes a subgame-perfect equilibrium.
36 / 36