Extensive-Form Games with Imperfect Information
Jeffrey Ely
May 6, 2015
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Example
r
2, 2
A b
Player 1 H
H
HH
r
3, 3
Player
1 r
pH
C p HH
p
p
D HHHr
H
p
H
B
0, 0
p
H
p
H
r
PlayerH
2H
1 pp
H
p
r 0, 0
HH
p
H
p
C
Down HH p
p PlayerHH
1 pr
HH
H
D HHHr
3, 3
Up
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Extensive-Form Games With Imperfect Information
Finite
No simultaneous moves: each node belongs to a single player
(possibly “chance.”)
Information Sets
I
I
The nodes h at which i moves are partitioned into sets H.
The interpretation is that player i knows that it is his move and some
node in the set has been reached, he does not know which one.
(Behavioral) Strategies
Subgames
I
I
I
Always begin with singleton information sets
Can’t break information sets
Nash and Subgame Perfect Nash Equilibria
Jeffrey Ely
Extensive-Form Games with Imperfect Information
The Weakness of SPNE
1b
HH
H
Stay out Enter, Green
r
(0, 0)
HH Enter, Red
H
HH
H
H
pH
p r
rp p p p p p p p p p 2p p p p p p p p H
A
A
Fight A Accom.
Fight A Accom.
A
A
A
A
A
A
Ar
r
Ar
r
(−1, −2)
(1, −1)
(−1, −2)
(1, −1)
This example has no proper subgames so SPNE is just NE.
So we need to extend the concept of “sequential rationality”
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Beliefs
Does the player’s continuation strategy maximize his continuation
payoff?
We need a way to pose the same question at a non-singleton
information set.
But what is the continuation payoff at a non-singleton information
set?
We need to specify the player’s beliefs over the nodes in the
information set.
Definition
A system of beliefs is a collection µ = (µ1 , µ2 , . . . , µN ) where
µi (·|H ) ∈ ∆H specifies a probability distribution over the nodes in any
information set H belonging to player i. We interpret µ(·|H ) ∈ ∆H as i’s
belief about which node in H has been reached.
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Continuation Payoffs
Given a behavioral strategy profile σ
At any node h belonging to i we can compute the probability
distribution over terminal nodes.
And thus we can compute the expected payoff to i conditional on
having arrived at h,
ui (σ |h )
Now to compute the continuation payoff at a non-singleton
information set H, we take expectations with respect to the belief
µ(·|H ):
ui (σ|H ) = ∑ µ(h |H )ui (σ|h )
h ∈H
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Sequential Rationality
Definition
A behavioral strategy profile σ is sequentially rational relative to a system
of beliefs µ if for each i and for each information set H belonging to i,
ui (σ|H ) ≥ ui (σi0 , σ−i |H )
for every alternative strategy σi0 .
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Assessments
In extensive form games with imperfect information a solution is described
by two objects
The strategy profile
The system of beliefs
A pair (σ, µ) consisting of the strategy profile σ and the system of beliefs
µ is called an assessment.
Solution concepts will impose requirements on the assessment:
Sequential rationality
Some condition on the beliefs.
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Sequential Rationality
1b
HH
HH
Stay out H Enter, Red
HH
Enter, Green
H
HH
2
r
rp p p p p p p p p p p p p p p p p p H
pH
p r
A
A
(0, 0)
A
A
Fight
Fight
A Accom.
A Accom.
A
A
A
A
r
Ar
r
Ar
(−1, −2)
(1, −1)
(−1, −2)
(1, −1)
Sequential rationality alone restores the intuitive solution of this game.
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Beliefs
r
1, −1
r
Player
1 pH
H p HH
p
H p
T HHHr
p
−1, 1
p
p
Player 2 b
p
1
H
p
HH
r −1, 1
p
p
H
p
HH
H
T
H pp H
r
PlayerH
1 pH
HH
T HHHr
1, −1
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Sequential Rationality Alone Is Not Enough
Sequential rationality alone is not enough for this game. Consider the
assessment in which both players play T but 2’s belief assigns probability
greater than 1/2 to the top node. It is sequentially rational. But it isn’t
even a Nash equilibrium. (This is Matching Pennies)
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Bayes’ Rule on the Path
Given a behavioral strategy profile, we say that a system of beliefs satisfies
Bayes’ rule on the path if at every information set H which is arrived at
with positive probability, the belief µi (·|H ) is the conditional probability
distribution.
Jeffrey Ely
Extensive-Form Games with Imperfect Information
(Weak) Perfect Bayesian Equilibrium
Definition
An assessment (σ, µ) is a (weak) Perfect Bayesian Equilibrium if it is
sequentially rational and satisfies Bayes’ rule on the path.
So in an extensive form game with imperfect information, to “solve” for a
Weak PBE is to
State the strategy profile σ.
State the system of beliefs µ.
Verify that the system of beliefs is derived from σ on the path.
Verify that the strategy profile is sequentially rational relative to µ.
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Problem with Weak PBE
r
−1/2, 0, 0
A b
Player 1 H
H
HH
r
−1, 1, −1
r
Player
3 pH
H p HH
p
H
p
T HHHr
H
p
H
B
1, −1, 1
p
H
p
H
r
PlayerH
2H
1 pp
H
p
r 1, −1, 1
HH
p
H
p
H
T HH p
p PlayerHH
3 pr
HH
H
T HHHr
−1, 1, −1
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Problems with Weak PBE
Consider the following assessment:
1 Plays A
2 Plays T
3 plays T
3’s belief assigns probability greater than 1/2 to his top node.
This is a weak PBE
Sequentially rational
Satisfies Bayes’ rule on the path.
But it is not a subgame perfect Nash equilibrium. (The subgame is
matching pennies.) The unique SPE has 1 playing B.
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Related Problem
r
2, 2
A b
Player 1 H
H
HH
r
3, 0
Player
1 r
pH
C p HH
p
p
D HHHr
H
p
H
B
0, 3
p
H
p
H
r
PlayerH
2H
1 pp
H
p
r 0, 3
HH
p
H
p
C
Down HH p
p PlayerHH
1 pr
HH
H
D HHHr
3, 0
Up
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Failure of OSDP for Weak PBE
Consider the following assessment.
1 Plays (A, C )
2 Plays Down
1 assigns probability 1 to the top node in his information set.
This is not sequentially rational for player 1
he can improve his payoff at the initial node by changing his strategy
to B, D.
But there is no profitable one stage deviation at any information set.
Jeffrey Ely
Extensive-Form Games with Imperfect Information
A Partial Fix
Bayes’ rule where possible. i.e. suppose
All of the nodes in an information set have a common immediate
predecessor
And that predecessor is a singleton information set
Then the mixed action at that predecessor should define the belief at
the information set.
There are other examples where it is possible to apply Bayes’ rule even at
off-path information sets. Applying the “where possible” principle is a bit
of a case-by-case matter.
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Application: Take it or leave it offers
Single object for sale.
Player 1 is the seller, Player 2 is a buyer.
The buyer has a value v > 0 for the object.
The seller has a prior belief µ about v .
The seller demands a price p for the good.
The buyer accepts or rejects.
Payoffs from sale:
I v − p for the buyer
I
p for the seller
Payoffs are zero if no sale.
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Extensive Form
Game begins with a move by Nature. (By the way this is how we put
incomplete information into an extensive-form game.)
Nature selects v with an exogenous probability distribution given by
the prior µ.
Player 1 does not observe Nature’s move (A single information set
includes them all.)
Player 2 observes the offer made as well as Nature’s move (singleton
information set.)
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Sequential Rationality For the Buyer
Because all of the buyer’s moves are at singleton information sets,
sequential rationality pins down his continuation strategy, except when
indifferent.
The buyer accepts any offer strictly below his value.
The buyer rejects any offer strictly above his value.
Sequential rationality alone does not restrict his response to prices
equaling his value.
(Think of the complete information ultimatum game.)
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Binary Values
Suppose v can take on two possible values {vl , vh } with vh > vl .
With probabilities µh and µl .
Bayes’ rule now pins down the seller’s belief at his information set.
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Sequential Rationality for the Seller
We can immediately rule out certain price offers
No price offer less than vl can be sequentially rational.
No price offer between vl and vh can be sequentially rational.
No price offer greater than vh can be sequentially rational.
So the only possible sequentially rational price offers are p = vl and
p = vh .
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Marginal Revenue
Think back to textbook undergrad micro. A monopoly who lowers price
faces a tradeoff
Lower sales price
More sales
Our seller faces the same tradeoff, but “more sales” means higher
probability of sale.
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Marginal Revenue
At price vh , a sale occurs with probability µh < 1.
At price vl :
I
Sale occurs with probability 1. The marginal sale earns revenue
µ l · vl
I
There is a loss of revenue on inframarginal sales equal to
µ h ( vh − vl )
Marginal revenue is the net:
µl · vl − µh (vh − vl )
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Marginal Revenue
The seller will cut the price to vl if and only if marginal revenue is positive:
vl −
µh
(vh − vl ) > 0
µl
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Perfect Bayesian Equilibrium
The seller offers vl if marginal revenue is positive, vh if negative.
(Either if zero.)
The buyer accepts the seller’s offer and any offer strictly greater than
his value.
The seller’s beliefs are given by Bayes’ rule
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Marginal Revenue Is Cleaner In The Continuum Case
Suppose now that v is distributed according to a continuous CDF given by
F with density f .
f (v )v − [1 − F (v )]
is the marginal revenue from reducing price below v (marginally). The
seller wants to sell to type v if and only if
v−
1 − F (v )
> 0.
f (v )
and so the optimal take-it-or-leave-it offer is the v at which marginal
revenue is zero.
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Kuhn’s Theorem
There is a sense in which we can interchangeably use mixed and behavioral
strategies. For example, in a game of perfect information:
Fix any mixed strategy.
There is a behavioral strategy which, regardless of the opponent’s
strategy, induces the same distribution over terminal nodes.
Likewise for any behavioral strategy there is an equivalent mixed
strategy.
What about games with imperfect information?
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Imperfect Recall
Wear
Wash
Wear
-1
Wash
0
1
Consider the laundry game.
Consider the behavioral strategy which randomizes with equal
probability.
This is the optimal strategy.
It has no equivalent mixed strategy.
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Kuhn’s Theorem
Theorem
In a finite extensive-form game with perfect recall, for every mixed
strategy there is at least one behavioral strategy which induces the same
distribution over terminal nodes against all strategy profiles of the
opponents. Likewise, for any behavioral strategy there is at least one
mixed strategy which induces the same distribution over terminal nodes
against any strategy profile of the opponents.
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Consistency
If the behavioral strategy profile is fully mixed then all information
sets are on the path.
Beliefs are defined everywhere by Bayes’ rule.
Thus, for any strategy profile, there are “nearby” strategy profiles
which would give uniquely defined beliefs.
Definition
An assessment (σ, µ) is consistent if there is a sequence of fully mixed
behavioral strategy profiles which converges to σ such that the
corresponding sequence of belief systems (derived using Bayes’ rule)
converges to µ.
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Sequential Equilibrium
Definition
An assessment (σ, µ) is a sequential equilibrium if it is sequentially
rational and consistent.
Every finite game has a sequential equilibrium.
A sequential equilbirium is subgame-perfect.
A sequential equilibrium has beliefs defined by Bayes’ rule wherever
possible.
When beliefs are consistent, the one-stage deviation principle holds.
Additional restrictions are implied.
Jeffrey Ely
Extensive-Form Games with Imperfect Information
Forward Induction
1
Out
In
2
2, 0
R
L
1
l
1, 3
r
0, 0
Jeffrey Ely
l
0, 0
r
3, 1
Extensive-Form Games with Imperfect Information