Dynamic Games
A dynamic game of complete information is:
• A set of players, i = 1, 2, ..., N
• A payoff function for each player that describes his payoff as a function of
the decisions of all the players
• A description of the timing of the game: Which player is allowed to move,
given the previous decisions of all the players?
• A set of feasible actions for each player, given the previous decisions of all
the players.
Extensive Forms
The extensive form has
• Decision points: The dots, or nodes, represent a decision that must be made
by some player. The name of the player is usually placed above or next to the
dot
• Actions: The lines, or branches, represent different actions that a player can
take at each decision point. The label is the name of the action, like “enter”
or “exit”, etc.
• Payoffs: The numbers at the bottom of the tree represent the payoffs to the
players. The convention is that the payoffs are given in order of the timing
at which that player moved first. So if Player 1 moves first, Player 2 moves
second, Player 3 moves third, and so on, the payoffs are given in the order:
Player 1, Player 2, Player 3...
Behavior Strategies
Definition 1. A behavior strategy for player i is a rule that gives that player’s
strategy at all of his decision points in the extensive form.
1
For example, Player 1 has two behavior strategies: L or R. Player 2, however,
has many behavior strategies that take the form, “If Player 1 just chose L, then
x, if Player 1 just chose R, then y.”
How should you think about behavior
strategies? Imagine that each player is writing down instructions for a lawyer, who
will play the strategies for them. The instructions must be sufficiently complete that
the lawyer knows what to do in every situation, and doesn’t run out of instructions.
(This lawyer analogy will be helpful later, too)
We often write behavior strategies as a1 a2 ...aK , where ak is the k-th decision
made by the player. For example, Player 2’s behavior strategies in the example
are ac, ad, bc, bd. You can imagine, though, how this becomes another one of
those problems with game theory where defining the notation correctly takes half
a page... defining the equilibrium concept in terms of the correct notation takes a
page... solving for the equilibrium in terms of the correct notation takes two pages...
There is another concept — “pure strategies” — where, instead, we write out
every possible “history” of previous moves, and associate with each history an action.
This involves a lot of notation and busywork, so we focus on the more intuitive idea
of a behavior strategy, where we think about the incentives facing players at each
decision point.
Throwing away the timing
Suppose we just ignored the timing. What happens? We use the set of
behavior strategies as the set of pure strategies for each player, and get a strategy
form:
L
R
ac
1,0
0,1
ad
1,0
-1,0
bc
2,3
0,1
bd
2,3
-1,0
So we can convert any dynamic game into a static game. This one has two Nash
equilibria: (L, bc) and (L, bd). But is (L, bd) a reasonable prediction? How did
we get this strategic form? First, we figured out all the behavior strategies for
each player. Now, we imagine these behavior strategies are the pure strategies of
a strategic-form game. Imagine that the two players submit their instructions to
two lawyers, who then play the received strategies for the players. Since the players
are simultaneously submitting their instructions to the lawyers, we can ignore the
timing. Now, the lawyers open the envelopes and follow the instructions. So if the
first lawyer receives L and the second lawyer receives bd, the first lawyer plays L
and the second plays B, giving the players payoffs of (2, 3).
Every James Bond movie
There is a Evil Rational Scientist, who is a perfect sociopath (maximizes his
own payoff with complete disregard to the welfare of others). He threatens the
United Nations by claiming he will detonate a nuclear weapon, destroying the
earth, including himself, unless he gets a billion dollars.
2
(The Entry Game)
A related, less silly game goes like this: Kmart is deciding on entering a
market where Walmart is the incumbent. Kmart can “enter” or “stay out”.
Walmart, upon entry, can “fight” the entrant by charging very low prices and
hurt them both, or “accomodate” the entrant and split the market. (Here, the
Evil Rational Scientist is Walmart)
Every James Bond movie
Let’s look at the equilibria of the static game:
P
DP
NN
-∞, −∞
-∞, −∞
ND
-∞, −∞
0,0
DN
-1,1
-∞, −∞
DD
-1,1
0,0
So we’ve got three pure-strategy Nash equilibria: (P, DN ), (P D, N D), and
(DP, DD). But are these all reasonable, given that the Evil Rational Scientist
is a perfect sociopath?
Subgames
Definition 2. A subgame is a set of decision points, actions, and payoffs, which
all follow from a single decision point to the end of the game.
To construct a subgame: Pick a single decision point, and circle all the
subsequent things that can happen, from that point to the end of the game.
3
Subgame Perfect Nash Equilibrium
Definition 3. A Nash equilibrium s∗ = (s∗1 , s∗2 , ..., s∗N ) is subgame perfect if it
is a Nash equilibrium in every subgame.
What we’re saying is that, “If you consider every subgame as a game in itself,
play must be Nash — i.e., no player has a profitable deviation from his proposed
strategy.” This means that while a dynamic game might have many Nash equilibria,
we are only interested in those where each player uses a strategy that does not
involve non-credible threats, like the ERS claiming he’ll blow up the world if he
isn’t paid — this simply isn’t a reasonable expectation of how the ERS will play,
as a perfect sociopath, since blowing up the world will kill him as well.
Backwards Induction
How do we solve for subgame perfect Nash equilibrium (SPNE)?
Definition 4. Backwards Induction is the process of “pruning the game tree”
described by the following algorithm:
• Step 1: Start at each of the final subgames in the game. Solve for the
players’ equilibrium behavior. Remove that subgame and replace it with
the payoffs that arise from players following their optimal course of action.
• Step 2: Repeat step 1 until you arrive at the first node in the extensive
form.
Theorem 5. The set of strategies constructed by backwards induction is a subgame perfect Nash equilibrium.
Example 1
Solve for all Nash and subgame perfect Nash equilibria. Are there are any Nash
equilibria that aren’t subgame perfect?
4
Example 2
Solve for all Nash and subgame perfect Nash equilibria. Are there are any Nash
equilibria that aren’t subgame perfect?
The ERS Game
Solve for all Nash and subgame perfect Nash equilibria. Are there are any Nash
equilibria that aren’t subgame perfect?
The Entry Game
Solve for all Nash and subgame perfect Nash equilibria. Are there are any Nash
equilibria that aren’t subgame perfect?
5
The Stackelberg Game
There are two firms, a Leader, L, and a Follower, F . The price in the
market is determined by the quantity chosen by the leader qf and the follower
ql as p(qf , ql ) = A − ql − qf ; the firms have the same total costs, C(q) = cq.
The leader chooses his quantity first, which the follower observes, and then the
follower chooses his quantity. What is the subgame perfect Nash equilibrium of
the game? Is consumer welfare greater or less than it would be in a standard
Cournot game? Compare the players’ strategies to the monopoly and perfectly
competitive outcomes.
Resale Price Maintenance
There is a manufacturer M and a retailer R. The manufacturer sells goods
to the retailer, who then sells goods to customers (think Best Buy selling Sony
t.v.’s at retail stores [but not franchises like JCrew that handle of the retailing
through franchised stores]). The price in the market is p(q) = A − q, and the
manufacturer sells units of the good to the retailer at a price h per unit. The
manufacturer’s total costs are C(q) = cq. What is the subgame perfect Nash
equilibrium of the game? Is the price to consumers higher or lower than it would
be if the two firms “vertically integrated” into a single monopoly?
Strategic Voting
There is an unpopular bill coming up for a vote which some politicians prefer
pass (like the budget bills associated with super-committee or the fiscal cliff),
but prefer not to vote for (they want to appear tough and uncompromising):
• If all the politicians vote yes, the bill passes and they get a payoff of 0
each.
• If all the politicians vote no, the bill fails and they get a payoff of −1 each.
• If two politicians vote yes and the third votes no, they get a payoff of 0
each, and the third gets a payoff of 1
• If two politicians vote no and the third votes yes, they get a payoff of −1
each, and the third gets a payoff of −2
Draw out an extensive form and solve for a subgame perfect Nash equilibrium
where politician 1 votes first, politician 2 votes second, and politician 3 votes
third; what is the probability the bill fails?
To solve for a symmetric mixed Nash strategy, we let p = pr[Vote yes] be the
probability a given politician votes yes. Then we consider the values of p that make
one of the other politicians indifferent between voting yes and no with probability
one. Or,
E[u(y, p)] = E[y(n, p)]
p2 (0) + 2p(1 − p)(0) + (1 − p)2 (−2) = p2 (1) + 2p(1 − p)(−1) + (1 − p)2 (−1)
{z
} |
{z
} | {z } |
{z
} |
{z
}
| {z } |
3 y, 0 n
2 y, 1 n
1 y, 2 n
2 y, 1 n
6
1 y, 2 n
0 y, 3 n
The left-hand side is, again, the expected payoff to voting yes for sure if my opponents vote yes with probability p, and the right-hand side is the expected payoff to
voting no for sure if my opponents vote yes with probability p. We’re looking for a
p that makes each politician indifferent between voting yes and no in expectation.
This reduces to
1
p = p2 +
4
Which is a quadratic equation, solvable by using the quadratic formula. Or, you
1
can just guess p∗ = , and check:
2
1
11 1
1
=
+ =
2
22 4
2
So each politician votes yes with probability .5. Then the probability that the bill
fails is
2
2 1
1
1
1
pr[0 y, 3 n] + pr[1 y, 2 n] =
=
+3
2
2
2
2
So the bill fails half the time in the super-committee.
The extensive form and SPNE are fairly straightforward to solve for.
7