Game Theory and Economic Applications
John Hillas
Dmitriy Kvasov
Aaron Schiff
Department of Economics, The University of Auckland, Private
Bag 92019, Auckland, New Zealand
E-mail address: [email protected]
Department of Economics, The University of Adelaide, Australia
E-mail address: [email protected]
Contents
Preface
1
Chapter 1. A Brief Historical Introduction
3
Part 1.
7
Noncooperative Game Theory
Chapter 2. Introduction to Normal Form Games
1. Examples of normal form games
2. Some terminology
3. What’s a Strategy?
4. “Self-enforcing uncertainty”
5. The Monty Hall Game
Problems
9
9
12
13
13
14
14
Chapter 3. Solving normal form games
1. Dominated strategies
2. Nash equilibrium
3. Utility functions
Problems
17
17
17
20
21
Chapter 4. A more formal treatment of normal form games
1. Formal definition of a normal form game
2. Best reply correspondences
3. Games where players have more than two pure strategies
4. Formal definitions of Nash equilibrium and best reply correspondences
Problems
23
23
24
26
34
35
Chapter 5. Two Player Constant Sum Games
1. Definition
41
41
Chapter 6. Extensive Form Games
1. Examples of extensive form games
2. Definition of an extensive form game
3. Perfect recall
45
45
46
49
Chapter 7. Solving extensive form games
1. Equilibria of extensive form games
2. The associated normal form
3. Behaviour strategies
4. Kuhn’s theorem
5. More than two players
6. Subgame perfect equilibrium
Problems
51
51
51
54
55
56
57
60
Chapter 8. Sequential equilibrium
1. Definition of Sequential Equilibrium
65
66
i
ii
CONTENTS
2. Signalling Games
Problems
Part 2.
Cooperative Game Theory
71
72
75
Chapter 9. Coalitional games
77
Chapter 10. The core
79
Chapter 11. The Shapley value
81
Chapter 12. Matching problems
1. Assignment Problem
2. The Marriage Problem
85
85
86
Bibliography
89
Preface
These lecture notes are to accompany the course ECON 212, Game Theory and
Economic Applications, taught at the University of Auckland. This is a second year
undergraduate course that provides a basic introduction to game theory. The course
is divided into two major sections: noncooperative game theory and cooperative
game theory.
Students will be introduced to the basic concepts and taught to analyse simple
examples. Emphasis will be very much on basic understanding rather than formal
statements of results. Some formal definitions will be examined, but only after a
thorough examination of the concepts in the context of examples. The students
should achieve a good understanding of the basic issues, an ability to solve simple
games, and a very basic introduction to the more formal aspects of the course.
About these notes: These notes were written in the following manner. In
the beginning, there were no notes. In the first semester of 2002, the first year that
this course was taught, Aaron, a PhD student, attended John’s classes and wrote
down everything that John said. Aaron then typed up the notes, rearranged them
a bit, and tried to make them readable. Thanks are due to Tina Kao, another PhD
student and the tutor for this paper in 2002, for providing notes for the lectures
that Aaron missed and for helping Aaron when his own notes were illegible. Any
errors in these notes should be blamed equally on John and Aaron, but no blame
should be assigned to Tina.
Then, in 2006, Dmitriy started teaching this course and added some notes of
his own. He can be blamed for some errors in the notes.
Dmitriy left the University of Auckland at the end of 2009. Since that time
John has continued to make changes to the notes.
John Hillas
January 2014
1
CHAPTER 1
A Brief Historical Introduction
The history of game theory is often said to start with the publication of Theory
of Games and Economic Behaviour by John von Neumann and Oskar Morgenstern
in 1944. This was certainly a major development and drew together a number of
earlier developments and laid out a programme for further developments, as well as
introducing the name “game theory” for the theory. There is however a substantial
pre-history. You might like to consult Paul Walker’s A Chronology of Game Theory
which you can find at
http://www.econ.canterbury.ac.nz/personal_pages/paul_walker/gt/hist.htm.
It provides more detail than is included here.
A number of the earlier contributions concern what we would now term twoperson zero-sum games. As we shall see later in the course these games have a very
special structure that allows a very strong solution. In such games each player has
a randomised strategy that guarantees him the “value” of the game, whatever the
other player does. This result is known as the minimax theorem.
The first known solution to such a two-person zero-sum game is contained in
a letter dated 13 November 1713 from James Waldegrave to Pierre-Remond de
Montmort concerning a two player version of the card game le Her. De Montmort
wrote to the mathematician Nicolas Bernoulli (perhaps best known, at least among
economists, for his formulation of the St. Petersburg Paradox).
The great French mathematician and probabilist, Emile Borel published four
notes on strategic games between 1921 and 1927. Borel gave the first formal definition of what we shall call a mixed strategy and demonstrated the existence of the
minimax solution to two player zero-sum games with either three or five possible
strategies. He initially conjectured that games with more strategies would not have
such a solution but, being unable to find a counter example, he later considered
that to be an open question.
That question was answered by the Hungarian (and later American) mathematician John von Neumann in 1928. Wikipedia describes von Neumann as
having “made major contributions to a vast range of fields, including set theory, functional analysis, quantum mechanics, ergodic theory, continuous geometry, economics and game theory, computer science, numerical analysis, hydrodynamics (of explosions), and statistics, as well as many other mathematical fields.”
(http://en.wikipedia.org/wiki/John_von_Neumann).
The 1928 proof by von Neumann of the minimax theorem made a nontrivial
use of deep results in topology and functional calculus. He later, in 1937, provided
another proof using Brouwer’s fixed point theorem, which would later prove central
to both Nash’s proof of the existence of strategic equilibrium in general finite games
and to the existence of competitive equilibrium by Arrow, Debreu, and McKenzie
in the early 1950s. In 1938 Jean Ville, a student of Borel, gave a new proof of
the minimax theorem, based centrally on the idea of convexity and in particular
on the important matematical idea of the separation of convex sets. The proof
of this result in the book by von Neumann and Morgenstern is a refinement of
Ville’s proof, as von Neumann and Morgenstern explicitly acknowledge. The first
3
4
1. A BRIEF HISTORICAL INTRODUCTION
completely algebraic proof of this result, not relying on any topological structure
or properties (that is having to do with notions of open and closed sets or of the
continuity of functions, in particular payoff functions, was by L. H. Loomis in 1946.
These results on the minmax theorem concern what we would now call two person zero sum normal (or strategic) form games. In 1913 the German mathematician
Ernst Zermelo, most famous for his axiomatisation of set theory, gave a statement
concerning a result about what we would now call extensive form games. Zermelo’s
Theorem is now usually stated as follows: In a game like chess either white can
guarantee that he wins or black can guarantee that he wins or both players can
guarantee at least a draw. The details of what Zermelo actually did and of later
developments in the 1920s by Denes Konig and Laszlo Kalmar are discussed by
Schwalbe and Walker [2001] (Ulrich Schwalbe and Paul Walker “Zermelo and the
Early History of Game Theory,” Games and Economic Behavior v34 no1, 123-37.)
Schwalbe and Walker also give an English translation of Zermelo’s original paper.
There were a number of substantial advances in the book by von Neumann
and Morgenstern: the axiomatic development of the theory of expected utility; the
formal definition of normal form games and extensive form games; the elaboration
of the minmax theorem for two-person zero sum games; and the definition of what
are now called cooperative or coalitional games. Missing was an adequate solution
of noncooperative games other than the two-person zero-sum case. Further, while
the solution proposed by von Nuemann and Morgenstern for coalitional games
(called by von Neumann and Morgenstern “the solution,” and later called “von
Neumann Morgenstern stable sets”) was extremely cumbersome, and perhaps not
as convincingly motivated as von Neumann and Morgenstern thought. Kenneth
Arrow [2003, p 17] comments that this solution “was hardly a useable one. The
few examples offered by the authors were laborious in the extreme.”
However more usable solutions were soon proposed. In the early 1950s John
Nash [1950,1951] proposed a definition of equilibrium, that we now call the Nash
equilibrium, that has become the central solution concept for noncooperative game
theory. We’ll study this solution in some detail in this course. Following Nash’s
definition it was realised that his definition could be thought of as generalising
the much earlier solution concept of Cournot equilibrium. (Cournot, Augustin A.
(1838), Recherches sur les Principes Mathematiquesde la Theorie des Richesses.
Paris: Hachette. English translation: Researches into the Mathematical Principles
of the Theory of Wealth. New York: Macmillan, 1897. (Reprinted New York:
Augustus M. Kelley, 1971.))
In coalition game theory Lloyd Shapley [1952] (Lloyd Shapley “Notes on the
N-Person Game III: Some Variants of the von-Neumann-Morgenstern Definition
of Solution, ”Rand Corporation research memorandum, RM- 817, 1952) and D.
B. Gillies [1953] (D. B. Gillies Some Theorems on N-Person Games, Ph.D. thesis,
Department of Mathematics, Princeton University, 1953) proposed the notion of the
core, the set of all utility allocations that cannot be blocked by any coalition. This
is a simplification, and in a sense a weakening, of the requirements for von Neumann
Morgenstern stable sets, but one that makes the the concept much more usable.
The core, like the Nash equilibrium, has a close connection to an earlier idea in the
Economics literature, in this case the contract curve of Edgeworth [1881]. (Francis
Ysidro Edgeworth, (1881), Mathematical Psychics: An Essay on the Application of
Mathematics to the Moral Sciences. London: Kegan Paul. Reprinted New York:
Augustus M. Kelley, 1967.) Edgeworth had already argued that in an economy
with two goods and two types of consumer the contract curve would shrink to
the set of competitive equilibria as the number of consumers became arbitrarily
large. This result was shown for more generally for the core by Martin Shubik
1. A BRIEF HISTORICAL INTRODUCTION
5
[1959] (Martin Shubik, (1959),“ Edgeworth Market Games,” in A. W. Tucker and
R. D. Luce, eds., Contributions to the Theory of Games, Volume IV, Princeton:
Princeton University Press, pp. 267-278.) for transferable utility games and by
Gerard Debreu and Herb Scarf [1963] (Gerard Debreu and Herbert Scarf (1963),
“A Limit Theorem on the Core of an Economy,” International Economic Review
4, 235-246.) for nontransferable utility games.
Part 1
Noncooperative Game Theory
CHAPTER 2
Introduction to Normal Form Games
We shall now give a fairly informal introduction to one of the two central models
of noncooperative game theory, normal form games. In fact, we shall, both now,
and for the rest of these notes, consider finite normal form games. That is situations
of strategic interaction in which there a finite number of players, each of whom has
a finite number of possible choices.
Instead of starting with even an informal definition of what a normal form
game is, we begin by considering some “famous” examples. These first examples
all consist of situations in which there are only two individuals or players and each
player has only two possible choices or strategies.
1. Examples of normal form games
1.1. The prisoners’ dilemma. A version of the story behind the prisoners’
dilemma is that two petty criminals are caught carrying burglary tools by the police.
This is, in itself, a (relatively minor) crime and the police take the two criminals
to the police station and put them in separate rooms. The police suspect that the
two are responsible for a spate of burglaries but cannot prove it. So they offer each
prisoner the following deal. The prisoner must choose between remaining quiet, or
testifying in court that the other prisoner was involved in the burglaries (finking).
If both prisoners remain quiet, the police will have no evidence that either was
involved in the burglaries, and so will only be able to prosecute them for possessing
burglary tools, which carries a sentence of a year in prison. If one prisoner finks and
the other remains quiet, the one who finked will be rewarded for his cooperation
and will be allowed to go free, while the other will go to prison for 5 years. If both
prisoners fink, the police will be able to prosecute both of them for the burglaries
but they will both get one year off their sentence for their cooperation and so both
will go to jail for 4 years. It is convenient to represent all this information in a
table, as shown in Figure 2.1.
Prisoner 2
Prisoner 1
Quiet
F ink
Quiet
1yr, 1yr
5yr, 0
F ink
0, 5yr
4yr, 4yr
Figure 2.1. The prisoners’ dilemma.
The prisoners are in separate rooms and independently decide whether to stay
quiet or to fink.1 Looking at Figure 2.1, it is clear that whatever Prisoner 2 does, it
is better for Prisoner 1 to fink. Similarly, whatever Prisoner 1 does, it is better for
1You may like to think about whether it would make any difference if the prisoners could
discuss the matter and come to some agreement, or whether one of the prisoners was asked first
and the other prisoner saw his choice before making his own choice.
9
10
2. INTRODUCTION TO NORMAL FORM GAMES
Prisoner 2 to fink. Thus we would predict that the outcome of this game would be
for both prisoners to fink. However, each prisoner would clearly prefer the outcome
in which both of them remain quiet. Alas, this does not happen, because if one
prisoner is remaining quiet, the other has an incentive to fink to reduce his jail
sentence. In the prisoners’ dilemma, individual incentives lead to an outcome that
is collectively bad.
We can make this game a little more abstract by replacing years in jail with
some numbers (utilities) that represent how the two prisoners feel about each outcome. A higher number associated with one outcome compared to another means
that the prisoner prefers the former outcome over the latter. The numbers that we
assign to each outcome are arbitrary, so long as we preserve the same individual
incentives as the original game. One such representation is shown in Figure 2.2.
Prisoner 2
Quiet F ink
Prisoner 1
Quiet
2, 2
0, 3
F ink
3, 0
1, 1
Figure 2.2. The prisoners’ dilemma with utilities instead of jail sentences.
The advantage of this kind of abstraction is illustrated by considering another,
very different, story which can be modeled by the same game. Suppose that we
have two competing firms which can each set either a high price or a low price. If
both set a high price they each do quite well. If both set a low price they each do
somewhat less well. If one sets a high price and the other a low price then the one
setting the high price does very badly, say getting no customers, and the one setting
the low price does very well. The game shown in Figure 2.3 has these features. You
might think of the numbers as representing profits in millions of dollars per month.
Firm 2
High Low
Firm 1
High
2, 2
0, 3
Low
3, 0
1, 1
Figure 2.3. An Industrial Organisation version of the prisoners’ dilemma.
Notice that the formal structure of this is exactly the same as the model of the
situation of the prisoners that we have been discussing. If we are able to analyse one
situation we do not need a new theory for the other situation. Moreover if we are
interested in studying the aspect of the problem involving the strategic interaction
of individuals concerned we may see the considerations more clearly in the abstract
setting.
1.2. Battle of the sexes. In battle of the sexes, a boy and a girl are going
on a date. Unfortunately, while agreeing to do something together they forgot to
arrange exactly what they would do and where they were supposed to meet and
and they cannot contact each other to find out (this game was invented before cell
phones existed). There are two possible locations to meet, either the fights (boxing)
or the ballet. The boy prefers the fights over the ballet and the girl prefers the ballet
over the fights. However both the boy and the girl prefer to go to the same place
as the other rather than ending up in different places. Lets also assume that if
1. EXAMPLES OF NORMAL FORM GAMES
11
they end up in different places they will be so sad that they won’t care which place
they are in. (Ah, young love.) One possible representation of this game is shown
in Figure 2.4. Here we immediately associate utilities with the various possible
outcomes, as we shall continue to do without further comment.
Girl
F ights Ballet
Boy
F ights
2, 1
0, 0
Ballet
0, 0
1, 2
Figure 2.4. Battle of the sexes.
Now let us try to predict the outcome of the battle of the sexes. First note
that, unlike in the prisoners’ dilemma, there is no one thing that is best for either
the boy or the girl to do, regardless of what the other does. Suppose that the boy
thinks that the girl will go to the fights. Then the boy should also go to the fights.
However, if the boy thinks that the girl will go to the ballet, he does best by going
to the ballet too. Similarly for the girl. Thus, both going to the fights or both
going to the ballet are two possible predictions that we could make. However, each
can only guess what the other is going to do and it is possible that either one of
them will guess wrong. Thus we might think that “confusion” is also a plausible
outcome.
1.3. Chicken. In the game of chicken, two boys (or girls, but usually girls
aren’t so stupid) are driving their cars towards each other at high speed. Each
can either choose to quit (by swerving out of the way) or to continue. If one boy
continues while the other quits, then the one who continued is a hero (to the boys)
while the other suffers terrible embarrassment. If both choose to continue, they
have a crash and possibly die, which is very bad. If both choose to quit then
neither of them gets to be the hero but it’s not as bad as having a crash or being
called a chicken. One possible representation of this game is given in Figure 2.5.
Boy 2
Quit
Continue
Boy 1
Quit
Continue
1, 1
−2, 2
2, −2
−4, −4
Figure 2.5. Chicken.
Note that the game as given in Figure 2.5 doesn’t quite correspond to the
reality of the story. In reality, both drivers are choosing whether to continue or
quit at any instant in time while they are driving towards each other. However, we
can save ourselves a lot of trouble while still getting some insight into what’s going
on by using the static model.
What would we predict as the outcome of chicken? We can see that if Boy 1
thinks that Boy 2 will quit, then Boy 1 should continue, while if Boy 1 thinks that
Boy 2 will continue, then Boy 1 should quit. Similar reasoning applies to Boy 2.
Thus we might predict (Continue, Quit) and (Quit, Continue) as possible outcomes.
As in battle of the sexes, however, there is the possibility for “confusion”, if each
boy is uncertain what the other will do.
12
2. INTRODUCTION TO NORMAL FORM GAMES
1.4. A pure coordination game. Unlike chicken, in a pure coordination
game there is no conflict of interest. Player 1 and Player 2 of this game are faced
with a choice between two things, A and B. Neither player has a preference over
A or B, they just care about choosing the same thing as the other player. One
possible representation of this game is given in Figure 2.6.
Player 2
Player 1
A
B
A
1, 1
0, 0
B
0, 0
1, 1
Figure 2.6. A pure coordination game.
In this game, we would predict that the outcome would be either (A, A) or
(B, B). Again, there is the possibility for confusion if each player is uncertain what
the other will do.
1.5. Matching pennies. In matching pennies there are two players each
holding a coin. Simultaneously they must put their coin down on a table either
facing heads or tails up. If both players choose the heads or both choose tails,
then Player 2 pays Player 1 $1. If the players choose differently then Player 1 pays
Player 2 $1. A representation of this game is given in Figure 2.7.
Player 2
Player 1
H
T
H
1, −1
−1, 1
T
−1, 1
1, −1
Figure 2.7. Matching pennies.
Could the outcome (H, H) be expected of this game? If Player 2 were expecting
Player 1 to play heads, then Player 2 would prefer to play tails. Thus (H, H) cannot
be an outcome of this game. Similar reasoning shows that any other combination,
i.e. (H, T ), (T, T ) and (T, H) cannot be an outcome of the game. Rather, the only
possibility is ‘confusion’, though perhaps the term confusion doesn’t seem quite
right here. Neither player is really confused about what the other is doing. It is
clear that each player is trying to conceal from the other exactly what it is that
he will do. Thus rather than confusion perhaps we should say there is uncertainty
about the outcome.
2. Some terminology
Consider the prisoners’ dilemma game from section 1. We shall call Prisoner 1
and Prisoner 2 players. The numbers in the matrix in Figure 2.2 are called payoffs.
The convention is to have Player 1 on the left-hand side of the matrix and Player 2
on the top, and to list the payoff for the player on the left-hand side first, so a payoff
of 0, 3 means a payoff of 0 to Player 1 (Prisoner 1) and 3 to Player 2 (Prisoner 2).
Finally, we shall call the choices available to each player, Quiet and Fink, that
player’s strategies.
4. “SELF-ENFORCING UNCERTAINTY”
13
3. What’s a Strategy?
In the examples that we’ve considered the nature of the stories we’ve told
suggests that the players simultaneously choose their actions or strategies. Most
games surely are not like this. In chess, for example, one player moves then the other
and then the first again, and so on. One of the great contributions of von Neumann
and Morgenstern was to suggest that we can analyse games like this by thinking
of players as forming plans as to how to play the game and then simultaneously
choosing their plans at the beginning of the game and then fairly mindlessly carrying
out their plans. They argue that if a player ever had an incentive to change his
plan he should have realised, before the game started, that he would have such and
incentive and so should have chose a different plan at the start of the game. They
termed such complete plans strategies.
We’ve treat this idea more formally when we come to consider extensive form
games later in the course. For now we simply say that a strategy is a complete
plan of how to play the game and that a normal or strategic form game specifies
a complete list of all such plans for each player, a specification of an outcome for
each possible profile of plans or strategies, and a specification in terms of a utility
function of how each player evaluates the possible outcomes.
4. “Self-enforcing uncertainty”
In all of the examples of normal form games given in section 1 except for the
prisoners’ dilemma, we listed ‘confusion’ or uncertainty as a possible outcome. This
occurs when neither player can be sure about what the other player will do and
thus has to take a guess, which may be wrong. Rather than calling this confusion,
let’s call it uncertainty. We’ll model uncertainty as probabilities.
Let’s think again about the matching pennies game. In section 1, we argued
that the only possible outcome of this game was ‘confusion’, or ‘uncertainty’ as
we are now calling it. We shall now show how this uncertainty can be quantified.
Suppose that in Player 1’s mind, he thinks that Player 2 will play H with probability
x and T with probability 1 − x. Player 1’s expected utility from playing H is
x − (1 − x) = 2x − 1. His expected utility from playing T is −x + (1 − x) = 1 − 2x.
If Player 1 thinks that x < 21 then Player 1 strictly prefers to play T , while if
x > 21 then Player 1 strictly prefers to play H. But, if Player 1 does play T then
Player 2 will play H which means that we cannot have x < 21 and Player 1 playing
T . Similarly if Player 1 does play H then Player 2 will play T which means we
cannot have x > 21 and Player 1 playing H.
So, it must be that the uncertainty in Player 1’s mind is such that Player 2 is
equally likely to play H or T , since any other quantification of Player 1’s uncertainty
about what Player 2 will do leads to a contradiction. Similar logic shows that the
uncertainty in Player 2’s mind must be that Player 1 is equally likely to play H or
T also. This equilibrium uncertainty can be represented as probabilities, as shown
in Figure 2.8.
Player 2
Player 1
H( 21 )
T ( 21 )
H( 12 )
T ( 21 )
1
4
1
4
1
4
1
4
Figure 2.8. The equilibrium uncertainty in matching pennies.
14
2. INTRODUCTION TO NORMAL FORM GAMES
In Figure 2.8, we can see that all of the outcomes are equally likely, i.e. all four
outcomes each occur with probability 41 . We have thus quantified the uncertainty
of the game.
In matching pennies there are only two possible outcomes: either Player 1 wins
or Player 2 wins. An implication of this is that it does not matter how we assign
utilities to the outcomes as long as we make the utility to Player 1 higher when he
wins than when he loses and similarly for Player 2. However, when there are more
than two possible outcomes, such as in Battle of the Sexes, it does matter. We’ll
return to this in Section 3 of the following chapter.
5. The Monty Hall Game
Before moving on to examining methods of solving games in the next chapter
we’ll look at one final less trivial game. This game was originally proposed as a
puzzle in probability theory. It was supposedly based on a real game show on
American TV, but there is some suggestion that the details of the puzzle are not
consistent with the way that the actual game show was played. We’ll also recast
the problem as a game and make the host, Monty, a nontrivial player.
There are three boxes and one prize. Monty moves first and without being
seen by the contestant puts the prize in one of the boxes. The contestant then
chooses one of the boxes. Monty then opens an empty box not chosen by the
contestant and shows the contestant that it is empty and offers the contestant the
possibility of switching from the box he had originally chose to the other unopened
box. The contestant then either switches or doesn’t. If the contestant ends up with
the box containing the prize then Monty pays the contestant the value of the prize,
otherwise Monty doesn’t pay him anything.
You might think about how many strategies each of the players would have in
this game and how you might solve this game. One of the problems below (possibly
the hardest one) takes you through how you might solve this game.
Problems
Problem 2.1. Consider the Monty Hall problem that we discussed above.
We’ll think of Monty choosing his actions probabilistically. Suppose that the three
boxes are numbered 1, 2, and 3, and the probability that the prize is in box i is
pi . If the contestant picks Box i and the prize is in Box j then Monte opens Box k
with probability qijk . Of course Monte never opens the Box the contestant picked,
and never opens the Box containing the prize.
(a) Say as much as you can about the qijk ’s. For example q111 = 0. Why?
(b) In Many treatments the prize is equally likely to be in each box and the Contestant originally picks a box at random and then has only two strategies, Switch
and Don’t Switch. Suppose that we also include the original choice of a box to
be a choice and allow the decision as to whether to switch to depend on which
box Monte opens. How many strategies does the contestant have and what are
they?
(c) For each strategy of the contestant what is the probability of winning the prize?
(d) What is the best strategy for the contestant?
(e) Suppose that Monty assumes that whatever he chooses for p1 , p2 , p3 and qijk
for i, j, k = 1, 2, 3 the Contestant will respond optimally. How should Monty
choose p and q?
Problem 2.2. Consider the game of noughts and crosses.
(a) Write out a set of rules that completely specify how this game is played and
who wins.
PROBLEMS
15
(b) Suppose that you had to instruct a friend on how to play this game. What
would you tell them? Consider each role separately.
(c) A strategy is a complete specification as to how to play the game. How would
you go about specifying one strategy for the player who moves first? Give a
serious attempt, but don’t go overboard. As an extra challenge you might try
to specify a good strategy.
(d) How many strategies does the Player who moves first have? If you cannot find
the actual answer you might try to give some bounds such as “at least a million
and no more than two million.”
Problem 2.3. Consider the game of chicken as shown in Figure 2.5.
(a) Is there a self-enforcing outcome of this game in which the players are uncertain
about what choice the other will make?
(b) If so, what is the probability (in Player 1’s mind) that Player 2 will play Quit?
(c) What does the matrix of probabilities look like?
Problem 2.4. Consider the game of Battle of the sexes as shown in Figure 2.4.
(a) Is there a self-enforcing outcome of this game in which the players are uncertain
about what choice the other will make?
(b) If so, what is the probability (in Boy’s’s mind) that Girl will play F ights?
What is the probability (in Girl’s mind) that Boy will play F ights?
(c) What does the matrix of probabilities look like?
(d) Now suppose we change the payoffs 2, 1 to 3, 1 and 1, 2 to 1, 3. Notice that
we have not changed the players’ preferences over the pure outcomes: Boy still
prefers that both go to the fights over both going to the Ballet over ending up
in different places. And similarly for Girl. Have the pure strategy equilibria of
this game changed? Repeat (b) and (c) for these new payoffs. Has your answer
changed? If so, why?
CHAPTER 3
Solving normal form games
In the next chapter we shall give some formal definitions examine quite systematically how to find the equilibria of games. In this chapter we are going to
examine things a little less formally.
1. Dominated strategies
A dominated strategy is a strategy for which there is some other strategy that
is always better whatever the other players are doing.
Let’s look again at the prisoners’ dilemma in Figure 2.2. In this game, Quiet
is a dominated strategy for both players because F ink is better (gives a higher
payoff) regardless of whether the other player chooses Quiet or F ink.
In some games, simply eliminating the dominated strategies leaves us with
a unique prediction of the outcome. In the prisoners’ dilemma, this gives us
(F ink, F ink) as the unique outcome of the game. In some sense there is no strategic interaction between the players. the solution to each player’s problem is the
same whatever the other player does.
In battle of the sexes, on the other hand, there are no dominated strategies. If
the boy thinks the girl will choose to go the fights, the boy should go to the fights,
while if the boy thinks the girl will go to the ballet then the boy should go to the
ballet. Thus the best thing for the boy to do depends on what he thinks the girl
will do, and neither strategy is dominated for the boy. Similar reasoning shows that
neither strategy is dominated for the girl. Elimination of dominated strategies has
no implications for the outcome in a game like battle of the sexes.
2. Nash equilibrium
We now define a Nash equilibrium, a concept that does have implications in
games like battle of the sexes. This is the central solution concept in noncooperative
game theory.
Definition. A Nash equilibrium is a profile (ordered list) of strategies, one for
each player, such that each player’s strategy gives him the highest possible utility,
given that the other players play their part of the profile.
Is (Fights, Fights) a Nash equilibrium of battle of the sexes as shown in Figure 2.4? Yes. Given that Girl plays Fights, playing Fights for Boy gives him a
utility of 2 while playing Ballet gives him a utility of zero. Similarly, given that
Boy plays Fights, Girl achieves her highest possible utility by also playing Fights.
The same kind of logic shows that (Ballet, Ballet) is also a Nash equilibrium of
battle of the sexes.
What about matching pennies, as shown in Figure 2.7? Under the above definition of Nash equilibrium, there is no Nash equilibrium of matching pennies. (H, H)
is not an equilibrium, since Player 2 could obtain a higher utility by switching her
strategy to T so the outcome would be (H, T ) and she would obtain a utility of 1
rather than −1. To accommodate games like matching pennies, we need to extend
the notion of strategy.
17
18
3. SOLVING NORMAL FORM GAMES
2.1. Mixed strategies. We now introduce the notion of a randomised or
Mixed strategy.
Definition. A mixed strategy of a player is a rule that assigns a probability
to each of that player’s pure strategies.
We now call the original strategies of the player pure strategies. An example
of a mixed strategy of Player 1 in matching pennies is a probability distribution
that assigns probability 41 to H and probability 43 to T . This is also an example of
a mixed strategy for Player 2.
2.2. Nash equilibrium in mixed strategies. Let us look again at the
matching pennies game of Figure 2.7 which, for convenience, we’ll repeat here as
Figure 3.1.
Player 2
H
T
Player 1
H
1, −1
−1, 1
T
−1, 1
1, −1
Figure 3.1. Matching pennies.
Although matching pennies has no Nash equilibrium in pure strategies, it does
have a Nash equilibrium that involves mixed strategies. In this game, a profile of
mixed strategies takes the form ((x, 1 − x), (y, 1 − y)) where (x, 1 − x) is Player 1’s
mixed strategy (i.e. x is the probability that Player 1 plays H) and (y, 1 − y) is
Player 2’s mixed strategy (i.e. y is the probability that Player 2 plays H). Such
a profile is a Nash equilibrium in mixed strategies if (x, 1 − x) is one of the best
mixed strategies of Player 1, given that Player 2 plays (y, 1 − y) and similarly for
Player 2.
Our informal analysis in section 4 of the previous chapter suggests that the
profile (( 12 , 21 ), ( 21 , 12 )) is a Nash equilibrium of matching pennies. Let us now confirm this. Suppose that Player 1 plays (1, 0) (i.e. Player 1 just plays H). Then
Player 1 gets payoff of 1 with probability y and −1 with probability 1 − y. Now
suppose Player 1 plays ( 21 , 21 ) (i.e. Player 1 plays H with probability 21 and T with
probability 12 ). Then Player 1 gets payoff of 1 with probability 21 y and payoff of −1
with probability 21 (1 − y). What would it mean for ( 21 , 12 ) to be best for Player 1?
Clearly, it must be better than both (1, 0) and (0, 1). Suppose that A = Player 1’s
payoff from H (i.e., (1, 0)), and B = Player 1’s payoff from T (i.e. (0, 1)) and C =
Player 1’s payoff from ( 12 , 12 ). Then if ( 12 , 12 ) is best for Player 1, we must have
C ≥ A and C ≥ B. Also note that C = 12 A + 21 B, i.e., that C is the average of A
and B. If you think about it for a moment, you will realise that if C is the average
of A and B, and if C ≥ A and C ≥ B then it must be the case that C = A = B.
That is, Player 1’s (expected) payoff from ( 12 , 12 ) must be the same as Player 1’s
(expected) payoff from just playing H or T , given what Player 2 is doing.
Fact: If a mixed strategy is best for a player then all pure strategies that are
played with positive probability in that mixed strategy are equally good (and are
best strategies).
To see why this fact must be true, just imagine if one pure strategy was a bit
better than the others. Then why would the player bother playing any of the other
pure strategies with any positive probability? He or she would be better off just
playing the pure strategy that is best with probability 1.
2. NASH EQUILIBRIUM
19
2.3. How do we find a (strictly) mixed strategy Nash equilibrium?
Suppose that we did not know what the equilibrium of matching Pennies was. Let
us see how we could find the equilibrium probabilities. What we are looking for is
two numbers, 0 < x < 1 and 0 < y < 1 where x is the probability that Player 1
plays H and y is the probability that Player 2 plays H, with (x, 1 − x) best for
Player 1 against (y, 1 − y) and (y, 1 − y) best for Player 2 against (x, 1 − x).
Since 0 < x < 1, we know that both H and T are best and so are equally good.
What does Player 1 get from playing H?
1 · y + (−1) · (1 − y).
And from T , Player 1 gets
(−1) · y + 1 · (1 − y).
We know that if Player 1 is playing a mixed strategy, the payoff from H and T
should be the same for Player 1. That is,
1 · y + (−1) · (1 − y) = (−1) · y + 1 · (1 − y)
or
2y = 2(1 − y)
or
y = 21 .
Thus we have shown that Player 1 is indifferent between the pure strategies H and
T if y = 21 , that is, if Player 2 plays the mixed strategy ( 12 , 12 ).
Note. We calculate Player 2’s equilibrium mixed strategy from the fact of
Player 1’s indifference between his pure strategies. We should think of Player 2’s
mixed strategy as Player 1’s uncertainty about what Player 2 is going to do.
In a similar fashion, since 0 < y < 1, both H and T must be equally good for
Player 2. This implies that
(−1) · x + 1 · (1 − x) = 1 · x + (−1) · (1 − x)
which gives
x = 12 .
Thus we have shown that the (only) Nash equilibrium of matching pennies is
(( 21 , 12 ), ( 12 , 12 )).
As another example consider the battle of the sexes game in Figure 2.4, which
we shall again repeat here as Figure 3.2.
Girl
F ights Ballet
Boy
F ights
2, 1
0, 0
Ballet
0, 0
1, 2
Figure 3.2. Battle of the sexes.
We already know that (Fights, Fights) is a pure strategy Nash equilibrium. We
could re-write this as a mixed strategy equilibrium in the form ((1, 0), (1, 0)). We
also know that (Ballet, Ballet), that is, ((0, 1), (0, 1)) is another Nash equilibrium.
Are there any other Nash equilibria of this game? When checking for other Nash
equilibria, we need to allow for the possibility that one player plays a strictly mixed
strategy and the other plays a pure strategy. Suppose that Boy plays (x, 1 − x) with
0 < x < 1 (i.e. Boy is playing a strictly mixed strategy) and Girl plays (y, 1 − y)
20
3. SOLVING NORMAL FORM GAMES
with 0 ≤ y ≤ 1 (i.e. Girl may be playing a pure strategy if y = 0 or y = 1).
Since 0 < x < 1, Boy must be indifferent between choosing Fights and Ballet. This
requires
2y + 0 · (1 − y) = 0 · y + 1 · (1 − y)
which implies
2y = 1 − y
which gives
y = 31 .
Therefore, Girl’s strategy ( 13 , 23 ) makes Boy indifferent between Fights and Ballet.
Since 0 < 31 < 1, Girl must also be indifferent between Fights and Ballet. This
requires
1 · x + 0 · (1 − x) = 0 · x + 2(1 − x)
which implies
x = 23 .
Thus Boy’s mixed strategy is ( 32 , 13 ). Therefore, a mixed strategy equilibrium of
the battle of the sexes game in Figure 3.2 is (( 23 , 13 ), ( 13 , 23 )).
3. Utility functions
When modern game theorists, or decision theorists, or economists talk about
utility they mean, or should mean, something quite particular. In particular “utility” is just a convenient representation of a well-behaved, or “rational” preference.
Moreover preference itself may, and I would argue should be thought of as just a
convenient representation of well-behaved choice behaviour.
When we talk about the ordinal properties of utility, we mean that we only
care about the ranking of utilities, not the size of the utilities themselves.
When we allow for mixed strategies, we are also interested in the ranking of
random outcomes. For example, in the prisoners’ dilemma game in Figure 2.2,
would Prisoner 1 prefer (Quiet, Quiet) to a 12 chance of (Quiet, Fink) and a 12
chance of (Fink, Quiet)? This depends on the cardinal properties of the utilities
(i.e., how big they are, or at least somewhat on how big they are).
To answer such questions we need a theory of utility over random outcomes or
lotteries. Suppose we have four outcomes: W , X, Y , and Z. A lottery is then a
vector of probabilities: (pW , pX , pY , pZ ), such that pW + pX + pY + pZ = 1. It was
shown by von Neumann and Morgenstern that if a decision maker’s preferences over
lotteries were well behaved in a certain precise sense then those preferences over
lotteries can be represented by the expected value of the decision maker’s utility.
That is, there is some way of assigning utility to the outcomes such that one lottery
is prefedded to another if and only if the expected utility of the lottery is greater
than the expected utility of the other lottery.
If we are concerned with the players’ preferences over lotteries, as we must be
when we consider mixed equilibria, then we cannot simply change the payoffs of a
game in ways that don’t change the preferences over pure outcomes. To illustrate
this, consider the battle of the sexes game shown in Figure 3.2. If Girl plays the
mixed strategy ( 31 , 23 ) then Boy does not prefer one of his strategies over the other
since they both give him an expected utility of 23 . Now consider the modified
battle of the sexes game shown in Figure 3.3. In both figures 2.4 and 3.3, Boy’s
preferences are that (F ights, F ights) is better than (Ballet, Ballet) which is better
than (Ballet, F ights) which is exactly as good as (F ights, Ballet). That is, Boy’s
preferences over the pure outcomes are the same for both games. Similarly, Girl’s
preferences are also the same for both games. However, the preferences over lotteries
of either Boy or Girl are not the same in these two games.
PROBLEMS
21
Girl
F ights Ballet
Boy
F ights
3, 1
0, 0
Ballet
0, 0
1, 3
Figure 3.3. Modified battle of the sexes. Players have the same
preferences over pure outcomes as the original game, but their
preferences over uncertain outcomes are changed.
To see this, suppose that Girl plays Fights with probability 0.3 and Ballet with
probability 0.7 (note these are not 31 and 32 ). In the original game in Figure 2.4,
Boy’s expected payoff from choosing Fights is 2·0.3+0·0.7 = 0.6 and from choosing
Ballet it is 0 · 0.3 + 1 · 0.7 = 0.7. Thus Boy will choose Ballet. However, in the
modified game in Figure 3.3, Boy’s expected payoff from Fights is 3·0.3+0·0.7 = 0.9
and his expected payoff from Ballet is 0 · 0.3 + 1 · 0.7 = 0.7, thus Boy will choose
Fights.
From the above example, we can see that changing the numbers that we put
for the payoffs can change the players’ preference over mixed outcomes, even if they
do not change the player’s preferences over certain outcomes.
Problems
Problem 3.1. Consider the following two person game.
Player 2
X
Y
Player 1
A
9, 9
0, 8
B
8, 0
7, 7
(1) Suppose that Player 1 thinks that Player 2 will play her strategy X with
probability y and her strategy Y with probability 1 − y. For what value
of y will Player 1 be indifferent between his two strategies?
(2) If y is less than this value what strategy will Player 1 prefer? If y is greater
than that value?
(3) Repeat this analysis with the roles of the players reversed. (To facilitate
the extensions below, suppose that Player 2 thinks that Player 1 will play
his strategy A with probability x and his strategy B with probability
1 − x.)
Problem 3.2. Now consider the following two person game.
Player 2
X
Y
Player 1
A
1, 1
0, 0
B
0, 0
7, 7
22
3. SOLVING NORMAL FORM GAMES
(1) Again suppose that Player 1 thinks that Player 2 will play her strategy X
with probability y and her strategy Y with probability 1 − y. For what
value of y will Player 1 be indifferent between his two strategies? Let this
value of y be called ȳ.
(2) If y is less than ȳ what (pure) strategy will Player 1 prefer? If y is greater
than ȳ? What mixed strategies will these pure strategies be equivalent
to?
(3) Graph, as a function of y, the mixed strategy of Player 2, what the best
values of x, the mixed strategies of Player 1, are. We call this the best reply correspondence of Player 1 and label it BR1 (·). That is BR1 (y) gives,
for a particular value of y, all those values of x for which (x, 1 − x) is a
best mixed strategy for Player 1 given Player 2 plays the mixed strategy
(y, 1 − y).
(4) Repeat this analysis with the roles of the players reversed. Let the probability that Player 2 assesses that Player 1 will play X be x.
Problem 3.3. In the previous problem you found two graphs. In this problem
you are asked to put the two graphs together. If you think about what is going
on you will see that these graphs show us precisely the Nash equilibria in mixed
strategies.
(1) Consider the graph of the best reply correspondence BR1 (·) that you
found in the previous problem. Explain what it means for a point (y, x)
to lie on the graph. What does it mean for the point (x, y) to lie on the
graph of BR2 (·)?
(2) Now draw both graphs on the same diagram. This will involve switching
the axes in one of the diagrams. Just so we all draw the same pictures,
lets put x, the mixed strategy of Player 1, on the horizontal axis.
(3) Mark on the graph all points that lie on both graphs.
(4) Explain what it means for a point to lie on both graphs and explain why
such a point is a Nash equilibrium.
CHAPTER 4
A more formal treatment of normal form games
1. Formal definition of a normal form game
Definition. A normal or strategic form game consists of:
(1) N , a finite set of players.
(2) For each player n ∈ N , a finite set of pure strategies Sn .
(3) For each player n ∈ N , a payoff function un that specifies a real number
for each profile of pure strategies s1 , . . . , sn , . . . , sN ).
Before giving examples, let us briefly discuss functions. A function is a rule
that associates to each member of one set (called the domain) a member of a second
set (called the codomain). For example, f : R → R, where f (x) = 3x2 . Then f
is a function that associates to each real number another real number, namely 3
times the square of the first number. As another example suppose we have a set
A = {a, b, c} and another set B = {x, y, z}. Then the following table represents a
function g : A → B:
a b
x x
c
y
Figure 4.1. A function g.
Returning to the definition of a normal form game, another way of writing part
(3) is that for each player n ∈ N there is a utility function un : S → R where
S = S1 × S2 × · · · × SN .
What does this mean? Consider again the sets A = {a, b, c} and B = {x, y, z}.
Then A × B is the so-called Cartesian product of the sets A and B, which is the
set of all ordered pairs (a, b) where a is an element of A and b is an element of B.
That is,
A × B = {(a, x), (b, x), (c, x), (a, y), (b, y), (c, y), (a, z), (b, z), (c, z)}.
Thus S = S1 × S2 × · · · × SN is the set of all strategy profiles (s1 , . . . , si , . . . , sN
and un : S → R takes any strategy profile and gives player n’s payoff from it.
In order to see exactly what this definition is saying let us consider again the
battle of the sexes game from Figure 2.4 and Figure 3.2, which we’ll repeat here
one final time as Figure 4.2.
Girl
Boy
F ights
Ballet
F ights
2, 1
0, 0
Ballet
0, 0
1, 2
Figure 4.2. Battle of the sexes.
23
24
4. A MORE FORMAL TREATMENT OF NORMAL FORM GAMES
In the battle of the sexes game we identify each element that we gave in Definition 1:
N = {Boy, Girl};
SBoy = {F ights, Ballet},
SGirl = {F ights, Ballet};
S = {(F ights, F ights), (F ights, Ballet), (Ballet, F ights), (Ballet, Ballet)};
And two payoff functions, uBoy : S → R and uGirl : S → R as shown in the
following table:
s
uBoy
uGirl
(F, F )
2
1
(F, B)
0
0
(B, F )
0
0
(B, B)
1
2
Figure 4.3. Payoff functions in the Battle of Sexes.
Observe that the information contained in this description is exactly the same
as the information contained in Figure 4.2, though perhaps not in a form that is
quite as easy to read.
2. Best reply correspondences
For two-player games such as we have been looking at so far, the best reply
correspondence of Player 1 is a rule that gives, for any mixed strategy of Player 2,
the set of Player 1’s best mixed strategies.
For example, in battle of the sexes as in Figure 2.4, suppose that Boy is playing
a mixed strategy (x, 1 − x) and Girl is playing a mixed strategy (y, 1 − y). If y is
“small” then Boy should choose Ballet. Similarly, if y is “large” then Boy should
choose Fights. In between, there is some y that makes Boy indifferent between
Ballet and Fights. What is this y? From Fights, Boy’s expected payoff is 2y and
from Ballet his expected payoff is 1 − y. Thus Boy is indifferent between Fights
and Ballet if 2y = 1 − y which implies y = 31 . So, when y = 13 , Boy’s best reply is
any mixed strategy (x, 1 − x) such that x ∈ [0, 1]. That is, we can write Boy’s best
reply as

if y < 31
 (0, 1)
{(x, 1 − x) : 0 ≤ x ≤ 1} if y = 31 .
BRBoy (y) =

(1, 0)
if y > 31
Boy’s best reply is shown graphically in Figure 4.4.
Similar calculations show that Girl’s best reply is

if x < 23
 (0, 1)
{(y, 1 − y) : 0 ≤ y ≤ 1} if x = 32 .
BRGirl (x) =

(1, 0)
if x > 32
Girls’ best reply is shown in Figure 4.5.
Figure 4.6 shows both the best replies of Boy and Girl on the same graph. Note
that if a point (x, y) is on the graph of BRGirl (·) then the mixed strategy (y, 1 − y)
is (one of) the best for Girl, given that Boy is playing (x, 1 − x). Similarly, if a
point (x, y) is on the graph of BRBoy (·) then the mixed strategy (x, 1 − x) is (one
of the) best for Boy, given that Girl is playing (y, 1 − y). Therefore, the points that
are on both best replies are Nash equilibria.
From Figure 4.6 we can see that there are three Nash equilibria of the battle
of the sexes game given in Figure 2.4. These are (( 32 , 13 ), ( 31 , 23 )), ((0, 1), (0, 1)),
2. BEST REPLY CORRESPONDENCES
25
x
16
BRBoy (·)
1 y
1
3
Figure 4.4. Boy’s best reply in the battle of the sexes.
y
16
BRGirl (·)
2
3
1 x
Figure 4.5. Girl’s best reply in the battle of the sexes.
y
16
BRGirl (·)
1
3
BRBoy (·)
2
3
1 x
Figure 4.6. The best replies of both Boy and Girl in the battle
of the sexes.
and ((1, 0), (1, 0)). Note that the equilibrium ((0, 1), (0, 1)) corresponds to (Ballet,
Ballet) and ((1, 0), (1, 0)) corresponds to (Fights, Fights).
At this point, you might be wondering what the possible configurations of
best-reply correspondences in a 2x2 game are. It turns out that there are only a
few such possibilities. Figure 4.7 shows the possible best-reply correspondences for
Player 2. There are eight different possibilities for one player. But for the other
player there are less than eight possibilities. By swapping the labels of the other
player’s strategies, we can see that (1) and (2) are equivalent. Similarly, (3) and
(4) are equivalent, and (5), (6), (7) and (8) are equivalent. Thus we only have three
different possibilities for the other player. This gives a total of 24 different possible
26
4. A MORE FORMAL TREATMENT OF NORMAL FORM GAMES
configurations for 2x2 games. Note that configurations (3) - (8) are ‘knife-edge’
and could be thought of as special cases of (1) and (2).
Now, we should be able to look at a best-reply graph and reconstruct the payoffs
that would give that best-reply. For example, for case (5) some possible payoffs are
shown in Figure 4.8 and for case (1) some possible payoffs are shown in Figure 4.9,
where Player 2’s indifference between L and R will be when Player 1 plays T with
probability 12 .
Problem 4.1. Give some payoffs of Player 2 that make Player 2 have a best
reply correspondence as in part (1) of Figure 4.7 and that make Player 2 indifferent
between L and R when Player 1 plays T with probability 13 .
Problem 4.2. We have already seen a game in which the best reply correspondences were such that one player’s looked like case (1) of Figure 4.7 and the other
player’s looked like case (2). What was the name of this game?
In fact, there’s one more knife-edge possibility, where the player is indifferent
over all of his mixed strategies. Such a best reply is shown in Figure 4.10 and
some payoffs of Player 2 that give rise to this best reply correspondence are given
in Figure 4.11.
3. Games where players have more than two pure strategies
So far we have only considered games with two players and two pure strategies.
Now consider the game in Figure 4.12 in which there are two players each having
three pure strategies. It is clear that (T, L) is a pure strategy equilibrium because
given that Player 2 is playing L, Player 1 does best by playing T , and given that
Player 1 plays T , Player 2 does best by playing L. Similarly, (M, C) and (B, R)
are also pure strategy equilibria.
Finding the mixed strategy equilibria of such a game is a little more complicated
than when the players had only two strategies. Suppose that Player 1 plays a
mixed strategy of playing T with probability xT , playing M with probability xM
and B with probability 1 − xT − xM . Since Player 1 has three pure strategies,
Player 1’s space of all possible mixed strategies is three dimensional and is shown
in Figure 4.13.
We can see that in this game, the mixed strategy space of Player 1 is a triangular
surface that ‘lives’ in a three-dimensional space. However, rather than drawing such
a complicated three-dimensional picture, we can represent any given mixed strategy
of Player 1 as a point in a triangle, as shown in Figure 4.14. This is simply the
triangle from Figure 4.13, drawn without the extra dimension. We denote the mixed
strategy space of Player 1 by Σ1 . Each of the corners of the triangle represents one
of Player 1’s pure strategies. Any other point within the triangle or on the edges
represents a mixed strategy in which more than one strategy is played with strictly
positive probability. In Figure 4.14, the points representing the mixed strategies
( 31 , 13 , 13 ) and ( 12 , 21 , 0) are shown.
Now that we know what Player 1’s mixed strategy space looks like, we can
draw Player 2’s best reply on it. Refer to Figure 4.15. If Player 1 plays B, Player 2
is indifferent between L and C. On the line between T and M , the probability
of Player 1 playing B is zero and if Player 1 plays the mixed strategy ( 23 , 13 , 0),
Player 2 is indifferent between L and C. Thus we can draw a line between B and
the point 23 T + 31 M and along this line Player 2 is indifferent between L and C. To
the left of this line Player 2 prefers L and to the right Player 2 prefers C. Similarly,
Player 2 is indifferent between C and R if Player 1 plays T or if the probability
of M is 23 times the probability of B. Thus we can draw a line between the point
T and the point 35 M + 52 B along which Player 2 is indifferent between C and R.
3. GAMES WHERE PLAYERS HAVE MORE THAN TWO PURE STRATEGIES
Σ2
16
Σ2
16
1 Σ1
1 Σ1
(1)
(2)
Σ2
16
Σ2
16
1 Σ1
1 Σ1
(3)
(4)
Σ2
16
Σ2
16
1 Σ1
1 Σ1
(5)
(6)
Σ2
16
Σ2
16
1 Σ1
(7)
1 Σ1
(8)
Figure 4.7. Eight possible best reply correspondences for
Player 2 in a 2 × 2 game.
27
28
4. A MORE FORMAL TREATMENT OF NORMAL FORM GAMES
Player 2
L
R
Player 1
T
x, 0
x, 0
B
x, 0
x, 1
Figure 4.8. Some possible payoffs for Player 2 that would give a
best reply correspondence like that of case (5) of Figure 4.7.
Player 2
L
R
Player 1
T
x, 3
x, 2
B
x, 1
x, 2
Figure 4.9. Some possible payoffs for Player 2 that would give a
best reply correspondence like that of case (1) of Figure 4.7.
Σ2
16
1 Σ1
Figure 4.10. One more knife-edge possibility where Player 2 is
indifferent among all of her mixed strategies.
Player 2
L
R
Player 1
T
x, 2
x, 2
B
x, 1
x, 1
Figure 4.11. Some possible payoffs for Player 2 that would give
a best reply correspondence like that in Figure 4.10.
Above this line Player 2 prefers R and below this line Player 2 prefers C. Finally,
Player 2 is indifferent between L and R if Player 1 plays M or if the probability
of T is 3 times the probability of B. Thus we can draw a line between the point
M and the point 43 T + 14 B along which Player 2 is indifferent between L and R.
Above this line Player 2 prefers R and below this line Player 2 prefers L.
The above analysis gives us Player 2’s best reply for any given mixed strategy
of Player 1. This is shown in Figure 4.16. In the regions marked L, C, and R,
Player 2’s only best reply is the marked pure strategy. Along the thick lines,
3. GAMES WHERE PLAYERS HAVE MORE THAN TWO PURE STRATEGIES
29
Player 2
L
C
R
T
1, 1
0, 0
0, 0,
Player 1 M
0, 0
2, 2
0, 0
B
0, 0
0, 0
3, 3
Figure 4.12. A coordination game with two players where both
players have three strategies.
xB
6
xM
(0, 0, 1) ∼ B
(0, 1, 0) ∼ M
(1, 0, 0) ∼ T xT
Figure 4.13. Player 1’s mixed strategy space in the coordination
game of Figure 4.12
(0, 0, 1) ∼ B
( 31 , 13 , 31 )
(1, 0, 0) ∼ T
( 12 , 21 , 0)
(0, 1, 0) ∼ M
Figure 4.14. Again, Player 1’s mixed strategy space in the coordination game of Figure 4.12.
Player 2 is indifferent between two of his pure strategies and thus his best reply is
a mixed strategy. For example, along the thick line between the point ( 34 , 0, 41 ) and
the intersection point of the thick lines, Player 2 is indifferent between L and R,
thus her best reply takes the form (y, 0, 1 − y). At the central intersection point of
the three thick lines, Player 2 is indifferent between L, C, and R and thus Player 2’s
best reply at this point is a mixed strategy in which all three of L, C, and R are
30
4. A MORE FORMAL TREATMENT OF NORMAL FORM GAMES
(0, 0, 1) ∼ B
LC
3T + 1B
4
4
(1, 0, 0) ∼ T
R
C
R
L
3M + 2B
5
5
(0, 1, 0) ∼ M
2T + 1M
3
3
Figure 4.15. Indifference lines for Player 2 in the game of Figure 4.12, drawn in Player 1’s mixed strategy space.
played with positive probability. To find this point, note that along the line that
makes Player 2 indifferent between L and C (the line extending from point B),
Player 1’s mixed strategy is of the form (2x, x, 1 − 3x). Similarly, along the line
that makes Player 2 indifferent between L and R (the line extending from point
M ), Player 1’s mixed strategy is of the form (3x, x, 1−4x). Thus at the point where
Player 2 is indifferent between all of L, C, and R, we must have 2x = 3(1 − 3x)
6
3
2
3
. Thus Player 1’s mixed strategy at this point is ( 11
, 11
, 11
).
which implies x = 11
(0, 0, 1) ∼ B
R
( 34 , 0, 41 )
(1, 0, 0) ∼ T
L
( 23 , 13 , 0)
(0, 35 , 25 )
C
(0, 1, 0) ∼ M
Figure 4.16. Player 2’s best replies in the game of Figure 4.12,
drawn in Player 1’s mixed strategy space.
To work out all the Nash equilibria of this game, we also need Player 1’s best
reply for any given mixed strategy of Player 2. Fortunately, since the game is
symmetric, we don’t really need to do any working. From what we have already
done, it’s fairly obvious that Player 1’s best reply will be as shown in Figure 4.17.
A Nash equilibrium of this game is a pair (X, Y ) where X is a mixed strategy
of Player 1 and Y is a mixed strategy of Player 2 and X is best for Player 1 against
Y and Y is best for Player 2 against X. We find the equilibria by searching for
them, that is, by checking all of the possible cases and seeing if an equilibrium can
exist in each case.
First, look at the region in Figure 4.16 in which L is Player 2’s only best reply
(that is, not including the thick lines along which Player 2 is indifferent between L
and one of her other pure strategies). Player 2’s best reply is (1, 0, 0). There is an
equilibrium in this region if and only if there is an X in this region that is a best reply
for Player 1 to (1, 0, 0). From Figure 4.17, we see that if Player 2 is playing (1, 0, 0)
3. GAMES WHERE PLAYERS HAVE MORE THAN TWO PURE STRATEGIES
31
(0, 0, 1) ∼ R
B
( 34 , 0, 41 )
(1, 0, 0) ∼ L
T
( 23 , 13 , 0)
(0, 35 , 25 )
M
(0, 1, 0) ∼ C
Figure 4.17. Player 1’s best replies in the game of Figure 4.12,
drawn in Player 2’s mixed strategy space.
then Player 1’s best reply is T . Thus we have (T, L) as a (pure strategy) Nash
equilibrium or ((1, 0, 0), (1, 0, 0)) as the equivalent mixed equilibrium. Similarly,
by looking at the regions in Figure 4.16 in which Player 2’s best reply is R or
C, we find that (M, C) and (B, R) are also (pure strategy) Nash equilibria with
((0, 1, 0), (0, 1, 0)) and ((0, 0, 1), (0, 0, 1)) as the equivalent mixed equilibria.
So far, we’ve found three pure strategy equilibria. What about equilibria in
which one or both of the players plays a nontrivial mixed strategy? First look at
the thick line in Figure 4.16 on which L and C are best replies for Player 2. Is there
an equilibrium in which Player 1’s strategy, X, lies on this line? There is, if and
only if there is some strategy here that is a best reply to some strategy of Player 2
that puts weight only on L and C. Looking at Figure 4.17, strategies of Player 2
that put weight only on L and C lie along the bottom line of the triangle. The
only strategy of Player 1 that is a best reply to a strategy of Player 2 that puts
weight only on L and C and that is on the thick line on which L and C are best
replies for Player 2 is the strategy ( 32 , 13 , 0). Thus one mixed strategy equilibrium is
(( 23 , 13 , 0), ( 32 , 31 , 0)). Similar logic shows (check for yourself) that (( 34 , 0, 41 ), ( 34 , 0, 14 ))
and ((0, 53 , 25 ), (0, 35 , 52 )) are also equilibria. Finally, the central intersection point of
6
3
2
6
3
2
the three thick lines, (( 11
, 11
, 11
), ( 11
, 11
, 11
)) is also a Nash equilibrium.
We’ve found the Nash equilibria of the game in Figure 4.12 by analysing the
best replies of the two players. This is a good way to understand what is going on,
but it is rather tedious. Now let’s see if we can find the equilibria computationally.
First, we define the support of a mixed strategy to be the set of pure strategies that
are played with strictly positive probability in the mixed strategy.
In this game, there are 7 possible supports for each player. For example, for
Player 1, the possible supports are {T }, {M }, {B}, {T, M }, {T, B}, {M, B}, and
{T, M, B}. In general, if a set has n elements then there are 2n possible subsets
(including the empty set) and 2n − 1 nonempty subsets. Since Player 1 has 3 pure
strategies, the number of possible supports is 23 − 1 = 7. Thus there are 7 × 7 = 49
possible combinations for us to consider. This sounds like a lot of work. However
we can eliminate many of these possibilities very quickly. For example, consider the
support {T, M } for Player 1 and {C, R} for Player 2. Are there any equilibria with
these supports? No. Looking at the payoffs in Figure 4.12, if Player 1 puts strictly
positive weight on T and M , Player 2 will not want to put any weight at all on R.
Thus there are no equilibria with these supports. More generally, Player 1 will not
play T with strictly positive probability unless Player 2 plays L with some strictly
positive probability. Similarly, Player 1 will only be willing to play M with strictly
32
4. A MORE FORMAL TREATMENT OF NORMAL FORM GAMES
positive probability if there is some chance of Player 2 playing C, etc. Therefore,
the only possibilities for equilibria in this game are where the two players’ supports
are the “same”, for example if Player 1’s support is {T, M } and Player 2’s support
is {L, C}. Thus there are only 7 possibilities for equilibria that we need to check
carefully.
First consider the case where Player 1’s support is {T }. By the above logic,
there is no possibility for an equilibrium unless Player 2’s support is {L}. These two
supports give us only one possible strategy profile: (T, L), which is a pure strategy
Nash equilibrium. Similarly, we can get the other pure strategy Nash equilibria
(M, C) and (B, R).
Now suppose that Player 1’s support is {T, M }. By the logic above, there is no
possibility for an equilibrium unless Player 2’s support is {L, C}. Suppose Player 1
plays T with probability x and M with probability 1 − x and Player 2 plays L
with probability y and C with probability 1 − y. Since Player 2 plays L and C
with strictly positive probability, the expected payoff from playing L must equal
the expected payoff from playing C. That is,
1 · x + 0 · (1 − x) = 0 · x + 2 · (1 − x)
which gives
x = 23 .
Similarly, since the game is symmetric, Player 1 is indifferent between T and M if
and only if y = 32 . Thus we’ve found a mixed strategy equilibrium: (( 23 , 13 , 0), ( 23 , 13 , 0)).
Similarly (check it for yourself), by considering the other supports, we can show that
(( 43 , 0, 41 ), ( 43 , 0, 14 )) and ((0, 53 , 25 ), (0, 53 , 25 )) are also mixed strategy Nash equilibria.
One final case remains, which is where Player 1’s support is {T, M, B} and
Player 2’s support is {L, C, R}. Suppose Player 1 plays T with probability x1 , M
with probability x2 and B with probability 1 − x1 − x2 , and Player 2 plays L with
probability y1 , C with probability y2 and R with probability 1 − y1 − y2 . Since
Player 1 plays all of T , M , and B with strictly positive probability, the expected
utility from each of T , M , and B must be the same. That is,
1 · x1 = 2 · x2 = 3 · (1 − x1 − x2 ).
Similarly, Player 2’s expected utility from each of L, C, and R must also be the
same. That is,
1 · y1 = 2 · y2 = 3 · (1 − y1 − y2 ).
Solving these equations (again, check it yourself) gives the mixed strategy equilib6
3
2
6
3
2
rium (( 11
, 11
, 11
), ( 11
, 11
, 11
)).
As another example of a game in which there are two players with three strategies each, consider the game of “rock-paper-scissors” shown in Figure 4.18. The
payoffs are such that if the outcome is a tie, both get zero, otherwise the winner
gets $1 from the loser. Intuition suggests that when playing this game, each player
needs to conceal completely from the other player what he or she is doing. Thus
(( 31 , 13 , 13 ), ( 13 , 31 , 13 )) should be the only Nash equilibrium.
Player 2
R
P
S
0, 0
−1, 1
1, −1
Player 1 P
1, −1
0, 0
−1, 1
S
−1, 1
1, −1
0, 0
R
Figure 4.18. The game of “rock-paper-scissors”.
3. GAMES WHERE PLAYERS HAVE MORE THAN TWO PURE STRATEGIES
33
To confirm this, let’s work out the best reply structure. Figure 4.19 shows
Player 2’s mixed strategy space. Suppose Player 2 plays R with probability yR , P
with probability yP and S with probability yS (= 1 − yR − yP ). Then Player 1 is
indifferent between R and P if
−yP + yS = yR − yS
which gives
2yS = yP + yR ,
or, since
yR + yP + yS = 1,
we have
1
.
3
This implies that Player 2 is playing S with probability 13 . Thus along the horizontal
line inside the triangle in Figure 4.19, Player 1 is indifferent between R and P .
Above this line Player 1 prefers R over P and below this line Player 1 prefers P
over R. Similarly, by considering when Player 1 is indifferent between R and S and
between P and S, we get the other lines inside the triangle in Figure 4.19.
yS =
(0, 0, 1) ∼ S
S
P
R
P
R
(1, 0, 0) ∼ R
S
(0, 1, 0) ∼ P
Figure 4.19. Working out Player 1’s best replies in the “rockpaper-scissors” game, drawn on Player 2’s mixed strategy space.
From Figure 4.19, we get Player 1’s best reply for any given mixed strategy
of Player 2. This is shown in Figure 4.20. To Obtain this figure we examine each
of the 6 regions our lines have divided the simplex in Figure 4.19 into and in each
region check which strategy is best for Player 1. For example, in the diamond
shaped region at the top of the figure we see that S is better than P , R is better
than S, and R is better than P . Thus R is the best. In the small triangle at the
bottom in the middle of the RP line we see that S is better than R, P is better
than R and P is better than S. Thus P is the best. The remaining comparisons
are left for you to complete.
Since the game is symmetric, Player 2’s best reply for any given mixed strategy
of Player 1 is the same. We can now search for equilibria. For example, suppose
Player 1 plays S, then Player 2’s best reply is R, to which Player 1’s best reply is
P . Thus there is no Nash equilibrium in which Player 1 plays S. Similarly we can
show that there are no Nash equilibria in which either player plays a pure strategy.
What about along the line where Player 2 plays only combinations of R and
S (that is, the left hand side of the simplex shown in Figure 4.20)? But to such
a mixed strategy, Player 1’s best reply is either P or R or a mixture of {P, R},
to which Player 2’s best reply is not {R, S}. (To see this, you need to think of
34
4. A MORE FORMAL TREATMENT OF NORMAL FORM GAMES
(0, 0, 1) ∼ S
R
S
P
(1, 0, 0) ∼ R
(0, 1, 0) ∼ P
Figure 4.20. Player 1’s best reply in the “rock-paper-scissors”
game, for any mixed strategy of Player 2.
the analog of Figure 4.20 giving Player 2’s best reply as a function of Player 1’s
mixed strategy, which will, in this game, look exactly the same.) Thus there are no
equilibria in which Player 2 plays only combinations of R and S.
With similar logic we can show that there are no equilibria in which either
player plays only two of their strategies with positive probability. Only the pair
of “dots” in the middle of the triangles, corresponding to the strategy profile
(( 13 , 13 , 13 ), ( 13 , 31 , 13 )), is a Nash equilibrium of this game.
4. Formal definitions of Nash equilibrium and best reply
correspondences
Recall that a strategic form game is (N, S, u). A mixed strategy, σn , of a player
n
n ∈ N is a distribution over Sn . That is, σn = (σn (s1n ), . . . , σn (sK
n )) where Kn is
the number of pure strategies of player n. We use Σn to denote the set of all of
player n’s mixed strategies. We call this the mixed strategy space of player n. We
can then define a payoff function for player n over mixed strategies. This is given
by
X
X
Y
un (σ1 , . . . , σN ) =
···
σn (s′n )un (s1 , s2 , . . . , sN ).
s1 ∈S1
sN ∈SN n′ =1,2,...,N
As an example, consider the coordination game from Figure 4.12. In this game,
N = 2, S1 = {T, M, B}, S2 = {L, C, R},
X
X
σ1 (s1 ) σ2 (s2 ) u1 (s1 , s2 ) ,
u1 (σ1 , σ2 ) =
s1 ∈{T,M,B} s2 ∈{L,C,R}
and
u2 (σ1 , σ2 ) =
X
X
σ1 (s1 ) σ2 (s2 ) u2 (s1 , s2 ) ,
s2 ∈{L,C,R} s1 ∈{T,M,B}
where the payoffs u1 (s1 , s2 ) and u2 (s1 , s2 ) are given by the numbers in the table
in Figure 4.12. Note that given the players are playing mixed strategies σ1 and σ2 ,
σ1 (s1 ) σ2 (s2 ) gives the probability that the strategy profile (s1 , s2 ) will be played.
∗
Definition. Given a game (N, S, u), a mixed strategy profile (σ1∗ , σ2∗ , . . . , σN
)
is a Nash equilibrium if for each n ∈ N and each sn ∈ Sn ,
∗
∗
∗
∗
.
> un σ1∗ , σ2∗ , . . . , sn , σn+1
, . . . , σN
un σ1∗ , σ2∗ , . . . , σn∗ , σn+1
, . . . , σN
PROBLEMS
35
Definition. Given a game (N, S, u), let Σn be the mixed strategy space of
player n and Σ = Σ1 × Σ2 × · · · ΣN be the space of mixed strategy profiles. Then
the best reply correspondence of player n, BRn is defined as
BRn (σ1 , . . . , σN ) = {σ n ∈ Σn |un (σ1 , . . . , σ n , . . . , σN )
≥ un (σ1 , . . . , sn , . . . , σN ) for any sn ∈ Sn }.
This allows us to restate the definition of a Nash equilibrium.
∗
Definition. Given a game (N, S, u), a mixed strategy profile (σ1∗ , σ2∗ , . . . , σN
)
is a Nash equilibrium if for each n ∈ N ,
∗
σn∗ ∈ BRn (σ1∗ , σ2∗ , . . . , σN
).
Problems
Problem 4.3. Consider the following two person game.
Player 2
X
Y
Player 1
A
9, 9
0, 8
B
8, 0
7, 7
(1) Suppose that Player 1 thinks that Player 2 will play her strategy X with
probability y and her strategy Y with probability 1 − y. For what value
of y will Player 1 be indifferent between his two strategies?
(2) If y is less than this value what strategy will Player 1 prefer? If y is greater
than that value?
(3) Repeat this analysis with the roles of the players reversed. (To facilitate
the extensions below, suppose that Player 2 thinks that Player 1 will play
his strategy A with probability x and his strategy B with probability
1 − x.)
Problem 4.4. Now consider the following two person game.
Player 2
X
Y
Player 1
A
1, 1
0, 0
B
0, 0
7, 7
(1) Again suppose that Player 1 thinks that Player 2 will play her strategy X
with probability y and her strategy Y with probability 1 − y. For what
value of y will Player 1 be indifferent between his two strategies? Let this
value of y be called ȳ.
(2) If y is less than ȳ what (pure) strategy will Player 1 prefer? If y is greater
than ȳ? What mixed strategies will these pure strategies be equivalent
to?
(3) Graph, as a function of y, the mixed strategy of Player 2, what the best
values of x, the mixed strategies of Player 1, are. We call this the best
36
4. A MORE FORMAL TREATMENT OF NORMAL FORM GAMES
reply correspondence of Player 1 and label it BR1 (·). That is BR1 (y)
gives, for a particular value of y, all those values of x for which (x, 1 − x) is
a best mixed strategy for Player 1 given Player 2 plays the mixed strategy
(y, 1 − y).
(4) Repeat this analysis with the roles of the players reversed. Let the probability that Player 2 assesses that Player 1 will play X be x.
Problem 4.5. In the previous problem you found two graphs. In this problem
you are asked to put the two graphs together. If you think about what is going
on you will see that these graphs show us precisely the Nash equilibria in mixed
strategies.
(1) Consider the graph of the best reply correspondence BR1 (·) that you
found in the previous problem. Explain what it means for a point (y, x)
to lie on the graph. What does it mean for the point (x, y) to lie on the
graph of BR2 (·)?
(2) Now draw both graphs on the same diagram. This will involve switching
the axes in one of the diagrams. Just so we all draw the same pictures,
lets put x, the mixed strategy of Player 1, on the horizontal axis.
(3) Mark on the graph all points that lie on both graphs.
(4) Explain what it means for a point to lie on both graphs and explain why
such a point is a Nash equilibrium.
Problem 4.6. Consider the following two person game.
Eve
Adam
L
R
T
9, 8
0, 0
B
6, 6
8, 9
(1) Find all the pure strategy equilibria for this game.
(2) There is one other equilibrium, in mixed strategies. In this equilibrium
both players play both of their pure strategies with strictly positive probability. Find that equilibrium and explain precisely why there are no other
equilibria, that is, explain why there are no equilibria in which one player
plays a strictly mixed strategy and the other plays a pure strategy.
(3) Graph the best reply correspondences and mark the three equilibria that
you found.
Problem 4.7. Now suppose that the payoffs become the same for each player
in the two pure strategy equilibria. That is, the game becomes as shown below.
Eve
Adam
L
R
T
9, 9
0, 0
B
6, 6
9, 9
(1) Find all the equilibria for this game, both pure and mixed.
PROBLEMS
37
(2) Graph the best reply correspondences, as you did in Homework 2, and
mark the three equilibria that you found.
(3) In the game of the previous problem one might imagine the players negotiating over the equilibria, with Adam arguing for (T, L) and Eve for
(B, R). If you were Adam in the game of Problem 2, i.e., where the payoffs
were the same in each of the pure strategy equilibria, which equilibrium
would you prefer to play, and why?
Problem 4.8. Consider the following two person game.
Barbara
S
H
B
S
9, 9
0, 8
0, 7.5
Alan H
8, 0
7, 7
8, 7.5
7.5, 0
7.5, 8
7.5, 7.5
B
This is a version of the stag hunt game in which the players have another
option, staying in bed. This is better than hunting hares if the other player is
hunting hares, but not as good as hunting hares if the other player is not.
(1) Find all the pure strategy equilibria for this game.
(2) Draw a diagram of the mixed strategy space of Barbara and mark on it
the best responses for Alan.
(3) Draw the same diagram for the mixed strategy space of Alan with the
best responses of Barbara and find all the Nash equilibria of this game.
Problem 4.9. Consider the following two person game.
Player 2
Player 1
X
Y
A
1, 1
1, 1
B
1, 1
0, 0
(1) Graph the best responses of Player 2 to Player 1’s mixed strategy. Pay
particular attention to the best responses when Player 1 plays A, that is,
when σ1 (A) = 1.
(2) Graph the best responses of Player 1 to Player 2’s mixed strategy.
(3) Place the two graphs that you found in the previous parts on the one
diagram and find the set of all equilibria.
(4) What strategy would you play if you were Player 1?
Problem 4.10. Consider the following two person game.
Player 2
Player 1
L
R
T
2, 0
1, 3
B
1, 5
4, 2
38
4. A MORE FORMAL TREATMENT OF NORMAL FORM GAMES
(1) Are there any pure strategy equilibria in this game? If so, find them. If
not, explain why there are not.
(2) Are there any additional equilibria in mixed strategies? If so, find them.
If not, explain why there are not.
(3) Graph the best reply correspondences and mark all the equilibria that
you found. Explain, in terms of the figure you have drawn, why each
equilibrium is an equilibrium.
Problem 4.11. Consider the game below in which each player has three strategies.
X
Eve
Y
T
3, 3
1, 1
1, 1
Adam M
1, 1
4, 2
2, 4
B
1, 1
2, 4
4, 2
(1) Are there any pure strategy equilibria in this game? If so, find them. If
not, explain why there are not.
(2) Draw a simplex (triangle) representing the mixed strategy space of Player
1. Carefully mark on it the regions in which each pure strategy of Player
2, and each subset of pure strategies, is best. Briefly explain how you
constructed each region. You should explain how you drew each line you
use and how the lines you draw are used to find the regions. If some
combinations of pure strategies are never best explicitly say that. Draw
another simplex representing the mixed strategy space of Player 2 and
carefully mark the regions in which each pure strategy of Player 1, and
each subset of pure strategies, is best.
(3) Carefully explain, in terms of the diagrams you have drawn, what the
mixed strategy equilibria (including those equivalent to pure strategy
equilibria) of this game are. Your explanation should show both why
the strategy profiles you claim are equilibria are equilibria and why there
are no other equilibria than the ones you have listed.
Problem 4.12. This problem is a bit more advanced that anything else that
we’ll do in these notes. Consider again the game of Problem 4.9. In this problem
we shall consider, somewhat informally, the robustness of the equilibria that you
found there.
(1) Consider again the best reply correspondence of Player 1. Let us look at
correspondences, F , that are “close to” the best reply correspondence in
the following sense.
(a) For every mixed strategy σ2 of Player 2 the set F (σ2 ) is close to the
set BR1 (σ2 ). (That is, every point in F (σ2 ) is close to some point in
the set BR1 (σ2 ) and every point in BR1 (σ2 ) is close to some point
in the set F (σ2 ). This means, among other things that there is some
point in F (σ2 ) for every value of σ2 .)
(b) For every value of σ2 the set F (σ2 ) is convex. In our case that means
that the set is an interval.
PROBLEMS
39
(c) The graph of the correspondence is a closed set. That is, the graph,
the set of points (σ2 , σ1 ) such that σ1 is one of Player 1’s best replies
to the strategy σ2 of Player 2, is a set that contains all the points on
its boundary.
Draw an example of such an F . What seem to be the essential features
of such F ?
(2) Repeat for the best reply correspondence of Player 2.
(3) What equilibria of the original game are such that if we replace the best
reply correspondences by “close” correspondences we will still have at least
one equilibrium close to the original equilibrium?
CHAPTER 5
Two Player Constant Sum Games
In the previous chapter we looked at the equilibria of normal form games. We
looked in detail only at two player games, and only at games in which the players
had either two or three strategies. However the ideas we developed are more general
and can be extended to games with more players and in which the players have more
strategies in a straightforward way.
In this chapter we shall look at a special class of two player game in which
Player 2 loses exactly what Player 1 gains. Thus the total payoff of the two players
remains constant whatever the outcome of the game. Often the level of the total
payoff is normalised to be zero and the games are called zero sum games, but
this is not an important feature. Such games are also sometimes described are
strictly competitive games, emphasising the fact that the incentives of the players
are strictly opposed.
Two player constant sum games were the focus of much of the earliest work
in game theory such as the work of Borel and von Neumann in the 1920s that we
briefly discussed in the introduction.
1. Definition
We begin with the obvious definition of two player constant sum games.
Definition. A two player normal from game (N, S, u) is said to be a two player
constant sum game if N contains exactly two distinct elements which we shall call
1 and 2 and u1 (s) + u2 (s) is constant for all s in S, that is does not depend on the
particular strategy profile s. If the constant is zero then the games may be called
zero sum games.
We have already seen examples of two player constant sum games. Matching
pennies and Rock-paper-scissors were two player constant sum games (indeed zero
sum games).
When we consider two person zero sum games there is perhaps not need to
write both payoffs, since the payoff to Player 2 is simply the negative of the payoff
to Player 1. The game in Figure 5.1 is such a game.
W
Player 2
X
Y
Z
A
12
2
7
3
B
8
6
9
7
Player 1 C
3
5
11
6
D
2
4
0
11
E
9
0
3
13
Figure 5.1. A Zero Sum Game
41
42
5. TWO PLAYER CONSTANT SUM GAMES
For each strategy of Player 1 let us see what is the least that Player could
get, that is, for each row what is the lowest or minimum number in that row. For
each strategy of Player 1 that minimum number represents what he is guaranteed
by playing that strategy. He might get more, but he is guaranteed at least that
minimum. If we now look for the strategy of Player 1 that gives the maximum od
those minima we see that if Player 1 were to play that strategy he would guarantee
that he would receive no less than that minimum. Surely he would expect no less
than that amount from playing the game. Similarly for each strategy of Player 2
let us see what is the most that Player could lose, that is, for each column what is
the highest or maximum number in that column. Again, if we look at the minimum
of those maxima then Player 2 could guarantee that she would lose no more than
that amount. If the amount that Player 1 can guarantee receiving is the same as
the amount that Player 2 can guarantee losing no more then then this would seem
to be the mutual value of the game to the players.
Let us look again at the game of Figure 5.1. In Figure 5.2 we add to that game
a column giving the minimum of the other columns and a row giving the maximum
of the other rows. We see that if Player 1 plays B he guarantees that he will receive
at least 6, while if Player 2 plays X she guarantee that she will lose no more than
6. We say that the value of this game is 6. One might argue that in such a game
since Player 1 can guarantee that he gets at least 6 and Player 2 can guarantee that
she loses at most 6 then Player 2 “paying” 6 to Player 1 must be the outcome of
the game. We call B the pure strategy maxmin strategy and strategy X the pure
strategy minmax strategy.
Player 1
W
Player 2
X
Y
Z
A
12
2
7
3
2
B
8
6
9
7
6
C
3
5
11
6
5
D
2
4
0
11
0
E
9
0
3
13
0
max
12
6
11
13
min
Figure 5.2. A Zero Sum Game with Pure Strategy Maxmin Equal
to Minmax
If the game is such that the minimum amount that Player 1 can guarantee
himself is the same as the maximum that Player 2 can guarantee she will pay no
more than then we call this common amount the value of the game. In the example
we have just analysed these two amounts are indeed the same. But are they always
the same? No. We have already seen an example in which they are not. Matching
pennies is a constant sum game (indeed a zero sum game). In Figure 5.3 we give
the matching pennies game using the convention that Player 2’s payoffs are not
given explicitly but rather are the negative of Player 1’s payoffs which are shown.
Notice that in each row the minimum payoff across the columns is −1 and so the
maximum of these minimums is also −1. Similarly in each column the maximum
payoff across the rows in 1 and so the minimum of these maximums is also 1. The
amount that Player 1 can guarantee using pure strategies is −1 while the amount
that Player 2 can guarantee is 1.
1. DEFINITION
43
Player 2
H
T
Player 1
H
1
−1
T
−1
1
Figure 5.3. Matching pennies.
However if each Player can use mixed strategies they can do better. If Player 1
plays the mixed strategy ( 12 , 12 ) then whichever pure strategy Player 2 uses Player
1 will obtain an expected payoff of 0. And hence also is Player 2 uses a mixed
strategy Player 1 will obtain 0. If Player 1 uses any other mixed strategy there
is some strategy of Player 2 that will lead Player 1 to get less than 0. Hence the
mixed strategy ( 21 , 12 ) is the unique strategy that maximises over values of σ1
min u(σ1 , σ2 ) = σ1 (H)σ2 (H) − σ1 (H)σ2 (T ) − σ1 (T )σ2 (H) + σ1 (T )σ2 (T ).
σ2
In fact, we do not need to consider mixed strategies for Player 2. For any mixed
strategy of Player 1
min u(σ1 , σ2 ) = min u(σ1 , s2 ).
σ2
s2 ∈S2
Similarly for Player 2 the mixed strategy ( 12 , 12 ) is the unique strategy that
minimises over values of σ2
max u(σ1 , σ2 ) = max u(s1 , σ2 ).
σ1
s1 ∈S1
When we alow players to used mixed strategies in this way the value that Player
1 can guarantee is necessarily the same as the value that Player 2 can guarantee. We
call the strategies that solve the above maximisation and minimisation problems
the optimal strategies of Players 1 and 2.
In a two player constant sum game if one of the players has only two strategies
however many strategies the other player has there is an easy graphical way to find
the optimal strategy for the Player who has only two pure strategies. And having
done that it’s then easy to find the optimal strategy for the other player. Consider
the game in Figure 5.4
Player 2
L R
T
8
0
Player 1 M
0
7
B
6
6
Figure 5.4. Another Zero Sum Game.
We want to find the value of σ2 that minimises
max s1 ∈ {T, M, B}u(s1 , σ2 ).
To see what’s going on, let us graph the expected payoff to each of Player 1’s
pure strategies as a function of σ2 (L).
44
5. TWO PLAYER CONSTANT SUM GAMES
u1 (s1 , σ2 (L))
6
8
7
6
s1 = T
s1 = B
1
7
3
4
s1 = M
1 σ2 (L)
From the graph, we can see that when σ2 (L) is in the interval [0, 1/7) Player 1’s
strategy M achieves the maximum, when σ2 (L) is in the interval (1/7, 3/4) Player
1’s strategy B achieves the maximum, and when σ2 (L) is in the interval (3/4, 1]
Player 1’s strategy T achieves the maximum. We also see that Player 2 minimises
what she has to pay to Player 1 by choosing a value of σ2 (L) is in the interval
(1/7, 3/4) in which case she pays no more than 6. Against such a strategy the best
that Player 1 can do is to play the pure strategy B.
From this example we see that optimal strategies are not necessarily unique.
We state now a number of facts about optimal strategies.
Fact. In a two person constant sum game for each player the set of optimal
strategies is a nonempty convex subset of his mixed strategies.
Fact. In a two person constant sum game, if (σ1 , σ2 ) is a Nash equilibrium
then σ1 is an optimal strategy for Player 1 and σ2 is an optimal strategy for Player
2.
Fact. In a two person constant sum game, if σ1 is an optimal strategy for
Player 1 and σ2 is an optimal strategy for Player 2 then (σ1 , σ2 ) is a Nash equilibrium.
The previous two facts imply that
Fact. In a two person constant sum game, if (σ̄1 , σ̄2 ) and (σ1∗ , σ2∗ ) are Nash
equilibria then so are (σ̄1 , σ2∗ ) and (σ1∗ , σ̄2 ) .
Recall that this last fact is not necessarily true for nonconstant sum games.
CHAPTER 6
Extensive Form Games
Again, we begin our discussion of extensive form games without formally defining what one is, but by giving some examples. The way we describe extensive form
games, and later define them, is a model developed by Harold Kuhn, a modification of the definition of von Neumann and Morgenstrn, together with a graphical
description that has been found by almost all game theorists to be very useful.
1. Examples of extensive form games
We interpret Figure 6.1 as follows. Each point where a player gets to move in
the game or at which the game ends is called a node. Nodes at which players move
are called decision nodes and shown by small black dots. Every decision node is
labeled with the name of the player who gets to move. Lines that connect two nodes
are called branches and represent moves available to players. The game starts at
a particular node, called the initial node or root. The nodes at which the game
ends are called terminal nodes. To each terminal node we associate a payoff to
each player. These payoffs tell us how the player evaluates the game ending at that
particular node.
4, 3, 7, 11
2, 9, 16, 3 10, 1, 9, 8
X
Y
1, 1, 1, 1
X
1, 10, 8, 9
Y
1, 1, 1, 1
4
U
D
A
2
B
3
L
R
1
Figure 6.1. An extensive form game.
In the game of Figure 6.1 we assume that the lowest node where Player 1 moves
is the initial node. Player 1 chooses between L or R. If Player 1 chooses L then
it is Player 2’s turn to move and choose between U or D. If Player 1 chooses R
then Player 3 moves and chooses between A and B. If Player 2 chooses U then the
game ends. If Player 2 chooses D then player 4 moves. If player 3 chooses B then
the game ends. If player 3 chooses A then Player 4 moves. When it’s Player 4’s
turn to move, he doesn’t see whether he is called upon to move because Player 1
45
46
6. EXTENSIVE FORM GAMES
chose L and Player 2 chose D or because Player 1 chose R and Player 3 chose A.
When a player cannot distinguish between several nodes at which he or she gets to
move we say that these two nodes are in the same information set of that player.
Thus, the two nodes at which player 4 moves are in the same information set and
we represent this by joining them with a dotted line. If it’s player 4’s turn to move,
he chooses between X and Y , after which the game ends.
−2, 2
2, −2 2, −2
S
D
1, −1
−2, 2
S
D
−1, 1
1
OUT
IN
In
2
Out
1
1
2
1
2
Nature
Figure 6.2. Another extensive form game. This game does not
have perfect recall.
As another example, consider the extensive form game in Figure 6.2. In this
game, the first mover is not a player but “Nature”. That is, at the beginning of
the game, there is a random selection of whether Player 1 or Player 2 gets to move,
each being chosen with probability 12 . Nodes at which Nature moves are indicated
by open circles and branches issuing from these nodes are labeled with probabilities
with which those moves are made. If Player 1 is chosen, Player 1 gets to choose
between In and Out, and if Player 2 is chosen, Player 2 gets to choose between IN
and OUT. If, once chosen to move, either Player 1 chooses Out or Player 2 chooses
OUT then the game ends. If Player 1 made the choice then Player 1 gets −1 and
Player 2 gets 1 while if Player 2 made the choice, Player 1 gets 1 and Player 2
gets −1. On the other hand, if either Player 1 chose In or Player 2 chose IN then
Player 1 gets to move again, but Player 1 does not know the identity of the player
who moved previously. This may seem a bit bizarre because it means that Player 1
forgets whether he moved previously in the game. Thus we say that this is a game
without perfect recall (see Section 3 below). One way to rationalise this is to think
of Player 1 as a ‘team’. So the first time Player 1 gets to move (if Player 1 was
chosen by Nature), it is one person or agent in the “Player 1 team” moving, and
the second time it is another agent in the “Player 1 team”, and neither agent in the
Player 1 team can communicate with the other. Finally, if Player 1 gets to move
for the second time, he gets to choose between S or D, and the appropriate payoffs
are given as in Figure 6.2.
2. Definition of an extensive form game
The essential part of a formal definition of an extensive form game is the
game tree. Formally, a game tree, g, is a finite connected graph with no loops
2. DEFINITION OF AN EXTENSIVE FORM GAME
47
and a distinguished initial node. What does this mean? A finite graph is a finite set of nodes, X = {x1 , x2 , . . . , xK } and a set of branches connecting them.
A branch is a set of two different nodes, {xi , xj } where xi 6= xj . In a game
tree of Figure 6.3, the set of nodes is X = {x1 , . . . , x9 } and the branches are
{x1 , x2 }, {x2 , x4 }, {x2 , x5 }, {x1 , x3 }, {x3 , x6 }, {x3 , x7 }, {x4 , x8 }, and {x4 , x9 }. The
initial node, or root, is x1 .
x6
x7
x3
x8
x9
x5
x4
x2
x1
Figure 6.3. A game tree.
We want to rule out trees that look like those in Figure 6.4. That is, we don’t
want to have trees with loops or trees with more than one initial node. A path is a
sequence of branches:
({x1 , x2 }, {x2 , x3 }, . . . , {xT −1 , xT }).
For example, in Figure 6.3, one path is ({x7 , x3 }, {x3 , x6 }). Then, to avoid problem
trees like those in Figure 6.4, we shall assume that every node except the root is
connected to the root by one and only one path.
x4
x2
x3
x4
x1
x2
x3
x1
Figure 6.4. Not game trees.
The terminal nodes in Figure 6.3 are {x5 , x6 , x7 , x8 , and x9 }. We denote the
set of terminal nodes by T . A node xt is a terminal node if there is a path
({x0 , x1 }, {x1 , x2 }, . . . , {xt−1 , xt }), where x0 is the initial node and there are no
paths that extend this.
48
6. EXTENSIVE FORM GAMES
An alternative way of describing game trees is by using the predecessor function.
Given a finite set of nodes X, we define the (immediate) predecessor function
p : X → X ∪ {∅},
to be the function that for every node gives the node that comes immediately before
in the game tree. For example, in Figure 6.3, p(x2 ) = x1 , p(x4 ) = x2 , p(x7 ) = x3 ,
and so on. To make sure that a game tree has only one initial node, we require that
there is a unique x0 ∈ X such that p(x0 ) = ∅, that is, there is only one node with no
immediate predecessor. To prevent loops we require that for every x 6= x0 , either
p(x) = x0 or p(p(x)) = x0 or . . . p(p(p . . . p(x))) = x0 . That is, by applying the
immediate predecessor function p to any node except for the initial node, eventually
(in a finite number of steps) we get to the initial node x0 . Also, we say that xt is
a terminal node if there is no node y ∈ X with p(y) = xt .
Now we can give a formal definition of an extensive form game.
Definition. An extensive form game consists of:
(1) A finite set of players, N = {1, 2, . . . , N }
(2) A finite set of nodes, X, and X is a game tree, where T ⊂ X is the set of
terminal nodes and x0 is the initial node.
(3) A set of actions, A, and a labeling function α : X\{x0 } → A where α(x)
is the action at the predecessor of x that leads to x. If p(x) = p(x′ ) and
x 6= x′ then α(x) 6= α(x′ ).
(4) A collection of information sets, H, and a function H : X\T → H that
assigns for every node, except the terminal ones, which information set
the node is in.
(5) A function n : H → N ∪ {0}, where player 0 denotes Nature. That is,
n(H) is the player who moves at information set H. Let Hn = {H ∈
H|n(H) = n} be the information sets controlled by player n.
(6) ρ : H0 × A → [0, 1] giving the probability that action a is taken at the
information set H of Nature. That is, ρ(H, a).
(7) (u1 , . . . , uN ) with un : T → R being the payoff to player n.
First, let C(x) = {a ∈ A|a = α(x′ ) for some x′ with p(x′ ) = x}. That is, C(x)
is the set of choices that are available at node x. Note that if x is a terminal node
then C(x) = ∅. If two nodes x and x′ are in the same information set, that is,
if H(x) = H(x′ ), then the same choices must be available at x and x′ , that is,
C(x) = C(x′ ).
Rooming
House
Home
L
R
x1
Cliff
L
R
x0
Figure 6.5. The forgetful driver.
Second, consider the “forgetful driver game” illustrates in Figure 6.5. A student
is driving home after spending the evening at a pub. He reaches a set of traffic lights
3. PERFECT RECALL
49
(node x0 ) and can choose to go left (L) or right (R). If he goes left, he falls off a
cliff. If he goes right, he reaches another set of traffic lights (node x1 ). However
when he gets to this second set of traffic lights, since he’s had a few drinks this
evening, he cannot remember if he passed a set of traffic lights already or not. At
the second traffic lights he can again either go left or right. If he goes left he gets
home and if he goes right he reaches a rooming house. The fact that he forgets at
the second set of traffic lights whether he’s already passed the first set is indicated
in Figure 6.5 by the nodes x0 and x1 being in the same information set. That is,
H(x0 ) = H(x1 ). We will assume that if two nodes are in the same information
set then neither should be a predecessor of the other. Then, this is not a proper
extensive form game.
Finally, we’ll assume that all H ∈ H0 are singletons, that is, sets consisting of
only one element. In other words, if H ∈ H0 and H(x) = H(x′ ) then x = x′ . This
says that Nature’s information sets always have only one node in them.
3. Perfect recall
Informally, a player has perfect recall if he remembers everything that he did
and everything that he knew in the past.
The informal definition is already quite helpful for figuring out whether a player
has perfect recall. If the player has only one information set then she has perfect
recall. The player effectively has no past so she has nothing to remember! We
can generalize this simple observation a little bit. If it is never the case that two
different information sets of the player both occur in any single play of the game
then the player has perfect recall.
If every information set is a singleton we say that the game has perfect information. In this case a player when called upon to move sees exactly what has
happened in the past. If the game has perfect information then each player has
perfect recall. Note, however, that the converse is not necessarily true. Not every
game in which every player has perfect recall is a game that has perfect information.
We will say that the game itself has perfect recall if every player in the game
has perfect recall. Now let us give a formal definition of perfect recall.
Definition. Given an extensive form game, we say that player n in that game
has perfect recall if whenever H(x) = H(y) ∈ Hn , that is, whenever x and y are
in the same information set and player n moves at that information set, with x′ a
predecessor of x with H(x′ ) ∈ Hn , that is, x′ comes before x and player n moves
at x′ , and a is the action at x′ on the path to x, then there is some y ′ ∈ H(x′ ) a
predecessor of y with a the action at y ′ on the path to y.
CHAPTER 7
Solving extensive form games
1. Equilibria of extensive form games
To find the Nash equilibria of an extensive form game, we have two choices.
First, we could find the equivalent normal form game and find all the equilibria
from that game, using the methods that we learned in the previous section. The
only trouble with this is that two different extensive form games can give the same
normal form. Alternatively, we can find the equilibria directly from the extensive
form using a concept (to be explained in subsection 6 below) called subgame perfect
equilibrium (SPE). Note that we cannot find the SPE of an extensive form game
from its associated normal form; we must find it directly from the extensive form.
The two approaches that we can take to finding the equilibria of an extensive form
game are shown in Figure 7.1.
Extensive
form
games
Normal
form
games
1
Normal form
function
Solutions
*
Equilibrium 1
correspondence
Subgame Perfect
Equilibrium correspondence
Figure 7.1. Methods for defining equilibria of extensive form
games. Note that two different extensive form games may have
the same normal form and that a single game may have multiple
equilibria, some of which may be subgame perfect and some of
which may not.
2. The associated normal form
Let us first consider the method for finding equilibria of extensive form games
whereby we find the Nash equilibria of the associated normal form. Consider the
extensive form game shown in Figure 7.2. To find the associated normal form of
this game, we first need to know what a strategy of a player is. As we said before,
51
52
7. SOLVING EXTENSIVE FORM GAMES
a strategy for a player is a complete set of instructions as to how to play in the
game. More formally, we have the following definition.
3, 3
0, 0
U
1, 2
D
1
L
2, 1
R
2
T
B
1
Figure 7.2. Yet another extensive form game.
Definition. A strategy for player n is a function that assigns to each information set of that player a choice at that information set.
Note that this definition doesn’t take account of the fact that certain choices by
the players may render some of their information sets irrelevant. For example, in
Figure 7.2, if Player 1 plays T at her first information set, her second information
set is never reached. However, a strategy for Player 1 must still specify what she
would do if her second information set were reached. Thus, strictly speaking, T U
and T D are distinct strategies for Player 1. However, two strategies that for any
fixed choice of the other player lead to the same outcome are said to be equivalent.
In the game in Figure 7.2, the strategies T U and T D are equivalent for Player 1.
Now that we know what a strategy in an extensive form game is, we can set
about deriving the associated normal form. Figure 7.3 shows the associated normal
form game of the extensive form game in Figure 7.2. Player 1’s strategies are T U ,
T D, BU and BD and Player 2’s strategies are L and R. We then construct the
associated normal form by simply writing all the strategies of Player 1 down the
left hand side and all the strategies of Player 2 across the top, and then filling in
each cell in the matrix with the appropriate payoff according to the strategies that
are played by each player.
Problem 7.1. Find all the Nash equilibria of the game in Figure 7.3.
As another example, consider the extensive form game in Figure 6.2. In this
game, Player 2’s strategy set is S2 = {IN, OUT} and Player 1’s strategy set is
S1 = {(In,S) , (In,D) , (Out,S) , (Out,D)}. In this game, because there is a random
move by Nature at the beginning of the game, a profile of pure strategies of the
player generates a probability distribution over the terminal nodes. To generate the
associated normal form of the game, we then use this probability distribution to
calculate the expected value of the terminal payoffs. Thus the associated normal
form of the extensive form game in Figure 6.2 is shown in Figure 7.4. As an
example of how the payoffs were calculated, suppose Player 1 plays (In,S) and
Player 2 plays IN. In this case, the payoff is (−2, 2) with probability 21 and (2, −2)
2. THE ASSOCIATED NORMAL FORM
53
Player 2
L
Player 1
R
TU
2, 1 2, 1
TD
2, 1 2, 1
BU
1, 2 3, 3
BD
1, 2 0, 0
Figure 7.3. The normal form game corresponding to the extensive form game in Figure 7.2.
with probability 21 . Thus the expected payoffs are 21 · (−2, 2) + 21 · (2, −2) = (0, 0).
If instead Player 1 plays (In,S) and Player 2 plays OUT, the expected payoffs are
1
1
1
1
2 · (1, −1) + 2 · (2, −2) = 1 2 , −1 2 . The other payoffs are calculated similarly.
Player 2
Player 1
IN
OU T
In, S
0, 0
3
3
2, −2
In, D
0, 0
− 21 , 12
Out, S
− 32 , 23
0, 0
Out, D
1
1
2, −2
0, 0
Figure 7.4. The normal form game corresponding to the extensive form game in Figure 6.2.
Let us complete this example by finding all the Nash equilibria of the normal form game in Figure 7.4. Note that for Player 1, the strategies (In,
D) and
(Out, S) are strictly dominated by, for example, the strategy 12 , 0, 0, 21 . To see
this, suppose that Player 1 plays (In, D) and Player 2 plays a mixed strategy
(y, 1− y) where 0 ≤ y ≤ 1. Then Player 1’s expected payoff from (In, D) is 0 · y +
− 12 · (1 − y) = − 21 + 12 y. Suppose instead that Player 1 plays the mixed strategy
1
1
expected payoff from this strategy is 12 0 · y + 1 21 · (1 − y) +
2 , 0, 0, 2 . Player 1’s
1 1
3
1
3
1
1
1
5
2 2 · y + 0 · (1 − y) = 4 − 2 y. Since 4 − 2 y − − 2 + 2 y = 4 − y ≥ 0 for any
1
1
0 ≤ y ≤ 1, 2 , 0, 0, 2 gives Player 1 a higher expected payoff than (In,D) whatever
Player 2 does. Similar calculations show that (Out,S) is also strictly dominated
for Player 1. Thus we know that Player 1 will never play (In,D) or (Out,S) in a
Nash equilibrium. This means that we can reduce the game to the one found in
Figure 7.5.
From Figure 7.5, we see that the game looks rather like that of matching pennies
from Figure 2.7. We know that in such a game there are no pure strategy equilibria.
In fact, there is no equilibrium in which either player plays a pure strategy. So we
only need to check for mixed strategy equilibria. Suppose that Player 1 plays (In,S)
with probability x and (Out,D) with probability 1 − x, and Player 2 plays IN with
probability y and OUT with probability 1 − y. Then x must be such that Player 2
is indifferent between IN and OUT, which implies
0 · x + − 12 · (1 − x) = −1 12 · x + 0. (1 − x)
54
7. SOLVING EXTENSIVE FORM GAMES
Player 2
Player 1
IN
OU T
In, S
0, 0
3
3
2, −2
Out, D
1
1
2, −2
0, 0
Figure 7.5. The normal form game corresponding to the extensive form game in Figure 6.2, after eliminating Player 1’s strictly
dominated strategies.
which implies x = 41 . Similarly, y must be such that Player 1 is indifferent between
(In,S) and (Out,D). This implies that
0 · y + 1 21 · (1 − y) =
1
2
· y + 0 · (1 − y)
which implies y = 43 . So, the only Nash equilibrium of the game in Figure 7.5, and
hence of the game in Figure 7.4 is
3 1 1
3
.
4 , 0, 0, 4 , 4 , 4
Problem 7.2. Consider the normal form game in Figure 7.4 and the Nash
equilibrium strategy that we have just found.
(1) If the players play their equilibrium strategies, what is the expected payoff
to each player?
(2) If Player 1 plays his equilibrium strategy, what is the worst payoff that
he can get (whatever Player 2 does)?
3. Behaviour strategies
Consider the extensive form game shown in Figure 7.6 and its associated normal
form shown in Figure 7.7. In this game, we need three independent numbers to
describe a mixed strategy of Player 2, i.e., (x, y, z, 1 − x − y − z). Suppose that
instead Player 2 puts off her decision about which strategy to use until she is called
upon to move. In this case we only need two independent numbers to describe
the uncertainty about what Player 2 is going to do. That is, we could say that at
Player 2’s left-hand information set she would choose L with probability x and R
with probability 1 − x, and at her right-hand information set she would choose W
with probability y and E with probability 1 − y. We can see that by describing
Player 2’s strategies in this way, we can save ourselves some work. This efficiency
increases very quickly depending on the number of information sets and the number
of choices at each information set. For example, suppose that the game is similar
to that of Figure 7.6, except that Player 1’s choice is between four strategies, each
of which leads to a choice of Player 2 between three strategies. In this case Player 2
has 3·3·3·3 = 81 different pure strategies and we’d need 80 independent numbers to
describe the uncertainty about what Player 2 is going to do! If instead we suppose
that Player 2 puts off her decision about which strategy to use until she is called
upon to move, we only need 4 · 2 = 8 independent numbers. This is really great!
When we say that a player “puts off” his or her decision about which strategy
to use until he or she is called upon to move, what we really mean is that we
are using what is called a behaviour strategy to describe what the player is doing.
Formally, we have the following definition.
Definition. In a given extensive form game with player set N , a behaviour
strategy for player n ∈ N is a rule, or function, that assigns to each information set of
4. KUHN’S THEOREM
3, 3
0, 0
L
55
2, 2
R
1, 4
W
E
2
2
T
B
1
Figure 7.6. An extensive form game.
Player 2
Player 1
LW
LE
RW
RE
T
3, 3
3, 3
0, 0
0, 0
B
2, 2
1, 4
2, 2
1, 4
Figure 7.7. The normal form game corresponding to the extensive form game in Figure 7.6.
that player a probability distribution over the choices available at that information
set.
4. Kuhn’s theorem
Remember we motivated behaviour strategies in subsection 3 as a way of reducing the amount of numbers we need compared to using mixed strategies. You
might be wondering whether we can always do this, that is, if we can represent
any arbitrary mixed strategy by a behaviour strategy. The answer is provided by
Kuhn’s theorem.
Theorem 7.1. (Kuhn) Given an extensive form game, a player n who has perfect recall in that game, and a mixed strategy σn of player n, there exists a behaviour
strategy bn of player n such that for any profile of strategies of the other players
(x1 , . . . , xn−1 , xn+1 , . . . , xN ) where xm , m 6= n, is either a mixed strategy of a behaviour strategy of player m, the strategy profiles (x1 , . . . , xn−1 , σn , xn+1 , . . . , xN )
and (x1 , . . . , xn−1 , bn , xn+1 , . . . , xN ) give the same distribution over terminal nodes.
In other words, Kuhn’s theorem says that, as long as player n has perfect recall,
for any σn there is some bn such that, given what the other players are doing, we
get the same distribution over terminal nodes from σn and bn . To see Kuhn’s
theorem in action, consider the game in Figure 7.8. Suppose Player 1 plays T with
probability p and B with probability 1 − p. Player 2’s pure strategies are XA, XB,
Y A, and Y B. Suppose Player 2 plays a mixed strategy of playing XA and Y B
with probability 12 and XB and Y A with probability 0. Thus, with probability 21 ,
Player 2 plays XA and we get to terminal node x with probability p and to terminal
node a with probability 1−p, and with probability 12 , Player 2 plays Y B and we get
to terminal node y with probability p and terminal node b with probability 1 − p.
This gives a distribution over terminal nodes as shown in the table in Figure 7.9.
56
7. SOLVING EXTENSIVE FORM GAMES
y
x
X
a
Y
b
A
B
2
2
T
B
1
Figure 7.8. An extensive form game.
Terminal node
x
y
a
b
Probability
p/2
p/2
(1 − p) /2
(1 − p) /2
Figure 7.9. Distributions over terminal nodes in the game of Figure 7.8 for the strategy profile given in the text.
Now suppose that Player 2 plays a behaviour strategy of playing X with probability 21 at his left-hand information set and A with probability 21 at his right-hand
information set. Thus we get to terminal node x with probability p · 21 , to terminal
node y with probability p · 21 , to terminal node a with probability (1 − p) · 12 and
to terminal node b with probability (1 − p) · 21 . Just as Kuhn’s theorem predicts,
there is a behaviour strategy that gives the same distribution over terminal nodes
as the mixed strategy.
5. More than two players
3, 3, 3
1, 1, 1
U
6, 2, 7
3
L
2, 7, 1
D
R
2
T
B
1
Figure 7.10. An extensive form game with three players.
6. SUBGAME PERFECT EQUILIBRIUM
57
It is easy to draw extensive form games that have more than two players, such
as the one shown in Figure 7.10, which has three players. How would we find the
associated normal form of such a game? Recall that a normal form game is given
by (N, S, u). For the game in Figure 7.10, we have N = {1, 2, 3}, S1 = {T, B},
S2 = {L, R} and S3 = {U, D}, hence
S = {(T, L, U ) , (T, L, D) , (T, R, U ) , (T, R, D) , (B, L, U ) , (B, L, D) ,
(B, R, U ) , (B, R, D)}.
Finally, the payoff functions of each player are shown in Figure 7.11.
S
u1
u2
u3
T, L, U
2
7
1
T, L, D
2
7
1
T, R, U
2
7
1
T, R, D
2
7
1
B, L, U
6
2
7
B, L, D
6
2
7
B, R, U
3
3
3
B, R, D
1
1
1
Figure 7.11. One way of presenting the normal form game associated with the extensive form game of Figure 7.10.
The form shown in Figure 7.11 is a perfectly acceptable exposition of the associated normal form. However, it’s more convenient to represent this three player
game as shown in Figure 7.12, where Player 3 chooses the matrix.
Player 3
U
D
Player 2
Player 2
L
Player 1
R
L
Player 1
R
T
2, 7, 1 2, 7, 1
T
2, 7, 1 2, 7, 1
B
6, 2, 7 3, 3, 3
B
6, 2, 7 1, 1, 1
Figure 7.12. A more convenient representation of the normal
form of the game in Figure 7.10.
6. Subgame perfect equilibrium
Subgame perfect equilibrium is an equilibrium concept that relates directly to
the extensive form of a game. The basic idea is that equilibrium strategies should
continue to be an equilibrium in each subgame (subgames will be defined formally
below).
Definition. A subgame perfect equilibrium is a profile of strategies such that
for every subgame the parts of the profile relevant to the subgame constitute an
equilibrium of the subgame.
To understand subgame perfect equilibrium, we first need to know what a
subgame is. Consider the game in Figure 7.10. This game has two proper subgames.
Strictly speaking, the whole game is a subgame of itself, so we call subgames that
are not the whole game proper subgames. The first is the game that begins from
Player 2’s decision node and the second is the game that begins from player 3’s
decision node. However, it would be a mistake to think that every game has proper
58
7. SOLVING EXTENSIVE FORM GAMES
subgames. Consider the extensive form game in Figure 7.13, which is the extensive
form version of matching pennies. In this game it doesn’t make sense to say there is
a subgame that begins at either of Player 2’s decision nodes. This is because both
of these nodes are in the same information set. If we said that a subgame started
at either one of them, then we would be saying that Player 2 knew which node she
is at, which violates the original structure of the game.
not a subgame
1, −1
−1, 1
H
−1, 1
T
1, −1
H
T
2
t
h
1
Figure 7.13. The extensive form version of matching pennies.
This game has no proper subgames.
As another example, consider the extensive form game shown in Figure 7.14.
In this game, the game starting from Player 2’s decision node is not a subgame,
because it splits player 3’s information set.
not a subgame
0, 0, 0
U
3, 2, 2
0, 0, 1
D
U
4, 4, 0
D
1, 1, 1
3
L
R
2
T
B
1
Figure 7.14. The game beginning at Player 2’s decision node is
not a subgame because it splits Player 3’s information set.
Let us now give a formal definition of a subgame.
6. SUBGAME PERFECT EQUILIBRIUM
59
Definition. A subgame of an extensive form game Γ is some node in the tree
of Γ and all the nodes that follow it, with the original tree structure but restricted
to this subset of nodes, with the property that any information set of Γ is either
completely in the subgame or completely outside the subgame. The rest of the
structure is the same as in Γ but restricted to the new (smaller) tree.
Notice that under this definition, Γ is a subgame of Γ. (Simply take the chosen
node in the definition of a subgame to be the initial node of Γ.) We call the
subgames that are not Γ itself the proper subgames of Γ.
And we are now in a position to give a more formal definition of subgame
perfect equilibrium.
Definition. A subgame perfect equilibrium of an extensive form game Γ with
perfect recall is a profile of behaviour strategies (b1 , b2 , . . . , bN ) such that, for every
subgame Γ′ of Γ, (b′1 , b′2 , . . . , b′N ) is a Nash equilibrium of Γ′ where b′n is bn restricted
to Γ′ .
6.1. Finding subgame perfect equilibria. An example of the simplest type
of extensive form game in which subgame perfect equilibrium is interesting is shown
in Figure 7.15. Notice that in this game, (T, R) is a Nash equilibrium (of the whole
game). Given that Player 1 plays T , Player 2 is indifferent between L and R. And
given that Player 2 plays R, Player 1 prefers to play T . There is something strange
about this equilibrium, however. Surely, if Player 2 was actually called upon to
move, she would play L rather than R. The idea of subgame perfect equilibrium is
to get rid of this silly type of Nash equilibrium. In a subgame perfect equilibrium,
Player 2 should behave rationally if her information set is reached. Thus Player 2
playing R cannot form part of a subgame perfect equilibrium. Rather, if Player 2 is
called upon to move, the only rational thing for her to do is to play L. This means
that Player 1 will prefer B over T , and the only subgame perfect equilibrium of
this game is (B, L). Note that (B, L) is also a Nash equilibrium of the whole game.
This is true in general: Subgame perfect equilibria are always Nash equilibria, but
Nash equilibria are not necessarily subgame perfect.
2, 2
0, 0
L
1, 1
R
2
T
B
1
Figure 7.15. An example of the simplest type of game in which
subgame perfect equilibrium is interesting.
As another example, consider the extensive form game in Figure 7.16. In this
game, players 1 and 2 are playing a prisoners’ dilemma, while at the beginning of
the game player 3 gets to choose whether players 1 and 2 will actually play the
prisoners’ dilemma or whether the game will end. The only proper subgame of this
game begins at Player 1’s node. We claim that (C, C, L) is a Nash equilibrium of
the whole game. Given that player 3 is playing L, players 1 and 2 can do whatever
60
7. SOLVING EXTENSIVE FORM GAMES
they like without affecting their payoffs and do not care what the other is playing.
And given that players 1 and 2 are both playing C, player 3 does best by playing
L. However, although (C, C, L) is a Nash equilibrium of the game, it is not a
subgame perfect equilibrium. This is because (C, C) is not a Nash equilibrium of
the subgame beginning at Player 1’s decision node. Given that Player 1 is playing
C, Player 2 would be better off playing D, and similarly for Player 1. The only
subgame perfect equilibrium is (D, D, A).
9, 9, 0
0, 10, 10
C
10, 0, 10
D
C
1, 1, 10
D
2
C
10, 10, 1
D
1
L
A
3
Figure 7.16. Player 3 chooses whether or not Players 1 and 2
will play a prisoners’ dilemma game.
In the previous example, we have seen that a extensive form games can have
equilibria that are Nash but not subgame perfect. You might then be wondering
whether it’s possible to have an extensive form game that has no subgame perfect
equilibria. The answer is no. Selten in 1965 proved that every finite extensive form
game with perfect recall has at least one subgame perfect equilibrium.
6.2. Backwards induction. Backwards induction is a convenient way of finding subgame perfect equilibria of extensive form games. We simply proceed backwards through the game tree, starting from the subgames that have no proper
subgames of themselves, and pick an equilibrium. We then replace the subgame by
a terminal node with payoff equal to the expected payoff in the equilibrium of the
subgame. Then repeat, as necessary.
As an illustration, consider the game in Figure 7.17. First Player 1 moves
and chooses whether she and Player 2 will play a battle of the sexes game or a
coordination game. There are two proper subgames of this game, the battle of the
sexes subgame and the coordination subgame. Following the backwards induction
procedure, one possible equilibrium of the battle of the sexes subgame is (F, F ),
and one possible equilibrium of the coordination subgame is (B, b). We then replace
each subgame by the appropriate expected payoffs, as shown in Figure 7.18. We
can then see that at the initial node, Player 1 will choose S. Thus one subgame
perfect equilibrium of this game is ((S, F, B) , (F, b)).
Problems
Problem 7.3. Draw the following two trees and give the function p by specifying what p(x) is for each x in X for both trees. [Note: the trees differ in only the
PROBLEMS
3, 1
0, 0
0, 0
1, 3
F
B
F
B
61
4, 4
0, 0
a
0, 0
a
b
2
F
1, 1
b
2
B
A
B
1
1
S
C
1
Figure 7.17. Player 1 chooses battle of the sexes or coordination
game.
3, 1
1, 1
S
C
1
Figure 7.18. Replacing the subgames with the equilibrium expected payoff in each subgame.
initial node.]
X = {x1 , x2 , x3 , x4 , x5 , x6 , x7 , x8 , x9 }
B = {{x1 , x2 }, {x1 , x3 }, {x1 , x4 }, {x4 , x5 }, {x4 , x6 }, {x6 , x7 }, {x6 , x8 }, {x6 , x9 }}
x0 = x1 ;
and
X = {x1 , x2 , x3 , x4 , x5 , x6 , x7 , x8 , x9 }
B = {{x1 , x2 }, {x1 , x3 }, {x1 , x4 }, {x4 , x5 }, {x4 , x6 }, {x6 , x7 }, {x6 , x8 }, {x6 , x9 }}
x0 = x6 .
Problem 7.4. The following is a formal description of a game tree using the
(X, p) notation. Draw the game tree and give the equivalent definition using the
(X, B, x0 ) notation.
X = {x, v, y, z, w, a, b, c, d, e, f }
p(x) = ∅
p(a) = w
p(d) = z
p(v) = x
p(b) = w
p(e) = z
p(y) = x
p(c) = w
p(f ) = z
p(z) = y
p(w) = y
62
7. SOLVING EXTENSIVE FORM GAMES
Problem 7.5. For the following formal description of an extensive form game
draw the extensive form game.
N = {1, 2}
X = {x, v, y, z, w, a, b, c, d, e, f }
p(x) = ∅
p(a) = w
p(d) = z
p(v) = x
p(b) = w
p(e) = z
α(v) = T
α(a) = L
α(d) = L
α(y) = B
α(b) = C
α(e) = C
p(y) = x
p(c) = w
p(f ) = z
p(z) = y
α(w) = U
α(c) = R
α(f ) = R
p(w) = y
α(z) = D
H = {H1 , H2 , H3 }
H(x) = H1
n(H1 ) = 1
H(y) = H2
n(H2 ) = 1
H(w) = H3
n(H3 ) = 2
H(z) = H3
And the utility function is given by
T
u1
u2
v
5
4
a
8
5
b c
0 6
0 3
d e f
0 7 6 .
0 6 3
Problem 7.6. Consider the extensive form game in Figure 7.19.
(1, 1)
(0, 0)
(1, 1)
(0, 0)
(1, 1)
R
W
R
W
R
(0, 0) (1, 1)
W
2
R
1
1
2
1
2
(1, 1)
me
me
him
2
me
him
1
Figure 7.19.
him
(0, 0)
W
PROBLEMS
63
(1) Give a formal description of the game tree, both in the form (X, B, x0 )
and in the form (X, p).
(2) Give a formal description of this extensive form game.
Problem 7.7. Consider the extensive form game in Figure 7.20.
−1, −1
4, 1
U
D
0, 0
1
L
R
2, 5
2
T
B
1
Figure 7.20.
(1) Give the associated normal form of this game.
(2) Find all the equilibria in mixed strategies of this game. [Hint: Be careful,
there are an infinite number.]
(3) Which of the equilibria are (equivalent to) subgame perfect equilibria of
the extensive form game.
Problem 7.8. Consider the extensive form game in Figure 7.21.
8, 5
0, 0
L
C
6, 3
0, 0
R
L
7, 6
C
6, 3
R
2
U
5, 4
D
1
T
B
1
Figure 7.21.
(1) Give the associated normal form game.
(2) Find all the equilibria of this game. What equilibria are subgame perfect?
CHAPTER 8
Sequential equilibrium
In the previous chapter we examined the most basic notions of backward induction and the concept of subgame perfection. It is not hard to find examples in
which the idea of backward induction seems to be pertinent but not captured by
the subgame perfection. In the game of 8.1 the only subgame perfect equilibrium is
(B, L). The argument in favour of the equilibrium (B, L) is quite straightforward.
If Player 2 is ever called upon to move then whatever equilibrium she was expecting
she knows that Player 1 has played B and that she will do better choosing L than
R and so should choose L. Moreover Player 1 knows this at the beginning of the
game and so knows that if he chooses B he can rely on Player 2 choosing L. So
Player 1 should expect to get 4 if he chooses B, while he only gets 2 if he chooses
T . It’s also clear that (B, L) is the only subgame perfect equilibrium.
4, 1
0, 0
L
R
2, 5
2
T
B
1
Figure 8.1
5, 1
0, 0
L
2, 5
4, 1
R
0, 0
L
R
2
M
B
T
1
Figure 8.2
However consider the game of Figure 8.2. Here the issues seem to be essentially
the same. There is an equilibrium in which Player 1 chooses T and so Player 2 never
65
66
8. SEQUENTIAL EQUILIBRIUM
gets to move. In that equilibrium Player 2’s strategy is R. That is, she is planning,
should she ever get to move to choose the action R. In the equilibrium she does
not move so her choice makes no difference to her payoffs. You can see this most
clearly if you write the strategic (or normal) form of the game. However you also
see quite clearly that should she get to move that L is better for her than R. No
matter whether she was called on to move because Player 1 played M or because
Player 1 played B she gets 1 from L and 0 from R. Thus she should choose L and
Player 1 can rely on this to be the case. So Player 1 should play M , confident that
Player 2 will choose L. Thus we argue that subgame perfection does not capture
all of what we intuitively understand by backward induction.
Problem 8.1. In the game of Figure 8.2 if we split Player 1’s choice into two
binary choices, having him first choose whether to play T and if he chose not to
play T then to choose whether to play M or B, we might think that we have not
very much changed the game. However in this case subgame perfection will rule
out the equilibria in which Player 1 chooses T . This problem asks you to work out
some of the details of this example.
(1) Give the associated normal form of the game of Figure 8.2 and find all
the Nash equilibria.
(2) What are the subgames of the game of Figure 8.2. Find all the subgame
perfect equilibria.
(3) Suppose that we split Player 1’s choice into two binary choices, having
him first choose whether to play T and if he chose not to play T then to
choose whether to play M or B. Draw the extensive form of this game,
give the associated normal form, and find all the Nash equilibria of the
new game.
(4) Find the subgame perfect equilibria of the new game. Remember that
subgame perfect equilibria are defined in terms of behaviour strategies
and in the new game Player 1 moves twice, so his mixed strategies and
behaviour strategies are different.
(5) Suppose that we change the game again so that Nature moves first and
that with probability one half the payoffs are as the were in the original
game and with probability one half the payoffs 5, 1 and 4, 1 are reversed
with 5, 1 being the payoff if Player 1 chooses B and Player 2 chooses L
and 4, 1 being the payoff if Player 1 chooses M and Player 2 chooses L.
Suppose that Player 1 observes Nature’s choice before making any moves
and Player 2 does not observe Nature’s choice. Draw the extensive form
games for these cases, both when Player 1 chooses between T , M , and
B at a single node and when we split Player 1’s choice into two binary
choices, having him first choose whether to play T and if he chose not
to play T then to choose whether to play M or B. Find the associated
normal forms and the Nash equilibria. In each case find the subgames and
all the subgame perfect equilibria.
1. Definition of Sequential Equilibrium
One way of capturing more of the idea of backward induction is by explicitly
requiring players to respond optimally at all information sets. The problem is, of
course, that, while in the game of Figure 8.2 it is clear what it means for Player 2 to
respond optimally, this is not generally the case. In the game of Figure 8.2 Player
2 should play L whichever node of his information set he believes himself to be
at. However in general we could have situations in which Player 2 would prefer
to Play L if he believed that Player 1 had played M and would prefer to play R
if he believed that Player 1 had played B. Of course, at this information set he
1. DEFINITION OF SEQUENTIAL EQUILIBRIUM
67
knows that Player 1 has either played M or B. However if we are looking at an
equilibrium in which Player 1 plays T then Player 1’s strategy does not tell Player
2 what to believe at his information set.
The concept of sequential equilibrium gets around this difficulty by expanding
the set of variables where e look for an equilibrium. Rather than defining an
equilibrium to be a profile of behaviour strategies as we do when defining subgame
perfect equilibria when defining a sequential equilibrium we also specify what each
player believes at each information set, or at least what each player believes about
which node in the information set he’s at. We define an assessment to be a pair
consisting of a behavioural strategy and a system of beliefs. A system of beliefs
gives, for each information set, a probability distribution over the nodes of that
information set.
Given an assessment we no longer have a problem in saying what we mean by
a player behaving rationally at each information set. Even if the information set is
not reached under a particular profile of behaviour strategies the system of beliefs
tells us where the player probabilistically believes he is and the behaviour strategy
tells us what will happen at each node following the information set. Thus for each
action at the information set we can calculate an expected utility from taking that
action. The behavioural strategy is said to be sequentially rational with respect
to the system of beliefs if, at every information set at which a player moves, it
maximises the conditional utility of the player, given his beliefs at that information
set and the strategies of the other players.
The reason that we introduced the notion of a system of beliefs was that the
behaviour strategy may not tell us what the player should believe about where he
is in some information sets. However sometimes the behaviour strategy will tell us
something about what the beliefs should be. As well as the strategy profile being
sequential rational given the beliefs we also require that the beliefs be consistent
with the strategy profile. The justification for defining consistency in the way that
we do is beyond the scope of this course. (And indeed probably not something you
should worry about until you’ve studied quite a bit more game theory.) For the
moment just take this as a definition that seems to behave quite reasonably in the
examples that we shall study.
A system of beliefs is said to be consistent with a behavioural strategy if it is
the limit of a sequence of beliefs each being the actual conditional distribution on
nodes of the various information sets induced by a sequence of completely mixed
behavioural strategies converging to the given behavioural strategy. If for some the
behaviour strategy a particular information set is reached with positive probability
then at that information set consistency will require that the beliefs will be the
conditional probability of that node, that is the probability that that node is reached
divided by the probability that the information set is reached, the sum of the
probability of each node in the information set. If an information set is reached
with zero probability under some behaviour strategy then it might be that any
beliefs at that information set are consistent with the behaviour strategy; it might
be that the beliefs are exactly specified at that information set; and it might be that
there are some restrictions on the beliefs that do not exactly identify the beliefs.
Sometimes it is nontrivial to work out the implications of consistency, but often it
is quite easy.
A sequential equilibrium is a pair such that the strategy is sequentially rational
with respect to the beliefs and the beliefs are consistent with the strategy.
We now state this all again, but a little bit more formally and using a few
symbols.
68
8. SEQUENTIAL EQUILIBRIUM
An assessment is a pair, (b, µ) where b is a profile of behaviour strategies and µ
is a system of beliefs. A system of beliefs, µ, is a profile of probability distributions,
one for each information set, over the nodes in the information set. The system
of beliefs specifies at each information set the conditional probabilities with which
each player believes he is at each of the nodes within each of his information sets,
conditional on being at that information set.
A sequential equilibrium is an assessment (b, µ) that satisfies two properties:
Sequential Rationality: At each information set, b puts positive weight
only on those actions that are optimal given the beliefs at that information
set specified by µ and behaviour at nodes following the information set
given by b.
Consistency: There is a sequence of completely mixed behaviour strategy
profiles bt converging to b with µt obtained from bt as conditional probability and µt converges to µ.
If we drop the consistency requirement at out of equilibrium information sets
we get another equilibrium concept called Perfect Bayesian Equilibrium, which we
won’t talk about in this course but you might come across in other books.
The concept of sequential equilibrium is a strengthening of the concept of subgame perfection. Any sequential equilibrium is necessarily subgame perfect, while
the converse is not the case. For example, it is easy to verify that in the game of
Figure 8.2 the unique sequential equilibrium involves Player 1 choosing M . And a
similar result holds for the modification of that game involving a move of Nature
discussed above.
Problem 8.2. In the game of Figure 8.2 find the unique sequential equilibrium.
Explain why none of the equilibria (in behaviour strategies) in which Player 1 plays
T are the strategy parts of sequential equilibria.
Notice also that the concept of sequential equilibrium, like that of subgame
perfection, is quite sensitive to the details of the extensive form. For example in
the extensive form game of 8.3 there is a sequential equilibrium in which Player 1
plays T and Player 2 plays L. However in a very similar situation (we shall later
argue strategically identical)—that of 8.4—there is no sequential equilibrium in
which Player 1 plays T .
2, 0
4, 1
L
3, 5
1, 1
R
2, 0
L
R
2
M
B
T
1
Figure 8.3
Let us consider one more example, the game shown in Figure 8.5. This game
is known as “Selten’s Horse”. (If you draw it with the initial node at the top you
might think it looks like a horse. It arguably looks at least as much like a horse
as the constellation Orion looks like a hunter.) In this game, there are no proper
1. DEFINITION OF SEQUENTIAL EQUILIBRIUM
2, 0
4, 1
L
1, 1
R
69
2, 0
L
R
2
M
B
3, 5
1
T
IN
1
Figure 8.4
subgames, hence all Nash equilibria are subgame perfect. Note that (T, R, D) is a
Nash equilibrium (and hence subgame perfect). But in this equilibrium, Player 2
isn’t really being rational, because if player 3 is really playing D then if Player 2
actually got to move, she would be better off playing L rather R. Thus we have
another “silly” Nash equilibrium, and subgame perfection is no help to us here to
eliminate it. This example was used by Selten (1975) lead to the development of
another equilibrium concept called perfect equilibrium. That solution concept is
very close to sequential equilibrium but doesn’t use beliefs and is in some examples
a little bit stronger.
0, 0, 0
U
3, 2, 2
0, 0, 1
D
U
4, 4, 0
D
1, 1, 1
3
L
R
2
T
B
1
Figure 8.5. “Selten’s Horse”.
Problem 8.3. Find all the Nash equilibria of the game in Figure 8.5.
Recall that the assessment (b, µ) is consistent if there is a sequence of completely
mixed behaviour strategy profiles bt → b and µt → µ with µt the conditional
distribution over the nodes induced by bt . To see how this works, consider the game
in Figure 8.6. Each player has two nodes in their (only) information set. Under
70
8. SEQUENTIAL EQUILIBRIUM
−2, 2
2, −2 2, −2
L
R
L
1, −1
OUT
−2, 2
R
−1, 1
2
IN
IN
OUT
1
1
2
1
2
Nature
Figure 8.6. Illustrating consistent beliefs.
sequential equilibrium we must specify the probabilities with which each player
believes they are at each node in each of their information sets. Furthermore, these
beliefs must be consistent with the equilibrium behaviour strategy. Suppose that
in this game, b = {(0, 1) , (1, 0)}, that is, that Player 1 plays OUT and Player 2
plays L. The profile of beliefs is µ = ((µ (x) , µ (y)) , (µ (a) , µ (b))). Clearly, since
the only move before Player 1’s information set is that of Nature, Player 1’s beliefs
must be that µ (x) = 12 and µ (y) = 12 . These beliefs will be consistent with
any behaviour strategy, since whatever behaviour strategy is being played will not
affect the probability that Player 1 is at either node in his information set. To
find Player 2’s beliefs µ (a) and µ (b) we need to find a completely mixed behaviour
strategy close to b. Actually, we only need to worry about Player 1’s behaviour
strategy, since there are no moves of Player 2 before Player 2’s information set.
for Player 1. Then,
Suppose we use bt = 1t , t−1
t
Pr (a) =
1 1
1
· =
2 t
2t
Pr (b) =
1 1
1
· = .
2 t
2t
and
Thus,
µt (a) =
1
2t
1
2t
+
1
2t
=
1
.
2
Similarly, we can show that µt (b) = 12 also.
Why couldn’t we just use Player 1’s behaviour strategy (0, 1)? Because in this
case Player 1 plays OUT and thus Player 2’s information set is not reached. Mathematically, we would have
1
·0
µ (a) = 1 2 1
·
0
+
2
2 ·0
which is not defined. So we must use completely mixed behaviour strategies close
to b to find the (consistent) beliefs, since if we use a completely mixed behaviour
strategy, every information set in the game will be reached with strictly positive
probability (even if the probability is very small).
2. SIGNALLING GAMES
71
2. Signalling Games
We remarked earlier that it could sometimes be quite easy to calculate the
implications of consistency for out of equilibrium information sets. And that sometimes it could be rather hard. One class of games for which the implications of
consistency for out of equilibrium information sets is easy to calculate are called
signalling games. In these games the implications are very easy to calculate because
consistency has no implications at out of equilibrium information sets.
In a signalling game there are two players who we call the Sender and the
Receiver. Nature moves first and chooses randomly one of a finite number of outcomes. The Sender sees the outcome, which for this reason we often call the Type
of the Sender. The Sender then chooses from a finite number of possibilities, which
are often called Messages. The Receiver sees the Message sent, but not the Type of
the Sender and chooses from another finite number of possibilities, which are often
called Responses. The payoff to each player can depend on the Type of the Sender,
the chosen Message, and the chosen Response. If the game is such that the payoffs
do not depend on the message, but may depend on both the Type of the Sender
and the Response of the Receiver the game is called a cheap talk game. We won’t
be much concerned with cheap talk games in this course.
Signalling games have been an important model in many areas of economics,
particularly Industrial Organisation and Information Economics. One particularly
influential model is the model of educational signalling, originally introduced by
Michael Spence (1973). Spence did not explicitly use game theory but his model
have since been reformulated using game theoretic language and that’s the way it’s
usually presented now. In that model there are many workers with private information about their own productivity. The firms who might employ them cannot
tell what the productivity of the worker is, but can observe how much education
the worker has undertaken. It is assumed that the amount of education the worker
undertakes does not affect his productivity but that it is easier for a worker of
high productivity to obtain a higher educational level. The firms are assumed be
competing for the workers and “want” to pay them their expected product since
otherwise they will go to some other firm.
Problem 8.4. Draw a signalling game to represent the following simplified
Spence educational signalling model. There are two types of the sender, L and H,
three possible messages, EL , EM , and EH , and three possible responses, WL , WM ,
and WH . The payoff to the sender is the sum of two components one depending
only on W and equal 10 for WH , 6 for WM and 2 for WL and one depending on
the type and the message and being negative (the cost of education) with the cost
being 0 for either type for EL , −1 for type H and −2 for type L for EM , and −2
for type H and −12 for type L for EH . The payoff for the receiver does not depend
on the message and is equal to 3 for WH , 2 for WM and 0 for WL if the type is H
and −3 for WH , −1 for WM and 0 for WL if the type is L.
There is a sequential equilibrium in which both types of the sender choose
EM with probability 1. What is the response of the receiver to EM in such an
equilibrium? What can you say about the probabilistic response of the receiver to
the messages EL and EH in such an equilibrium? Pick one set of responses that
make both types sending EM an equilibrium. What can you say about what the
beliefs of the receiver at each of the information sets must be that equilibrium? An
equilibrium like this is called a pooling equilibrium because both types send the
same message.
There is another sequential equilibrium in which the L type chooses EL with
probability 1 and the H type chooses EH with probability 1. What can you say
72
8. SEQUENTIAL EQUILIBRIUM
about the response of the receiver and the beliefs of the receiver at each of his
information sets?
Are there any other sequential equilibria? If so find them. [This last part
is probably unreasonably hard for this course and is here as a challenge for any
student who is looking for such a challenge.]
Problems
The problems here concern signalling games. Signalling games are often drawn
such that they are not strictly extensive form games as we have defined them. You
can make them so by adding a move of Nature at the beginning that gives the two
“initial nodes” with the given probabilities.
Problem 8.5. Consider the following two-person extensive form game.
0, 0
m
′
[ 21 ]
r1
r2
1a
0, 0
m′
[ 21 ]
1, 3
m
2
r1
−2, 0
−2, 0
m
r2
1b
1, 3
(a) Suppose that both types of the sender choose m′ , that is, the behaviour strategy
of Player 1 is ((1, 0), (1, 0)). What restrictions, if any, does this put on the beliefs
of the receiver, in a consistent assessment, at the information set following the
choice m?
(b) There are three (sets of) sequential equilibria: one equilibrium in which type
1a chooses m and type 1b chooses m′ ; one equilibrium in which type 1a chooses
m′ and type 1b chooses m; and a set of equilibria in which both types choose
m′ . Explicitly construct one of these equilibria.
(c) Explicitly construct all of the other sequential equilibria.
(d) Give the normal form of this game and say which strategy profile is equivalent
to the equilibrium you found in part (b).
Problem 8.6. Consider the following two-person extensive form signaling game.
0, 0
L
[ 21 ]
U
D
1a
0, 0
L
[ 21 ]
1b
−2, 0
R
2
U
2, 2
−2, 2
R
D
−2, 0
(a) Suppose that both types of the sender choose L, that is, the behaviour strategy
of Player 1 is ((1, 0), (1, 0)). What restrictions, if any, does this put on the beliefs
PROBLEMS
73
of the receiver, in a consistent assessment, at the information set following the
choice R?
(b) There are two (sets of) sequential equilibria: one equilibrium in which type 1a
chooses R and type 1b chooses L; and a set of equilibria in which both types
choose L. Explicitly construct one of these sets of equilibria.
(c) Explicitly construct the other set of sequential equilibria.
(d) Give the normal form of this game. Are there any pure strategy equilibria other
than the sequential equilibria you found above?
Problem 8.7. Consider the following signalling games.
0, 0
m
′
[ 21 ]
r1
r2
r3
m
1a
0, 0
m′
[ 21 ]
2
m
1b
3, 1 don’t
duel
1, 0
duel
2, 1 don’t
duel
0, 2
[.9]
beer
r1
r2
r3
−1, 3
1, 2
−1, 0
1, 0
1, 2
−2, 3
don’t 2, 1
duel
quiche
AS
BQ
BB
[.1]
beer
quiche
duel
don’t 3, 1
duel
AW
duel
0, 0
duel
1, 2
a In each case give the normal form of the game.
b In the first game suppose that both types of the sender choose m′ , that is, the
behaviour strategy of Player 1 is ((1, 0), (1, 0)). What restrictions, if any, does this
put on the beliefs of the receiver, in a consistent assessment, at the information
set following the choice m? Show how your answer follows from the definition of
a consistent assessment.
c For each game find the sequential equilibria.
Part 2
Cooperative Game Theory
CHAPTER 9
Coalitional games
A coalitional game is a model of interacting decision makers that focuses on
the behaviour of groups of players. Each group of players is called a coalition and
the coalition of all players is called the grand coalition.
Definition. Given a set of players N = {1, 2, . . . , N }, a coalition is a nonempty
subset of N . The set N itself is called the grand coalition. For a given player set
N the collection of all possible coalitions is denoted by C(N ).
Consider for example the prisoners’ dilemma game, repeated here in Figure 9.1.
We already know that (F, F ) with corresponding payoffs of 1 to each player is the
only Nash equilibrium of this game.
Player 2
Player 1
Q
F
Q
2, 2
0, 3
F
3, 0
1, 1
Figure 9.1. The prisoners’ dilemma.
But what if the players could somehow cooperate and write a contract that
specified how each was going to play? The solid dots in Figure 9.2 show the four
possible contracts. If we allow for randomisations over all such contracts, the shaded
area shows all possible expected utilities of each player, with Player 1’s expected
utility plotted horizontally and Player 2’s expected utility plotted vertically.
In the prisoners’ dilemma example we have N = {1, 2}. For coalitional games
we also specify a worth function, v(·) which gives the payoffs that any coalition of
players can get. For this example we have v({1}) = 1, v({2}) = 1 and v({1, 2}) =
anything in the grey set in Figure 9.2. This is an example of a game without side
payments, also called a game with non-transferrable utility (NTU). For the rest of
this chapter we’ll deal with games with side payments, or transferrable utility (TU)
games. If side payments are allowed in the prisoners’ dilemma game with contracts,
all the points on the diagonal line from (0, 4) to (4, 0) in Figure 9.2 are possible.
Definition. A transferrable utility coalitional game is a pair (N, v) where
N = {1, 2, . . . , N } is a set of players and v : C(N ) → R is the worth function.
For example, in the prisoners’ dilemma cooperative game with transferrable
utility, N = {1, 2}, v(1) = 1, v(2) = 1, and v(1, 2) = 4. At this point, let us give
some other example of transferrable utility coalitional games. We shall refer back
to these examples.
Example 9.1. Gloves. N = L ∪ R, i.e. L is the set of left-handed gloves that
you have and R is the set of right-handed gloves. Then v(S) = min{#S ∩L, #S ∩R}
for any coalition S. That is, only pairs of gloves are valuable. So for any given
77
78
9. COALITIONAL GAMES
u2
6
4
3
2
1
1
2
3
4
u1
Figure 9.2. Possible expected utilities if contracts can be used in
the Prisoners’ Dilemma game.
coalition of gloves, the value of the coalition is the minimum of the number of
left-handed and right-handed gloves in the coalition.
Example 9.2. UN Security Council. N = P ∪ R where P is the set of
permanent members of the security council and R is the set of rotating members.
In this game, v(S) equals 1 if P ⊂ S and #S ≥ 7, and equals 0 otherwise. Thus,
v(N ) = 1.
CHAPTER 10
The core
For coalitional games such as gloves and security council, we would like to be
able to say what the outcome is likely to be. To do this, we need a solution concept.
One such concept is called the core. Basically, an outcome is in the core of a game
if no subset of players could make themselves all better off by breaking away and
playing the game amongst themselves.
Definition. An allocation is a vector x = (x1 , x2 , . . . , xN ) that is feasible, i.e.,
i=1 xi ≤ v(N ), and individually rational, i.e., xn ≥ v(n).
PN
The definition of an allocation says that the sum of what we give to everyone
cannot be greater than what is available in total, and what everyone gets in the
allocation must make them at least as well off as they could be by themselves.
Definition. An allocation x = (x1 , x2 , . . . , xN ) is in the core of the game
(N, v) if for every coalition S ⊂ N
X
xi ≥ v(S).
i∈S
The definition of allocations that are in the core says that every such allocation
must make all the members of every possible coalition at least as well off as they
would be in the coalition itself.
As an illustration of the core, consider the gloves game with N = {1, 2, 3},
L = {1, 2} and R = {3}. Thus v(1) = v(2) = v(1, 2) = v(3) = 0, and v(1, 3) =
v(2, 3) = v(1, 2, 3) = 1. Thus if an allocation (x1 , x2 , x3 ) is in the core, it must be
that x1 ≥ 0, x2 ≥ 0, x3 ≥ 0, x1 + x2 + x3 = v(N ) = 1, x1 + x3 ≥ 1, x2 + x3 ≥ 1,
and x1 + x2 ≥ 0. Note that x1 + x3 ≥ 1 and x2 + x3 ≥ 1 together imply that
x1 + x2 + 2x3 ≥ 2 and since x1 + x2 + x3 = 1, we must have x3 ≥ 1, x1 ≥ 0 and
x2 ≥ 0. Thus the only allocation that is in the core is (0, 0, 1).
As another example, consider the following 3-person simple majority game. Let
N = {1, 2, 3}, and
1 if #S ≥ 2,
v(S) =
0 otherwise.
In this game an allocation (x1 , x2 , x3 ) is in the core if x1 ≥ 0, x2 ≥ 0, x3 ≥ 0,
x1 + x2 + x3 = 1, x1 + x2 ≥ 1, x1 + x3 ≥ 1, and x2 + x3 ≥ 1. Note that the last
three inequalities together imply 2(x1 + x2 + x3 ) ≥ 3 but since x1 + x2 + x3 = 1,
we have 2 ≥ 3, which is clearly not true. Thus the core of this game is the empty
set, that is, there are no allocations in the core. This example with the empty core
suggests that while the core may be a good solution concept for many economic
problems it may not for political problems.
Definition. A simple game is a game (N, v) with v(N ) = 1 and v(S) either
0 or 1 for all coalitions S.
In a simple game, a coalition S for which v(S) = 1 is called a winning coalition.
A player who is in every winning coalition is called a veto player.
79
80
10. THE CORE
Example 10.1. Let N = {1, 2, 3} and let v(1, 2, 3) = 1 and v(S) = 0 for all
other coalitions S. This is a simple game and in this game every player is a veto
player. The core of this game is the set {(x1 , x2 , 1 − x1 − x2 ) : xn ≥ 0}.
Example 10.2. Let N = {1, 2, 3}, v(1, 2, 3) = v(1, 2) = v(1, 3) = v(2, 3) = 1,
and v(1) = v(2) = v(3) = 0. This is a simple game and in this game there are no
veto players. The core of this game is empty.
Example 10.3. Let N = {1, 2, 3} and let v(1, 2, 3) = v(1, 2) = v(1, 3) = 1, and
v(2, 3) = v(1) = v(2) = v(3) = 0. This is a simple game and Player 1 is the only
veto player. The core of this game is (1, 0, 0).
You may have noticed that in the above three examples, only the games in
which there is at least one veto player have non-empty cores. In fact this is a
general result for simple games.
Theorem 10.1. A simple game has a non-empty core if and only if it has veto
players.
Proof. It’s easy, try to do it yourself.
If a simple game has veto players then the core is the set of individually rational
allocations that give everything to the veto players.
CHAPTER 11
The Shapley value
Consider again example 3 from the previous section. In that game, the only
allocation in the core is (1, 0, 0), i.e., Player 1 gets everything. In other words, the
marginal worth of what Players 2 and 3 contribute is zero. But you wouldn’t expect
in such a game that Player 1 would get everything, since Player 1 by himself is worth
zero, i.e., v(1) = 0. This kind of problem leads us to an alternative solution concept
called the Shapley value. The Shapley value is an axiomatic solution concept. That
is, we define some things (axioms) that we think a solution concept should satisfy.
Then we show that the Shapley value satisfies these axioms (in fact, it’s the only
thing that satisfies them).
Before giving the axioms, let us explain what the Shapley value is. Consider
again example 10.3 from above. A pivotal player is a player who, when added
to a coalition, turns that coalition from a losing coalition into a winning one. In
example 10.3, let’s list all the possible orderings of players and list who is the pivotal
player in each case. This is shown in Figure 11.1.
Order Pivotal Player
123
2
213
1
231
1
132
3
312
1
321
1
Figure 11.1. The possible player orderings and the pivotal player
in each case from example 10.3.
The Shapley value of player n is then defined to be
φn =
# orders in which n is pivotal
.
total # orders
Thus for this game we have φ1 = 46 = 32 , φ2 = 61 and φ3 = 16 . So, the Shapley value
for the game is φ(N, v) = ( 32 , 61 , 61 ) .
Now let us give the axioms that the Shapley value satisfies. First, suppose in
a game (N, v) it’s true that for any coalition S, v(S) = v(T ∩ S) for some coalition
T , and that v(∅) = 0. That is, all the action in the game happens in the coalition
T . Then we call coalition T a carrier of the game (N, v). Now we specify our first
axiom that we think solution concepts should satisfy.
Axiom 11.1. If S is a carrier of (N, v) then
X
fn (N, v) = v(S)
n∈S
where fn is the solution function, which tells what each player gets in the solution.
81
82
11. THE SHAPLEY VALUE
Axiom 11.1 says that if S is a carrier of the game then what the members of
S get in the solution should be the same as what they could get as a group by
themselves.
A permutation of the player set N is a one-to-one and onto function π :
{1, 2, . . . , N } → {1, 2, . . . , N }. 1 That is, the function π just re-labels the players with different numbers. Given a game (N, v) we define the permuted game,
(N, πv) by πv(S) = v(π −1 (S)) where π −1 ‘undoes’ the permutation, i.e.,
π −1 (S) = {m ∈ N : m = π(n) for some n ∈ S}.
Axiom 11.2. For any permutation π and any n ∈ N , fπ(n) (πv) = fn (v).
Axiom 11.2 says that changing the labels of the players (e.g., calling player ‘1’
player ‘2’ and calling player ‘2’ player ‘1’) should not essentially change the solution,
i.e., the players should get the same payoff in the solution even when their ‘names’
are swapped around.
Axiom 11.3. For any two games (N, v) and (N, w), the game (N, (u + v)) is
defined by (v + w)(S) = v(S) + w(S) and fn (v + w) = fn (v) + fn (w).
Axiom 11.3 is a technical thing and we won’t try to explain it. Anyway, amazingly, it can be proved that the solution concept called the Shapley value is the
only solution that satisfies Axioms 11.1 – 11.3.
At this point, there are some useful observations that we can make from the
axioms. First, axiom 11.1 implies that if n is a dummy player, that is if v(S ∪
{n}) = v(S) for any coalition S, then φn (v) = 0. That is, dummy players do
not get any payoff in the Shapley value. Second, axiom 11.1 also implies that
P
n∈N φn (v) = v(N ), that is, that there is no wastage in the Shapley value, i.e., all
of the available surplus is allocated. Finally, axiom 11.2 implies that if players n
and m are substitutes, that is, that v(S ∪ {m}) = v(S ∪ {n}) for any S such that
n∈
/ S and m ∈
/ S then φn (v) = φm (v). That is, if any two players add the same
value to any given coalition, then both of these players will get the same payoff in
the Shapley value.
Let us do another example that we’ll call “Australia”. In the Australian federal
government each of the 6 states have one vote each, and the federal government itself
has two votes plus one extra vote if there is a tie. Let’s say that player 1 is the federal
government and the other 6 players are the states. Thus N = {1, 2, 3, 4, 5, 6, 7}.
Then S is a winning coalition if 1 ∈ S and #S ≥ 3, or if 1 ∈
/ S and S ≥ 5. It is a
bit tedious to solve this game directly, so let’s see if we can use some of the above
observations to help us out. First, when is player 1 pivotal? In any ordering, player
1 is pivotal if it is in positions 3, 4 or 5, and player 1 is in one of these positions in
3
3
7 of the possible orders. Thus φ1 (v) = 7 . Now, since players 2 . . . 7 are substitutes,
we must have φ2 (v) = φ3 (v) = · · · = φ7 (v). Furthermore, efficiency requires that
P7
4
2
n=1 φn (v) = 1. Thus φ2 (v) = φ3 (v) = · · · = φ7 (v) = 7 /6 = 21 .
As yet another example, suppose that N = {1, 2, 3}, v(1) = v(2) = v(3) = 0,
v(1, 2) = 6, v(1, 3) = 7, v(2, 3) = 2, and v(1, 2, 3) = 8. This is no longer a
simple game. However the idea of the Shapley value is still similar. In this case
we calculate the marginal contribution of each player in each possible ordering
and then average these over all possible orderings to get the Shapley value. The
marginal contributions of each player in this game for each ordering are shown in
10
13
Figure 11.2. Thus we get φ1 (v) = 25
6 , φ2 (v) = 6 , and φ3 (v) = 6 .
Before doing a final example, let us note some useful facts about orderings:
1A function f is one-to-one if for every x 6= y, f (x) 6= f (y). A function f is onto if every
element of the range of f is the image of some element of the domain of f .
11. THE SHAPLEY VALUE
Order
123
132
213
231
312
321
Total
Player 1
0
0
6
6
7
6
25
83
Player 2 Player 3
6
2
1
7
0
2
0
2
1
0
2
0
10
13
Figure 11.2. Marginal contributions of each player in our nonsimple Shapley value example.
(1) If you have T objects and T ordered boxes to put them in there are
T · (T − 1) · (T − 2) · · · 2 · 1 = T ! ways of doing this.
(2) A particular object j is in the kth box T1 fraction of the times, or (T − 1)!
times.
Now consider the following example. A farmer (F ) owns some land that is
worth $1,000,000 to her to use as a farm. It’s worth $2,000,000 to an industrialist
(I) and it’s worth $3,000,000 to a developer (D). So, N = {F, I, D} and v(F ) = 1,
v(I) = v(D) = 0, v(F, I) = 2, v(F, D) = 3, v(F, D, I) = 3, v(D, I) = 0, and
v(∅) = 0. Now let R be some ordering and let SRn be the coalition of all players
coming before n in the order R. Then, for example, if the ordering is R = F ID
then v(SRF ∪ v(F )) − v(SRF ) = 1 is the value that F adds to the empty coalition.
Or, if R = F DI then v(SRD ∪ v(D)) − v(SRD ) = 2 is the value that D adds to
the coalition consisting of F . Then the calculation of the Shapley value is shown
1
4
in Figure 11.3. We have φF (v) = 13
6 , φI (v) = 6 , and φD (v) = 6 .
Order
F ID
F DI
IF D
IDF
DF I
DIF
Total
Farmer
1
1
2
3
3
3
13
Industrialist
1
0
0
0
0
0
1
Developer
1
2
1
0
0
0
4
Figure 11.3. Calculating the Shapley value in the FarmerIndustrialist-Developer example.
Problem 11.1. In the farmland example described above, is the Shapley value
13 1 4
φ(N, v) = ( , , )
6 6 6
in the core of the game?
CHAPTER 12
Matching problems
1. Assignment Problem
There are N players and N different houses. Initially, each player owns a house
which has the same name as the player.
Definition. An allocation is assignment of houses to owners such that no one
is homeless.
Each player ranks the houses from her first choice to the last choice, indifference
is not permitted. It is convenient to summarize players’ preferences in the table.
The top row of the table is the names of the players, and the corresponding columns
list houses starting from the most preferred one to the least preferred one. For
example, in the Figure 12.1 for the Player 2 House 4 is the most preferred one,
House 1 is the second best, and so on.
1
h3
h2
h4
2
h4
h1
h2
3
h1
h4
h3
4
h3
h2
h1
Figure 12.1. A house assignment problem.
Within any coalition players can exchange their houses in any way. But it is
assumed that players have no money, so they cannot do any side payments! This
is an example of a game with non-transferable utility, or NTU-game.
Note that there exist possibilities for mutually beneficial exchanges given the
initial allocation. For example, players 1 and 3 can exchange their houses and, as a
result, both will own their favorite houses. Players 3 and 4 can also engage in the
exchange ending up with better (from their respective points of view) houses than
they start with. There are also some mutually beneficial exchanges that involve
group of 3 players. (Try to find some!)
For this game, core is a very appealing and intuitive solution concept. An
allocation is in the core if there are no coalitions that can engage in mutually beneficial exchange. But, given our previous experience, one starts to wonder whether
the core is not empty!? After all, we haven’t imposed any restrictions of players’
preferences. Amazingly, the core of this game is always non-empty and, moreover,
there exist a very clever way of finding an allocation that belongs to the core, called
the Top Trading Cycles algorithm, due to David Gale.
Gale’s Top Trading Cycles Algorithm
Step 1: Each player ‘points’ at the owner of her favorite house. Since there is a
finite number of players, there is at least one cycle.
Step 2: Each player in a cycle is assigned the house of the player she points at and
removed from the market with her assignment. If there is at least one remaining
player, repeat Step 1. Note that the cycle can be of length 1 when a player owns
85
86
12. MATCHING PROBLEMS
her favourite house, in which case this player is not trading.
Let’s run the algorithm for the problem in Figure 12.1, starting with Player 1.
The Player 1 points at the owner of her favourite house, Player 3, who points at
the owner of her favourite house, Player 1. This is already a cycle, so we allocate
house 3 to Player 1 and house 1 to Player 3. At the second step we again start with
some player, say Player 2. Player 2 points to Player 4. Player 4’s favourite house,
house 3, is already out of the market, so she points to owner of the next house on
her list, Player 2. This is a cycle, so Player 2 is assigned house 4 and Player 4 is
assigned house 2.
The allocation obtained by Gale’s Top Trading Cycles Algorithm has the following properties:
(1) it is independent of the starting point;
(2) it is the unique allocation in the core;
(3) the algorithm is strategy-proof.
Let’s do one more example (Figure 12.2)
1
h3
h2
h1
h6
h10
h4
h7
h8
h9
h5
2
h1
h6
h9
h2
h5
h8
h3
h4
h7
h10
3
h10
h2
h4
h6
h8
h3
h5
h7
h9
h1
4
h6
h1
h5
h2
h7
h3
h8
h4
h10
h9
5
h7
h2
h8
h4
h10
h5
h9
h6
h3
h1
6
h4
h8
h2
h6
h10
h3
h7
h1
h5
h9
7
h1
h4
h3
h8
h5
h2
h9
h6
h7
h10
8
h6
h4
h2
h10
h8
h3
h1
h9
h7
h5
9
h8
h10
h6
h9
h5
h7
h4
h3
h1
h2
10
h6
h5
h1
h4
h10
h8
h9
h2
h3
h7
Figure 12.2. Another house assignment problem.
The unique core allocation is (h3 , h9 , h10 , h6 , h7 , h4 , h1 , h2 , h8 , h5 ).
2. The Marriage Problem
Players M = {m1 , . . . , mN } are men and players W = {w1 , . . . , wN } are
women. Each man ranks women in the order of his preference. Similarly, each
woman ranks men in the order of her preference. We will again assume that all
preferences are strict. An example is given in Figure 12.3.
m1
w2
w1
w3
m2
w1
w3
w2
m3
w1
w2
w3
w1
m1
m3
m2
w2
m3
m1
m2
w3
m1
m3
m2
Figure 12.3. A marriage problem.
Definition. A matching is a mapping µ : M → W such that it is one-to-one
and onto, that is every man is married to one and only one woman.
Thus, for every mi , µ(mi ) is the wife of man mi in matching µ and, for every
wj , µ−1 (wj ) is the husband of wj in matching µ.
2. THE MARRIAGE PROBLEM
87
Definition. A matching µ is stable if there is no man mi and woman wj who
are not married to each other but prefer each other to their actual partners.
If such a pair exists, the matching is not stable and such pair is said ‘to block’
a matching µ. Consider the preferences given in Figure 12.3. For example, the
matching (m1 , w1 ), (m2 , w2 ), (m3 , w3 ) is not stable, because it is blocked by a pair
(m1 , w2 ) or by a pair (m3 , w2 ).
Theorem 12.1 (Gale and Shapley, 1962). There exists a stable matching in
the marriage problem.
First, each man proposes to his favorite woman. Each woman who receives
more than one proposal rejects all but her favorite from among those who have
proposed to her. However, she does not accept him yet, but keeps him on string to
allow for the possibility that someone better may come along later.
Second, those men who have been rejected now propose to their second choices.
Each woman receiving proposals chooses her favorite from the new proposers and
the man on her string, if any. She rejects all the rest.
Every next stage proceeds in the same manner. Those who are rejected at the
previous stage propose to their next choices, and the women again reject all but
the best proposal they have had so far.
Eventually, every woman will have received a proposal, for as long as any
woman has not been proposed to there will be rejections and new proposals, but
since no man can propose to the same woman more than once, every woman will
get a proposal. As soon as all girls are proposed to, the procedure stops and each
woman accepts the man on her string.
The resulting matching is stable. Suppose John and Mary are not married to
each other but John prefers Mary to his wife. The John must have proposed to
Mary at some stage and has been rejected in favor of someone whom Mary liked
better. Thus, Mary prefers her husband to John and there is no instability.
The stable matching generated by Gale-Shapley algorithm with men proposing
is (weakly) preferred by every man to any other stable matching, in particular to a
matching generated by Gale-Shapley algorithm with women proposing. Similarly,
the stable matching generated by Gale-Shapley algorithm with women proposing
is (weakly) preferred by every woman to any other stable matching.
This mechanism is not in general strategy-proof. Consider the following example (Figure 12.4).
m1
w2
w1
w3
m2
w1
w2
w3
m3
w1
w2
w3
w1
m1
m3
m2
w2
m3
m1
m2
w3
m1
m2
m3
Figure 12.4. Another marriage problem.
The stable matching generated by Gale-Shapley algorithm with men proposing
is (m1 , w2 ), (m2 , w3 ), (m3 , w1 ). Now, suppose that woman 1 acts as if her first choice
is m1 , the second choice is m2 , and the third choice is m3 . In that case, the outcome
of Gale-Shapley algorithm with men proposing will be (m1 , w1 ), (m2 , w3 ), (m3 , w2 ).
Thus, by misrepresenting her preferences, m1 is able to achieve the better (from
her point of view) match.
Bibliography
Robert J. Aumann (1974): “Subjectivity and Correlation in Randomized Strategies,” Journal of Mathematical Economics, 1, 67–96.
Robert J. Aumann (1976): “Agreeing to Disagree,” The Annals of Statistics, 4,
1236–1239.
Robert J. Aumann (1987a): “Correlated Equilibria as an Expression of Bayesian
Rationality,” Econometrica, 55, 1–18.
Robert J. Aumann (1987b): “Game Theory,” in The New Palgrave Dictionary
of Economics, edited by J. Eatwell, M. Milgate, and P. Newman, 460–482, W.W.
Norton, New York.
Robert J. Aumann (1990): “Nash Equilibria Are Not Self-Enforcing,” in Economic Decision-Making: Games, Econometrics, and Optimization, edited by J.J.
Gabszewicz, J.-F. Richard, and A.L. Wolsey, 201–206, Elsevier, Amsterdam.
Robert J. Aumann (1992): “Irrationality in Game Theory,” in Economic Analysis
of Markets and Games, edited by P. Dasgupta, D. Gale, O. Hart, and E. Maskin,
214–227, MIT Press, Cambridge, MA.
Robert J. Aumman (1987): “What is Game Theory Trying to Accomplish,” in
Frontiers of Economics, edited by Kenneth J. Arrow and S. Honkapohja, 28–100,
Basil Blackwell, Cambridge.
Ken Binmore (1987): “Modeling Rational Players, Part I,” Journal of Economics
and Philosophy, 3, 179–214.
Ken Binmore (1988): “Modeling Rational Players, Part II,” Journal of Economics
and Philosophy, 4, 9–55.
Adam Brandenburger (1992): “Knowledge and Equilibrium in Games,” Journal
of Economic Perspectives, 6(4), 83–102.
Augustin A. Cournot (1838): Recherches sur les principes mathématiques de
la théorie des richesse, M. Riviere, Paris, Translated in Researches into the
Mathematical Principles of Wealth, A. M. Kelly, New York, 1960.
Melvin Dresher, Llyod S. Shapley, and A. W. Tucker, editors (1964):
Advances in Game Theory (Annals of Mathematics Studies, 52), Princeton University Press, Princeton.
Melvin Dresher, A. W. Tucker, and P. Wolfe, editors (1957): Contributions to the Theory of Games, Volume III (Annals of Mathematics Studies, 39),
Princeton University Press, Princeton.
Francis Ysidro Edgeworth (1881): Mathematical Psychics: An Essay on
the Application of Mathematics to the Moral Sciences, Kegan Paul, London,
(Reprinted New York: Augustus M. Kelley, 1967).
John Geanakoplos (1992): “Common Knowledge,” Journal of Economic Perspectives, 6(4), 53–82.
D. B. Gillies (1953): Some Theorems on N-Person Games, Ph.D. thesis, Department of Mathematics, Princeton University.
D. B. Gillies (1959): “Solutions to General Non-Zero-Sum Games,” in Contributions to the Theory of Games, Volume IV (Annals of Mathematics Studies,
40), edited by A. W. Tucker and R. D. Luce, 47–85, Princeton University Press,
89
90
Bibliography
Princeton.
John C. Harsanyi (1967-1968): “Games with Incomplete Information Played by
‘Bayesian’ Players, I–III,” Management Science, 14, 159–182, 320–334, 486–502.
John C. Harsanyi (1973): “Games with Randomly Distributed Payoffs: A
New Rationale for Mixed Strategy Equilibrium Points,” International Journal
of Game Theory, 2, 1–23.
John C. Harsanyi and Reinhard Selten (1988): A General Theory of Equilibrium Selection in Games, MIT Press, Cambridge MA.
Elon Kohlberg (1990): “Refinement of Nash Equilibrium: The Main Ideas,” in
Game Theory and Applications, edited by Tatsuro Ichiishi, Abraham Neyman,
and Yair Tauman, Academic Press, San Diego CA.
David M. Kreps (1987): “Nash Equilibrium,” in The New Palgrave Dictionary of
Economics, edited by J. Eatwell, M. Milgate, and P. Newman, 460–482, W.W.
Norton, New York.
David M. Kreps (1990): Games Theory and Economic Modeling, Oxford University Press, New York, NY.
David M. Kreps and Robert Wilson (1982): “Sequential Equilibria,” Econometrica, 50, 863–894.
H. W. Kuhn and A. W. Tucker, editors (1950): Contributions to the Theory
of Games, Volume I (Annals of Mathematics Studies, 24), Princeton University
Press, Princeton.
H. W. Kuhn and A. W. Tucker, editors (1953): Contributions to the Theory
of Games, Volume II (Annals of Mathematics Studies, 28), Princeton University
Press, Princeton.
Harold W. Kuhn (1953): “Extensive Games and the Problem of Information,”
in Contributions to the Theory of Games, Vol. 2, 193–216, Princeton University
Press, Princeton NJ.
L. H. Loomis (1946): “On a Theorem of von Neumann,” Proceedings of the National Academy of Sciences of the United States of America, 32, 213–215.
R. Duncan Luce and Howard Raiffa (1957): Games and Decisions: Introduction and Critical Survey, John Wiley & Sons, New York.
John Maynard Smith and George R. Price (1973): “The Logic of Animal
Conflict,” Nature, 246, 15–18.
Roger Myerson (1978): “Refinement of the Nash Equilibrium Concept,” International Journal of Game Theory, 7, 73–80.
John Nash (1950a): “The Bargaining Problem,” Econometrica, 18, 155–162.
John Nash (1950b): “Equilibrium Points in N -Person Games,” Proceedings of the
National Academy of Sciences, 36, 48–49.
John Nash (1951): “Non-Cooperative Games,” Annals of Mathematics, 54, 286–
295.
John Nash (1953): “Two Person Cooperative Games,” Econometrica, 21, 128–140.
Philip J. Reny (1992): “Rationality in Extensive Form Games,” Journal of Economic Perspectives, 6(4), 103–118.
Ariel Rubinstein (1989): “The Electronic Mail Game: Strategic Behavior Under
‘Almost Common Knowledge’,” American Economic Review, 79, 385–391.
Ariel Rubinstein (1991): “Comments on the Interpretation of Game Theory,”
Econometrica, 59, 909–924.
Thomas C. Schelling (1960): The Strategy of Conflict, Harvard University Press,
Cambridge, Mass.
Ulrich Schwalbe and Paul Walker (2001): “Zermelo and the Early History
of Game Theory,” Games and Economic Behavior, 34, 123–137.
Reinhard Selten (1965): “Speiltheoretische Behandlung eines Oligopolmodells
Bibliography
91
mit Nachfragetragheit,” Zeitschrift fur die gesamte Staatswissenschaft, 121, 301–
324, 667–689.
Reinhard Selten (1975): “Reexamination of the Perfectness Concept for Equilibrium Points in Extensive Games,” International Journal of Game Theory, 4,
25–55.
Reinhard Selten (1978): “The Chain-Store Paradox,” Theory and Decision, 9,
127–159.
Lloyd S. Shapley (1953a): “A Value for n-Person Games,” in Contributions to
the Theory of Games, Volume II (Annals of Mathematics Studies, 28), edited by
H. W. Kuhn and A. W. Tucker, 47–85, Princeton University Press, Princeton.
Llyod S. Shapley (1953b): “Stochastic Games,” Proceedings of the National
Academy of Sciences of the United States of America, 39, 1095–1100.
Llyod S. Shapley and Martin Shubik (1954): “A Method for Evaluating
The Distribution of Power in a Committee System,” American Political Science
Review, 48, 787–792.
Martin Shubik (1959a): “Edgeworth Market Games,” in Contributions to the
Theory of Games, Volume IV (Annals of Mathematics Studies, 40), edited by
A. W. Tucker and R. D. Luce, 267–278, Princeton University Press, Princeton.
Martin Shubik (1959b): Strategy and Market Structure: Competition, Oligopoly,
and the Theory of Games, Wiley, New York.
A. W. Tucker and R. D. Luce, editors (1959): Contributions to the Theory of
Games, Volume IV (Annals of Mathematics Studies, 40), Princeton University
Press, Princeton.
Jean Ville (1938): “Note sur la theorie generale des jeux ou intervient l’habilite
des jouers,” in Applications aux jeux de hasard, Tome IV, Fascicule II of Traite
du calcul des probabilities et de ses applications, edited by Emile Borel, 105–113,
Gauthier-Villars, Paris.
John von Neumann (1928): “Zur Theorie der Gesellschaftsspiele,” Mathematische
Annalen, 100, 295–320, (Translated as “On the Theory of Games of Strategy”,
pp.13-42 in Contributions to the Theory of Games, Volume IV (Annals of Mathematics Studies, 40) (A. W. Tucker and R. D. Luce, eds.), Princeton University
Press, Princeton, 1959).
John von Neumann and Oskar Morgenstern (1944): Theory of Games and
Economic Behavior, Princeton University Press, Princeton NJ.
Paul
Walker
(2012):
“A
Chronology
of
Game
Theory,”
http://www.econ.canterbury.ac.nz/personal_pages/paul_walker/gt/hist.htm.
E. Zermelo (1913): “Über eine Anwendungen der Mengenlehre auf die Theorie
der Schachspiels,” in Proceedings of the International Fifth Congress of Mathematicians, Cambridge, 1912, Vol. II, 501–504, Cambridge, Cambridge University
Press.