Outline
Introduction
Complexity
Finding equilibria in 2-player games
Computing Nash-equilibria in 2-player games and
beyond
Algorithms Reading Group
Aditya Menon
June 3, 2007
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Introduction
Nash equilibrium
A formal definition
Complexity
Characterizing complexity
PPAD
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Nash equilibrium
A formal definition
Nash-equilibrium - Recap
I
I
Refers to a special kind of state in an n-player game
No player has an incentive to unilaterally deviate from his
current strategy
I
I
A kind of “stable” solution
Existence depends on the type of game
I
I
If strategies are “pure” i.e. deterministic, does not have to
exist in the game
If strategies are “mixed” i.e. probabilistic, then it always exists
I
Yet how do we find it!?!
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Nash equilibrium
A formal definition
Notation
I
Suppose that player p follows the mixed strategy
xp = (xp1 , . . . , xpnp )
I
I
I
The ith entry gives the probability that player p plays move i
Let x := (x1 , . . . , xn ) be the collection of strategies for all
players
Let the function Up (x) denote the expected utility or payoff
that player p gets when each player uses the strategy dictated
in x
I
Based on up (s1 , . . . , sn ), which is the utility when player q
plays sq
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Nash equilibrium
A formal definition
Formal definition
I
We say that x∗ = (x1 ∗ , . . . , xn ∗ ) is a Nash equilibrium if...
I
“No player has an incentive to unilaterally deviate from his
current strategy”
I
If player p decides to switch to a strategy yp , then write the
resulting strategy set as x−p ; yp
I
So, x∗ is a N.E. if, for every player p, and for any mixed
strategy yp for that player, we have
Up (x∗ ) ≥ Up (x∗−p ; yp )
I
A more symmetric version:
Up (x∗−p ; x∗p ) ≥ Up (x∗−p ; yp )
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Nash equilibrium
A formal definition
Why the equilibrium exists
I
Based on Kakutani’s fixed point thereom
I
Generalizes the following: take two pieces of paper (which are
3D objects) and imagine placing them on top of each other,
so that the “points” align. Crumple the top sheet without
tearing it, and place it on top of the other sheet. Then, there
is always a point that has not changed position on the two
sheets.
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Characterizing complexity
PPAD
Questions about finding Nash equilibria
I
I
Proof of existence is non-constructive
So how do we find it?
I
I
Can we find it?
Can we find it efficiently ?
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Characterizing complexity
PPAD
Do we really care?
I
I
Obviously an “interesting” problem, but is there a significant
benefit in being able to compute it?
Arguably, equilibrium is not useful unless we can find it
I
I
Consider the case of a market
“If your laptop can’t find it [the equilibrium], then neither can
the market” [Papadimitriou]
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Characterizing complexity
PPAD
Importance of the problem
I
“Together with factoring, the complexity of finding a Nash
equilibrium is in my opinion the most important concrete open
question on the boundary of P today” [Papadimitriou]
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Characterizing complexity
PPAD
Complexity of the problem
I
Need to define a special class, PPAD, for this problem
I
Turns out that finding the Nash-equilibrium is PPAD-complete
What about NP?
I
I
I
Probably not NP-complete
The decision version is in P
I
Why?
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Characterizing complexity
PPAD
Complexity of the problem
I
Need to define a special class, PPAD, for this problem
I
Turns out that finding the Nash-equilibrium is PPAD-complete
What about NP?
I
I
I
Probably not NP-complete
The decision version is in P
I
I
Why?
Because the equilibrium always exists!
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Characterizing complexity
PPAD
Polynomial parity argument
I
Simple fact: Every graph has an even number of odd-degree
nodes
I
Proof: Let W = {v ∈ V : v has odd degree }
X
X
2|E | =
deg(v ) +
deg(v )
v ∈W
=
X
v ∈W
/
odd + even
v ∈W
I
Corollary: If a graph has maximum degree 2, then it must
have an even number of leaves
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Characterizing complexity
PPAD
The class TFNP
I
PPAD is based on the class TFNP
I
Suppose we have a set of binary predicates P(x, y ) where
(∀x)(∃y ) : P(x, y ) = TRUE
I
Problems in TFNP: Given an x, find a y so that P(x, y ) is
TRUE
I
Subclasses defined based on how we decide (∃y ) : P(x, y ) is
TRUE
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Characterizing complexity
PPAD
PPAD in terms of TFNP
I
PPAD is defined by the following problem
I
I
I
I
Suppose we have an exponential-size graph where the
in-degree and out-degree of each node is at most 1
Given any node v , we have a polynomial-time algorithm that
finds the neighbours of v
Problem: Given a leaf node, output another leaf node
Existence is guaranteed by the parity argument
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Characterizing complexity
PPAD
Completeness of finding Nash equilibrium
I
Finding a Nash equilibrium is PPAD-complete
I
For 4-player games... [Papadimitriou]
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Characterizing complexity
PPAD
Completeness of finding Nash equilibrium
I
Finding a Nash equilibrium is PPAD-complete
I
I
For 4-player games... [Papadimitriou]
...and 3-player games [Papadimitriou]
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Characterizing complexity
PPAD
Completeness of finding Nash equilibrium
I
Finding a Nash equilibrium is PPAD-complete
I
I
I
For 4-player games... [Papadimitriou]
...and 3-player games [Papadimitriou]
...and even for 2-player games! [Chen]
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Characterizing complexity
PPAD
Completeness of finding Nash equilibrium
I
Finding a Nash equilibrium is PPAD-complete
I
I
I
For 4-player games... [Papadimitriou]
...and 3-player games [Papadimitriou]
...and even for 2-player games! [Chen]
I
So finding the Nash equilibrium even for 2-player games is no
easier than doing it for n-players!
I
At the moment, however, not much known about how “hard”
a class PPAD is
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Approaches to finding equilibria
I
No P algorithms known!
I
Most approaches are based on solving non-linear programs
(for general n)
I
Completeness result means even 2-player games are not (yet)
“easy” to solve
I
One of the earliest algorithms for finding equilibria in 2-player
games: Lemke-Howson algorithm
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Lemke-Howson algorithm
I
I
An algorithm for finding the Nash equilibrium for a game with
2 players
Developed in 1964
I
Independent proof of why equilibrium must exist
I
Worst-case exponential time, but in practise quite good
performance
I
Still viewed favourably, although it has been beaten of late
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
How we proceed
I
I
We need to redefine a Nash equilibrium for 2-players
Try and make a graph that lets us find equilibria easily
I
Exploiting the convenience of the alternate definition
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Utility for 2-players
I
Suppose that for a 2-player game, we have the mixed
strategies x = (s, t)
I
Label the strategies by I = {1, . . . , m} for player 1, and
J = {m + 1, . . . , m + n} for player 2
I
Expected utility for player p must be
XX
Up (x) =
Pr[player 1 chooses i] × Pr[player 2 chooses j]
i
j
× Payoff for player p when 1 plays i and 2 plays j
XX
=
s(i)t(j)up (i, j)
i
j
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Nash equilibria for two players
I
We call x∗ = (s∗ , t∗ ) a Nash equilibrium iff
XX
XX
(∀s)
s∗ (i)t∗ (j)u1 (i, j) ≥
s(i)t∗ (j)u1 (i, j)
i
(∀t)
j
XX
i
i
s∗ (i)t∗ (j)u2 (i, j) ≥
j
XX
i
Aditya Menon
j
s∗ (i)t(j)u2 (i, j)
j
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Claim
I
Claim: If in a Nash equilibrium player p can play strategy i
(non-zero probability), then if player p chooses to play
strategy i, it is the best possible strategy
I
Mathematically,
s∗ (i) > 0 =⇒ (∀i0 )
X
t∗ (j)u1 (i, j) ≥
j
t∗ (j) > 0 =⇒ (∀j0 )
X
X
t∗ (j)u1 (i0 , j) (1)
j
s∗ (i)u2 (i, j) ≥
i
Aditya Menon
X
s∗ (i)u2 (i, j0 ) (2)
i
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Lemma
I
Lemma: Let πp,i denote the mixed strategy (0, . . . , 1, . . . , 0) i.e. we
deterministically choose strategy i for player p. Then, x is a Nash
Equilibrium iff
(∀p, πp,i )Up (x) ≥ Up (x−p ; πp,i )
I
Proof: (note that =⇒ direction is by definition)
(
)
X
X
x1 (i1 ) . . . xn (in )up (x−p ; πp,ip )
yp (ip )
Up (x−p ; yp ) =
=
ip
i1 ...in
X
yp (ip )Up (x−p ; πp,ip )
ip
≤ Up (x) since
Aditya Menon
X
yp (i) = 1
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Proof of claim
I
Use the lemma: Up (x∗ ) ≥ Up (x−p ; πp,i )
X
X
Up (x∗ ) =
x∗p (i)Up (x∗ ) since
x∗p (i) = 1
X
≥
x∗p (i)Up (x∗−p ; πp,i )
= Up (x∗ )
I
So, we have to conclude that
X
X
x∗p (i)Up (x∗ ) =
x∗p (i)Up (x∗−p ; πp,i )
I
Taking terms to one side,
x∗p (i) > 0 =⇒ Up (x∗ ) = Up (x∗−p ; πp,i )
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Reformulation of Nash equilibrium - I
I
So, x∗ is a Nash equilibrium iff
I
I
For player 1, equation 1 holds or Pr[strategy i] = 0
For player 2, equation 2 holds or Pr[strategy j] = 0
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Reformulation of Nash equilibrium - II
I
Define
S i = {s : s(i) = 0}, S j = {s :
X
s(i)u2 (i, j) ≥
X
i
T j = {t : t(j) = 0}, T i = {t :
i
X
t(j)u1 (i, j) ≥
j
I
s(i)u2 (i, j0 )}
X
t(j)u1 (i0 , j)}
j
Then, x∗ is a Nash equilibrium iff
(∀i)s ∈ S i ∨ t ∈ T i
(∀j)s ∈ S j ∨ t ∈ T j
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Labelling
I
We are claiming that x = (s, t) is an equilibrium iff...
I
I
For any k ∈ I ∪ J, either s or t (or maybe both) is in the
appropriate region S k or T k
Can think of these k’s as labels of points
I
I
I
Labels(s) = {k ∈ I ∪ J : s ∈ S k }
Labels(t) = {k ∈ I ∪ J : t ∈ T k }
Note: This definition also makes sense even when s, t aren’t
valid strategies
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Reformulation of Nash equilibrium - III
I
Natural label for x = Labels(s)∪ Labels(t)
I
So, x is a Nash equilibrium iff it is completely labelled
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Strategy simplex
I
m strategies =⇒ valid strategy space is an (m − 1)
dimensional simplex
(0, 0, 1)
(0, 0, 0)
(1, 0, 0)
(0, 1, 0)
I
With labelling, we can split up the simplex into regions
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Labelling - example
I
For the payoff matrix A = [0 6; 2 5; 3 3]
I
Label the strategy space for player 2:
(0, 1)
1, 4
1, 2
4
1 2
,
3 3
2, 3
2 1
,
3 3
3, 5
5
Aditya Menon
(1, 0)
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Reformulation of problem
I
I
Using the labelling definition, x is a Nash equilibrium iff it is
completely labelled
∼ Point that lie in the intersection of many regions
I
I
I
But here there are two variables, and two sets of spaces
Can “sacrifice” one for the sake of the other
New problem: How do we find points that are completely
labelled?
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
High-level solution
I
With labelling, we have divide the strategy spaces into
sub-spaces
I
We want to end up with a pair of strategies that together “lie
in the intersection of the spaces”
I
Search problem: starting off with some initial pair of
strategies, we want to find the equilibrium
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
High-level solution
I
Think of the space as a graph
I
Vertices should correspond to strategy pairs
Edges should define some well-defined change in the strategies
I
I
Which makes it easy to find equilibria
I
Following a well-defined path on this graph should take us to
equilibria
I
Problem: How do we make such a graph?
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Graph construction
I
Form the graphs GS = (VS , ES ), GT = (VT , ET ) where:
I
I
I
I
P
VS ← {s ∈ Rm
si ≤ 1 ∧ s has exactly m labels }
+ :
ES between s1 , s2 if they differ in exactly one label
Similarly for VT , ET
Note: This is now “filling” the strategy simplex
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Example
I
Payoff B = [1 0; 0 2; 4 3]
I
Vertices lie on simplex, except for 0
(0, 0, 1) L = 1, 2, 4
0, 13 , 23
L = 1, 2, 4
(0, 0, 0)
(1, 0, 0)
L = 2, 3, 5
L = 1, 2, 3
2 1
, ,0
3 3
L = 3, 4, 5
(0, 1, 0)
L = 1, 3, 5
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Graph construction
I
Form the product graph G = GS × GT
I
I
V = {(s, t) : s ∈ VS , t ∈ VT }
E = {(s1 , t1 ) → (s1 , t2 ) : t1 → t2 } ∪ {(s1 , t1 ) → (s2 , t1 ) :
s1 → s2 }
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Graph and equilibria
I
So now, have a graph where vertices are strategy pairs
I
Know that the equilibria is one of the vertices
I
Question: How do we find this vertex?
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Two definitions
I
Define:
I
I
L− (k) := vertices that have all labels except, possibly, k
L := vertices that have all labels
I
By definition, L ⊆ L− (k)
I
L = (0, 0) ∪ {Equilibria}
I
We can prove some properties about these sets...
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Fact 1
I
Fact: For any k, every member of L is adjacent to exactly one
member of L− (k) − L
I
For any label, every (pseudo) equilibrium is adjacent to exactly
one strategy pair that is missing that label
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Fact 1
I
Fact: For any k, every member of L is adjacent to exactly one
member of L− (k) − L
I
I
For any label, every (pseudo) equilibrium is adjacent to exactly
one strategy pair that is missing that label
Proof:
I
I
I
I
Let (s, t) ∈ L. Then the label k must apply to either s or t, by
definition
Suppose that s is labelled with k. Then, there must be an
edge between (s, t) and the point (s0 , t) where s0 is missing the
label k
There is only one such s0 that is missing the label k - hence
the neighbour is unique
Similar argument if t is labelled with k
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Fact 2
I
Fact: For any k, every member of L− (k) − L is adjacent to
exactly two members of L− (k)
I
Every strategy pair missing exactly one label is adjacent to
exactly two other strategy pairs that are potentially missing
the same label
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Fact 2
I
Fact: For any k, every member of L− (k) − L is adjacent to
exactly two members of L− (k)
I
I
Every strategy pair missing exactly one label is adjacent to
exactly two other strategy pairs that are potentially missing
the same label
Proof:
I
I
I
I
Since
|Labels(s, t)| = m + n − 1 6= |Labels(s)| + |Labels(t)| = m + n,
there must be a duplicate label, `
In the graph GS , we must have an edge from s to some other
point s0 , where s0 does not have the label `
Then, the edge (s, t) → (s0 , t) must belong to E
Similarly for GT - this means that the graph G has exactly two
edges that change the labelling
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Putting the facts together
I
L− (k) describes a subgraph of G containing (disjoint) paths
and loops of G
I
The endpoints of a path in G are (pseudo) equilibria
Problem: How do we find this set quickly?
I
I
Look at this later...
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Finding equilibria
I
Start off at the “artificial equilibrium” (0, 0)
I
Choose an arbitrary label ` ∈ I ∪ J
I
Follow the path generated by the set L− (`)
I
When we reach the end of the path, we necessarily have
stopped at an equilibria
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Example
I
Payoff matrices

0
A = 2
3

1
B = 0
4
I

6
5
3

0
2
3
Choose the label 2 to be dropped i.e. move along L− (2)
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Example
I
Start off at the artificial equilibrium, ((0, 0, 0), (0, 0)) → labels
{1, 2, 3}, {4, 5}
(0, 1) L = 1, 4
(0, 0, 1) L = 1, 2, 4
0, 13 , 23
1 2
,
3 3
L = 1, 2
L = 1, 2, 4
(0, 0, 0)
(1, 0, 0)
L = 2, 3, 5
L = 1, 2, 3
2 1
, ,0
3 3
(0, 0)
L = 4, 5
2 1
,
3 3
L = 2, 3
(1, 0)
L = 3, 5
L = 3, 4, 5
(0, 1, 0)
L = 1, 3, 5
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Example
I
Step 1: ((0, 1, 0), (0, 0)) → labels {1, 3, 5}, {4, 5}; duplicate is
5
(0, 1) L = 1, 4
(0, 0, 1) L = 1, 2, 4
0,
1 2
,
3 3
1 2
,
3 3
L = 1, 2
L = 1, 2, 4
(0, 0, 0)
(1, 0, 0)
L = 2, 3, 5
L = 1, 2, 3
2 1
, ,0
3 3
(0, 0)
L = 4, 5
2 1
,
3 3
L = 2, 3
(1, 0)
L = 3, 5
L = 3, 4, 5
(0, 1, 0)
L = 1, 3, 5
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Example
I
Step 2: ((0, 1, 0), (0, 1)) → labels {1, 3, 5}, {1, 4}; duplicate is
1
(0, 1) L = 1, 4
(0, 0, 1) L = 1, 2, 4
0,
1 2
,
3 3
1 2
,
3 3
L = 1, 2
L = 1, 2, 4
(0, 0, 0)
(1, 0, 0)
L = 2, 3, 5
L = 1, 2, 3
2 1
, ,0
3 3
(0, 0)
L = 4, 5
2 1
,
3 3
L = 2, 3
(1, 0)
L = 3, 5
L = 3, 4, 5
(0, 1, 0)
L = 1, 3, 5
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Example
I
Step 3: (( 23 , 13 , 0), (0, 1)) → labels {3, 4, 5}, {1, 4}; duplicate is
4
(0, 1) L = 1, 4
(0, 0, 1) L = 1, 2, 4
0,
1 2
,
3 3
1 2
,
3 3
L = 1, 2
L = 1, 2, 4
(0, 0, 0)
(1, 0, 0)
L = 2, 3, 5
L = 1, 2, 3
2 1
, ,0
3 3
(0, 0)
L = 4, 5
2 1
,
3 3
L = 2, 3
(1, 0)
L = 3, 5
L = 3, 4, 5
(0, 1, 0)
L = 1, 3, 5
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Example
I
Step 4: (( 32 , 13 , 0), ( 13 , 32 )) → labels {3, 4, 5}, {1, 2}
(0, 1) L = 1, 4
(0, 0, 1) L = 1, 2, 4
0, 13 , 23
1 2
,
3 3
L = 1, 2
L = 1, 2, 4
(0, 0, 0)
(1, 0, 0)
L = 2, 3, 5
L = 1, 2, 3
2 1
, ,0
3 3
(0, 0)
L = 4, 5
2 1
,
3 3
L = 2, 3
(1, 0)
L = 3, 5
L = 3, 4, 5
(0, 1, 0)
L = 1, 3, 5
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Example
I
Step 4: (( 32 , 13 , 0), ( 13 , 32 )) → labels {3, 4, 5}, {1, 2}
I
Completely labelled, and so an equilibria
(0, 1) L = 1, 4
(0, 0, 1) L = 1, 2, 4
0, 13 , 23
1 2
,
3 3
L = 1, 2
L = 1, 2, 4
(0, 0, 0)
(1, 0, 0)
L = 2, 3, 5
L = 1, 2, 3
2 1
, ,0
3 3
(0, 0)
L = 4, 5
2 1
,
3 3
L = 2, 3
(1, 0)
L = 3, 5
L = 3, 4, 5
(0, 1, 0)
L = 1, 3, 5
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Algorithm summary
I
Consider strategies sk that have all labels except, possibly, some
label k
I
I
Clearly, every equilibrium belongs to this set
So too does the “artificial equilibrium”, 0
I
Construct a graph from all strategies
I
Then, one can show:
I
I
I
Each strategy missing a label is adjacent to exactly two such
strategies
Each equilibrium is adjacent to only one strategy sk
It follows that:
I
Equilibria are endpoints of paths on the graph
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Generating L− (`)
I
Second problem...
I
How do we find adjacent strategies?
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Generating L− (`)
I
“Pivoting” approach
I
Able to implicitly generate the graph, on-the-fly
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Optimization problem
I
Suppose we know player 2’s strategy to be t
I
Suppose we have payoff matrices A, B
I
We need to choose our strategy to maximize the payoff, i.e.
XX
U1 (x) =
s(i)t(j)A(i, j)
i
=
X
j
s(i)A(i, :)t
i
T
= s (At)
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Optimization problem
I
Primal:
maximize sT (At) :
I
X
s(i) = 1, s ≥ 0
Dual:
minimize u : u1 ≥ At
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Linear system
I
We need to solve
s.1 = 1
t.1 = 1
u1 − At ≥ 0
v 1 − BT s ≥ 0
s, t ≥ 0
I
Equilibrium iff the above, and
sT (u1 − At) = 0
tT (v 1 − B T s) = 0
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Linear system with slackness
I
Introduce slack variables z, w:
s.1 = 1
t.1 = 1
−u1 + At + w.1 = 0
−v 1 + B T s + z.1 = 0
s, z ≥ 0
t, w ≥ 0
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
System in matrix form
I
Can write system as a matrix equation
I
Consider player 1’s equations - same as:
s(1) + . . . + s(m) = 1
bT
i .s − v + z(i) = 0, i = 1, . . . , n
s, z ≥ 0
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
System in matrix form
I
Can write as Dr = c, where


1
1 ...
1
0 0 ... 0
b11 b21 . . . bm1 −1 1 . . . 0


D=

..


.
b1n b2n . . . bmn −1 0 . . . 1


s(1)
 .. 
 
 . 
1


s(m)
0


 

r=
.
 v , and c = 
 .. 
 z(1) 


0
 .. 
 . 
z(n)
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Connection with labelling
I
I
Let the basis of the column space of D be {Dβ }
Call rj ...
I
I
I
Labels correspond to non-basic variables
I
I
I
A basic variable if Dj is in the basis
A non-basic variable otherwise; let rj = 0 in this case
r1 , . . . , rm = 0 =⇒ s(i) = 0 i.e. strategy is not played
rm+2 , . . . , rm+n+1 = 0 =⇒ z(i) = 0 i.e. strategy is
best-response
So, removing a label corresponds to making a variable strictly
positive
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Rewriting system
I
System is the same as
Dβ−1 (Dr) = Dβ−1 c
I
Fact: Dβ−1 D = I +
I
System is the same as
P
j ∈β
/
Dβ−1 Dj
rβ = Dβ−1 c −
X
Dβ−1 Dj rj
j ∈β
/
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Why rewrite?
I
I
What is wrong with the standard representation of the
system?
Nothing “wrong” with it...
I
I
I
Just that the new system makes it easier to find strategies
with different labels
i.e. Generating the set L− (`)
This is the “pivot” operation
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Example
I
Use earlier payoff matrix
I
Matrix D for player 1 is


1 1 1 0 0 0
D = 1 0 4 −1 1 0
0 2 3 −1 0 1
I
Choose column-space {2, 4, 5}
I
I
Setting x1 , x3 = 0
Setting z2 = 0


1 0 0
Dβ = 0 −1 1
2 −1 0
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Example
I
Solution:
   
 
 
1
1
1
0







rβ = 2 − 2 r1 − −1 r3 − −1 r6
2
3
3
−1
I
Or equivalently:
x2 = 1 − x1 − x 3
v = 2 − 2x1 + x3 + z2
z1 = 2 − 3x1 − 3x3 + z2
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Example
I
Can read off solution from here
I
I
I
s = (0, 1, 0)
v =2
z1 = 2, z2 = 0
I
Can read the labels: 1, 3, 5
I
Do the same thing for player 2
I
Result: We get a pair of strategies (s, t), and we know what
their labels are
I
Question: How do we find the “adjacent” strategy, as per the
graph?
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Finding adjacent strategy
I
Traversal step in the algorithm was...
I
I
I
I
Find the duplicate label in the two strategies
Drop the label from player 1 and player 2 alternately
Not hard to find duplicate (just read off from solution)
How to drop label?
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Finding adjacent strategy
I
Traversal step in the algorithm was...
I
I
I
I
Find the duplicate label in the two strategies
Drop the label from player 1 and player 2 alternately
Not hard to find duplicate (just read off from solution)
How to drop label?
I
Just change the appropriate variable to be positive
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Finding adjacent strategy
I
Traversal step in the algorithm was...
I
I
I
I
Find the duplicate label in the two strategies
Drop the label from player 1 and player 2 alternately
Not hard to find duplicate (just read off from solution)
How to drop label?
I
I
Just change the appropriate variable to be positive
As large as possible while satisfying constraints
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Example of pivoting
I
Starting equation for player 1:
x2 = 1 − x1 − x 3
v = 2 − 2x1 + x3 + z2
z1 = 2 − 3x1 − 3x3 + z2
I
Starting equation for player 2:
y2 = 1 − y1
u = 6 − 6y1 + w1
w2 = 1 − 3y1 + w1
w3 = 3 − 6y1 + w1
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Example of pivoting
I
Solve to get ((0, 1, 0), (0, 1))
I
I
I
x1 = 0 and w1 = 0, so label 1 is duplicate
I
I
I
s = (0, 1, 0), z = (2, 0)
w = (0, 1, 3), t = (0, 1)
So we look to get rid of it
=⇒ Modify x1 while satisfying non-negativity constraints
Relevant equations:
x 2 = 1 − x1
z1 = 2 − 3x1
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Example of pivoting
I
For first equation: x1 ∈ (0, 1]
I
For second equation: x1 ∈ 0, 32
I
Clearly, x1 →
I
I
I
I
2
3
x2 → 13
But, z1 → 0!
As a result, we see that label 4 is duplicate
Another duplicate =⇒ we repeat the process!
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Updating system
I
Now x1 should appear on the LHS, since it is no longer zero
I
New column basis is {D1 , D2 , D4 }


1 1 0
Dβ = 1 0 −1
0 2 −1
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Updating system
I
New system is
2
2
1
− 3x3 − z1 + z2
3
3
3
1
1
1
x1 = − x3 − z1 + z2
3
3
3
1 1
1
x2 = + z1 − z2
3 3
3
v=
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Complementary pivoting
I
Newly added label will be duplicate
I
e.g. Label 4 is added, and is a duplicate
Make y1 positive
I
I
I
I
y2 = 1 − y1 =⇒ y1 ∈ (0, 1]
w2 = 1 − 3y1 + w1 =⇒ y1 ∈ 0, 13
So, new strategy is
I
I
1 2
3, 3
Now completely labelled
Hence, we have found the equilibria
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Performance of Lemke-Howson
I
I
Worst-case exponential running time
In practise, reasonably fast
I
Of late, been beaten by a few competing algorithms e.g.
[SHNP]
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Problems?
I
Does not generalize to n > 2 players
I
Sometimes, equilibria may be out of reach
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Other approaches
I
Many more techniques...
I
I
I
I
I
I
Local-search techniques [SHNP]
Mixed integer programming
Global Newton optimization
Computer algebra
Markov Random Fields
etc...
I
Quite a few generalize to more than 2 players
I
Nothing (as yet) tells us about the boundary of P!
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
Xi Chen, Xiaotie Deng, and Shang hua Teng.
Settling the complexity of computing two-player nash
equilibria.
Technical report, 2007.
Ryan Porter, Eugene Nudelman, and Yoav Shoham.
Simple search methods for finding a nash equilibrium.
In AAAI, pages 664–669, 2004.
R. Savani and B. von Stengel.
Exponentially many steps for finding a nash equilibrium in a
bimatrix game, 2004.
Lloyd S. Shapely.
A note on the Lemke-Howson algorithm.
Mathematical Programming Study 1:Pivoting and Extensions,
pages 175 – 189, 1974.
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond
Outline
Introduction
Complexity
Finding equilibria in 2-player games
Lemke-Howson algorithm
Other approaches
B. von Stengel.
Computing equilibria for two-person games.
Handbook of Game Theory, 3, 2002.
Aditya Menon
Computing Nash-equilibria in 2-player games and beyond