Game Theory for Networks
Mainak Chatterjee
University of Central Florida
Mainak Chatterjee
Decisions in daily life
• Our decisions in daily life always depend on payoff/benefit
–
–
–
–
–
–
–
Price paid on ebay
Deals by Wireless service providers
Introduction of Vanilla Pepsi after Vanilla Pepsi
Price competition of airline tickets
Decisions made by ad hoc / sensor nodes
Decisions for MAC protocols
Etc.
• Benefit may be long term or short term
• Does the benefit depend on only what I do or what others
also do?
• We need a theory to analyze these decision problems.
Mainak Chatterjee
What is Game Theory?
• Game Theory is a part of (applied) mathematics
that describes and studies interactive decision
problems.
• In an interactive decision problem the
decisions made by each decision maker affect
the outcomes and, thus, the resulting situation
for all decision makers involved.
Mainak Chatterjee
Early Inventors
John von Neumann
(December 28,1903 – February 8,1957)
John Forbes Nash, Jr.
(born June 13, 1928)
Winner of 1994 the Nobel Prize in Economics
Mainak Chatterjee
What is a Game?
•
A game is model (mathematical
representation) of an interactive decision
situation.
•
Its purpose is to create a formal framework
that captures the relevant information in such a
way that is suitable for analysis.
•
Different situations indicate the use of different
game models.
Mainak Chatterjee
Game Components?
• A Game consists of
– at least two players
– a set of strategies for each player
– a preference relation over possible outcomes
• Player is general entity
– individual, company, nation, protocol, animal, etc
– intelligent and rational
• Strategies
– actions which a player chooses to follow
• Outcome
– determined by mutual choice of strategies
• Preference relation
– modeled as utility (payoff) over set of outcomes
Mainak Chatterjee
Important Questions?
1. Does the game have a steady state?
 (NE Existence)
2. What are those steady states?
 (NE Characterization)
3. Is the steady state(s) desirable?
 (NE Optimality)
4. When will the play reach the steady state(s)?
 (Convergence)
Mainak Chatterjee
Normal Form Game Model
Components
1. A set of 2 or more players
2. A set of actions for each player
3. A set of utility functions that describe the
players’ preferences over the action space
Player 1 Actions {a, b}
Player 2 Actions {A, B}
A
B
a
1,-1
0,2
b
-1,1
2,2
Mainak Chatterjee
Matrix Representation of Game
Strategy set
for Player 1
Player 1
Strategy set
for Player 2
Player 2
A
B
C
A
(2, 2)
(0, 0)
(-2, -1)
B
(-5, 1)
(3, 4)
(3, -1)
• Simultaneous play
Payoff to
Player 1
Payoff to
Player 2
– players analyze the game and write their strategy on a paper
• Combination of strategies determines payoff
Mainak Chatterjee
Why study Games?
• In the majority of markets firms interact with
competitors
Oligopoly market
• Each firm has to consider rival’s actions
– strategic interaction in prices, outputs, advertising…
• This kind of interaction can be best analyzed
using game theory
– assumes that “players” are rational
Mainak Chatterjee
Oligopoly theory
• No one-to-one interaction
– employs game theoretic tools that are appropriate
– outcome depends upon information available
• Need a concept of equilibrium
– players choose strategies, one or more for each player
– combination of strategies determines outcome
– outcome determines pay-offs (profits?)
Equilibrium first formalized by Nash:
No firm wants to change its current strategy given
that no other firm changes its current strategy.
Mainak Chatterjee
The Good News: Nash’s Theorem
• Every two person game has at least one
equilibrium in either pure or mixed strategies
• Proved by Nash in 1950 using fixed point
theorem
– generalized to N person game
• An outcome of a game is a NEP (Nash
equilibrium point) if no player can unilaterally
change its strategy and increase his payoff.
Mainak Chatterjee
Nash equilibrium
• Equilibrium need not be “nice”
– firms might do better by coordinating but such coordination may
not be possible (or legal)
• Some strategies can be eliminated on occasions
– they are never good strategies no matter what the rivals do
• These are dominated strategies
– they are never employed and so can be eliminated
– elimination of a dominated strategy may result in another being
dominated: it also can be eliminated
• One strategy might always be chosen no matter what the
rivals do: dominant strategy
Mainak Chatterjee
An example
• Two airlines set the prices for tickets
• Compete for departure times
• 70% of consumers prefer evening departure, 30% prefer
morning departure
• If the airlines choose the same departure times they share
the market equally
• Pay-offs to the airlines are determined by market shares
• Represent the pay-offs in a pay-off matrix
Mainak Chatterjee
The Payoff Matrix
What is the
equilibrium for this
game?
The left-hand
American
number is the
pay-off to
Morning
Evening
Delta
Morning
(15, 15)
(30, 70)
(70, 30)
The right-hand
number is the
(35, 35)
pay-off
to
American
Delta
Evening
Mainak Chatterjee
The Payoff Matrix
If American
If American
chooses a morning chooses an evening
The morning departure
departure, Delta
The morning
departure,departure
Delta
also choose is also a dominated
will choose
is will
a dominated
strategy for American
evening
strategyevening
for Delta
American
Both airlines
choose an
Morning
Evening
evening
departure
Morning
(15, 15)
(30, 70)
Delta
Evening
(70, 30)
(35, 35)
Mainak Chatterjee
Let’s change the problem
• Now suppose that Delta has a frequent flier
program
• When both airline choose the same departure
times Delta gets 60% of the travelers
• This changes the pay-off matrix
Mainak Chatterjee
Modified
If DeltaPayoff Matrix
However, a
The
Pay-Off Matrix morning departure
chooses
a morning
departure, American
is still a dominated
But
if
Delta
American has
will no
choose
strategy for Delta
American
chooses
an evening
dominated
strategy
evening
departure,
American
American knows
willand
choose
this
so
Morning
Evening
morning
chooses
a morning
departure
Morning
(18, 12)
(30, 70)
Delta
Evening
(70,
(70,30)
30)
(42, 28)
Mainak Chatterjee
Dominated & Dominant Strategies
• In the first case
– (35,35) was the Nash Equilibrium
• In the second case
– (70,30) was the Nash Equilibrium
• In both cases, we made use of the ability to rule
out dominated strategies
• We focus on dominant strategy
– If there is one, use it
– This outperforms all other strategies no matter what
rivals do
Mainak Chatterjee
A Very Popular Example:
The Prisoner’s Dilemma
• One of the most studied and used games
– proposed in 1950s
• Two suspects arrested for joint crime
– each suspect when interrogated separately, has
option to confess or remain silent
Suspect 2
Suspect 1
S
C
S
2, 2
10, 1
C
1, 10
5, 5
better outcome
payoff is years in jail
(smaller is better)
single NEP
Mainak Chatterjee
Applications of Game Theory
• Game theory aimed at modelling situations with
conflicting decision makers
• It has been used primarily in Economics
–
–
–
–
–
Government properties
Firms
Markets
Politics
Etc.
• Recently, Networking and Communications
Mainak Chatterjee
Why Game theory for networks?
•Self-Organization very important
• facilitate large-scale and rapid deployment
• reduce costs
•Nodes make decisions on their own that affects others.
•Resource contention
•Bargaining
•Some types of networks are distributed in nature
• Mesh, ad hoc, sensor networks, etc.
•Distributed protocols (routing, MAC etc)
Mainak Chatterjee
Classification of Games
• Some major classification of games:
1. Non-cooperative vs. Cooperative
2. One-shot vs. Repeated (iterative)
3. Pure Strategy vs. Mixed Strategy
4. Complete information vs. Incomplete Information
5. Static vs. Dynamic
6. Zero-sum vs. Non-zero-sum
7. 2-person vs. N-person
Mainak Chatterjee
Non-cooperative vs. Cooperative
• Non-cooperative games
– Where players cannot make agreements
– “Cheap talk” agreements, but not binding
– Expect inefficient outcomes in one-shot setting
(Prisoner’s Dilemma)
• Cooperative games
– Where players can make agreements
– Expect efficient outcomes
Mainak Chatterjee
One-shot vs. Repeated (iterative)
• One-shot games
– Only played once, no repetition
• Repeated games
– A game which consists of some number of repetitions
of some base game, called a stage game
– Repeated games may be repeated finitely or infinitely
many times
• Finite horizon
• Infinite horizon
Mainak Chatterjee
Pure Strategy vs. Mixed Strategy
• Pure strategy games
– Restrict each player to choice of one strategy or
another
– A player chooses a strategy with probability 1
• Mixed strategy games
– Different strategies are chosen randomly
– Probability is associated with every strategy
– Some probability distribution is defined over the set of
allowed actions
Mainak Chatterjee
Complete information vs.
Incomplete Information
• Complete information games
– A game in which knowledge of players is available to all
players
– Every player knows the payoffs and strategies available to
other players
• Incomplete information games
– A game in which knowledge of payoff and strategies of other
players is not available to a player
– for example, in the context of Prisoner's Dilemma game, the
game is of incomplete information since one prisoner does
not know the action of the other player
Mainak Chatterjee
Static vs. Dynamic
• Static games
– Similar to the one-slot games, where players
have only one move to make
• Dynamic games
– Played in multiple moves where players move
simultaneously or sequentially
– Players change their strategies dynamically
Mainak Chatterjee
Zero-sum vs. Non-zero-sum
• Zero-sum games
– The total payoffs to all the players in the game, for
every combination of strategies, always adds to zero
– The increase in payoff of one player comes only at
the expense of decrease in payoff of other player(s)
• Non-zero-sum games
– The total payoffs to all the players in the game may
be greater or less than zero
– Increase in payoff by one player does not necessarily
correspond with a loss in payoff by another player
Mainak Chatterjee
2-person vs. N-person
• 2-person games
– Only 2 players in the game
– One is the opponent of other
• N-person games
– Generalization of the 2-person game
– N players in the game
– Each player plays against the remaining (N-1) players
Mainak Chatterjee
Example problems from networks
1.
Forwarder’s dilemma (Routing problem in wireless
mesh networks)
2.
Shared medium access
3.
Jamming game
4.
Power control in ad-hoc / sensor networks
5.
Resource pricing for wireless service providers
Mainak Chatterjee
Example problem 1
Forwarder’s dilemma in wireless mesh networks
Forwarder
for S2 R2
Forwarder
for S1 R1
R2
S1
S2
Should I
forward or drop?
Should I
forward or drop?
R1
Mainak Chatterjee
Example problem 2
Shared medium access problem
Should I
transmit or wait?
R2
Shared wireless
medium
S1
R1
S2
Should I
transmit or wait?
Mainak Chatterjee
Example problem 3
Jamming game problem
Which channel
should I
transmit on?
wireless
transmission
channel 1
Which channel
should I
jam?
Jammer
Transmitter
wireless
transmission
channel 2
Mainak Chatterjee
Example problem 4
Infrastractureless Sensor Networks
• Sensor nodes are randomly
deployed
• Distributed: No coordination on
networking activities
• Nodes contend with each other
for resources
Mainak Chatterjee
Example problem 5
Games among spectrum broker
and service providers
Games among service broker
and end-users
Mainak Chatterjee
From Problems to Games
Mainak Chatterjee
Example 1 Revisited:
Forwarder’s dilemma in wireless mesh networks
Non-cooperative vs. Cooperative
One-shot vs. Repeated (iterative)
Pure Strategy vs. Mixed Strategy
Complete information vs. Incomplete Information
Static vs. Dynamic
Zero-sum vs. Non-zero-sum
2-person vs. N-person
Rule of the game between players
S1
VS.
S2
• Game is played only once
• Whatever players once do is final for them
• Players do not have strategy information about other player
• Players are rational and greedy: they try to maximize their profit
Mapping
Mainak Chatterjee
Problem formulation using game
Game G = < {P}, {S}, {U} >
{P} : set of players
{S} : set of strategies taken by players
{U} : set of utilities obtained by players
Defn: U = benefit – cost of forwarding
Benefit = 2 for successful packet transmission
Benefit = 0 for dropped packet
Cost = 1 for forwarding a packet
Forwarder’s dilemma in wireless mesh networks
S2
forward
Drop
Forward
2-1, 2-1
-1, 2
drop
2, -1
0, 0
S1
Normal Form Game Model
Mainak Chatterjee
Solving forwarder’s dilemma game
• A player’s aim is to find strict dominance in such games
• Strict dominance is defined as the strategy which can not be
dominated by the strategies of other players
ui ( s , s )  ui ( si , si )
*
i
Where,
*
i
ui U ,
and,
si , si* , si , s*i  S
si
denotes the strategies of all other players except i
si*
denotes the strict dominance strategy of player i
• Our aim is to investigate the players’ strict dominance in this game
Solved by iterative dominance elimination mechanism
Mainak Chatterjee
Finding Strict Dominance:
Forwarder’s dilemma game
forward
S2
Drop
S1
S2
forward
Drop
S1
Forward
2-1, 2-1
-1, 2
Forward
2-1, 2-1
-1, 2
drop
2, -1
0, 0
drop
2, -1
0, 0
S2
forward
Drop
Forward
2-1, 2-1
-1, 2
drop
2, -1
0, 0
S1
Strict Dominance
Strategy
for both players
Nash equilibrium
point
for both players
Mainak Chatterjee
Sequential packet forwarding game
Game G = < {P}, {S}, {U} >
{P} : set of players
{S} : set of strategies taken by players
{U} : set of utilities obtained by players
Source
Defn: U = benefit – cost of forwarding
S1
S2
Should I
forward or drop?
Should I
forward or drop?
Destination
Benefit = 2 for successful packet arrival
Benefit = 0 for dropped packet
Cost = 1 for forwarding a packet
What is the strict dominance strategy here?
Mainak Chatterjee
Solving sequential packet
forwarding game
Normal Form Game Model
S2
forward
Drop
S1
Weak dominance strategy
No strict dominance strategy!!!
Forward
2-1, 2-1
-1, 0
drop
0, 0
0, 0
• Weak dominance in such games
• Weak dominance is defined as the strictly better strategy for at
least one strategy of the opponent player
u (s , s )  u (s , s )
*
*
i
i
i
i
i
i
Weak dominance is not unique in general!
Mainak Chatterjee
Example 2:
Shared medium access problem
No strict or weak dominance
Strategy!
Concept of best response!
S2
transmit
wait
-1, -1
2-1, 0
Two pure strategy Nash
Equilibrium points where no
Player has the motivation to
deviate alone
S1
Transmit
wait
0, 2-1
0, 0
What is mixed
Strategy NEP?
Mainak Chatterjee
Example 3:
Jamming game problem
Game G = < {P}, {S}, {U} >
{P} : set of players
{S} : set of strategies taken by players
{U} : set of utilities obtained by players
Transmitter:
U = c for successful packet transmission
U = -c for jammed transmission
Jammer:
U = -c for successful packet transmission
U = c for jammed transmission
Channel 1
Channel 2
Channel 1
-c, c
c, -c
Channel 2
c, -c
-c, c
jammer
transmitter
•
•
Example of zero-sum game
All previous examples: non-zero-sum
No pure strategy Nash
Equilibrium!!!
Mainak Chatterjee
Example 3:
Solving jamming game problem
• Introducing probability – mixed strategy decision
p: probability of transmitting on channel 1 by transmitter
q: probability of transmitting on channel 1 by jammer
E[U transmitter ]  p(1  q)c  pqc  (1  p)qc  (1  p)(1  q)c
E[U jammer ]  pqc  p(1  q)c  (1  p)qc  (1  p)(1  q)c
Solve for p and q – mixed strategy NEP
P = ½ and q = ½ is a mixed strategy NEP
Nash theorem (1950): Every finite strategy game has a mixed strategy
Nash equilibrium.
Mainak Chatterjee
Equilibrium selection
• So far, the aim has been to investigate the existence of
Nash equilibrium
• But there might be more than one NEP
• The question is which equilibrium should we choose and
follow?
• In this regard, we need to investigate into the goodness
of the equilibrium point
• For a single NEP also we need to study the goodness
We must study the efficiency of NEP
Mainak Chatterjee
Pareto-optimality:
Efficiency of Nash equilibrium
• We revisit the case study: sequential packet forwarding game
Multiple NEP
S2
forward
Drop
Forward
2-1, 2-1
-1, 0
drop
0, 0
0, 0
S1
Which NEP is better?
• Pareto-optimality to used measure the efficiency of NEPs
• Pareto-optimality: A strategy is Pareto-optimal if it is not possible to
increase the payoff of any player without decreasing the payoff of
another player.
Mainak Chatterjee
Pareto-optimality:
Forwarder’s dilemma
S2
forward
Drop
Forward
2-1, 2-1
-1, 2
drop
2, -1
0, 0
Pareto-optimal
Strategy
for both players
S1
Nash equilibrium
point
for both players
• Forwarder’s dilemma game: NEP is not Pareto-optimal.
• Pareto-optimal strategy is also not NEP.
Mainak Chatterjee
Pareto-optimality:
Shared medium access problem
transmit
wait
Transmit
-1, -1
2-1, 0
wait
0, 2-1
0, 0
S2
S1
Both pure strategy profiles are NEP and Pareto-optimal.
Mainak Chatterjee
So far, we have considered
static games played at one shot
What if the games are played
dynamically and repeatedly?
Mainak Chatterjee
Dynamic Games
Mainak Chatterjee
Dynamic Multi-stage Games
• Players make their moves multiple times in
contrast to single move in static games
– Need to check the history of the games
– Only first move does not have any history
– For all other moves, players have information about
the past strategies of other players
Sequential vs. Simultaneous
Dynamic games
Finite vs. Infinite
Horizon Dynamic games
Mainak Chatterjee
Definitions
• Sequential interaction dynamic games: Move of one
player is conditioned by move of other players
• Simultaneous interaction dynamic games: Players make
their moves simultaneously, i.e., players may or may not
have information about histories but not current strategy
of other player
• Finite horizon games: A game is referred to as the finite
horizon game if the number of stages of the game is
finite in number
• Infinite horizon games: A game is referred to as the
infinite horizon game if the number of stages of the game
is infinite
Mainak Chatterjee
Sequential interaction dynamic
games: Perfect information
• Move of one player is conditioned by move of other
player(s) in multi-stages and sequentially
• Is the normal form matrix representation sufficient
enough to represent these dynamic games?
transmit
wait
Transmit
-1, -1
2-1, 0
wait
0, 2-1
0, 0
S2
S1
Not possible to represent the
Multi-stage behavior
Mainak Chatterjee
Extensive form representation
• In the extensive form, the game is depicted as a tree
• Root of the tree is the start of the game (blank circle)
• Rest of the nodes of the tree shows the stages of the
games (shown with solid circles)
• Each leaf node determines the potential outcome of the
game and is associated with a payoff
Root (start of the game)
History h
Leaf (potential outcome of the game)
Mainak Chatterjee
Sequential shared medium access game:
problem revisited
• Assumptions:
– Multi-stage game i.e., the game
is not a one-shot game
– Finite horizon game
– S1 and S2 are not synchronized,
i.e., their moves are sequential
– Perfect information game, i.e.,
one player knows what move the
other player made
• Modified dynamic version of the multiple access game
• Need to build the extensive form representation
Mainak Chatterjee
Sequential shared medium access game:
Extensive form representation
S1
transmit
wait
S2
transmit
[-1, -1]
wait
[2-1, 0]
S2
transmit
[0, 2-1]
wait
[0, 0]
• 2-stage game (S1 playing first)
• S2 observes move of S1 before own move
• Possible strategies of S1: [transmit], [wait]
• Possible strategies of S2:
[transmit, transmit], [transmit, wait], [wait, transmit], [wait, wait]
• Question: Is there any equilibrium and how to find that?
Mainak Chatterjee
Nash equilibrium
S1
transmit
wait
S2
transmit
[-1, -1]
•
•
•
•
wait
[2-1, 0]
S2
transmit
[0, 2-1]
wait
[0, 0]
Three pure strategy Nash equilibrium
{ [transmit],[wait, transmit] }
Empty threat strategy
{ [transmit],[wait, wait] }
{ [wait],[transmit, transmit] }
Mainak Chatterjee
Stackelberg Game
S1
transmit
wait
S2
transmit
[-1, -1]
wait
[2-1, 0]
S2
transmit
[0, 2-1]
wait
[0, 0]
• The dynamic game described so far can be best
explained by Stackelberg game
– Also known as leader-follower game
• Stackelberg equilibrium can be best solved by backward
induction
Mainak Chatterjee
Stackelberg equilibrium
S1
transmit
wait
S2
transmit
[-1, -1]
wait
[2-1, 0]
S2
transmit
[0, 2-1]
wait
[0, 0]
• S1 plays [transmit]; best response of S2 will be:
[wait, wait] or [wait, transmit]
• Payoff vector = [2-1, 0]
• S1 plays [wait]; best response of S2 will be:
[transmit, wait] or [transmit, transmit]
• Payoff vector = [0, 2-1]
Mainak Chatterjee
Stackelberg equilibrium
S1
transmit
wait
S2
transmit
[-1, -1]
wait
[2-1, 0]
S2
transmit
wait
[0, 2-1]
[0, 0]
• As the leader of the game, S1 will exploit this game and
thus Stackelberg equilibrium strategies will be given by:
[transmit], [wait, wait] or [transmit], [wait, transmit]
• Example of leader exploiting the game
Mainak Chatterjee
Stackelberg equilibrium for
Forwarder’s dilemma
S1
forward
drop
S2
forward
[2-1, 2-1]
S2
forward
Drop
Forward
2-1, 2-1
-1, 2
drop
2, -1
0, 0
drop
[-1, 2]
forward
[2, -1]
S2
drop
[0, 0]
S1
Nash equilibrium
point
for both players
Stackelberg
Equilibrium point
for both players
Single shot normal form representation
Mainak Chatterjee
Stackelberg equilibrium for
Forwarder’s dilemma
S1
forward
drop
S2
forward
[2-1, 2-1]
drop
[-1, 2]
forward
S2
drop
[2, -1]
[0, 0]
Stackelberg
Equilibrium point
for both players
• In this case, the equilibrium point does not change for
both static and dynamic sequential games
• The game can NOT be exploited by any of leader or
follower players
Mainak Chatterjee
Stackelberg equilibrium for
sequential packet forwarding
S1
forward
drop
S2
forward
[2-1, 2-1]
drop
[-1, 0]
S2
drop
forward
[0, 0]
[0, 0]
Stackelberg equilibrium
S2
forward
Drop
Forward
2-1, 2-1
-1, 0
drop
0, 0
0, 0
Weak dominance strategy
S1
Single shot normal form presentation
Mainak Chatterjee
Stackelberg equilibrium for
sequential packet forwarding
S1
forward
drop
S2
forward
[2-1, 2-1]
drop
[-1, 0]
forward
[0, 0]
S2
drop
[0, 0]
Stackelberg equilibrium
• Locating the equilibrium point has become more
efficient with Stackelberg sequential game
• Pareto-optimality achieved
Mainak Chatterjee
Sequential interaction dynamic
games: Imperfect information
• So far, we have explored the dynamic games with
sequential moves and perfect information
• The players always knew about the moves of other
player(s) before their own move
• But how about if a player does not have complete
information?
• These games are illustrated using Imperfect information
games
Mainak Chatterjee
Example of a sequential interaction dynamic
game with imperfect information
• Revisit the problem of shared
medium access
• If the players are not
synchronized, they might make
their moves one after another
• The players have knowledge
about other players’ move in
previous stages
• However, the move of other
player in a stage is not known –
as signal still not received
Game of Imperfect Information
Mainak Chatterjee
Sequential shared medium access
S1
transmit
wait
S2
transmit
[-1, -1]
wait
[2-1, 0]
S2
transmit
[0, 2-1]
wait
[0, 0]
• Extensive form representation
• The only difference is the dashed box
• S2 does not have information about S1’s move:
Imperfect information
Mainak Chatterjee
Sequential shared medium access
S1
transmit
mi(h(n))
wait
S2
transmit
wait
mi(h(n))
S2
transmit
h(n)=Φ
wait
h(n)=Φ
[-1, -1]
[2-1, 0]
[0, 2-1]
[0, 0]
• Strategy of player i assigns a move mi(h(n)) to
every non-terminal node n
– h(n) is the history/information set at the node n
– carries information about the previous moves, if any
Mainak Chatterjee
Sequential shared medium access
S1
transmit
wait
S2
transmit
wait
S2
transmit
wait
S1
transmit
wait
S2
transmit
wait
[-2, -2]
[2-2, -1]
[2-1, 0]
S2
transmit
[-1, 2-2]
[0, 2-1]
[0, 0]
wait
[-1, -1]
• As the game unfolds in multiple stages, it is difficult to solve the
game with backward induction
• The strategies of the players are increasing multiplicatively
Mainak Chatterjee
Subgame
• It is clear that sequential multiple access game
with retransmissions can not be analyzed with
backward induction
• The reason behind it is the imperfect information
at the 2nd stage for the example considered
• In this regard, Selten introduced the concept of
subgame perfection
Mainak Chatterjee
Subgame definition
• The game G’ is a proper subgame of an extensive-form
game G if it consists of a single node in the extensiveform tree and all of its successors down to the leaves.
• Formally, if a node n is in G’ and n’ is in h(n), then n’ is in
G’ . The information sets and payoffs of the subgame G’
are inherited from the original game G: this means that n
and n’ are in the same information set in G’ if they are in
the same information set in G; and the payoff function of
G’ is the restriction of the original payoff function to G’ .
Mainak Chatterjee
Subgame definition
S1
transmit
wait
S2
transmit
wait
S2
transmit
wait
S1
transmit
wait
S2
transmit
[-2, -2]
wait
[2-2, -1]
[2-1, 0]
S2
transmit
[-1, 2-2]
[0, 2-1]
[0, 0]
wait
subgame
[-1, -1]
Mainak Chatterjee
Solving using subgame perfection
• Steps to solve dynamic multiple access game:
– Replace the multiple access subgame with one of the
pure-strategy Nash equilibrium
– Obtain simple reduced extensive form game tree
– Solve the reduced form with backward induction
Mainak Chatterjee
Solving using subgame perfection
S1
transmit
wait
S2
transmit
[-1, 2-2]
wait
[2-1, 0]
S2
transmit
[0, 2-1]
wait
[0, 0]
• Replace the subgame with pure strategy Nash equilibria
{wait, transmit} and obtain the reduced extensive form
• Apply backward induction on the reduced form tree
Mainak Chatterjee
Solving using subgame perfection
S1
transmit
wait
S2
transmit
[2-2, -1]
wait
[2-1, 0]
S2
transmit
[0, 2-1]
wait
[0, 0]
• Replace the subgame with pure strategy Nash equilibria
{transmit, wait} and obtain the reduced extensive form
• Apply backward induction on the reduced form tree
Mainak Chatterjee
Subgame perfection disadvantages
• Concept of subgame perfection has
disadvantages regarding equilibrium selection
• It is not clear how the players will be able to
determine their moves if several Nash
equilibrium exist in a given subgame
Mainak Chatterjee
Dynamic games: Simultaneous moves
• All the dynamic games played so far were sequential in nature
– Players made their moves one after another
• Now, we look at games where players make moves simultaneously
and they interact repeatedly
– Each interaction is called a stage
– Repetition can be finite (finite horizon games) or infinite times (infinite
horizon games)
• Inherently, these games are of most complex nature due to their
incomplete information
• Players do not have information about the other players’ strategies
in the same stage and may or may not have history h(n) of other
players’ moves
• Most popular wireless networking decision problems are of this
game category…will be presented later
Mainak Chatterjee
Forwarder’s dilemma: Revisited with
dynamic finite-horizon games
• Denote, mi(n) is the strategy of i-th
player at the n-th stage
• Then the history for 2 players in this
game at n-th stage can be given by,
h(n)  {m1 (n), m2 (n),......, m1 (0), m2 (0)}
•
•
•
•
• Then the strategy to be taken at (n+1)th stage can be given by,
Recall, S1 and S2 playing
against each other
We assume the moves made by
S1 and S2 are simultaneous
•
S1 and S2 do not know what
the other player will do
We assume S1 and S2 know
each other’s history h(n)
mi (n  1)  (h(n))
The payoff of a player can then be
given by
K
max imize  U (mi (k ))
k 0
Mainak Chatterjee
Myopic vs. long sighted games
• In some cases, players aim might be to
maximize their payoffs for the next stage only
– These games are referred to as myopic or short
sighted optimizers U (mi (n  1))
• In contrast, the players aim to maximize the total
utility throughout the game
– These games are referred to as long sighted finite
K
optimizers  U (mi (k ))
k 0
Mainak Chatterjee
Solving forwarder’s dilemma for
long sighted finite games
• In the repeated forwarder’s dilemma, the
strategy profile {drop, drop} is a Nash
equilibrium
– Can be proved by backward induction by looking at all
the strategies
– But can be computationally infeasible if the strategy
space is large
– Hence, one heuristic choice would be strategies of
history-1 (forgetful strategy)
– Same Nash equilibrium as with the previous case
Mainak Chatterjee
Infinite-horizon games
• Unlike the previous case, the end of the game is
not defined
• In other words, these games are also known as
long sighted infinite games where players aim is
to maximize

 U ( m ( k ))
k 0
i
Mainak Chatterjee
Infinite-horizon games with
discounting factor
• More generalized form of these games are given
by infinite horizon games with discounting

maximize
k
U
(
m
(
k
)
)

 i
k 0
• In order to model the unpredictable end of the
game, the payoffs of the future stages are
decreased by a factor (discount factor)
0   1
Mainak Chatterjee
Example problem 4:
Power control in sensor network
Mainak Chatterjee
Aim: Conserving Energy through
Transmitting Power Control
• With what power sensor nodes should transmit?
– Low transmitting power
• Saves energy BUT increases failure probability
– High transmitting power
• Increases success probability BUT drains battery
• Few centralized power control algorithms already
available for cellular systems BUT not applicable
to sensor networks
Mainak Chatterjee
Distributed Sensor Networks
• Incomplete Information
– Each node has information
about own power level, SINR
and channel condition only
– Each node acts in a “selfish”
manner and creates
interference for other nodes
No global knowledge
Mainak Chatterjee
Distributed sensor networks
• Non-cooperative vs. Cooperative
• One-shot vs. Repeated (iterative)
• Pure Strategy vs. Mixed Strategy
• Complete information vs. Incomplete Information
• Static vs. Dynamic
• Zero-sum vs. Non-zero-sum
• 2-person vs. N-person
Mainak Chatterjee
Game Formulation
• Assumptions
– Time-slotted CDMA based network
– Power chosen by all (N+1) nodes from a common
pool S
– S consists of all permissible power levels
– Node 1 chooses transmitting power level s1  S
– Node 2 chooses transmitting power level s2  S
• Aim
– Investigate Nash equilibrium
Mainak Chatterjee
Payoff with fixed channel
• Payoff is then given by
Utility – Cost of transmission
• Does there exists a threshold power Pt, which
the nodes must comply with?
• Exceeding this threshold will introduce nonbeneficial payoff
• Nash equilibrium exists for this power threshold
Mainak Chatterjee
Payoff with varying channel
• Cost function of transmission decreases
– Good channel condition  Low cost of transmission
– Bad channel condition  High cost of transmission
• Existence of a channel threshold
– Transmitting below threshold will induce nonbeneficial net utility
• Nash equilibrium exists for channel threshold
Mainak Chatterjee
Other Problems
• MAC layer decisions
– Should a node transmit or not
– If not, how long should a node wait
– How to maintain QoS; at what cost
• Routing layer decisions
– Should a node forward a packet or not
– What kind of incentives does a node have to forward other’s
packets
– Should they co-operate
– Can a node rely on the forwarding node
– Can we build a reputation profile based on the history
– Reputation and go to infinity  so use discounting
Mainak Chatterjee
Example problem 5:
Resource pricing for wireless
service providers
Mainak Chatterjee
Current (Static) Scenario
Spectrum Band
Service
Provider 1
Service
Provider 2
Users with
SP 1
Users with
SP 2
…….
…….
Service
Provider n
Users with
SP n
Mainak Chatterjee
Complete Paradigm Shift
Coordinated Access Band
Spectrum Broker
Service
Provider 1
Service
Provider 2
…….
Service
Provider n
Service Broker
Common User Pool
Mainak Chatterjee
Cyclic Dependency
Wireless
Service Providers
(WSP)
Service
Requested
Service
Allocated
Dynamically Setting Price of Service
Spectrum
Requested
Broker
Spectrum
Allocated
Estimating Demand of Bandwidth And
Reservation Price
Spectrum
Subscribers
(End users)
Mainak Chatterjee
Game Model
Service
Provider 1
Service
Provider 2
…….
Service
Provider n
Service Broker
Common User Pool
• Service providers compete against other service providers for users
• Determine price of service depending on amount of spectrum won and
price paid for it
• Incomplete information game
• Users obtain the price from providers and request for bandwidth
• How to select the best service provider?
Mainak Chatterjee
Utility Functions
• Users’ utility function are formed and given by,
Net Utility = Benefit obtained
– (direct price paid)
– (cost incurred due to other users)
– (others costs)
• Service provider’s utility function is also formed
Revenue earned from users – Infrastructure cost
Mainak Chatterjee
Users’ Strategies
• Users need to maximize their utilities
• Under varying channel conditions
– a rational user should be active (transmit/receive) only when the
channel condition is better than the minimum channel quality
threshold
– maintaining this strategy helps the users reaching Nash
equilibrium.
• For prices advertised by providers, compute the resource
vector that would maximize utilities from all the providers
– User should then connect that provider gives the maximum value
in the maximized utility vector.
Mainak Chatterjee
Providers’ Strategies
• With users’ maximization strategy, providers should also maintain a
price threshold for maximizing their utility
• With users’ and other providers’ strategy unchanged, one provider
can not unilaterally change their strategy of pricing threshold to
improve their utility thus proving the existence of Nash equilibrium
• With this pricing threshold in effect, providers should constantly try
to update their resource amount by buying more spectrum, which
acts as incentive for both providers and users
Mainak Chatterjee
What about the downside of game theory?
Mainak Chatterjee
Limitations of Game Theory
• No unified solution to general conflict resolution
• Real-world conflicts are complex
– models can at best capture important aspects
• Players are (usually) considered rational
– determine what is best for them given that others are
doing the same
• No unique prescription
– not clear what players should do
It can provide intuitions, suggestions and partial prescriptions
– One of the best mathematical tool we currently have
Mainak Chatterjee