Achieving Network Optima Using
Stackelberg Routing Strategies
Yannis A. Korilis, Member, IEEE
Aurel A. Lazar, Fellow, IEEE
&
Ariel Orda, Member IEEE
IEEE/ACM transactions on networking, vol. 5, No. 1, February 1997
Sanjeev Kohli
EE 228A
Presentation Outline







Introduction to non cooperative networks
Overview of approach
Model and Problem Formulation
Non cooperative User & Manager
Single Follower Stackelberg Routing game
Multi Follower Stackelberg Routing game
Issues
Non-cooperative Networks
Non-cooperative Networks

Users take control decisions individually to max
own performance
Non-cooperative Networks


Users take control decisions individually to max
own performance
Similar to non cooperative games
Non-cooperative Networks



Users take control decisions individually to max
own performance
Similar to non cooperative games
Operating points of such networks are
determined by Nash equilibria
Non-cooperative Networks




Users take control decisions individually to max
own performance
Similar to non cooperative games
Operating points of such networks are
determined by Nash equilibria
Nash Equilibria – Unilateral deviation doesn’t
help any user
Non-cooperative Networks





Users take control decisions individually to max
own performance
Similar to non cooperative games
Operating points of such networks are
determined by Nash equilibria
Nash Equilibria – Unilateral deviation doesn’t
help any user
Inefficient, leads to sub optimal performance
Non-cooperative Networks






Users take control decisions individually to max
own performance
Similar to non cooperative games
Operating points of such networks are
determined by Nash equilibria
Nash Equilibria – Unilateral deviation doesn’t
help any user
Inefficient, leads to sub optimal performance
Better solution needed !
Network Manager
Network Manager

Architects the n/w to achieve efficient equilibria
Network Manager


Architects the n/w to achieve efficient equilibria
Run time phase
Network Manager



Architects the n/w to achieve efficient equilibria
Run time phase
Awareness of users behavior
Network Manager




Architects the n/w to achieve efficient equilibria
Run time phase
Awareness of users behavior
Aims to improve overall system performance
through maximally efficient strategies
Network Manager





Architects the n/w to achieve efficient equilibria
Run time phase
Awareness of users behavior
Aims to improve overall system performance
through maximally efficient strategies
Maximally efficient strategy

Optimizes overall performance
Network Manager





Architects the n/w to achieve efficient equilibria
Run time phase
Awareness of users behavior
Aims to improve overall system performance
through maximally efficient strategies
Maximally efficient strategy


Optimizes overall performance
Individual users are well off at this operating point
[Pareto Efficient]
Presentation Outline







Introduction to non cooperative networks
Overview of approach
Model and Problem Formulation
Non cooperative User & Manager
Single Follower Stackelberg Routing game
Multi Follower Stackelberg Routing game
Issues
Overview of this approach
Overview of this approach

Total flow: Flow of users + Flow of manager
Overview of this approach


Total flow: Flow of users + Flow of manager
Example of manager’s flow
•
Traffic generated by signaling/control mechanism
Overview of this approach


Total flow: Flow of users + Flow of manager
Example of manager’s flow
•
•
Traffic generated by signaling/control mechanism
Users traffic that belongs to virtual networks
Overview of this approach


Total flow: Flow of users + Flow of manager
Example of manager’s flow
•
•

Traffic generated by signaling/control mechanism
Users traffic that belongs to virtual networks
Manager optimizes system performance by
controlling its portion of flow
Overview of this approach


Total flow: Flow of users + Flow of manager
Example of manager’s flow
•
•


Traffic generated by signaling/control mechanism
User traffic that belongs to virtual networks
Manager optimizes system performance by
controlling its portion of flow
Investigates manager’s role using routing as a
control paradigm
Non Cooperative Routing Scenario

IPv4/IPv6 allow source routing
•
User determines the path its flow follows from sourcedestination
Goal of Manager

Optimize overall network performance according
to some system wide efficiency criterion
Capability of Manager

It is aware of non cooperative behavior of users
and performs its routing based on this information
Central Idea
Central Idea

Manager can predict user responses to its routing
strategies
Central Idea


Manager can predict user responses to its routing
strategies
Allows manager to choose a strategy that leads of
optimal operating point
Central Idea



Manager can predict user responses to its routing
strategies
Allows manager to choose a strategy that leads of
optimal operating point
Example of Leader-Follower Game [Stackelberg]
MAN
VP’s k
VP’s k
VP’s k
Org1
Org n
Org2
User 1
User p
User 2
User 3
Need to derive

A necessary and sufficient condition that
guarantees that the manager can enforce an
equilibrium that coincides with the network
optimum

Above condition requires –
Manager’s flow Control > Threshold
Need to derive

A necessary and sufficient condition that
guarantees that the manager can enforce an
equilibrium that coincides with the network
optimum

Above condition requires –
Manager’s flow Control > Threshold
If the above criterion is met, we can show that the maximally
efficient strategy of manager is unique and we will specify its
structure explicitly
Presentation Outline







Introduction to non cooperative networks
Overview of approach
Model and Problem Formulation
Non cooperative User & Manager
Single Follower Stackelberg Routing game
Multi Follower Stackelberg Routing game
Issues
Model and Problem Formulation


I = {1,…..,I}
L= {1,.....,L}
User set
Communication Links
1
Source
2
L
Destination
Model and Problem Formulation (contd)










Manager is referred at user 0
I0 = I U {0}
cl = capacity of link l
c = (c1,….cL) :
capacity configuration
C = lL cl
:
total capacity of the system
of parallel links
c1 >= c2 >= …. >= cL
Each i  I0 has a throughput demand ri > 0
r1 >= r2 >= …. >= rI
r = iI ri
R = r + r0
Demand is less than capacity of links  R < C
Model and Problem Formulation (contd)




User i  I0 splits its demand ri over the set of parallel
links to send its flow
Expected flow of user i on link l is fli
Routing strategy of user i  fi = (f1i,….fLi)
Strategy space of user i 
Fi = {fi  IRL : 0 <= fli <= cl, l  L; lL fli = ri}


Routing strategy profile  f = {f0, f1,….,fI)
System strategy space  F = iIo Fi
Model and Problem Formulation (contd)

Cost function quantifying GoS of user i’s flow is
Ji : F  IR





i  I0
Cost of user i under strategy profile f is Ji(f)
Ji(f) = lL fliTl(fl); Tl(fl) is the average delay on
link l, depends only on the
total flow fl = iIo fli on that link
Tl(fl) = (cl - fl)-1,
fl < cl
= ,
fl >= cl
Total cost J(f) = iIo Ji(f) = lL fl / (cl - fl)
Higher cost  lower GoS provided to the user’s flow,
higher average delay
Model and Problem Formulation (contd)


l

f l (cl  f l ) 1
is a convex function of (f1, …, fL)
a unique link flow configuration exists – min cost
(f1*,….fL*) ;

f l*  0 & l f l*  R
Above is solution to classical routing opt problem,
routing of all flow (users+manager) is centrally
controlled; referred to as network optimum.
Kuhn – Tucker Optimality conditions

(f1*,….fL*) is the network optimum if and only if there exists
a Lagrange Multiplier *, such that for every link l L
* 
cl
*
if
f
l 0
2
(cl  f l )
1
 
cl
*
if f l *  0
Presentation Outline







Introduction to non cooperative networks
Overview of approach
Model and Problem Formulation
Non cooperative User & Manager
Single Follower Stackelberg Routing game
Multi Follower Stackelberg Routing game
Issues
Non cooperative users
Non cooperative users

Each user tries to find a routing strategy fi  Fi that
minimizes its cost Ji (average time delay)
Non cooperative users


Each user tries to find a routing strategy fi  Fi that
minimizes its cost Ji (average time delay)
This minimization depends on strategies of the manager and
other users, described by strategy profile
f-i = (f0 , f1 ,… fi-1, fi+1 ,… fI )
Non cooperative users


Each user tries to find a routing strategy fi  Fi that
minimizes its cost Ji (average time delay)
This minimization depends on strategies of the manager and
other users, described by strategy profile
f-i = (f0 , f1 ,… fi-1, fi+1 ,… fI )

Routing strategy of manger is FIXED  f0
Non cooperative users


Each user tries to find a routing strategy fi  Fi that
minimizes its cost Ji (average time delay)
This minimization depends on strategies of the manager and
other users, described by strategy profile
f-i = (f0 , f1 ,… fi-1, fi+1 ,… fI )

Routing strategy of manger is FIXED  f0

Each user adjusts its strategy to other users actions
Non cooperative users


Each user tries to find a routing strategy fi  Fi that
minimizes its cost Ji (average time delay)
This minimization depends on strategies of the manager and
other users, described by strategy profile
f-i = (f0 , f1 ,… fi-1, fi+1 ,… fI )

Routing strategy of manger is FIXED  f0


Each user adjusts its strategy to other users actions
Can be modeled as a non cooperative game, any operating
point is Nash Equilibrium; dependent on f0 !
Non cooperative users


Each user tries to find a routing strategy fi  Fi that
minimizes its cost Ji (average time delay)
This minimization depends on strategies of the manager and
other users, described by strategy profile
f-i = (f0 , f1 ,… fi-1, fi+1 ,… fI )

Routing strategy of manger is FIXED  f0


Each user adjusts its strategy to other users actions
Can be modeled as a non cooperative game, any operating
point is Nash Equilibrium; dependent on f0 !
From users view point, manager reduces capacity on each
link l by fl0 , the system reduces to a set of parallel links
with capacity configuration c – f0  has a unique Nash
Equilibrium

f0  f -0 ……. N0(f0)
Non cooperative users

For a given strategy profile f-i of other users in I0, the cost of
i, Ji(f) = lL fliTl(fl), is a convex fn of its strategy fi , hence
the following min problem has a unique solution
i
i
i
f i  arg min
J
(
g
,
f
),
i
i
g F
iI
Kuhn – Tucker Optimality conditions

fi is the optimal response of user i if and only if there exists
a (Lagrange Multiplier) i , such that for every link l L, we
have
cl  f l  f l i
 
,
2
(cl  f l )
if f l i  0
1
 
,
(cl  f l )
if f l i  0
i
i
Non cooperative users


f-0  F-0 is a Nash Equilibrium of the self optimizing users
induced by strategy f0 of the manger.
The function N0 : F0  F-0 that assigns the induced
equilibrium of the user routing game (to each strategy of the
manger) is called the Nash Mapping. It is continuous.
Role of the Manager




It has knowledge of non cooperative behavior of users;
determines the Nash Equilibrium N0(f0) induced by any
routing strategy it f0 chosen by him
Acts as Stackelberg leader, that imposes its strategy on the
self optimizing users that behave as followers
Aims to optimize the overall network performance, plays a
social rather than selfish role
To find f0 such that if f-0 = N0(f0), then iIo fli = fl* for all l
This f0 is called maximally efficient strategy of manager
It is Pareto efficient !
Outline of Results






In case of a single user, the manager can always enforce
network optimum; its MES is specified explicitly
In case of any no of users, the manager can enforce the
network optimum iff its demand is higher that some
threshold r0, in which case the MES is specified explicitly
r0 is feasible if total demand of users plus r0 is less than C
It is easy for manager to optimize heavily loaded networks
as r0 is small
As the no of user increases, threshold increases i.e. harder
for manager to enforce network optimum
The higher the difference in throughput demands of any two
users, the easier it is for manager to enforce network
optimum



Network optimum:
(f1*,….fL*)
Flow on link l, fl* is decreasing in link no l  L
There exists some link L*, such that fl* > 0 for l <= L* and
fl* = 0 for l > L* ; L* is determined by (from [1] & [2]),
GL*  R  GL* 1
where
l 1
l 1
n 1
n 1
Gl   cn  cl  cn
and G1=0, GL+1=ln=1cn = C

cl >= cl+1 
Gl <= Gl+1
l  2,..., L

Using Lagrange Multiplier’s equations, we get,
cl  f l *  cl 1  f l *1
l  1,....., L  1
 
cl 
*
l A
 
 ,
 lA (cl  f l ) 
A  1,..., L* 
2

Network Optimum is given by [2]
L

f l  cl   n 1 cn  R 


*
f l*  0
cl
*

L*
n 1
,
l  L*
cn
l  L*



Best reply fi of user i  I0 to the strategies of manager and
other users, described by f-i, can be determined as network
optimum for a system of parallel links with capacity
configuration (c1i,…, cLi)
Assuming cli >= cl+1i ,
l=1,…,L-1
the flow fli is decreasing in the link no l  L
There exists some link Li, such that fli > 0 for l <= Li and
fli = 0 for l > Li ; The threshold Li is determined by
GLi i  r i  GLi i 1
where
l 1
G  c  c
i
l
n 1
i
n
i
l
and G  0, G
i
1
i
L 1
l 1

n 1
cni ,
l  2,..., L
 n 1 cni  C  ( R  r i )
Gli  Gli1 for all l  L
L

Best reply fi of user i to strategy profile f-i of the other users
in I0 is given by
f l i  cli  (m 1 cmi  r i )
cli
i
L
f l i  0,

i
L
m 1
,
l  Li
cmi
l  Li

Best reply doesn’t depend on detailed description of f-i but
only on residual capacity cli seen by user on every link l  L

In practice, residual capacity info can be acquired by
measuring the link delays using an appropriate estimation
technique
Presentation Outline







Introduction to non cooperative networks
Overview of approach
Model and Problem Formulation
Non cooperative User & Manager
Single Follower Stackelberg Routing game
Multi Follower Stackelberg Routing game
Issues
Single Follower Stackelberg Routing Game
Single Follower Stackelberg Routing Game
l 1
Hl  
n 1
f l * l 1
f 
 cn ,
cl n 1
*
n
l  2,..., L
and H1  0, H L 1  n 1 f n*  R
L
H l  Gl / *cl , l  L*
H l  R,
l  L*
thus H l  H l 1 , l  1,..., L
In this game, there exists a MES of the manager then it is unique and is given
by
L1
*
1
f

r

n
f l 0  cl n 1 L1
, l  L1
n1 cn
f l 0  f l* ,
H L1  r 1  H L1 1
l  L1 where L1 is determined by
Single Follower Stackelberg Routing Game
 The best reply f1 of the follower is
 f

L1
f l1  f l *  f10  f l *  cl
n 1
*
n
 r1
1
L
,
l  L1
c
n 1 n
 0,
l  L1
 Therefore, {1,…,L1} is the set of links over which the follower
sends its flow when manager implements f0.
 For manager: Send flow fl* on every link l that will not receive
any flow from the follower
Split the rest of its flow among the links that will
receive user flow proportional to their capacities
Presentation Outline







Introduction to non cooperative networks
Overview of approach
Model and Problem Formulation
Non cooperative User & Manager
Single Follower Stackelberg Routing game
Multi Follower Stackelberg Routing game
Issues
Multi Follower Stackelberg Routing Game
Multi Follower Stackelberg Routing Game

An arbitrary number I of self optimizing users share the
system of parallel links
Multi Follower Stackelberg Routing Game


An arbitrary number I of self optimizing users share the
system of parallel links
Maximally Efficient Strategy of manager (if it exists) and the
corresponding Nash Equilibrium of non cooperative users is:
*
i
f

r
n1 n
Li
f l 0  cl 

i
L
 ( I l  1) f l * , l  L
c
n 1 n
where for every user i  I , Li is determined by
H Li  r i  H Li 1
and for every link l  L, I l  {i  I : l  Li } and I l  I l
Multi Follower Stackelberg Routing Game

Equilibrium strategy fi of user i I is described by
f l i  f l *  cl
 0,


Li
*
i
f

r
n 1 n
,
l  Li
l  Li
If a MES exists, then the induced Nash equilibrium of the
followers has precisely the same structure with the best reply
follower in the single follower case
Remarks - M F Stackelberg Routing Game
Remarks - M F Stackelberg Routing Game

{1,…., Li} is the set of links that receive flow from follower
i I
Remarks - M F Stackelberg Routing Game


{1,…., Li} is the set of links that receive flow from follower
i I
Il is the set of followers that send flow on link l. Since
H1 = 0 < ri, i  I, all users send flow on link 1  I1 = I
Remarks - M F Stackelberg Routing Game



{1,…., Li} is the set of links that receive flow from follower
i I
Il is the set of followers that send flow on link l. Since
H1 = 0 < ri, i  I, all users send flow on link 1  I1 = I
For f0 to be admissible, fl0 >= 0, for all l  L
Remarks - M F Stackelberg Routing Game




{1,…., Li} is the set of links that receive flow from follower
i I
Il is the set of followers that send flow on link l. Since
H1 = 0 < ri, i  I, all users send flow on link 1  I1 = I
For f0 to be admissible, fl0 >= 0, for all l  L
If fl0 < 0  fl-10 < 0
Remarks - M F Stackelberg Routing Game





{1,…., Li} is the set of links that receive flow from follower
i I
Il is the set of followers that send flow on link l. Since
H1 = 0 < ri, i  I, all users send flow on link 1  I1 = I
For f0 to be admissible, fl0 >= 0, for all l  L
If fl0 < 0  fl-10 < 0
Admissible condition reduces to f10 >= 0
Remarks - M F Stackelberg Routing Game






{1,…., Li} is the set of links that receive flow from follower
i I
Il is the set of followers that send flow on link l. Since
H1 = 0 < ri, i  I, all users send flow on link 1  I1 = I
For f0 to be admissible, fl0 >= 0, for all l  L
If fl0 < 0  fl-10 < 0
Admissible condition reduces to f10 >= 0
f10 is an increasing function of the throughput demand r0 of
leader, r0  [0, C - r] ………. [3]
Theorem

There exists some r0, with 0 < r0 < C – r, such that the leader
in multi follower Stackelberg routing game can enforce the
network optimum if and only if its throughput demand r0
satisfies r0 < r0 < C – r. The maximally efficient strategy of
leader is given by
f

c 

Li
fl 0
n 1
l
*
n
 ri
i
L
 ( I l  1) f l * , l  L
c
n 1 n
where for every user i  I , Li is determined by
H Li  r i  H Li 1
and for every link l  L, I l  {i  I : l  Li } and I l  I l
Presentation Outline







Introduction to non cooperative networks
Overview of approach
Model and Problem Formulation
Non cooperative User & Manager
Single Follower Stackelberg Routing game
Multi Follower Stackelberg Routing game
Issues
Properties of Leader Threshold r0
Properties of Leader Threshold r0

r0 of the leader is a unique solution of the equation
“f10(r0) = 0” in r0  [0, C - r]
Properties of Leader Threshold r0

r0 of the leader is a unique solution of the equation
“f10(r0) = 0” in r0  [0, C - r]

When r  C,
r0  0 i.e. in heavily loaded networks,
controlling a small portion of flow can drive the system into
the network optimum
Properties of Leader Threshold r0

r0 of the leader is a unique solution of the equation
“f10(r0) = 0” in r0  [0, C - r]

When r  C,
r0  0 i.e. in heavily loaded networks,
controlling a small portion of flow can drive the system into
the network optimum

With throughput demand r fixed, the leader threshold r0
increases with increase in no of users.
Properties of Leader Threshold r0

r0 of the leader is a unique solution of the equation
“f10(r0) = 0” in r0  [0, C - r]

When r  C,
r0  0 i.e. in heavily loaded networks,
controlling a small portion of flow can drive the system into
the network optimum

With throughput demand r fixed, the leader threshold r0
increases with increase in no of users.

Leader threshold r0 decreases with increase in difference in
user demands
Set of Parallel links with capacity conf (12,7,5,3,2,1), shared by I identical
followers with total demand r
Set of Parallel links with capacity conf (12,7,5,3,2,1), shared by 100
identical self optimizing users with total demand r and the manager
r0 = r0
Set of Parallel links with capacity conf (12,7,5,3,2,1), shared by 100
identical self optimizing users with total demand r and the manager
r0 = r0
Set of Parallel links with capacity conf (12,7,5,3,2,1), shared by 100
identical self optimizing users with total demand r and the manager
r0 = r0
Set of Parallel links with capacity conf (12,7,5,3,2,1), shared by 100
identical self optimizing users with total demand r and the manager
r0 = r0
Scalability
Scalability

To determine maximally efficient strategy, manager needs
throughput demand ri of every user.
Scalability


To determine maximally efficient strategy, manager needs
throughput demand ri of every user.
In many networks, user declare average rate ri during
negotiation phase
Scalability



To determine maximally efficient strategy, manager needs
throughput demand ri of every user.
In many networks, user declare average rate ri during
negotiation phase
Alternatively, the manager can estimate average rates by
monitoring the behavior of users
Scalability




To determine maximally efficient strategy, manager needs
throughput demand ri of every user.
In many networks, user declare average rate ri during
negotiation phase
Alternatively, the manager can estimate average rates by
monitoring the behavior of users
Manager can adjust its strategy to maximally efficient one
whenever a user departs or a new one joins the network
Scalability





To determine maximally efficient strategy, manager needs
throughput demand ri of every user.
In many networks, user declare average rate ri during
negotiation phase
Alternatively, the manager can estimate average rates by
monitoring the behavior of users
Manager can adjust its strategy to maximally efficient one
whenever a user departs or a new one joins the network
User not necessarily mean a single user, it can be a group of
users joining the network as an organization. It also reduces
threshold r0
References
[1]
[2]
[3]
A. Orda, R. Rom, and N. Shimkin, “Competitive routing in multiuser communication networks,” IEEE/ACM Trans. Networking,
vol. 1, pp. 510-521, Oct. 1993.
Y.A. Korilis, A.A. Lazar, and A. Orda, “Capacity allocation under
non cooperative routing,” IEEE Trans. Automat. Contr.
Y.A. Korilis, A.A. Lazar, and A. Orda, “Achieving network optima
using Stackelberg routing strategies,” Center for
Telecommunications Research, Columbia University, NY, CTR
Tech. Rep. 384-94-31, 1994.
THANK YOU