Episode 7. Modeling Network
Traffic using Game Theory
Baochun Li
Department of Electrical and Computer Engineering
University of Toronto
Objective in this episode
Sending packets through the Internet involves
fundamentally game-theoretic reasoning —
Rather than simply choosing a route in isolation,
individuals must evaluate routes in the presence of the
congestion resulting from the decisions made by
themselves and everyone else
Objective: to develop models for network traffic using
the game-theoretic ideas developed thus far
Baochun Li, Department of Electrical and Computer Engineering, University of Toronto
2
Consider a transportation network
Edges are highways, and nodes are exits on highways
Everyone wants to get from A to B
using
game theory
Eachmodeling
edge has anetwork
designatedtraffic
travel time
(in minutes)
when
there are x cars traveling on this highway
C
45
x/100
A
B
45
x/100
D
Baochun Li, Department of Electrical and Computer Engineering, University of Toronto
3
Consider a transportation network
Suppose that 4,000 cars want to get from A to B
If the cars divide up evenly between the two routes, the total
travel time is 65 minutes
modeling network traffic using game theory
So what do we expect to happen?
C
45
x/100
A
B
45
x/100
D
Baochun Li, Department of Electrical and Computer Engineering, University of Toronto
4
The traffic game
The traffic model we’ve described is really a game in
which the players correspond to the drivers, and each
player’s possible (two) strategies consist of the possible
routes from A to B
The payoff for a player is the negative of his or her travel
We have focused primarily on games with two players, whereas
the current traffic game will generally have an enormous
number of players (4,000 in our example)
There is no dominant strategy in this game
Either route has the potential to be the best choice for a player if
all the other players are using the other route!
Baochun Li, Department of Electrical and Computer Engineering, University of Toronto
5
Equilibrium traffic
The traffic game does have Nash equilibria —
Any list of strategies in which the drivers balance themselves evenly
between the two routes (2,000 on each) is a Nash equilibrium, and
these are the only Nash equilibria
What does equal balance yield a Nash equilibrium?
With an even balance between the two routes, no driver has an
incentive to switch over to the other route.
Why do all Nash equilibria have equal balance?
Consider a list of strategies in which x drivers use the upper route and
the remaining 4000 − x drivers use the lower route.
Then if x is not equal to 2000, the two routes will have unequal travel
times, and any driver on the slower route would have an incentive to
switch to the faster one — not a Nash equilibrium!
Baochun Li, Department of Electrical and Computer Engineering, University of Toronto
6
Braess’s Paradox
Suppose that the city government decides to build a new, very fast
highway from C to D — with a travel time of 0!
It would stand to reason that people’s travel time from A to B ought
to get better after this edge from C to D is added
There is indeed a unique Nash equilibrium in this new highway
network
braess’s paradox
209
But it leads to a worse travel time for everyone!
C
45
x/100
0
A
45
B
x/100
D
Baochun Li, Department of Electrical and Computer Engineering, University of Toronto
7
Braess’s Paradox
At equilibrium, every driver uses the route through both C and
D
As a result, the travel time for every driver is 80
Why this is an equilibrium?
No driver can benefit
by changing
their route: with traffic
braess’s
paradox
209
snaking through C and D the way it is, any other route would
now take 85 minutes!
C
45
x/100
0
A
45
B
x/100
D
Baochun Li, Department of Electrical and Computer Engineering, University of Toronto
8
Braess’s Paradox
The creation of the edge from C to D has in fact made the
route through C and D a dominant strategy for all drivers:
regardless of the current traffic pattern, you gain by
switching your route to go through C and D!
In the new network, there is no way, given individually selfinterested behaviour by the drivers, to get back to the evenbalance solution that was better for everyone
Adding resources to a transportation network can sometimes
hurt performance at equilibrium! — Braess’s Paradox
Just like adding the option of “confess” doesn’t improve the payoff
in the Prisoner’s Dilemma
Network traffic at equilibrium may not be socially optimal!
Baochun Li, Department of Electrical and Computer Engineering, University of Toronto
9
The social cost of traffic at equilibrium
In any general network, how far from optimal traffic
can be at equilibrium?
The network can be any directed graph
There is a set of drivers, and different drivers may have
different starting points and destinations
each edge e has a travel-time function Te(x) = aex + be
A traffic pattern is simply a choice of a path by each driver
The social cost of a given traffic pattern is the sum of the
travel times incurred by all drivers when they use this
traffic pattern
Baochun Li, Department of Electrical and Computer Engineering, University of Toronto
10
212
modeling network traffic u
Social welfare maximizers
Social cost = 7 x 4 = 28
C
5
x
0
A
5
B
x
A
D
Baochun Li, Department of Electrical and Computer Engineering, University of Toronto
11
Unique Nash equilibrium
Social cost = 8 x 4 = 32
C
5
x
0
A
5
B
x
D
Baochun Li, Department of Electrical and Computer Engineering, University of Toronto
12
Two questions and their answers
Q1: Is there always an equilibrium traffic pattern?
Pure strategy equilibria may not exist
But the answer is yes in the traffic game
Q2: Is there always an equilibrium traffic pattern whose
social cost is not much more than the social optimum?
Roughgarden and Tardos: There is always an equilibrium
whose social cost is at most 4/3 that of the optimum
If we change 45 to 40 in our example, we get the upper
bound of “penalty”: 4/3 (an increase in travel times from 60
to 80)
It is as bad as it can get!
Baochun Li, Department of Electrical and Computer Engineering, University of Toronto
13
How to find a traffic pattern at equilibrium?
Using an algorithm called best-response dynamics
Start from any traffic pattern
If it is an equilibrium, done!
Otherwise, there is at least one driver whose best response
is some alternate path (with a strictly lower travel time)
Check again to see if it is an equilibrium, and continue
iterating until the algorithm stops at an equilibrium
But why should the algorithm stop?
In the Matching Pennies game, if only pure strategies are
allowed, two players with switch between H and T forever
Baochun Li, Department of Electrical and Computer Engineering, University of Toronto
14
s the traffic pattern from the social optimum to the inferior equilibrium in t
aradox).
So
in
general,
as
best-response
dynamics
runs,
the
social
cost
of
t
Analyzing best-response dynamics
affic pattern can oscillate between going up and going down, and it’s not cle
is Idea:
related
to
our
progress
toward
an
equilibrium.
define a measure that shows progress in the algorithm
d, we’re going to define an alternate quantity that initially seems a bit my
Social cost
maysee
notthat
be ita suitable
progress
measure
However,
we will
has the property
that
it strictly decreases w
-response
update,
and worse
so it can
be used
to track
the progress of best-respon
As it may
become
in the
Braess’s
Paradox
s [303].
will
referother
to this
quantity
the progress
potential energy of a traffic patte
WeWe
need
some
measure
to as
show
otential energy of a traffic pattern is defined edge-by-edge, as follows. If
Potential energy of a traffic pattern
urrently has x drivers on it, then we define the potential energy of this ed
If an edge e currently has x drivers on it, then the potential energy of
this edge is:
Energy(e) = Te (1) + Te (2) + · · · + Te (x).
Interpreted as the “cumulative” quantity in which we imagine drivers
crossing the edge one by one
The potential energy of a traffic pattern is the sum of the potential
energies of all the edges
Baochun Li, Department of Electrical and Computer Engineering, University of Toronto
15
(a)
modeling
network traffic
using
traffic
using game
game theory
theory
Potential
energy
in best-response
dynamics
214
C
C
energy
energyx=
= 1+2
1+2x
A
A
A
5
5
0
0
26
55
00
55
D
xx
D
D
energy
energy =
= 5+5
5+5
B
B
ergyenergy
= 1+2 = 1+2
x
A
A
A
x
xx
energy
5 energy
5 == 00
C
C
230
0
x
55
D
ergyenergy
= 5+5 = 5+5
D
D
D
energy
energy =
= 5+5
5+5
(c)
B
(c)
(c)
(c)
x
xx
A
A
A
A
5
x
21
C
C
B
D
B
D
Dx
energy = 5
x B
B
x
(c)
(d)
x
= 1+2+3
energyenergy
= 1+2+3
energy = 1+2+3+4
(b)
C
energy
energy == 1+2+3
1+2+3
C energyenergy
=0 =0
energy
== 00
energy
5
5
0
00
5
energy = 0
B
0
20
A
B
energy = 0
5
x
55
0
5
energy = 1+2+3+4
D
D
(b)
x
B
B
5
D
0
A
x
05
C
x
5
energy
== 55 B
energy
5
24
energy
energy == 1+2+3
1+2+3
B
5
energy
5 = 05+5
0
energy
energy == 5+5
5+5
xx
C
0
C
C
energy = 1+2+3
energyenergy
=5 =5
5
0
xx
5
A
A
55
00
5
A
A
Cx
(b)
(b)
= 1+2+3 C
energyenergy
= 1+2+3
C energyenergy
=0 =0
energy
energy =
= 1+2
1+2
x
Ax
= 5+5 = 5+5
energyenergy
= 1+2 = 1+2 energyenergy
(a)
(a)
C
energy
energy == 1+2
1+2
55
energy
energy == 1+2
1+2
(a)
(a)
B
B
x
x
D
energy
nergy
= 5+5= 5+5
A
5 energy = 5+5
energy = 5+5
5
C
C
xx
energy = 1+2
energyenergy
= 5+5 = 5+5 energy = 1+2
energy = 0
C
energy = 1+2
energy
nergy
= 1+2= 1+2
A
(b)
B
x
D
energy = 1+2+3+4
(e)
BB
5Figure5 8.5. We can track the progress of best-response dynamics in the tr
x
x
5
5how the potentialxenergy changes: (a) initial traffic pattern (potential ene
x
step of best-response
dynamics (potential energy is 24); (c) after two ste
D
D
23); (d) after
three steps (potential energy is 21); and (e) after four step
D
energyenergy
= 1+2+3+4
= 1+2+3+4
D
energy
energyenergy
= 5 = reached
=is 1+2+3+4
(potentialenergy
energy
20). = 1+2+3+4
5
energy
energy == 1+2+3+4
1+2+3+4
energy
=
5
energy
=
1+2+3+4
energy = 5
energy = 1+2+3+4
(d)
(d)
(d)
(d) has no drivers on it, its potential energy is defined to b
If an edge
potential energy
energy of a traffic pattern is then simply the sum of the potentia
edges, with their current numbers of drivers in this traffic pattern.
energy
=
1+2+3+4
C
energy
== 00
energy
=
1+2+3+4
C
energy
=
energy
=
1+2+3+4
C
energy
energyand
= 1+2+3+4
energy
0 = 0energy of each edge for the five traffic patterns that16bes
Baochun Li, Department of Electrical
Computer Engineering,CUniversity of
Toronto
potential
The potential energy strictly decreases
From one traffic pattern to the next, the only change is that
one driver abandons his current path and switches to a new
one
First releases potential energy as the driver leaves the system
The change in potential energy on edge e is Te(x), exactly the travel
time that the driver was experiencing on e
Then adds potential energy as he rejoins
The increase of Te(x + 1) is exactly the new travel time the driver
experiences on this edge
The net change in potential energy is simply his new travel
time minus his old travel time
With best-response dynamics, this must be negative!
Baochun Li, Department of Electrical and Computer Engineering, University of Toronto
17
of an edge by Energy(e), and we recall that, when there are x drive
The potential energy is the area under the shaded rectangles; it is alway
al
energy
is defined
Comparing
traffic
to social
ralhand,
ofwhich
theequilibrium
xby
experiences
travel
time
ofoptimum
T (x), and so th
traveleach
time,
isdrivers
the area
inside
the aenclosing
rectangle.
e
ime experienced by all drivers on the edge is
Recall that Energy(e) = Te (1) + Te (2) + · · · + Te (x).
y, and
we can see this
by a bit of simple=algebra,
Total-Travel-Time(e)
xTe (x).recalling that Te (x) = a
other hand, each of the x drivers experiences a travel time of Te (x), and
Te (1)experienced
+ Te (2) + ·by
· ·all
+ drivers
Te (x) =onathe
+2+
e (1 edge
vel time
is · · · + x) + be x
s of comparison with the potential energy, it is useful to write this a
ae x(x + 1)
+ be x
=
Total-Travel-Time(e)
= xTe (x).
2
!
"
Total-Travel-Time(e)
=
T
(x)
+
T
(x)
+
·
·
·
+
T
(x)
.
e
e
e
a
(x
+
1)
e
poses of comparison with the
energy,
it is $useful to write
"#
! potential
= x x terms
+ be
:
2
1each have x terms, but the terms
tential energy
and
the
total
travel
time
Total-Travel-Time(e) = Te≥(x) +
Tee(x)
+b·e )· · + Te (x) .
x(a
x
+
$
pression are at least as large as the! terms
former, we have
2 in the"#
x terms
1
xTe (x).
= time
Energy(e)
≤ Total-Travel-Time(e).
he potential energy
and the
total travel
each have x terms, but the te
2
er expression are at least as large as the terms in the former, we have
f energies and total travel times, this says
Baochun Li, Department of Electrical and Computer Engineering, University of Toronto
18
we saw in the previous section that the potential energy decreases as best-resp
same relationships govern
the potential energy and social cost of
′
mics
moves from Zequilibrium
to Z , and so
Comparing
traffic to social optimum
Energy(Z ′ ) ≤ Energy(Z).
1
[Social-Cost(Z)]
≤ Energy(Z)
≤ Social-Cost(Z).
nd, the2quantitative
relationships
between energies
and social cost say that
′
′
) ≤ 2[Energy(Z
)]
Westart
startSocial-Cost(Z
fromaasocially
socially
optimaltraffic
traffic
pose that we
from
optimal
pattern Z, and
patternto
Z, run
allow
best-response
esponse dynamics
until
it stops at an equilibrium traffic pa
dynamics
toas
run
stops at an dynamics, but the
ost may start
increasing
wetill
runitbest-response
Energy(Z)
≤pattern
Social-Cost(Z).
equilibrium
traffic
Z’, then
we be more than t
nly go down. And because the social cost
can never
have:
we just
chain
these inequalities
that cost from eve
ergy,
this
shrinking
potentialtogether,
energy concluding
keeps the social
wice
as
high
as
′where it started.
′ Therefore, the social cost of the equ
Social-Cost(Z ) ≤ 2[Energy(Z )] ≤ 2[Energy(Z)] ≤ 2[Social-Cost(Z)].
at most twice the cost of the social optimum we started with; he
that
this
really
is
the
same
argument
that
we
made
in
words
in
the
prev
librium with at most twice the socially optimal cost, as we wanted
graph: the potential energy decreases during best-response dynamics, and
e
this
argument
out
in
terms
of
the
inequalities
on
energies
and
soc
ease prevents the social cost from ever increasing by more than a factor of 2.
whus,
in the
previous
section
thatisthe
energy
decreases
as besttracking
potential
energy
notpotential
only useful
for showing
that best-resp
′
oves
fromreach
Z toanZequilibrium;
, and so by relating this potential energy to the social
mics must
Baochun Li, Department of Electrical and Computer Engineering, University of Toronto
19
Chapter 8
Baochun Li, Department of Electrical and Computer Engineering, University of Toronto