Asaf Cohen
(joint with Erhan Bayraktar and Amarjit Budhiraja)
Department of Mathematics
University of Michigan
Math Finance, Probability, and PDE Conference
Rutgers University
May 2017
1
Contents
The Queueing Model
The Mean-Field Game
Asymptotic Nash Equilibrium
Numerical Scheme
2
Contents
The Queueing Model
The Mean-Field Game
Asymptotic Nash Equilibrium
Numerical Scheme
2
The Queueing Model
Literature
Rate control in communication systems:
(1) H. J. Kushner. Heavy traffic analysis of controlled queueing and communication networks,
2001.
(2) B. Ata, J. M. Harrison, and L. A. Shepp. Drift rate control of a Brownian processing system.
AAP, 2005
(3) A. Budhiraja, A. P. Ghosh, and C. Lee. Ergodic rate control problem for single class
queueing networks, SICON, 2011
Rate control in limit order books:
(1) E. Bayraktar and M. Ludkovski. Liquidation in limit order books with controlled intensity.
Math. Finance, 2014.
(2) O. Gueant and C.A. Lehalle. General intensity shapes in optimal liquidation. Math. Finance,
2015.
(3) A. Lachapelle, J. Lasry, C.A. Lehalle, and P. Lions. Efficiency of the price formation process
in presence of high frequency participants: a mean field game analysis, Arxiv 305.6323, 2015.
Strategic servers:
(1) R. Gopalakrishnan, S. Doroudi, A. R. Ward, and A. Wierman. Routing and staffing when
servers are strategic. Oper. Res., 2016.
(2) J. Li, R. Bhattacharyya, S. Paul, S. Shakkottai, and V. Subramanian. Incentivizing sharing in
real time d2d streaming networks: A mean field game perspective. IEEE(INFOCOM), 2015
3
The Queueing Model
Basic definitions
size of the
i-th queue
controlled
arrival
process
controlled
service
process
queues
capacity
queue 1
4
queue 2
queue 3
queue 4
The Queueing Model
Basic definitions
size of the
i-th queue
controlled
arrival
process
controlled
service
process
queues
capacity
queue 1
4
queue 2
queue 3
queue 4
The Queueing Model
Scaling
size of the
i-th queue
controlled
arrival
process
controlled
service
process
queues
• arrival rates
• service rates
where
•
is the control of Server
is the scaled queue length
•
is the empirical distribution of the scaled
•
queue lengths
5
heavy traffic
The Queueing Model
Scaling
size of the
i-th queue
controlled
arrival
process
controlled
service
process
queues
• arrival rates
• service rates
where
•
is the control of Server
is the scaled queue length
•
is the empirical distribution of the scaled
•
queue lengths
5
heavy traffic
The Queueing Model
Scaling
reflections from
below and above
where
5
The Queueing Model
The control problem
6
The Queueing Model
The control problem
Representation by the Skorokhod map:
Given
the cost is
and
initial states
strategies
MIN
6
The Queueing Model
Assumptions a)-b)
a) The functions
and are uniformly Lipschitz with respect to all of their
arguments.
b) The Hamiltonian has a unique minimizer:
Main Result
There exists an asymptotic Nash equilibrium. That is, there is a sequence of admissible
strategies
, such that for every player , and every sequence of admissible
strategies
for that player, one has
7
Contents
The Queueing Model
The Mean-Field Game
Asymptotic Nash Equilibrium
Numerical Scheme
8
The Mean-Field Game
Literature
Mean-field games:
(1) J.-M. Lasry and P.-L. Lions. Jeux à champ moyen. I. Le cas stationnaire. C. R. Math. Acad.
Sci. Paris, 2006..
(2) J.-M. Lasry and P.-L. Lions. Jeux à champ moyen. II. Horizon ni et contrôle optimal. C. R.
Math. Acad. Sci. Paris, 2006.
(3) J.-M. Lasry and P.-L. Lions. Mean field games. Jpn. J. Math., 2007.
(4) M. Huang, P. E. Caines, and R. P. Malhame. The Nash certainty equivalence principle and
Mckean-Vlasov systems: An invariance principle and entry adaptation. Decision and
Control, IEEE, 2007.
(5) M. Huang, R. P. Malhame, and P. E. Caines. Large population stochastic dynamic games:
Closed loop McKean-Vlasov systems and the Nash certainty equivalence principle. Commun.
Inf. Syst., 2006.
(6) P. Cardaliaguet. Notes on mean field games. 2013
(7) R. Carmona and F. Delarue. Probabilistic analysis of mean-field games. SICON, 2013.
(8) R. Carmona and D. Lacker. A probabilistic weak formulation of mean field games and
applications. AAP, 2015.
Brownian control problem:
(1) J.M. Harrison, Brownian models of queueing networks with heterogeneous customer
populations, IMA Volumes in Mathematics and its Applications, 1988.
9
The Mean-Field Game
Intuition
The idea is to approximate the queueing process
By the drift-controlled reflected diffusion process
weak
formulation
In the heavy-traffic literature, the limit problem is often called Brownian control
problem. We now define a Brownian control problem of mean-field type, that is, a
mean-field game.
Notice that the processes in the pre-limit are discrete, whereas the limit is continuous!
10
The Mean-Field Game
Scheme
(1) Fix a flow of measures
.
Solve the control problem with
replacing the flow of empirical
measures
. Denote the value function by
.
(2) Find the optimal control
(3) Find a fixed point of the mapping
and the distribution of
under that control.
.
Theorem 1
Under Assumptions a)-b), there is a solution to the MFG.
The fixed
point is
called a
solution of
the MFG
Remark
Under standard assumptions in MFG theory uniqueness of the MFG
solution is attained.
11
Contents
The Queueing Model
The Mean-Field Game
Asymptotic Nash Equilibrium
Numerical Scheme
12
Asymptotic Nash Equilibrium
Assumptions a)-d)
a) The functions
and are uniformly Lipschitz with respect to all of their
arguments.
b) The Hamiltonian has a unique minimizer:
c) The function
d) There exists
is independent of the mean-field term.
s.t. for every ,
Theorem 2 (main result)
There exists an asymptotic Nash equilibrium. That is, there is a sequence of admissible
controls
, such that for every player , and every sequence of admissible
strategies
for that player, one has
13
Asymptotic Nash Equilibrium
(1) Same control for all the players
•
The controls are defined by
a solution of the MFG.
•
We show that the sequence
where
is
derivative
Weak
of the
value
formulation
of
the MFG
is tight and by reducing to a subsequence using de-Finetti type of argument,
and
•
Using the regularity of
•
By the SLLN, we get
we get that
and the continuity of the Skorokhod mapping, we get
, and by the continuity of the cost functions
Value of the MFG with
14
Asymptotic Nash Equilibrium
(2)
of one
(1) Deviation
Same control
for player
all the (Player
players 1)
•
The controls
by the others
Server
1 usesare defined
, while
a solution of the MFG.
•
We show that the sequence
where
is
relaxed
controls
is tight and by reducing to a subsequence using
usingde-Finetti
de-Finettitype
typeof
ofargument,
argument,
•
Using the convergence
regularity of of andand
thethe
continuity
continuity
of the
of the
Skorokhod
Skorokhod
mapping,
mapping,
we we
getget
•
Onethe
player
has
impact on the limit of the
empirical
before,
we
By
SLLN,
weno
get
, and
by the distribution,
continuity of so
theascost
functions
get get that
, and by the continuity of the cost functions we get that
we
Value of theValue
MFG with
of the MFG with
14
Contents
The Queueing Model
The Mean-Field Game
Asymptotic Nash Equilibrium
Numerical Scheme
15
Numerical Scheme
Literature
(1) Y. Achdou and I. Capuzzo-Dolcetta. Mean-Field Games: Numerical Methods, SIAM J.
Numer. Anal, 2010
(2) Y. Achdou, F. Camilli, and I. Capuzzo-Dolcetta. Mean-Field Games: Numerical Methods for
the Planning Problem, SICON, 2012
(3) S. Cacace, F. Camilli, C.A. Marchi. A Numerical Method for Mean-Field Games on Networks.,
ESAIM: Mathematical Modelling and Numerical Analysis, 2017
16
Numerical Scheme
Approximating Markov chain
Fix
and
buffer
size
time
MDP
MIN
17
Numerical Scheme
Assumptions
Fix a sequence
Assumptions a)-e)
a) The functions
and are uniformly Lipschitz with respect to all of their
arguments.
b) The Hamiltonian has a unique minimizer:
c) The function is independent of the mean-field term.
d) There exists
s.t. for every ,
e) The MFG has a unique solution.
18
is Lipschitz
Numerical Scheme
First attempt
Fix a sequence
Iterations:
(1) Fix an arbitrary function
optimal
and set
tightness
(subsequence)
(2) Define
Now, the tightness of
implies that
is relatively tight.
Finally, we need to make sure that is optimal. To this end, we fix an arbitrary control
and by an additional discretization argument, and using the optimality of
we show
that is suboptimal.
However, this is not working! Tightness forces us to reduce to subsequences and we
cannot obtain the limit .
19
Numerical Scheme
Solution
(1) Using the same scheme for a fixed h:
(a) show contraction of the mapping
(b) find a fixed point of
(for the fixed h), denoted by
. So,
and
(c) take
and use the tightness argument.
(2) Using the same scheme for a fixed h:
(a) show “almost” contraction of the mapping
(b) after enough iterations as before, there is
such that
and
(c) take
20
and use the tightness argument and the last inequality.
Numerical Scheme
Numerical study
We studied a linear quadratic MFG with penalty for rejections:
and
.
Also,
where
21
is the mean of .
Numerical Scheme
Numerical study
22
23
Summary
(1) We consider a rate control problem for large symmetric queuing systems in
heavy-traffic with strategic servers.
(2) We introduce an MFG for controlled reflected diffusions, establish its
solvability and prove unique solvability under additional conditions.
(3) We use the solution of MFG to construct an asymptotically optimal Nash
equilibrium for the n-player game.
(4) We develop a convergent numerical scheme to solve the MFG.
23
The Mean-Field Game
Scheme
(1) Fix a flow of measures
Solve the control problem
.
s.t.
Find the optimal control
and the distribution of
(2) Find a fixed point of the mapping
.
Theorem 1
Under Assumptions a)-b), there is a solution to the MFG.
11
under that control.
The fixed
point is
called a
solution of
the MFG