From the master equation to mean field game limits, fluctuations, and large deviations
From the master equation to mean field
game limits, fluctuations, and large
deviations
Daniel Lacker
Division of Applied Mathematics, Brown University
June 16, 2017
Joint work with Francois Delarue and Kavita Ramanan
From the master equation to mean field game limits, fluctuations, and large deviations
Overview
Overview
A mean field game (MFG) will refer to a game with a continuum
of players.
In various contexts, we know rigorously that the MFG arises as the
limit of n-player games as n → ∞.
From the master equation to mean field game limits, fluctuations, and large deviations
Overview
Overview
A mean field game (MFG) will refer to a game with a continuum
of players.
In various contexts, we know rigorously that the MFG arises as the
limit of n-player games as n → ∞.
This talk: Refined MFG asymptotics in the form of a central limit
theorem and large deviation principle, as well as non-asymptotic
concentration bounds.
Key idea: Use the master equation to quantitatively relate
n-player equilibrium to n-particle system of McKean-Vlasov type,
building on idea of Cardaliaguet-Delarue-Lasry-Lions ’15.
From the master equation to mean field game limits, fluctuations, and large deviations
Interacting diffusion models
Interacting diffusions
Suppose particles i = 1, . . . , n interact through their empirical
measure according to
n
dXti
=
b(Xti , ν̄tn )dt
+
dWti ,
ν̄tn
1X
δXtk ,
=
n
k=1
where W 1 , . . . , W n are independent Brownian motions.
From the master equation to mean field game limits, fluctuations, and large deviations
Interacting diffusion models
Interacting diffusions
Suppose particles i = 1, . . . , n interact through their empirical
measure according to
n
dXti
=
b(Xti , ν̄tn )dt
+
dWti ,
ν̄tn
1X
δXtk ,
=
n
k=1
where W 1 , . . . , W n are independent Brownian motions.
Under “nice” assumptions on b, we have ν̄tn → νt , where νt solves
the McKean-Vlasov equation,
dXt = b(Xt , νt )dt + dWt ,
νt = Law(Xt ),
From the master equation to mean field game limits, fluctuations, and large deviations
Interacting diffusion models
Interacting diffusions
Suppose particles i = 1, . . . , n interact through their empirical
measure according to
n
dXti
=
b(Xti , ν̄tn )dt
+
dWti ,
ν̄tn
1X
δXtk ,
=
n
k=1
where W 1 , . . . , W n are independent Brownian motions.
Under “nice” assumptions on b, we have ν̄tn → νt , where νt solves
the McKean-Vlasov equation,
dXt = b(Xt , νt )dt + dWt ,
or
νt = Law(Xt ),
d
1
hνt , ϕi = hνt , b(·, νt )∇ϕ(·) + ∆ϕ(·)i.
dt
2
From the master equation to mean field game limits, fluctuations, and large deviations
Interacting diffusion models
Empirical measure limit theory
There is a rich literature on asymptotics of ν̄tn :
1. LLN: ν̄ n → ν, where ν solves a McKean-Vlasov equation.
(Oelschläger ’84, Gärtner ’88, Sznitman ’91, etc.)
From the master equation to mean field game limits, fluctuations, and large deviations
Interacting diffusion models
Empirical measure limit theory
There is a rich literature on asymptotics of ν̄tn :
1. LLN: ν̄ n → ν, where ν solves a McKean-Vlasov equation.
(Oelschläger ’84, Gärtner ’88, Sznitman ’91, etc.)
√
2. Fluctuations: n(ν̄tn − νt ) converges to a distribution-valued
process driven by space-time Brownian motion.
(Tanaka ’84, Sznitman ’85, Kurtz-Xiong ’04, etc.)
From the master equation to mean field game limits, fluctuations, and large deviations
Interacting diffusion models
Empirical measure limit theory
There is a rich literature on asymptotics of ν̄tn :
1. LLN: ν̄ n → ν, where ν solves a McKean-Vlasov equation.
(Oelschläger ’84, Gärtner ’88, Sznitman ’91, etc.)
√
2. Fluctuations: n(ν̄tn − νt ) converges to a distribution-valued
process driven by space-time Brownian motion.
(Tanaka ’84, Sznitman ’85, Kurtz-Xiong ’04, etc.)
3. Large deviations: ν̄ n has an explicit LDP.
(Dawson-Gärtner ’87, Budhiraja-Dupius-Fischer ’12)
From the master equation to mean field game limits, fluctuations, and large deviations
Interacting diffusion models
Empirical measure limit theory
There is a rich literature on asymptotics of ν̄tn :
1. LLN: ν̄ n → ν, where ν solves a McKean-Vlasov equation.
(Oelschläger ’84, Gärtner ’88, Sznitman ’91, etc.)
√
2. Fluctuations: n(ν̄tn − νt ) converges to a distribution-valued
process driven by space-time Brownian motion.
(Tanaka ’84, Sznitman ’85, Kurtz-Xiong ’04, etc.)
3. Large deviations: ν̄ n has an explicit LDP.
(Dawson-Gärtner ’87, Budhiraja-Dupius-Fischer ’12)
4. Concentration: Finite-n bounds are available for
P(d(ν̄ n , ν) > ), for various metrics d.
(Bolley-Guillin-Villani ’07, etc.)
From the master equation to mean field game limits, fluctuations, and large deviations
Interacting diffusion models
Empirical measure limit theory
There is a rich literature on asymptotics of ν̄tn :
1. LLN: ν̄ n → ν, where ν solves a McKean-Vlasov equation.
(Oelschläger ’84, Gärtner ’88, Sznitman ’91, etc.)
√
2. Fluctuations: n(ν̄tn − νt ) converges to a distribution-valued
process driven by space-time Brownian motion.
(Tanaka ’84, Sznitman ’85, Kurtz-Xiong ’04, etc.)
3. Large deviations: ν̄ n has an explicit LDP.
(Dawson-Gärtner ’87, Budhiraja-Dupius-Fischer ’12)
4. Concentration: Finite-n bounds are available for
P(d(ν̄ n , ν) > ), for various metrics d.
(Bolley-Guillin-Villani ’07, etc.)
The idea: Use the more tractable McKean-Vlasov system to
analyze the large-n-particle system.
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field games
A class of mean field games
Agents i = 1, . . . , n have state process dynamics
dXti = αti dt + dWti ,
with W 1 , . . . , W n independent Brownian, (X01 , . . . , X0n ) i.i.d.
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field games
A class of mean field games
Agents i = 1, . . . , n have state process dynamics
dXti = αti dt + dWti ,
with W 1 , . . . , W n independent Brownian, (X01 , . . . , X0n ) i.i.d.
Agent i chooses αi to minimize
Z T 1
f (Xti , µ̄nt ) + |αti |2 dt + g (XTi , µ̄nT ) ,
Jin (α1 , . . . , αn ) = E
2
0
n
1X
µ̄nt =
δXtk .
n
k=1
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field games
A class of mean field games
Agents i = 1, . . . , n have state process dynamics
dXti = αti dt + dWti ,
with W 1 , . . . , W n independent Brownian, (X01 , . . . , X0n ) i.i.d.
Agent i chooses αi to minimize
Z T 1
f (Xti , µ̄nt ) + |αti |2 dt + g (XTi , µ̄nT ) ,
Jin (α1 , . . . , αn ) = E
2
0
n
1X
µ̄nt =
δXtk .
n
k=1
Say (α1 , . . . , αn ) form an -Nash equilibrium if
Jin (α1 , . . . , αn ) ≤ + inf Jin (. . . , αi−1 , β, αi+1 , . . .), ∀i = 1, . . . , n
β
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field games
The n-player HJB system
The value function vin (t, x ), for
n-player game solves
∂t vin (t, x ) +
x = (x1 , . . . , xn ), for agent i in the
n
1X
1
∆xk vin (t, x ) + |Dxi vin (t, x )|2
2
2
k=1
+
X
k6=i
Dxk vkn (t, x ) · Dxk vin (t, x ) = f
n
1X
δxk
xi ,
n
k=1
!
.
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field games
The n-player HJB system
The value function vin (t, x ), for
n-player game solves
∂t vin (t, x ) +
x = (x1 , . . . , xn ), for agent i in the
n
1X
1
∆xk vin (t, x ) + |Dxi vin (t, x )|2
2
2
k=1
+
X
Dxk vkn (t, x ) · Dxk vin (t, x ) = f
n
1X
δxk
xi ,
n
k=1
k6=i
A Nash equilibrium is given by
αti = −Dxi vin (t, Xt1 , . . . , Xtn ).
!
.
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field games
The n-player HJB system
The value function vin (t, x ), for
n-player game solves
∂t vin (t, x ) +
x = (x1 , . . . , xn ), for agent i in the
n
1X
1
∆xk vin (t, x ) + |Dxi vin (t, x )|2
2
2
k=1
+
X
Dxk vkn (t, x ) · Dxk vin (t, x ) = f
n
1X
δxk
xi ,
n
k=1
k6=i
A Nash equilibrium is given by
αti = −Dxi vin (t, Xt1 , . . . , Xtn ).
But vin is generally hard to find, especially for large n.
!
.
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field games
Mean field limit n → ∞?
The problem
Given a Nash equilibrium (αn,1 , . . . , αn,n ) for each n, can we
describe the asymptotics of (µ̄nt )t∈[0,T ] ?
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field games
Mean field limit n → ∞?
The problem
Given a Nash equilibrium (αn,1 , . . . , αn,n ) for each n, can we
describe the asymptotics of (µ̄nt )t∈[0,T ] ?
Previous results, limited to LLN
Lasry/ Lions ’06, Feleqi ’13, Fischer ’14, L. ’15,
Cardaliaguet-Delarue-Lasry-Lions ’15, Cardaliaguet ’16...
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field games
Mean field limit n → ∞?
The problem
Given a Nash equilibrium (αn,1 , . . . , αn,n ) for each n, can we
describe the asymptotics of (µ̄nt )t∈[0,T ] ?
Previous results, limited to LLN
Lasry/ Lions ’06, Feleqi ’13, Fischer ’14, L. ’15,
Cardaliaguet-Delarue-Lasry-Lions ’15, Cardaliaguet ’16...
A related, better-understood problem
Find a mean field game solution directly, and use it to construct an
n -Nash equilibrium for the n-player game, where n → 0.
See Huang/Malhamé/Caines ’06 & many others.
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field games
Proposed mean field game limit
A deterministic measure flow (µt )t∈[0,T ] ∈ C ([0, T ]; P(Rd )) is a
mean field equilibrium (MFE) if:

hR
i
T
1
α
∗
2
α

∈ arg minα E 0 f (Xt , µt ) + 2 |αt | dt + g (XT , µT ) ,
α

dXtα



= αt dt + dWt ,
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field games
Proposed mean field game limit
A deterministic measure flow (µt )t∈[0,T ] ∈ C ([0, T ]; P(Rd )) is a
mean field equilibrium (MFE) if:

hR
i
T
1
α
∗
2
α

∈ arg minα E 0 f (Xt , µt ) + 2 |αt | dt + g (XT , µT ) ,
α

dXtα


µ
t
= αt dt + dWt ,
∗
= Law(Xtα ).
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field games
Proposed mean field game limit
A deterministic measure flow (µt )t∈[0,T ] ∈ C ([0, T ]; P(Rd )) is a
mean field equilibrium (MFE) if:

hR
i
T
1
α
∗
2
α

∈ arg minα E 0 f (Xt , µt ) + 2 |αt | dt + g (XT , µT ) ,
α

dXtα


µ
t
= αt dt + dWt ,
∗
= Law(Xtα ).
Law of large numbers
Under strong assumptions, there exists a unique MFE µ, and
µ̄n → µ in probability in C ([0, T ]; P(Rd )).
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
Constructing the MFG value function
1. Fix t ∈ [0, T ) and m ∈ P(Rd ).
2. Solve the MFG starting from (t, m), i.e., find (α∗ , µ) s.t.

hR
i
T
∗
α , µ ) + 1 |α |2 ds + g (X α , µ ) ,

α
∈
arg
min
E
f
(X

α
s
T
s
T

2 s
t
α
dXs = αs ds + dWs , s ∈ (t, T )


∗
µ
= Law(Xsα ),
µt = m
s
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
Constructing the MFG value function
1. Fix t ∈ [0, T ) and m ∈ P(Rd ).
2. Solve the MFG starting from (t, m), i.e., find (α∗ , µ) s.t.

hR
i
T
∗
α , µ ) + 1 |α |2 ds + g (X α , µ ) ,

α
∈
arg
min
E
f
(X

α
s
T
s
T

2 s
t
α
dXs = αs ds + dWs , s ∈ (t, T )


∗
µ
= Law(Xsα ),
µt = m
s
3. Define the value function, for x ∈ Rd , by
U(t, x, m)
Z
=E
t
T
∗
f (Xsα , µs )
α∗
1 ∗2
α∗
+ |αs | ds + g (XT , µT ) Xt = x
2
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
Constructing the MFG value function
1. Fix t ∈ [0, T ) and m ∈ P(Rd ).
2. Solve the MFG starting from (t, m), i.e., find (α∗ , µ) s.t.

hR
i
T
∗
α , µ ) + 1 |α |2 ds + g (X α , µ ) ,

α
∈
arg
min
E
f
(X

α
s
T
s
T

2 s
t
α
dXs = αs ds + dWs , s ∈ (t, T )


∗
µ
= Law(Xsα ),
µt = m
s
3. Define the value function, for x ∈ Rd , by
U(t, x, m)
Z
=E
t
T
∗
f (Xsα , µs )
α∗
1 ∗2
α∗
+ |αs | ds + g (XT , µT ) Xt = x
2
Note: This definition requires uniqueness!
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
Toward the master equation
The strategy is analogous to classical stochastic optimal control:
1. Show the value function satisfies a dynamic programming
principle (DPP).
2. Use the DPP to identify a PDE for the value function.
3. Use this PDE to construct optimal controls.
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
Toward the master equation
The strategy is analogous to classical stochastic optimal control:
1. Show the value function satisfies a dynamic programming
principle (DPP).
2. Use the DPP to identify a PDE for the value function.
3. Use this PDE to construct optimal controls.
The second step requires a notion of derivative on the space
P(Rd ) of probability measures as well as an analog of Itô’s formula
for certain measure-valued processes.
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
Derivatives on P(Rd )
Definition
δu
u : P(Rd ) → R is C 1 if ∃ δm
: P(Rd ) × Rd → R continuous s.t.
e − m)) − u(m)
u(m + t(m
lim
=
h↓0
t
Z
Rd
δu
e − m)(y ).
(m, y ) d(m
δm
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
Derivatives on P(Rd )
Definition
δu
u : P(Rd ) → R is C 1 if ∃ δm
: P(Rd ) × Rd → R continuous s.t.
e − m)) − u(m)
u(m + t(m
lim
=
h↓0
t
Define also
Z
Dm u(m, y ) = Dy
Rd
δu
e − m)(y ).
(m, y ) d(m
δm
δu
(m, y ) .
δm
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
Derivatives on P(Rd )
Definition
δu
u : P(Rd ) → R is C 1 if ∃ δm
: P(Rd ) × Rd → R continuous s.t.
e − m)) − u(m)
u(m + t(m
lim
=
h↓0
t
Define also
Z
Dm u(m, y ) = Dy
Rd
δu
(m, y ) .
δm
Key lemma: For x1 , . . . , xn ∈ Rd ,
!
n
1X
1
Dxi u
δxk = Dm u
n
n
k=1
δu
e − m)(y ).
(m, y ) d(m
δm
n
1X
δxk , xi
n
k=1
!
.
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
Key tool: The master equation
Using the DPP along with an Itô formula for functions of
measures, one derives the master equation:
Z
∂t U(t, x, m) −
Dx U(t, y , m) · Dm U(t, x, m, y ) m(dy )
Rd
1
1
+ f (x, m) − |Dx U(t, x, m)|2 + ∆x U(t, x, m)
2
2
Z
1
+
divy Dm U(t, x, m, y ) m(dy ) = 0,
2 Rd
Refer to Cardaliaguet-Delarue-Lasry-Lions ’15,
Chassagneux-Crisan-Delarue ’14, Carmona-Delarue ’14,
Bensoussan-Frehse-Yam ’15
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
Key tool: The master equation
Using the DPP along with an Itô formula for functions of
measures, one derives the master equation:
Z
∂t U(t, x, m) −
Dx U(t, y , m) · Dm U(t, x, m, y ) m(dy )
Rd
1
1
+ f (x, m) − |Dx U(t, x, m)|2 + ∆x U(t, x, m)
2
2
Z
1
+
divy Dm U(t, x, m, y ) m(dy ) = 0,
2 Rd
Assume henceforth that there is a smooth classical solution with
bounded derivatives!
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
Key tool: The master equation
Using the DPP along with an Itô formula for functions of
measures, one derives the master equation:
Z
∂t U(t, x, m) −
Dx U(t, y , m) · Dm U(t, x, m, y ) m(dy )
Rd
1
1
+ f (x, m) − |Dx U(t, x, m)|2 + ∆x U(t, x, m)
2
2
Z
1
+
divy Dm U(t, x, m, y ) m(dy ) = 0,
2 Rd
Assume henceforth that there is a smooth classical solution with
bounded derivatives!
Assume also E[exp(κ|X01 |2 )] < ∞ for some κ > 0.
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
Key tool: The master equation
Using the DPP along with an Itô formula for functions of
measures, one derives the master equation:
Z
∂t U(t, x, m) −
Dx U(t, y , m) · Dm U(t, x, m, y ) m(dy )
Rd
1
1
+ f (x, m) − |Dx U(t, x, m)|2 + ∆x U(t, x, m)
2
2
Z
1
divy Dm U(t, x, m, y ) m(dy ) = 0,
+
2 Rd
Assume henceforth that there is a smooth classical solution with
bounded derivatives!
See also explicitly solvable models: Carmona-Fouque-Sun ’13,
L.-Zariphopoulou ’17
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
A first n-particle approximation
The MFE µ is the unique solution of the McKean-Vlasov equation
dXt = −Dx U(t, Xt , µt ) dt + dWt ,
{z
}
|
α∗t
µt = Law(Xt ).
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
A first n-particle approximation
The MFE µ is the unique solution of the McKean-Vlasov equation
dXt = −Dx U(t, Xt , µt ) dt + dWt ,
{z
}
|
µt = Law(Xt ).
α∗t
Old idea: Consider the system of n independent processes,
dXti = −Dx U(t, Xti , µt ) dt + dWti .
|
{z
}
αit
These controls αti can be proven to form an n -equilibrium for the
n-player game, where n → 0.
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
A better n-particle approximation
Key idea of Cardaliaguet et al.: Consider the McKean-Vlasov
system
n
1X
i
n
i
i
n
ν̄t =
dYt = −Dx U(t, Yt , ν̄t ) dt + dWt ,
δYtk .
{z
}
|
n
αit
k=1
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
A better n-particle approximation
Key idea of Cardaliaguet et al.: Consider the McKean-Vlasov
system
n
1X
i
n
i
i
n
ν̄t =
dYt = −Dx U(t, Yt , ν̄t ) dt + dWt ,
δYtk .
{z
}
|
n
k=1
αit
Classical theory says that ν̄ n → ν, where ν solves the
McKean-Vlasov equation,
dYt = −Dx U(t, Yt , νt )dt + dWt ,
νt = Law(Yt ).
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
A better n-particle approximation
Key idea of Cardaliaguet et al.: Consider the McKean-Vlasov
system
n
1X
i
n
i
i
n
ν̄t =
dYt = −Dx U(t, Yt , ν̄t ) dt + dWt ,
δYtk .
{z
}
|
n
k=1
αit
Classical theory says that ν̄ n → ν, where ν solves the
McKean-Vlasov equation,
dYt = −Dx U(t, Yt , νt )dt + dWt ,
νt = Law(Yt ).
We had the same equation for the MFE µ, so uniqueness implies
µ ≡ ν.
So to prove µ̄n → µ, it suffices to show µ̄n and ν̄ n are close.
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
A better n-particle approximation
Key result of Cardaliaguet et al. ’15
Recalling that µ̄nt denotes the n-player Nash equilibrium empirical
measure, µ̄n and ν̄ n are very close.
Note: This requires smoothness assumptions on the master
equation U, but not on the n-player HJB system!
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
A better n-particle approximation
Key result of Cardaliaguet et al. ’15
Recalling that µ̄nt denotes the n-player Nash equilibrium empirical
measure, µ̄n and ν̄ n are very close.
Note: This requires smoothness assumptions on the master
equation U, but not on the n-player HJB system!
Proof idea: Show that
uin (t, x1 , . . . , xn )
n
:= U
1X
δxk
t, xi ,
n
k=1
nearly solves the n-player HJB system.
!
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
The n-player HJB system revisited
We defined
n
uin (t, x1 , . . . , xn )
:= U
1X
δxk
t, xi ,
n
!
.
k=1
Use the master equation U to find
∂t uin (t, x ) +
n
1X
1
∆xk uin (t, x ) + |Dxi uin (t, x )|2
2
2
k=1
+
X
k6=i
Dxk ukn (t,
x) ·
Dxk uin (t,
x) = f
n
1X
δxk
xi ,
n
where rin is continuous, with krin k∞ ≤ C /n.
k=1
!
+ rin (t, x ),
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
Nash system vs. McKean-Vlasov system
The n-player Nash equilibrium state processes solve
dXti = −Dxi vin (t, Xt1 , . . . , Xtn )dt + dWti .
Compare this to the McKean-Vlasov system,
dYti = −Dx U(t, Yti , ν̄tn )dt + dWti , where ν̄tn =
1X
δYtk .
n
k=1
Use Lipshitz property of Dx U and Gronwall to bound
n
n Z
1X i
CX t
i 2
|Xt − Yt | ≤
|(Dxi vin − Dxi uin )(s, Xs1 , . . . , Xsn )|2 ds.
n
n
0
i=1
i=1
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
Nash system vs. McKean-Vlasov system
We have estimated
n
n Z
1X i
C X t i,i
i,i
|Zs − Z s |2 ds,
|Xt − Yti |2 ≤
n
n
0
i=1
where
Yti = vin (t, Xt ),
Y t = uin (t, Xt ),
i
i=1
Zti,j = Dxj vin (t, Xt ),
Z t = Dxj uin (t, Xt ).
i,j
The rest of the argument relies on BSDE-type estimates.
From the master equation to mean field game limits, fluctuations, and large deviations
The master equation
Nash system vs. McKean-Vlasov system
We have estimated
n
n Z
1X i
C X t i,i
i,i
|Zs − Z s |2 ds,
|Xt − Yti |2 ≤
n
n
0
i=1
where
i=1
Yti = vin (t, Xt ),
Y t = uin (t, Xt ),
i
Zti,j = Dxj vin (t, Xt ),
Z t = Dxj uin (t, Xt ).
i,j
The rest of the argument relies on BSDE-type estimates.
P
Key observation: Recalling uin (t, x ) = U(t, xi , n1 nk=1 δxk ), the
bounds on master equation derivatives yield
i,i
|Z t | ≤ C ,
i,j
|Z t | ≤ C /n, for i 6= j.
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field game asymptotics
Toward refined mean field game asymptotics
Main idea: Estimate the “distance” between the Nash EQ
empirical measure µ̄n and the McKean-Vlasov empirical measure
ν̄ n , and then transfer known results on McKean-Vlasov limits.
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field game asymptotics
Toward refined mean field game asymptotics
Main idea: Estimate the “distance” between the Nash EQ
empirical measure µ̄n and the McKean-Vlasov empirical measure
ν̄ n , and then transfer known results on McKean-Vlasov limits.
Note: In linear-quadraticRsystems, we can instead describe the
asymptotics of the mean Rd x d µ̄nt (x) in a self-contained manner.
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field game asymptotics
Fluctuations
Theorem
√
√
The sequences n(µ̄nt − µt ) and n(ν̄tn − µt ) both “converge” to
the unique solution of the SPDE:
p
∂t St (x) = A∗t,µt St (x) − divx ( µt (x)Ḃ(t, x)),
where B is a space-time Brownian motion and
Z
δ
(Dx U(t, y , m)) (x) · ∇ϕ(y ) m(dy ),
At,m ϕ(x) := Lt,m ϕ(x) −
Rd δm
1
Lt,m ϕ(x) := −Dx U(t, x, m) · ∇ϕ(x) + ∆ϕ(x).
2
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field game asymptotics
Fluctuations
Theorem
√
√
The sequences n(µ̄nt − µt ) and n(ν̄tn − µt ) both “converge” to
the unique solution of the SPDE:
p
∂t St (x) = A∗t,µt St (x) − divx ( µt (x)Ḃ(t, x)),
where B is a space-time Brownian motion and
Z
δ
(Dx U(t, y , m)) (x) · ∇ϕ(y ) m(dy ),
At,m ϕ(x) := Lt,m ϕ(x) −
Rd δm
1
Lt,m ϕ(x) := −Dx U(t, x, m) · ∇ϕ(x) + ∆ϕ(x).
2
Provides a second-order approximation µ̄nt ≈ µt +
√1 St .
n
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field game asymptotics
Proof idea
√
Show Stn = n(µ̄nt − ν̄tn ) → 0 , then use Kurtz-Xiong ’04 to
√
identify limit of n(ν̄tn − µt ). For nice ϕ,
n
1 X
|hStn , ϕi| ≤ √
|ϕ(Xti ) − ϕ(Yti )| ≤ . . .
n
i=1
n Z t
X
C
≤√
|Xsi − Ysi | + W2 (µ̄ns , ν̄sn )
n
0
i=1
+ |Dxi v n,i (s, Xs ) − Dx U(s, Xsi , µ̄ns )| ds.
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field game asymptotics
Proof idea
√
Show Stn = n(µ̄nt − ν̄tn ) → 0 , then use Kurtz-Xiong ’04 to
√
identify limit of n(ν̄tn − µt ). For nice ϕ,
n
1 X
|hStn , ϕi| ≤ √
|ϕ(Xti ) − ϕ(Yti )| ≤ . . .
n
i=1
n Z t
X
C
≤√
|Xsi − Ysi | + W2 (µ̄ns , ν̄sn )
n
0
i=1
+ |Dxi v n,i (s, Xs ) − Dx U(s, Xsi , µ̄ns )| ds.
Key point: Master equation estimates yield
"
#
n
1X
C
E sup |Xti − Yti | ≤ ,
n
n
t∈[0,T ]
i=1
√
not C / n ! Similarly for other terms.
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field game asymptotics
Proof idea
√
Show Stn = n(µ̄nt − ν̄tn ) → 0 , then use Kurtz-Xiong ’04 to
√
identify limit of n(ν̄tn − µt ). For nice ϕ,
n
1 X
|hStn , ϕi| ≤ √
|ϕ(Xti ) − ϕ(Yti )| ≤ . . .
n
i=1
n Z t
X
C
≤√
|Xsi − Ysi | + W2 (µ̄ns , ν̄sn )
n
0
i=1
+ |Dxi v n,i (s, Xs ) − Dx U(s, Xsi , µ̄ns )| ds.
Key point: Master equation estimates yield
"
#
n
1X
C
E sup |Xti − Yti | ≤ ,
n
n
t∈[0,T ]
i=1
√
√
not C / n ! Similarly for other terms. Yields E|hStn , ϕi| ≤ C / n.
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field game asymptotics
Large deviations
Theorem
The sequences µ̄n and ν̄ n both satisfy a large deviation principle on
C ([0, T ]; P(Rd )), with the same (good) rate function.
( RT
1
k∂t mt − L∗t,mt mt k2S dt if m abs. cont.
I (m· ) = 2 0
∞
otherwise,
where k · kS acts on Schwartz distributions by
kγk2S = sup hγ, ϕi2 /hγ, |∇ϕ|2 i.
ϕ∈Cc∞
Heuristically:
n
P (µ̄ ∈ A) ≈ exp −n inf I (m· ) .
m· ∈A
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field game asymptotics
Large deviations
Proof idea: Show exponential equivalence
!
1
n n
sup W2 (µ̄t , ν̄t ) > = −∞, ∀ > 0,
lim log P
n→∞ n
t∈[0,T ]
where W2 is Wasserstein distance, then identify LDP ν̄ n using
Dawson-Gärtner ’87 or Budhiraja-Dupuis-Fischer ’12.
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field game asymptotics
Large deviations
Proof idea: Show exponential equivalence
!
1
n n
sup W2 (µ̄t , ν̄t ) > = −∞, ∀ > 0,
lim log P
n→∞ n
t∈[0,T ]
where W2 is Wasserstein distance, then identify LDP ν̄ n using
Dawson-Gärtner ’87 or Budhiraja-Dupuis-Fischer ’12.
Key challenge: Bounding W2 (µ̄nt , ν̄tn ) requires exponential
estimates for terms like
n
n Z
1 XX T
|(Dxj vin − Dxj uin )(t, Xt1 , . . . , Xtn )|2 dt.
n
0
i=1 j=1
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field game asymptotics
Non-asymptotic estimates
Theorem (Dimension-free concentration)
∃C , δ > 0 such that for ∀ a > 0, ∀ n ≥ C /a and all 1-Lipshitz
functions Φ : (C ([0, T ]; Rd ))n → R we have
2
2
P |Φ(X 1 , . . . , X n ) − E Φ(X 1 , . . . , X n )| > a ≤ 2ne −δna + 2e −δa .
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field game asymptotics
Non-asymptotic estimates
Theorem (Dimension-free concentration)
∃C , δ > 0 such that for ∀ a > 0, ∀ n ≥ C /a and all 1-Lipshitz
functions Φ : (C ([0, T ]; Rd ))n → R we have
2
2
P |Φ(X 1 , . . . , X n ) − E Φ(X 1 , . . . , X n )| > a ≤ 2ne −δna + 2e −δa .
Corollary
∃C , δ > 0 such that for ∀ a > 0, ∀ n ≥ C /a we have
2 2
2
P sup W2 (µ̄nt , µt ) > a ≤ 2ne −δn a + 2e −δna .
t∈[0,T ]
Proof idea.
The map (x1 , . . . , xn ) 7→ W2 ( n1
Pn
i=1 δxi , µt )
is n−1/2 -Lipschitz.
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field game asymptotics
Non-asymptotic estimates
Quantitatively compare n-player and k-player games:
Corollary
∃C , δ > 0 such that for ∀ a > 0, ∀ n, k ≥ C /a we have
P sup W2 (µ̄nt , µ̄kt ) > a
t∈[0,T ]
≤ 2ne −δn
2 a2
2
+ 2e −δna + 2ke −δk
2 a2
2
+ 2e −δka .
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field game asymptotics
Non-asymptotic estimates
Proof of concentration theorem.
Use McKean-Vlasov results after showing
v

u n
u1 X
P t
kX i − Y i k2∞ > a ≤ 2n exp(−δa2 n2 ).
n
i=1
Justify dimension-free concentration for McKean-Vlasov systems
by showing Pn := Law(Y 1 , . . . , Y n ) satisfies a transport-entropy
inequality with constant independent of n, i.e., ∃C > 0 s.t.
p
W1 (Pn , Q) ≤ CH(Q|Pn ), ∀Q Pn .
Use results of Djellout-Guillin-Wu ’04.
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field game asymptotics
The moral of the story
Sufficiently smooth solution of master equation
=⇒ refined asymptotics for mean field game equilibria,
by comparing the n-player equilibrium to an n-particle system and
then applying existing results on McKean-Vlasov systems.
From the master equation to mean field game limits, fluctuations, and large deviations
Mean field game asymptotics
The moral of the story
Sufficiently smooth solution of master equation
=⇒ refined asymptotics for mean field game equilibria,
by comparing the n-player equilibrium to an n-particle system and
then applying existing results on McKean-Vlasov systems.
Major challenges
I
Requires a lot of regularity for the master equation, permitting
Lipshitz-BSDE-type estimates.
I
Are there counterexamples without smoothness? E.g., can we
always expect µ̄n and ν̄ n to be exponentially equivalent?