INTRODUCTION
GENERAL PRESENTATION
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
DIFFERENT POPULATIONS
COALITIONS
1/110
MEAN FIELD GAMES AND MEAN FIELD TYPE
CONTROL THEORY
Alain Bensoussan
1
1 International
Jens Frehse
2
Phillip Yam
3
Center for Decision and Risk Analysis
Jindal School of Management, University of Texas at Dallas
and Department of Systems Engineering and Engineering department,City
University Hong Kong
2 Institute
3 Department
for Applied Mathematics, Bonn University
of Statistics,The Chinese University of Hong Kong
August 16, 2013
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, Ha
INTRODUCTION
GENERAL PRESENTATION
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
DIFFERENT POPULATIONS
COALITIONS
OUTLINE I
1
2
3
4
2/110
INTRODUCTION
GENERAL PRESENTATION
MODEL AND ASSUMPTIONS
DEFINITION OF THE PROBLEMS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL
PROBLEM
HJB-FP APPROACH
STOCHASTIC MAXIMUM PRINCIPLE
TIME CONSISTENCY APPROACH
THE MEAN VARIANCE PROBLEM
TIME CONSISTENCY
DIFFERENT POPULATIONS
GENERAL CONSIDERATIONS
MULTI-CLASS AGENTS
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, Ha
INTRODUCTION
GENERAL PRESENTATION
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
DIFFERENT POPULATIONS
COALITIONS
OUTLINE II
MAJOR PLAYER
5
3/110
COALITIONS
SYSTEM OF HJB-FP EQUATIONS
APPROXIMATE NASH EQUILIBRIUM FOR LARGE
COMMUNITIES
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, Ha
INTRODUCTION
GENERAL PRESENTATION
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
DIFFERENT POPULATIONS
COALITIONS
ACKNOWLEDGEMENTS
Research supported by the Research Grants Council of Hong Kong
Special Administrative Region (City U 500113)
4/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, Ha
INTRODUCTION
GENERAL PRESENTATION
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
DIFFERENT POPULATIONS
COALITIONS
GENERAL COMMENTS
Mean eld games reduces to a standard control problem and
an equilibrium (xed point)
Dynamic Programming: coupled HJB and FP equations
Mean eld type control is a non standard control problem.
Stochastic Maximum Principle ( time inconsistency)
Time inconsistency
Major Playor
Coalitions
5/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, Ha
INTRODUCTION
GENERAL PRESENTATION
MODEL AND ASSUMPTIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
DEFINITION OF THE PROBLEMS
DIFFERENT POPULATIONS
COALITIONS
MODEL
Probability space Ω, A , P , ltration F t generated by an
n -dimensional standard Wiener process w (t ).
The state space is R n and the control space is R d .
( , , ) : R n × L1 (R n ) × R d → R n ;
g x m v
f
(x , m, v ) : R n × L1 (R n ) × R d → R ;
σ (x ) : R n → L (R n ; R n )
( , ) : R n × L1 (R n ) → R
h x m
σ (x ), σ −1 (x ) bounded
m
6/110
(1)
(2)
is a probability density on R n
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, Ha
INTRODUCTION
GENERAL PRESENTATION
MODEL AND ASSUMPTIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
DEFINITION OF THE PROBLEMS
DIFFERENT POPULATIONS
COALITIONS
STATE EQUATION
7/110
( ) ∈ C (0, T ; L1 (R n )) given . Feedback control v (x , t ).
m t
state of the system
dx
x
= g (x (t ), m(t ), v (x (t ))dt + σ (x (t ))dw (t )
(3)
(0) = x0
is a random variable independent of the Wiener process,
probability density m = m(0).
To the pair v (.), m(.) we associate the control objective
x0
0
( (.), m(.)) = E [
Z T
J v
f
(x (t ), m(t ), v (x (t )) dt + h(x (T ), m(T )]
0
Alain Bensoussan, Jens Frehse, Phillip Yam
(4)
Dierential games, Nash equilibrium, Mean Field, Ha
INTRODUCTION
GENERAL PRESENTATION
MODEL AND ASSUMPTIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
DEFINITION OF THE PROBLEMS
DIFFERENT POPULATIONS
COALITIONS
MEAN FIELD GAME
8/110
Find a pair v̂ (.),m(.) such that, denoting by x̂ (.) the solution
of
d x̂
x̂
= g (x̂ (t ), m(t ), v̂ (x̂ (t )))dt + σ (x̂ (t ))dw (t )
(5)
(0) = x0
then
( ) is the probability distribution of x̂ (t ), ∀t ∈ [0, T ]
m t
J (v̂
(6)
(.), m(.)) ≤ J (v (.), m(.))∀v (.)
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, Ha
INTRODUCTION
GENERAL PRESENTATION
MODEL AND ASSUMPTIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
DEFINITION OF THE PROBLEMS
DIFFERENT POPULATIONS
COALITIONS
MEAN FIELD TYPE CONTROL PROBLEM
9/110
For any feedback v (.), let x (t ) = xv (.) (t ) be the solution of (3)
with m(t ) =probability distribution of xv (.) (t ). So (3) becomes
a McKean-Vlasov equation. Denote by mv (.) (t ) =probability
distribution of xv (.) (t ), we thus have
dx
x
v (.) = g (xv (.) (t ), mv (.) (t ), v (xv (.) (t ))dt + σ (xv (.) (t ))dw (t ) (7)
(0) = x0
v (.) (t ) = probability distribution of xv (.) (t )
m
(8)
Find v̂ (.) such that
J (v̂
(.), mv̂ (.) (.)) ≤ J (v (.), mv (.) (.)) ∀v (.)
Alain Bensoussan, Jens Frehse, Phillip Yam
(9)
Dierential games, Nash equilibrium, Mean Field, Ha
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
NOTATION
Set
1
2
and the 2nd order dierential operator
( ) = σ (x )σ ∗ (x )
(10)
ϕ(x ) = −tr a(x )D 2 ϕ(x )
(11)
a x
A
The Dual Operator is
∗
A
ϕ(x ) = −
n
∂2
(akl (x )ϕ(x ))
k ,l =1 ∂x ∂x
∑
k
10/110
Alain Bensoussan, Jens Frehse, Phillip Yam
(12)
l
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
GATEAUX DIFFERENTIABILITY
11/110
Assume that the
m
→ f (x , m, v ), g (x , m, v ), h(x , m)
are dierentiable in m ∈ L
2
Notation
d
d
θ
(R n )
(13)
∂f
(x , m, v )(ξ ) to represent the derivative, so that
∂m
Z
f
(x , m + θ m̃, v )|θ =0 =
∂f
(x , m, v )(ξ )m̃(ξ ) d ξ
∂
R m
Alain Bensoussan, Jens Frehse, Phillip Yam
n
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
OBJECTIVE FUNCTIONAL I
12/110
Consider a feedback v (x ) and the corresponding trajectory
dened by (7), the probability distribution mv (.) (t ) of xv (.) (t )
is solution of the FP equation
∂ mv (.)
+ A∗ mv (.) + div (g (x , mv (.) , v (x ))mv (.) ) = 0
∂t
mv (.) (x , 0) = m0 (x )
(14)
and the objective functional J (v (.), mv (.) ) can be expressed as
follows
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
OBJECTIVE FUNCTIONAL II
13/110
( (.), mv (.) (.)) =
Z TZ
J v
0
Z
+
R
R
f
n
(x , mv (.) (x ), v (x ))mv (.) (x )dxdt + (15)
( , v (.) (x , T ))mv (.) (x , T )dx
h x m
n
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
FURTHER DIFFERENTIABILITY I
Consider an optimal feedback v̂ (x ) and the corresponding
probability density mv̂ (.) (x ) = m(x ).Let then v (.) be any
feedback and v̂ (x ) + θ v (x ). We want to compute
v̂ (.)+θ v (.) (x )
dm
d
θ
|θ =0 = m̃(x )
We can check that
14/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
FURTHER DIFFERENTIABILITY II
∂ m̃
+ A∗ m̃ + div (g (x , m, v̂ (x ))m̃) +
∂t
(16)
Z
+div([
15/110
∂g
∂g
(x , m, v̂ (x ))(ξ )m̃(ξ )d ξ +
(x , m, v̂ (x ))v (x )]m(x )) = 0
∂m
∂v
m̃ (x , 0) = 0
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
COST DIFFERENTIABILITY I
16/110
dJ (v̂
(.) + θ v (.), mv̂ (x )+θ v (x ) (.))
|θ =0 =
dθ
Z TZ
0
Z TZ
0
R
R
f
(17)
(x , m, v̂ (x ))m̃(x )dtdx +
n
Z
n
∂f
(x , m, v̂ (x ))(ξ )m̃(ξ )m(x )dtd ξ dx +
∂
R m
Z TZ
∂f
(x , m, v̂ (x ))v (x )m(x )dtdx +
0
R ∂v
n
n
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
COST DIFFERENTIABILITY II
17/110
Z
+
Z
+
R
R
( , ( ))m̃(x , T )dx
h x m T
n
Z
n
∂h
(x , m(T ))(ξ )m̃(ξ , T )m(x , T )d ξ dx
∂
R m
n
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
FUNCTION
u(x , t )
Introduce the function u (x , t ) solution of
Z
∂u
∂g
−
+ Au − g (x , m, v̂ (x )).Du −
Du (ξ ).
(ξ , m, v̂ (ξ ))(x )m(ξ )d ξ
∂t
∂m
R
Z
∂f
= f (x , m, v̂ (x ))+
(ξ , m, v̂ (ξ ))(x )m(ξ )d ξ
R ∂m
n
n
(18)
Z
( , ) = h(x , m(T ))+
u x T
∂h
(ξ , m(T ))(x )m(ξ , T )d ξ
∂
R m
n
18/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
NECESSARY CONDITION I
19/110
(.) + θ v (.), mv̂ (x )+θ v (x ) (.))
|θ =0 =
dθ
Z TZ
∂f
(x , m, v̂ (x ))v (x )m(x )dtdx +
0
R ∂v
Z TZ
∂g
(x , m, v̂ (x ))v (x )m(x )dtdx
+
Du (x ).
∂v
0
R
dJ (v̂
n
n
Since v̂ (.) is optimal, this expression must vanish for any v (.).
Hence necessarily
∂f
∂g ∗
(x , m, v̂ (x )) +
(x , m, v̂ (x ))Du (x ) = 0
∂v
∂v
Alain Bensoussan, Jens Frehse, Phillip Yam
(19)
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
REWRITING I
It follows that ( at least with convexity assumptions)
( ) = v̂ (x , m, Du (x ))
(20)
(x , m, v̂ (x )) + g (x , m, v̂ (x )).Du = H (x , m, Du )
(21)
v̂ x
We note that
f
Z
∂f
∂g
(ξ , m, v̂ (ξ ))(x ) + Du (ξ ).
(ξ , m, v̂ (ξ ))(x )]m(ξ )d ξ =
∂m
R ∂m
[
n
(22)
Z
∂H
(ξ , m, Du (ξ ))(x )m(ξ )d ξ
R ∂m
n
20/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
REWRITING II
21/110
( , , ( )) = g (x , m, v̂ (x , m, Du (x ))) = G (x , m, Du )
g x m v̂ x
Alain Bensoussan, Jens Frehse, Phillip Yam
(23)
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
HJB-FP SYSTEM
We can nally write the system of HJB-FP P.D.E.
−
Z
∂H
∂u
+ Au = H (x , m, Du ) +
(ξ , m, Du (ξ ))(x )m(ξ )d ξ
∂t
∂m
R
Z
∂h
u (x , T ) = h (x , m (T )) +
(ξ , m(T ))(x )m(ξ , T )d ξ (24)
∂
R m
n
n
∂m
+ A∗ m + div (G (x , m, Du )m) = 0
∂t
m (x , 0) = m0 (x )
22/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
SPIKE MODIFICATION I
Instead of changing v̂ (x , t ) into v̂ (x , t ) + θ v (x , t ) one can use
a spike modication
v̄ (x , s ) = v
( , )
v̂ x s
s
s
∈ (t , t + ε)
6∈ (t , t + ε)
similar to the proof of Pontryagin maximum principle.
One proves directly that v̂ (x , t ) minimizes the Lagrangian in v ,
instead of simply being a stationary point
23/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
NOTATION
24/110
From the optimal feedback v̂ (x ) and the probability distribution
( ) we construct stochastic processes X (t ) ∈ R n , V (t ) ∈ R d ,
n
n n
Y (t ) ∈ R , Z (t ) ∈ L (R ; R ) which are adapted, dened as follows
m t
( ) = x̂ (t ),
X t
( ) = PX (t )
m t
We next dene
Y
(t ) = Du (X (t ), t ),
( ) = v̂ (X (t ), PX (t ) , Y (t ))
V t
and nally
( ) = D 2 u σ (X (t ), t )
Z t
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
STOCHASTIC MAXIMUM PRINCIPLE I
=g (X (t ), PX (t ) , V (t ))dt + σ (X (t ))dw (t )
∂H
−dY =
(X (t ), PX (t ) , V (t ), Y (t ))+
(25)
∂x
∂ 2H
∂ σ (X (t )) ∗
Z (t )
dt
E[
(X (t ), PX (t ) , V (t ), Y (t ))](X (t )) + tr
∂ x∂ m
∂x
− Z (t )dw (t )
(26)
2
∂ h(X (T ), PX (T ) )
∂ h
X (0) = x0 , Y (T ) =
+E[
(X (T ), PX (T ) ] (X (T )
∂x
∂ x∂ m
dX
25/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
STOCHASTIC MAXIMUM PRINCIPLE I
( ) minimizes H (X (t ), PX (t ) , v , Y (t )) in v
V t
(27)
When we write
∂ 2f
(X (t ), PX (t ) , V (t ))](X (t ))
∂ x∂ m
∂f
we mean that we take the function
(ξ , m, v )(x ), where ξ and v
∂m
are parameters and we take the gradient in x , denoted by
∂ 2f
(ξ , m, v )(x ).
∂ x∂ m
We then consider ξ = X (t ), v = V (t ) and take the expected value
∂ 2f
E
(X (t ), m, V (t ))(x ).
∂ x∂ m
E
26/110
[
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
STOCHASTIC MAXIMUM PRINCIPLE II
We take m = PX (t ) (note that it is a deterministic quantity) and
∂ 2f
thus get E
(X (t ), PX (t ) , V (t ))(x ).
∂ x∂ m
Finally, we take the argument x = X (t ).
27/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
POSSIBLE CONFUSION I
∂f
To emphasize the diculty of confusion, consider (x , m, v ). If
∂x
we want to take the derivative with respect to m, then we should
consider x , v as parameters, so change the notation to ξ and
∂ 2f
compute
(ξ , m, v )(x ).
∂ m∂ x
Clearly
∂ 2f
∂ 2f
(ξ , m, v )(x ) 6=
(ξ , m, v )(x )
∂ m∂ x
∂ x∂ m
28/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
PARTICULAR CASE I
29/110
We discuss here the following particular mean eld type problem
dx
x
= g (x (t ), v (x (t ))dt + σ (x (t ))dw (t )
(28)
(0) = x0
Z T
( (.), m(.)) = E [
J v
+
Z T
f
(x (t ), v (x (t )) dt + h(x (T ))]
0
(
(29)
( ))dt + Φ(Ex (T ))
F Ex t
0
We consider a feedback v (x , t ) and m(t ) = mv (.) (t ) is the
probability density of xv (.) (t ) the solution of (28).
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
PARTICULAR CASE II
The functional becomes J (v (.), mv (.) (.)). It is clearly a particular
case of mean eld type control problem.
30/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
NOTATION I
31/110
We have indeed
Z
f
(x , m, v ) = f (x , v ) + F (
ξ m(ξ )d ξ )
Z
( , ) = h(x ) + Φ(
h x m
ξ m(ξ )d ξ )
Therefore
Z
( , , ) = H (x , q ) + F (
H x m q
ξ m(ξ )d ξ )
where
( , ) = inf (f (x , v ) + q .g (x , v ))
H x q
v
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
NOTATION II
32/110
Considering v̂ (x , q ) which attains the inmum in the denition of
H (x , q ) and setting
( , ) = g (x , v̂ (x , q ))
G x q
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
HJB-FP SYSTEM I
The coupled system HJB-FP becomes , see (24),
−
∂u
+ Au = H (x , Du ) + F (
∂t
Z
Z
∂F
( ξ m(ξ )d ξ )xk
k ∂ xk
Z
Z
∂Φ
( ξ m(ξ )d ξ )xk
u (x , T ) = h (x ) + Φ(
ξ m(ξ )d ξ ) + ∑
k ∂ xk
ξ m(ξ )d ξ ) + ∑
(30)
∂m
+ A∗ m + div (G (x , Du )m) = 0
∂t
m (x , 0) = δ (x − x0 )
33/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
REWRITING I
We can reduce slightly this problem, using the following step:
introduce the vector function Ψ(x , t ; s ), t < s , solution of
−
∂Ψ
+ AΨ − D Ψ.G (x , Du ) = 0, t < s
∂t
Ψ(x , s ; s ) = x
(31)
then
Z
ξ m(ξ , t )d ξ = Ψ(x0 , 0; t )
so (30) becomes
34/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
REWRITING II
−
∂u
∂F
+ Au = H (x , Du ) + F (Ψ(x0 , 0; t )) + ∑
(Ψ(x0 , 0; t ))xk
∂t
∂
k xk
∂Φ
(Ψ(x0 , 0; T ))xk (32)
∂
k xk
( , ) = h(x ) + Φ(Ψ(x0 , 0; T )) + ∑
u x T
35/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
PRECOMMITMENT
We now have the system (31), (32). We can also look at u (x , t ) as
the solution of a non-local HJB equation, depending on the initial
state x . The optimal feedback
0
( , ) = v̂ (x , Du (x , t ))
v̂ x t
depends also on x . Note that it does not depend on any
intermediate state. This type of optimal control is called a
pre-commitment optimal control.
0
36/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
GAME CONCEPT
In [8], the authors introduce a new concept, in order to dene an
optimization problem among feedbacks which do not depend on the
initial condition.
A feedback will be optimal only against spike changes, but not
against global changes.
Game interpretation. Players are attached to small periods of
time ( eventually to each time, in the limit). Therefore, if one
uses the concept of Nash equilibrium, decisions at dierent
times correspond to decisions of dierent players, and thus out
of reach.
37/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
NOTATION I
In the spirit of Dynamic Programming, and the invariant
embedding idea, we consider a family of control problems indexed
by the initial conditions, and we control the system using feedbacks
only. So if v (x , s ) is a feedback, we consider the state equation
( ) = xxt (s ; v (.))
x s
dx
= g (x (s ), v (x (s ), s ))ds + σ (x (s ))dw (t )
(33)
( )=x
x t
and the payo
38/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
NOTATION II
39/110
Z T
J
x ,t (v (.)) = E [
+
Z T
t
t
(
f
(x (s ), v (x (s ), s )) ds + h(x (T )] +
(34)
( ))ds + Φ(Ex (T ))
F Ex s
Consider a specic control v̂ (x , s ) which will be optimal . We
dene x̂ (.) to be the corresponding state, solution of (33) and set
( , ) = Jx ,t (v̂ (.))
V x t
Alain Bensoussan, Jens Frehse, Phillip Yam
(35)
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
SPIKE MODIFICATION I
40/110
We make a spike modication and dene
v̄ (x , s ) = v
( , )
v̂ x s
t
< s < t +ε
s > t +ε
where v is arbitrary. The idea is to evaluate Jx ,t (v̄ (.)) and to
express that it is larger than V (x , t ). We introduce the function
Ψ(x , t ; s ) = E x̂xt (s ), t < s
which is the solution of
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
SPIKE MODIFICATION II
41/110
−
∂Ψ
+ AΨ − D Ψ.g (x , v̂ (x , t )) = 0, t < s
∂t
Ψ(x , s ; s ) = x
(36)
We note the important property
( ) = E Ψ(x (t + ε), t + ε; s ), ∀s ≥ t + ε
E x̄ s
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
COMPARISON I
Therefore
x ,t (v̄ (.)) = E [
Z t +ε
J
+
Z T
t +ε
f
t
(x (s ), v )ds +
x (t +ε),t +ε (s ), v̂ (x̂x (t +ε),t +ε (s ), s ))ds + h(x̂x (t +ε),t +ε (T ))]
f (x̂
+
Z t +ε
t
(
( ))ds +
F Ex s
Z T
t +ε
( Ψ(x (t + ε), t + ε; s ))ds +
F E
+Φ(E Ψ(x (t + ε), t + ε; T )
42/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
COMPARISON I
The next point is to compare F (E Ψ(x (t + ε), t + ε; s )) with
EF (Ψ(x (t + ε), t + ε; s )). This is a simple application of Ito's
formula
EF
(Ψ(x (t + ε), t + ε; s )) − F (E Ψ(x (t + ε), t + ε; s )) =
ε ∑ aij (x ) ∑
ij
∂
2
F
kl ∂ xk ∂ xl
(Ψ(x , t ; s ))
(37)
∂ Ψk ∂ Ψl
(x , t ; s )) + 0(ε)
∂ xi ∂ xj
We can similarly compute the dierence
E
43/110
Φ(Ψ(x (t + ε), t + ε; T )) − Φ(E Ψ(x (t + ε), t + ε; T ).
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
EVALUATION OF THE PAYOFF I
44/110
J
x ,t (v̄ (.)) = EV (x (t + ε), t + ε) + ε[f (x , v ) + F (x )−
− ∑ aij (x )
ij
Z T
t
∑∂
kl
∂ 2F
∂ Ψk ∂ Ψl
(Ψ(x , t ; s ))
(x , t ; s ))ds −
xk ∂ xl
∂ xi ∂ xj
∂ 2Φ
∂ Ψk ∂ Ψl
(Ψ(x , t ; T ))
(x , t ; T ))] + 0(ε)
∂ xi ∂ xj
kl ∂ xk ∂ xl
− ∑ aij (x ) ∑
ij
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
HJB EQUATION I
45/110
−
Z T
− ∑ aij (x )[
ijkl
+
t
∂V
+ AV = H (x , DV ) + F (x )−
∂t
∂ 2F
∂ Ψk ∂ Ψl
(Ψ(x , t ; s ))
(x , t ; s ))ds +
∂ xk ∂ xl
∂ xi ∂ xj
(38)
∂ 2Φ
∂ Ψk ∂ Ψl
(Ψ(x , t ; T ))
(x , t ; T ))]
∂ xk ∂ xl
∂ xi ∂ xj
( , ) = h(x ) + Φ(x )
V x T
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
FUNCTION
46/110
Ψ
I
Moreover the equation for Ψcan be written as
−
∂Ψ
+ AΨ − D Ψ.G (x , DV ) = 0, t < s
∂t
Ψ(x , s ; s ) = x
(39)
The optimal feedback obtained from the system (38), (39) is time
consistent.
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
STATEMENT OF THE PROBLEM I
The mean-variance problem is the extension in continuous time for
a nite horizon of the Markowitz optimal portfolio theory. Without
referring to the background of the problem, it can be stated as
follows, mathematically. The state equation is
dx
x
= rxdt + xv .(α dt + σ dw )
(40)
(0) = x0
( ) is scalar, r is a positive constant, α is a vector in R m and σ is
a matrix in L (R d ; R m ). All can depend on time and they are
deterministic quantities. v (t ) is the control in R m .
x t
We note that, conversely to our general framework, the control
aects the volatility term. The objective function is
47/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
STATEMENT OF THE PROBLEM II
γ
( (.)) = Ex (T ) − var(x (T ))
J v
2
(41)
which we want to maximize.
48/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
MEAN FIELD TYPE CONTROL PROBLEM I
Because of the variance term, the problem is not a standard
stochastic control problem. It is a mean eld type control problem,
since one can write
γ
γ
( )2 ) + (Ex (T ))2
(42)
2
We consider a feedback control v (x , s ) and the corresponding state
xv (.) (t ) solution of (40) when the control is replaced by the
feedback.
We associate the probability density mv (.) (x , t ) solution of
( (.)) = E (x (T ) −
J v
49/110
2
x T
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
MEAN FIELD TYPE CONTROL PROBLEM II
∂ mv (.)
∂
1 ∂2 2
+
(xmv (.) (r + α.v (x ))) −
(x mv (.) |σ ∗ v (x )|2 ) = 0
∂t
∂x
2 ∂ x2
(43)
v (.) (x , 0) = δ (x − x )
m
0
The functional (42) can be written as
Z
( (.)) =
J v
50/110
v (.) (x , T )(x −
m
γ
2
x
2
γ
)dx + (
Alain Bensoussan, Jens Frehse, Phillip Yam
2
Z
v (.) (x , T )xdx ) (44)
m
2
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
NECESSARY CONDITIONS I
Let v̂ (x , t ) be an optimal feedback, and m(t ) = mv̂ (.) (t ). Using the
mean eld type control approach, we get a pair u (x , t ),m(x , t )
satisfying
∂u 2
∂u 1 (∂x ) ∗
∂u
−
− xr
+
α (σ σ ∗ )−1 α = 0
∂t
∂ x 2 ∂ 2u
∂ x 2Z
γ 2
u (x , T ) = x −
x + γx
m (ξ , T )ξ d ξ
(45)
2
51/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
NECESSARY CONDITIONS II
52/110

∂u

∂m
∂ (xm)
∂ 
m ∂ x  α ∗ (σ σ ∗ )−1 α −
+r
−
2

∂t
∂x
∂x
∂ u
2
 ∂x

∂u 2
( ) 
1 ∂2 
m ∂ x  α ∗ (σ σ ∗ )−1 α = 0
−
2 ∂ x 2  ∂ 2u 2 
( 2)
∂x
m (x , 0) = δ (x − x0 )

Alain Bensoussan, Jens Frehse, Phillip Yam
(46)
(47)
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
OPTIMAL FEEDBACK I
The optimal feedback is dened by
∂u
∂ x (σ σ ∗ )−1 α
v̂ (x , t ) = −
∂ 2u
x
∂ x2
(48)
We can solve explicitly the system (45), (46). We look for
( , )=−
u x t
We also dene
1
P (t )x + s (t )x + ρ(t )
2
2
Z
( )=
q t
53/110
m
(ξ , t )ξ d ξ
Alain Bensoussan, Jens Frehse, Phillip Yam
(49)
(50)
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
SOLUTION I
54/110
We obtain
( ) = γ exp
P t
Z T
t
(2r − α ∗ (σ σ ∗ )−1 α)d τ
( ) = (1 + γ q (T )) exp
s t
ρ(t ) =
Z T
Z T
t
(51)
(r − α ∗ (σ σ ∗ )−1 α)d τ
1s ∗
α (σ σ ∗ )− α(τ)d τ
t 2P
2
1
We have to x q (T ). Equation (46) becomes
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
SOLUTION II
55/110
∂m
∂ (xm)
∂ s
+r
−
) α ∗ (σ σ ∗ )−1 α
m (x −
∂t
∂x
∂x
P
1 ∂2 s 2
−
m (x −
)
α ∗ (σ σ ∗ )−1 α = 0
2 ∂ x2
P
m (x , 0) = δ (x − x0 )
Alain Bensoussan, Jens Frehse, Phillip Yam
(52)
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
OPTIMAL FEEDBACK
If we test this equation with x we obtain easily
( ) = x0 exp
Z T
rd
q T
0
1
τ + [exp
γ
Z T
0
α ∗ (σ σ ∗ )−1 α)d τ − 1]
(53)
This completes the denition of the function u (x , t ). The optimal
feedback is dened by , see (48)
∗ −1
( , ) = −(σ σ )
v̂ x t
α+
1 1 + γ q (T )
x
γ
exp −
Z T
t
rd
τ
(54)
We see that this optimal feedback depends on the initial condition
x0
56/110
.
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
TIME CONSISTENCY APPROACH I
If we take the time consistency approach, we consider the family of
problems
dx
(55)
= rxds + xv (x , s ).(α dt + σ dw ), s > t
( )=x
x t
and the pay-o
J
γ
γ
x ,t (v (.)) = E (x (T ) − 2 x (T ) ) + 2 (Ex (T ))
2
2
(56)
Denote by v̂ (x , s ) an optimal feedback and set V (x , t ) = Jx ,t (v̂ (.)).
57/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
FUNCTION
Ψ
We dene
Ψ(x , t ; T ) = E x̂xt (T )
where x̂xt (s ) is the solution of (55) for the optimal feedback.
58/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
FUNCTION
V
I
The function Ψ(x , t ; T ) is the solution of
∂Ψ ∂Ψ
1 ∂ 2Ψ
+
(rx + x v̂ (x , t )∗ α) + x 2 2 |σ ∗ v̂ (x , t )|2 = 0
∂t
∂x
2 ∂x
Ψ(x , T ; T ) = x
We can write
( , ) = E (x̂xt (T ) −
V x t
59/110
γ
γ
x̂ (T ) ) +
(Ψ(x , t ; T ))
2 xt
2
Alain Bensoussan, Jens Frehse, Phillip Yam
2
2
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
SPIKE MODIFICATION I
We consider a spike modication
v̄ (x , s ) = v
( , )
v̂ x s
t
< s < t +ε
s > t +ε
then
J
γ
x ,t (v̄ (.)) = E ((x̂x (t +ε),t +ε (T ) − 2 x̂x (t +ε),t +ε (T ) )
2
γ
+ (E Ψ(x (t + ε), t + ε; T ))2
2
where x (t + ε) corresponds to the solution of (55) at time t + ε for
the feedback equal to the constant v .
60/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
APPROXIMATION I
61/110
We note that
( ( + ε), t + ε) = E ((x̂x (t +ε),t +ε (T ) −
EV x t
+
γ
2
E
γ
2
x̂
(T ) )
2 x (t +ε),t +ε
(Ψ(x (t + ε), t + ε; T ))2
so we have to compare (E Ψ(x (t + ε), t + ε; T )) with
E (Ψ(x (t + ε), t + ε; T )) . We see easily that
2
2
(E Ψ(x (t + ε), t + ε; T ))2 − E (Ψ(x (t + ε), t + ε; T ))2 =
= −ε x 2
∂ 2Ψ
(x , t ; T )|σ ∗ v |2 + 0(ε)
∂ x2
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
HJB EQUATION I
We obtain the HJB equation
2
∂V ∂V
∂V ∗
x
∂ 2V
∂ 2Ψ
+
rx + max[x
v α +
( 2 − γ 2 (x , t ; T ))v ∗ σ σ ∗ v ] = 0
v
∂t
∂x
∂x
2 ∂x
∂x
(57)
( , )=x
V x T
A direct checking shows that
1 ZT ∗
V (x , t ) = x exp r (T − t ) +
α (σ σ ∗ )α ds
2γ t
Ψ(x , t ; T ) = x exp r (T − t ) +
62/110
Alain Bensoussan, Jens Frehse, Phillip Yam
1Z T
γ t
(58)
α ∗ (σ σ ∗ )α ds
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
HJB-FP APPROACH
GENERAL PRESENTATION
STOCHASTIC MAXIMUM PRINCIPLE
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
TIME CONSISTENCY APPROACH
DIFFERENT POPULATIONS
THE MEAN VARIANCE PROBLEM
COALITIONS
TIME CONSISTENCY
HJB EQUATION II
and
( , )=
v̂ x t
exp −r (T − t )
x
γ
(σ σ ∗ )α
(59)
This optimal control satises the time consistency property ( it
does not depend on the initial condition).
63/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
GENERAL CONSIDERATIONS I
In the preceding slides, we have considered a single population,
composed of a large number of individuals, with identical behavior.
In real situations, we will have several populations. The natural
extension to the preceding developments is to obtain mean eld
equations for each population. A much more challenging situation
will be to consider competing populations. We present rst the
approach of multi-class agents, as described in [16], [18].
64/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
MODEL I
Instead of functions f (x , m, v ),g (x , m, v ), h(x , m), σ (x ) we consider
functions fk (x , m, v ),gk (x , m, v ), hk (x , m), σk (x ), k = 1, · · · K .
The index k represents some characteristics of the agents, and a
class corresponds to one value of the characteristics. So there are
K classes. In the model discussed previously, we have considered a
single class. In the sequel, when we consider an agent i ,he will have
a characteristics α i ∈ (1, · · · , K ). Agents will be dened with upper
indices, so i = 1, · · · N with N very large. α i is a known
information. The important assumption is
K
1 N
N
65/110
∑ 1Iα =k → πk , as
i=
i
N
→ +∞
(60)
1
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
MODEL II
and πk is a probability distribution on the nite set of
characteristics, which represents the probability that an agent has
the characteristics k .
66/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
FURTHER NOTATION I
67/110
Generalizing the case of a single class, we dene
1
ak (x ) =
σ (x )σk (x )∗ and the operator
2 k
k ϕ(x ) = −trak (x )D ϕ(x )
A
2
We dene Lagrangians, Hamiltonians indexed by k ,namely
k (x , m, v , q ) = fk (x , m, v ) + q .gk (x , m, v )
L
L (x , m , v , q )
k (x , m, q ) = inf
v k
H
and v̂k (x , m, q ) denotes the minimizer in the dention of the
Hamiltonian. We also dene
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
FURTHER NOTATION II
68/110
k (x , m, q ) = gk (x , m, v̂k (x , m, q ))
G
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
SYSTEM OF HJB EQUATIONS I
69/110
Given a function m(t ) we consider the HJB equations, indexed by k
−
∂ uk
+ Auk = Hk (x , m, Duk )
∂t
uk (x , T ) = hk (x , m (T ))
(61)
and the FP equations
∂ mk
+ A∗ mk + div (Gk (x , m, Duk )mk ) = 0
∂t
mk (x , 0) = mk 0 (x )
Alain Bensoussan, Jens Frehse, Phillip Yam
(62)
(63)
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
SYSTEM OF HJB EQUATIONS II
in which the probability densities mk are given. A mean eld game
equilibrium for the multi class agents problem is attained whenever
0
( , )=
m x t
K
∑ πk
k=
k (x , t ), ∀x , t
m
(64)
1
70/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
GENERAL COMMENTS I
We consider here a problem initiated by Huang [15], in the L.Q.
case. In a recent paper Nourian and Caines [23] have studied a non
linear mean eld game with a major player. In both papers, there is
a simplication in the coupling between the major player and the
representative agent. We will describe here the problem in full
generality and explain the simplication which is done in [23].
The new element is that, besides the representative agent there is a
major player. This major player inuences directly the mean eld
term. Since the mean eld term also impacts the major playor, he
will takes this into account to dene his decisions. On the other
hand, the mean eld term can no longer be deterministic, since it
depends on the major player decisions. This coupling creates new
diculties.
71/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
MODEL OF MAJOR PLAYER I
72/110
We introduce the following state evolution for the major player
dx0
x0
(65)
= g0 (x0 (t ), m(t ), v0 (t ))dt + σ0 (x0 )dw0
(0) = ξ0
We assume that x (t ) ∈ Rn0 , v (t ) ∈ Rd0 . The process w (t ) is a
standard Wiener process with values in Rk0 and ξ is a random
variable in Rn0 independent of the Wiener process. The process
m (t ) is the mean eld term, with values in the space of
probabilities on Rn . This term will come from the decisions of the
representative agent.
However, It will be linked to x (t ) since the major player inuences
the decision of the representative agent.
0
0
0
0
0
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
MODEL OF MAJOR PLAYER II
If we dene the ltration
F 0t = σ (ξ0 , w0 (s ), s ≤ t )
(66)
then m(t ) is a process adapted to F t . But it is not external.
0
73/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
MODEL OF MAJOR PLAYER I
74/110
We will describe the link with the state x in analyzing the
representative agent problem. The control v (t ) is also adapted to
F t . The objective functional of the major player is
0
0
0
( (.)) = E [
Z T
J0 v0
( ( ), m(t ), v0 (t ))dt +
f0 x0 t
0
(67)
+h0 (x0 (T ), m(T ))]
The functions g , f , σ , h are deterministic. We do not specify the
assumptions, since our treatment is formal.
0
0
0
0
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
MODEL OF REPRESENTATIVE AGENT I
The representative agent has state x (t ) ∈ Rn and control
d
v (t ) ∈ R .We have the evolution
dx
x
= g (x (t ), x0 (t ), m(t ), v (t ))dt + σ (x (t ))dw
(68)
(0) = ξ
in which w (t ) is a standard Wiener process with values in Rk and ξ
is a random variable with values in Rn independent of w (.).
Moreover, ξ , w (.) are independent of ξ , w (.). We dene
0
0
F t = σ (ξ , w (s ), s ≤ t )
Gt = F t ∪Ft
0
75/110
Alain Bensoussan, Jens Frehse, Phillip Yam
(69)
(70)
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
MODEL OF REPRESENTATIVE AGENT II
76/110
The control v (t ) is adapted to G t . The objective functional of the
representative agent is dened by
( (.), x0 (.), m(.)) = E [
Z T
J v
f
(x (t ), x0 (t ), m(t ), v (t ))dt +
0
(71)
+h(x (T ), x0 (T ), m(T ))]
Conversely to the major player problem, in the representative agent
problem, the processes x (.),m(.) are external. In (67) m(t )
depends on x (.).
0
0
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
CONDITIONAL PROBABILITY DENSITY OF THE
REPRESENTATIVE AGENT I
The representative agent's problem is similar to the standard
situation except for the presence of x (t ).
We begin by limiting the class of controls for the representative
agent to belong to feedbacks v (x , t ) random elds adapted to F t
. The corresponding state, solution of (68) is denoted by xv (.) (t ).
Of course, this process depends also of x (t ), m(t ) . Note that
t
x (t ), m (t ) is independent from F , therefore the conditional
probability density of xv (.) (t ) given the ltration ∪t F t is the
solution of the F.P. equation with random coecients
0
0
0
0
0
77/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
CONDITIONAL PROBABILITY DENSITY OF THE
REPRESENTATIVE AGENT II
78/110
∂ pv (.)
+ A∗ pv (.) + div(g (x , x0 (t ), m(t ), v (x , t ))pv (.) ) = 0
∂t
pv (.) (x , 0) = ϖ(x )
(72)
in which ϖ(x ) is the density probability of ξ .
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
OBJECTIVE FUNCTIONAL OF THE REPRESENTATIVE
AGENT I
We can then rewrite the objective functional J (v (.), x (.), m(.)) as
follows
0
Z TZ
( (.), x0 (.), m(.)) = E [
J v
p
0
Z
+
Rn
v (.),x0 (.),m(.) (x , t )f (x , x (t ), m(t ), v (x , t ))d
(73)
v (.),x0 (.),m(.) (x , T )h(x , x (T ), m(T ))dx ]
p
Rn
0
0
We can give an expression for this functional. Introduce the
random eld χv (.) (x , t ) solution of the stochastic backward PDE:
79/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
OBJECTIVE FUNCTIONAL OF THE REPRESENTATIVE
AGENT II
80/110
−
∂ χv (.)
+ Aχv (.) = f (x , x0 (t ), m(t ), v (x , t )) + g (x , x0 (t ), m(t ), v (x , t )).D χ
∂t
(74)
χv (.) (x , T ) = h(x , x0 (T ), m(T ))
then we can assert that
Z TZ
v (.),x0 (.),m(.) (x , t )f (x , x (t ), m(t ), v (x , t ))dxdt +
p
0
Rn
0
Z
p
Rn
v (.),x0 (.),m(.) (x , T )h(x , x (T ), m(T ))dx =
Alain Bensoussan, Jens Frehse, Phillip Yam
0
Z
Rn
χv (.) (x , 0)ϖ(x )
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
COMPUTING THE OBJECTIVE FUNCTION I
We get
Z
( (.), x0 (.), m(.)) =
J v
Rn
ϖ(x )E χv (.) (x , 0)dx
(75)
Now dene
u
v (.) (x , t ) = E
F 0t
χv (.) (x , t )
From equation (74) we can assert that
−E F
0t
∂ χv (.)
+ Auv (.) = f (x , x0 (t ), m(t ), v (x , t )) + g (x , x0 (t ), m(t ), v (x , t )
∂t
(76)
v (.) (x , T ) = h(x , x (T ), m(T ))
u
81/110
0
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
BACKWARD SPDE I
82/110
On the other hand
v (.) (x , t ) −
Z t
u
E
F 0s
0
∂ χv (.)
(x , s )ds
∂s
is a F t martingale. Therefore we can write
0
u
v (.) (x , t )−
Z t
E
0
F 0s
Z t
∂ χv (.)
(x , s )ds = uv (.) (x , 0)+ Kv (.) (x , s )dw0 (s )
∂s
0
where Kv (.) (x , s ) is F s measurable, and uniquely dened. It is
then easy to check that the random eld uv (.) (x , t ) is solution of
the backward stochastic PDE (BSPDE) :
0
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
BACKWARD SPDE II
83/110
−∂t uv (.) (x , t ) + Auv (.) (x , t )dt = f (x , x0 (t ), m(t ), v (x , t ))dt +
(77)
+g (x , x0 (t ), m(t ), v (x , t )).Duv (.) (x , t )dt − Kv (.) (x , t )dw0 (t )
v (.) (x , T ) = h(x , x (T ), m(T ))
u
Alain Bensoussan, Jens Frehse, Phillip Yam
0
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
NECESSARY CONDITION I
84/110
From ( 75) we get immediately
Z
( (.), x0 (.), m(.)) =
J v
Rn
ϖ(x )Euv (.) (x , 0)dx
(78)
We then write a necessary condition of optimality for a control
v̂ (x , t ).Setting u (x , t ) = uv̂ (.) (x , t ), K (x , t ) = Kv̂ (.) (x , t ) we obtain
the stochastic HJB equation
−∂t u (x , t ) + Au (x , t )dt = H (x , x0 (t ), m(t ), Du )dt − K (x , t )dw0
(79)
( , ) = h(x , x0 (T ), m(T ))
u x T
and
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
NECESSARY CONDITION II
85/110
( , ) = v̂ (x , x0 (t ), m(t ), Du (x , t ))
v̂ x t
Alain Bensoussan, Jens Frehse, Phillip Yam
(80)
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
FP EQUATION I
86/110
We next have to express the mean eld game condition
( ) = pv̂ (.),x0 (.),m(.) (., t )
m t
we obtain from (72) the FP equation
∂m
+ A∗ m + div(G (x , x0 (t ), m(t ), Du (x , t ))m) = 0
∂t
m (x , 0) = ϖ(x )
(81)
The coupled pair of HJB-FP equations (79),(81) allow to dene the
reaction function of the representative agent to the trajectory x (.)
of the major player. One denes the random elds u (x , t ), m(x , t )
and the optimal feedback is given by (80).
0
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
MAJOR PLAYER
I
Consider now the problem of the major player. In [23] and also [15]
for the L.Q. case it is limited to (65), (67) since m(t ) is external.
However since m(t ) is coupled to x (t ) through equations (79),
(81) one cannot consider m(t ) as external, unless limiting the
decision of the major player. So in fact the major player has to
consider three state equations (65), (79), (81). For a a given v (.)
adapted to F t we associate x ,v0 (.) (.), uv0 (.) (., .), mv0 (.) (., .)
solution of the system (65), (79), (81).
Introduce the notation
0
0
0
0
87/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
MAJOR PLAYER
88/110
II
( , , ) = inf [f0 (x0 , m, v0 ) + p .g0 (x0 , m, v0 )]
H0 x0 m p
( , , )
v̂0 x0 m p
( , , ) =
G0 x m p
v0
minimizes the expression in brackets
( , , ( , , ))
g0 x0 m v̂0 x0 m p
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
NECESSARY CONDITIONS FOR THE MAJOR PLAYER I
We have 3 adjoint equations
k0
−dp = [H0,x0 (x0 (t ), m(t ), p (t )) + ∑ σ0∗l ,x0 (x0 (t ))ql (t )
l=
1
Z
+
Z
∗
Gx (x , x0 (t ), m (t ), Du (x , t ))D η(x , t )m (x , t )dx
0
(82)
k0
ζ (x , t )Hx0 (x , x0 (t ), m(t ), Du (x , t )dx ]dt − ∑ ql dw0l
l=
89/110
+
Alain Bensoussan, Jens Frehse, Phillip Yam
1
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
NECESSARY CONDITIONS FOR THE MAJOR PLAYER II
90/110
( ) = h0,x0 (x0 (T ), m(T )) +
p T
Z
ζ (x , T )hx0 (x , x0 (T ), m(T ))dx
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
NECESSARY CONDITIONS FOR THE MAJOR PLAYER I
91/110
−∂t η + Aη(x , t )dt = [
∂ H0
(x0 (t ), m(t ), p (t ))(x )
∂m
+D η(x , t ).G (x , x0 (t ), m(t ), Du (x , t ))+
Z
+
D
η(ξ , t ).
Z
+
ζ (ξ , t )
∂G
(ξ , x0 (t ), m(t ), Du (ξ , t ))(x )m(ξ , t )d ξ
∂m
∂H
(ξ , x0 (t ), m(t ), Du (ξ , t ))(x )d ξ ]dt −
∂m
∂ h0
η(x , T ) =
(x0 (T ), m(T ))(x )+
∂m
Z
Alain Bensoussan, Jens Frehse, Phillip Yam
ζ (ξ , T )
∑ µl (
l
(83)
, )
l (t )
x t dw0
∂h
(ξ , x0 (T ), m(T ))(x )d ξ
∂m
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
NECESSARY CONDITIONS FOR THE MAJOR PLAYER I
∂ζ
+ A∗ ζ (x , t ) + div (G (x , x0 (t ), m(t ), Du (x , t ))ζ (x , t ))
∂t
+div(Gq∗ (x , x0 (t ), m(t ), Du (x , t ))D η(x , t ) m(x , t )) = (84)
0
ζ (x , 0) = 0
Next x (t ) satises
0
dx0
x0
92/110
=
( ( ), m(t ), p (t ))dt + σ0 (x0 (t ))dw0
G0 x0 t
(85)
(0) = ξ0
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
GENERAL CONSIDERATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
MULTI-CLASS AGENTS
DIFFERENT POPULATIONS
MAJOR PLAYER
COALITIONS
NECESSARY CONDITIONS FOR THE MAJOR PLAYER II
So, in fact the complete solution is provided by the 6 equations
(85),(82), (79), (84), (81), (83) and the feedback of the
representative agent and the contol of the major player are given by
(80) and
( ) = v̂0 (x0 (t ), m(t ), p (t ))
v̂0 t
93/110
Alain Bensoussan, Jens Frehse, Phillip Yam
(86)
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
SYSTEM OF HJB-FP EQUATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
APPROXIMATE NASH EQUILIBRIUM FOR LARGE COMM
DIFFERENT POPULATIONS
COALITIONS
SYSTEM OF HJB-FP EQUATIONS I
We can introduce more general problems
−
∂ ui
+ Au i = H i (x , m, Du )
∂t
i
i
u (x , T ) = h (x , m (T ))
∂ mi
+ A∗ mi + div (G i (x , m, Du )mi ) = 0
∂t
i
i
m (x , 0) = m0 (x )
(87)
(88)
in which m = (m , · · · , mN ) and the functions H i , G i depend on the
full vector m. The interpretation is much more elaborate.
1
94/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
SYSTEM OF HJB-FP EQUATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
APPROXIMATE NASH EQUILIBRIUM FOR LARGE COMM
DIFFERENT POPULATIONS
COALITIONS
DESCRIPTION OF THE GAME I
95/110
We want to associate to problem (87), (88) a dierential game for
N communities, composed of very large numbers of agents. We
denote the agents by the index i , j where i = 1, · · · N and
j = 1, · · · M . The number M will tend to +∞. Each player
i ,j (x ), x ∈ Rn . The state of player i , j is
i , j chooses a feedback v
denoted by x i ,j (t ) ∈ Rn . We consider independent standard Wiener
processes w i ,j (t ) and independent replicas x i ,j of the random
variable x i , whose probability density is mi . They are independent
of the Wiener processes. We denote
0
0
0
v
.j
(.) = (v 1,j (.), · · · , v N ,j (.))
The trajectory of the state x i ,j is dened by the equation
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
SYSTEM OF HJB-FP EQUATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
APPROXIMATE NASH EQUILIBRIUM FOR LARGE COMM
DIFFERENT POPULATIONS
COALITIONS
DESCRIPTION OF THE GAME II
i ,j = g i (x i ,j , v .j (x i ,j ))dt + σ (x i ,j )dw i ,j
i ,j (0) = x i ,j
x
dx
(89)
0
96/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
SYSTEM OF HJB-FP EQUATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
APPROXIMATE NASH EQUILIBRIUM FOR LARGE COMM
DIFFERENT POPULATIONS
COALITIONS
DESCRIPTION OF THE GAME I
97/110
The trajectories are independent. The player i , j trajectory is
inuenced by the feedbacks v k ,j (x ), k 6= i acting on his own state.
When we focus on player i we use the notation
v
.j
(.) = (v i ,j (.), v̄ i ,j (.))
in which v̄ i ,j (.) represents all feedbacks v k ,j (x ), k 6= i . The notation
v (.) represents all feedbacks.
We now dene the objective functional of player i , j . It is given by
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
SYSTEM OF HJB-FP EQUATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
APPROXIMATE NASH EQUILIBRIUM FOR LARGE COMM
DIFFERENT POPULATIONS
COALITIONS
DESCRIPTION OF THE GAME II
J i ,j (v (.)) = E
i ( i ,j (t ),
f0 x
1
M
M
− 1 l =∑
16=j
Z T
[f i (x i ,j (t ), v .j (x i ,j (t ))) +
0
δx , (t ) ))]dt + Ehi (x i ,j (T ),
1
i l
(90)
M
δx , (T ) )
− 1 l =∑
16=j
i l
M
We look for a Nash equilibrium.
98/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
SYSTEM OF HJB-FP EQUATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
APPROXIMATE NASH EQUILIBRIUM FOR LARGE COMM
DIFFERENT POPULATIONS
COALITIONS
APPROXIMATE NASH EQUILIBRIUM I
99/110
Consider next the system of pairs of HJB-FP equations (87), (88)
and the feedback v̂ (x ).
We can show that the feedback
v̂
i ,j (.) = v̂ i (.)
is an approximate Nash equilibrium.
If we use this feedback in the state equation (89) we get
d x̂
i ,j = g i (x̂ i ,j , v̂ (x̂ i ,j ))dt + σ (x̂ i ,j )dw i ,j
i ,j (0) = x i ,j
x̂
0
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
SYSTEM OF HJB-FP EQUATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
APPROXIMATE NASH EQUILIBRIUM FOR LARGE COMM
DIFFERENT POPULATIONS
COALITIONS
APPROXIMATE NASH EQUILIBRIUM II
100/110
and the trajectories x̂ i ,j become independent replicas of x̂ i solution
of
d x̂
i = g i (x̂ i , v̂ (x̂ i ))dt + σ (x̂ i )dw i
i
i
x̂ (0) = x
0
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
SYSTEM OF HJB-FP EQUATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
APPROXIMATE NASH EQUILIBRIUM FOR LARGE COMM
DIFFERENT POPULATIONS
COALITIONS
APPROXIMATE NASH EQUILIBRIUM I
101/110
The probability density of x̂ i (t ) is mi (t ). We rst prove
J i ,j (v̂ (.)) − J i (v̂ (.), mi (.)) → 0, as M → +∞.
We now focus on player 1, 1 to x the ideas. Suppose he uses a
feedback v , (x ) 6= v̂ , (x ), and the other players use
i ,j (x ) = v̂ i (x ), ∀i ≥ 2, ∀j or ∀i , ∀j ≥ 2. We set v (x ) = v , (x ). Call
v̂
1 1
1 1
1
1 1
this set of controls ṽ (.). By abuse of notation, we also write
ṽ
(.) = (v 1 (.), v̂ 2 (.), · · · v̂ N (.)) = (v 1 (.), v̂ (.))
1
The corresponding trajectories are denoted by y 1,j (t ) solutions of
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
SYSTEM OF HJB-FP EQUATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
APPROXIMATE NASH EQUILIBRIUM FOR LARGE COMM
DIFFERENT POPULATIONS
COALITIONS
APPROXIMATE NASH EQUILIBRIUM II
102/110
dy
y
,
1 1
,
1 1
1
= g 1 (y 1,1 , v 1 (y 1,1 ), v̂ (y 1,1 ))dt + σ (y 1,1 )dw 1,1
(0) = x0
(91)
,
1 1
and y ,j = x̂ ,j for j ≥ 2.
We can then prove that
1
1
J 1,1 (ṽ (.)) ≥ J 1 (v̂ (.), m1 (.)) − O (M )
and this concludes the approximate Nash equilibrium property.
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
SYSTEM OF HJB-FP EQUATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
APPROXIMATE NASH EQUILIBRIUM FOR LARGE COMM
DIFFERENT POPULATIONS
COALITIONS
REFERENCES I
Andersson,D.,Djehiche,B. (2011) A Maximum Principle for
SDEs of Mean Field Type, Applied Mathematics and
Optimization 63, p.341-356
Bensoussan,A., Frehse,J. (2002) Regularity Results for
Springer Applied
Mathematical Sciences, Vol 151, 2002
Bensoussan,A., Frehse,J. (1995) Ergodic Bellman Systems for
Stochastic Games in arbitrary Dimension, Proc. Royal Society,
London, Mathematical and Physical sciences A, 449, p. 65-67
Nonlinear Elliptic Systems and Applications,
Bensoussan,A., Frehse,J. (2002) Smooth Solutions of Systems
of Quasilinear Parabolic Equations, ESAIM: Control,
Optimization and Calculus of Variations , Vol.8, p. 169-193
103/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
SYSTEM OF HJB-FP EQUATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
APPROXIMATE NASH EQUILIBRIUM FOR LARGE COMM
DIFFERENT POPULATIONS
COALITIONS
REFERENCES II
Bensoussan,A., Frehse,J. (2012), Control and Nash Games
with Mean Field eect, Chinese Annals of Mathematics, to be
published
Bensoussan,A., Frehse,J., Vogelgesang, J., (2010) Systems of
Bellman Equations to Stochastic Dierential Games with
Noncompact Coupling, Discrete and Continuous Dynamical
Systems, Series A, 274, p. 1375-1390
Bensoussan,A., Frehse,J., Vogelgesang, J., (2012) Nash and
Stackelberg Dierential Games, Chinese Annals of
Mathematics, Series B
Björk T., Murgoci A., A General Theory of Markovian Time
Inconsistent Stochastic Control Problems, SSRN: 1694759
104/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
SYSTEM OF HJB-FP EQUATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
APPROXIMATE NASH EQUILIBRIUM FOR LARGE COMM
DIFFERENT POPULATIONS
COALITIONS
REFERENCES III
Bensoussan,A.,Sung, K.C.J., Yam, S.C.P. ,Yung, S.P. ,
Linear-Quadratic Mean Field Games, Technical report
Buckdahn,R., Djehiche, B.,Li, J., (2011) A General Stochastic
Maximum Principle for SDEs of Mean-eld Type, Appl Math
Optim, 64: 197-216
Buckdahn,R., Li, J.,Peng, SG., (2009) Mean-eld backward
stochastic dierential equations and related partial dierential
equations, Stochastic Processes and their Applications, 119,
p.3133-3154
Cardaliaguet, P., (2010) Notes on Mean Field Games,
Technical Report
105/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
SYSTEM OF HJB-FP EQUATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
APPROXIMATE NASH EQUILIBRIUM FOR LARGE COMM
DIFFERENT POPULATIONS
COALITIONS
REFERENCES IV
Carmona, R., Delarue, F., (2012), Probabilistic Analysis of
Mean-Field Games, Working Paper
Guéant,O., Lasry,J.-M., Lions,P.-L.(2011) Mean Field Games
and applications, in A,R. Carmona et al.(eds), Paris-Princeton
Lectures on Mathematical Sciences 2010, 205-266
Huang,M., (2010) Large-Population LQG Games Involving a
Major Player: The Nash Certainty Equivalence Principle, SIAM
J. Control, Vol. 48, 5, p.3318-3353.
106/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
SYSTEM OF HJB-FP EQUATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
APPROXIMATE NASH EQUILIBRIUM FOR LARGE COMM
DIFFERENT POPULATIONS
COALITIONS
REFERENCES V
Huang,M.,Malhamé, R.P. ,Caines,P.E. (2006) Large Population
Stochastic Dynamic Games: Closed-Loop MCKean-Vlasov
Systems and the Nash Certainty Equivalence Principle,
Communications in Information and Systems, Vol 6, nº 3, p.
221-252
Huang,M., Caines,P.E., Malhamé, R.P. (2007)
Large-population cost-coupled LQG problems with nonuniform
agents: individual-mass behavior and decentralized ε−Nash
equilibria, IEEE Transactions on Automatic Control, 52 (9),
1560-1571
Huang,M., Caines,P.E., Malhamé, R.P. (2007) An Invariance
Principle in Large Population Stochastic Dynamic Games,
Journal of Systems Science and Complexity, 20 (2), 162-172
107/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
SYSTEM OF HJB-FP EQUATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
APPROXIMATE NASH EQUILIBRIUM FOR LARGE COMM
DIFFERENT POPULATIONS
COALITIONS
REFERENCES VI
Lasry,J.-M., Lions,P.-L. (2006). Jeux à champ moyen I- Le cas
stationnaire, Comptes Rendus de l'Académie des Sciences,
Series I, 343,619-625.
Lasry,J.-M., Lions,P.-L. (2006). Jeux à champ moyen II- Horizn
ni et contrôle optimal, Comptes Rendus de l'Académie des
Sciences, Series I, 343,679-684.
Lasry,J.-M., Lions,P.-L. (2007). Mean Field Games, Japanese
Journal of Mathematics 2(1),229-260.
McKean, H.P., (1966), A class of Markov processes associated
with nonlinear parabolic equations, Proc. Natl. Acad. Sci. USA
56, 1907-1911
108/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
SYSTEM OF HJB-FP EQUATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
APPROXIMATE NASH EQUILIBRIUM FOR LARGE COMM
DIFFERENT POPULATIONS
COALITIONS
REFERENCES VII
Nourian, M., Caines P.E. (2012) e-Nash Mean Field Game
Theory for nonlinear stochastic dynamical systems with major
and minor agents, submitted to SIAM J. Control Optim.
109/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H
INTRODUCTION
GENERAL PRESENTATION
SYSTEM OF HJB-FP EQUATIONS
DISCUSSION OF THE MEAN FIELD TYPE CONTROL PROBLEM
APPROXIMATE NASH EQUILIBRIUM FOR LARGE COMM
DIFFERENT POPULATIONS
COALITIONS
Thanks!
110/110
Alain Bensoussan, Jens Frehse, Phillip Yam
Dierential games, Nash equilibrium, Mean Field, H