Strong equilibrium definition
Theorem
Example
Strong Equilibrium in Differential Games
with three Players
N. A. Zenkevich and A. V. Zyatchin
Graduate school of management
Faculty of applied mathematics and control processes
09.09.2011
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
1/ 34
Strong equilibrium definition
Theorem
Example
Contents
1
Strong equilibrium definition
2
Theorem
3
Example
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
2/ 34
Strong equilibrium definition
Theorem
Example
Strong equilibrium definition
Consider the differential game Γ(x0 , T − t0 ) with initial state x0
and finite duration T − t0 , where t0 ≥ 0, T ≥ t0 . Denote set of
players as N = {1, . . . , i , . . . , n}.
The dynamics of the game Γ(x0 , T − t0 ) has the following form:
ẋ = f [t, x, u1 , . . . , un ] ,
x(t0 ) = x0 ,
(1)
where x(t) ∈ R.
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
3/ 34
Strong equilibrium definition
Theorem
Example
Strong equilibrium definition
Let ui (t) – control function of player i ∈ N at the moment t,
n
Q
ui (t) ∈ Ui ⊆ R,
Ui = UN ⊆ R n and f [t, x, u1 , . . . , un ] –
i =1
continuous differentiable functions on [t0 , T ] × R × UN and there
exist positive constants C1 and C2 such that:
|f (t, x, u)| ≤ C1 (1 + |x| + |u|) ,
f (t, x ′ , u) − f (t, x, u) ≤ C2 x ′ − x (1 + |u|)
for all t, x, x ′ ∈ E , u ∈ UN .
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
4/ 34
Strong equilibrium definition
Theorem
Example
Strong equilibrium definition
The payoff of player i ∈ N is defined as:
Ji (x0 , u1 (·), u2 (·), . . . , un (·)) =
=
ZT
gi [t, x(t), u1 (t), u2 (t), . . . , un (t)] dt + qi [x(T )] ,
t0
where ui (·) is continuous function. Suppose that gi [t, x, u1 , . . . , un ]
and qi [x(T )] are differentiable functions in the range of definition
and there exists positive functions C3 > 0 and k > 0 such that:
|gi (t, x, u)| ≤ C3 (1 + |x| + |u|)k ,
|q(x)| ≤ C3 (1 + |x|)k .
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
5/ 34
Strong equilibrium definition
Theorem
Example
Strong equilibrium definition
Every player i ∈ N tends to maximize
Ji (x0 , u1 (·), . . . , ui (·), . . . , un (·)).
The solution of the game Γ(x0 , T − t0 ) is to be found in feedback
strategies:
ui = φi (t, x), x ∈ R, t ∈ [t0 , T ],
where φi (t, x) is differentiable on [t0 , T ] × R functions, i ∈ N.
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
6/ 34
Strong equilibrium definition
Theorem
Example
Strong equilibrium definition
Consider a couple
φ(t, x) = (φ1 (t, x), . . . , φi (t, x), . . . , φn (t, x) ) ,
(t, x) ∈ [t0 , T ]×R.
Let S ⊆ N is a coalition in the game Γ(x0 , T − t0 ). Denote strategy
of the coalition S as φS (·) = {φi (·)}i ∈S . A strategy of additional
coalition N\S denote as φN\S (·) or φ−S (·).
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
7/ 34
Strong equilibrium definition
Theorem
Example
Strong equilibrium definition
Definition.
The set of strategies {φ∗i (t, x), (t, x) ∈ [t0 , T ] × R, i ∈ N} is said to
constitute a strong equilibrium for differential game Γ(x0 , T − t0 ), if ∀M ⊆ N,
∀φM (·) the following is not hold: ∀i ∈ M
Ji (x0 , φM (·), φ∗−M (·)) =
ZT
t0
≥
ZT
h
i
h
i
gi t, x [M] (t), φM (t, x), φ∗−M (t, x) dt+qi x [M] (T ) ≥
gi [t, x ∗ (t), φ∗M (t, x), φ∗−M (t, x)] dt + qi [x ∗ (T )] = Ji (x0 , φ∗M (·), φ∗−M (·))
t0
and ∃i0 ∈ M such that:
Ji0 (x0 , φM (·), φ∗−M (·))
=
ZT
t0
>
ZT
h
i
h
i
gi0 t, x [M] (t), φM (t, x), φ∗−M (t, x) dt+qi0 x [M] (T ) >
gi0 [t, x ∗ (t), φ∗M (t, x), φ∗−M (t, x)] dt + qi0 [x ∗ (T )] = Ji0 (x0 , φ∗M (·), φ∗−M (·)),
t0
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
8/ 34
Strong equilibrium definition
Theorem
Example
Strong equilibrium definition
where
h
i
ẋ [M] (t) = f t, x [M] (t), φM (t, x), φ∗−M (t, x) ,
ẋ ∗ (t) = f t, x ∗ (t), φ∗M (t, x), φ∗−M (t, x) ,
x [M] (t0 ) = x0 ,
x ∗ (t0 ) = x0 .
The set of all strong equilibrium situations in the game
Γ(x0 , T − t0 ) we define as
SME (Γ(x0 , T − t0 )) .
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
9/ 34
Strong equilibrium definition
Theorem
Example
Lemma
Consider the following vectors
[n,i ]
[n,i ]
[n,i ]
λ[n,i ] = λ1 , . . . , λi , . . . , λn
∈ E n,
[n,i ]
where λj
[n,i ]
= 0, j 6= i and λi
= 1.
Lemma 1.
An n-tuple of strategies
φ∗N (·) = (φ∗1 (·), φ∗2 (·), . . . , φ∗n (·)) ∈ SME (Γ(x0 , T − t0 )) ,
i.e. provides a feedback strong equilibrium solution to the game
Γ(x0 , T − t0 ) if for any coalition S ⊆ N there exist such number i0S
as for any strategy φS (·) 6= φ∗S (·) the following inequality holds:
n
n
X
X
[n,i S ]
[n,i S ]
λi 0 Ji (x0 , φ∗S (·), φ∗−S (·)) >
λi 0 Ji (x0 , φS (·), φ∗−S (·)).
i =1
N. A. Zenkevich and A. V. Zyatchin
(2)
i =1
Strong Equilibrium in Differential Games
10/ 34
Strong equilibrium definition
Theorem
Example
Lemma
Remark.
The number of players n is finite, therefore existence of i0S it is
possible to determine by enumerating vectors λ[n,i ] , i = 1, 2, . . . , n
for any coalition S ⊆ N.
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
11/ 34
Strong equilibrium definition
Theorem
Example
Theorem
Theorem.
If for any coalition S ⊆ N, S 6= ∅ in a game Γ(x0 , T − t0 ) there exist a number
i0S ∈ S and continuously differentiable on [0, T ] × R functions V [S] , satisfying
the following set of partial differential equations:
(
)
n
X
[n,i S ]
[S]
Vt (t, x)+max f [t, x, uS , φ∗−S (t, x)] Vx[S] (t, x) +
λi 0 gi [t, x, uS , φ∗−S (t, x)] =
uS
=
i =1
[S]
Vt (t, x)
n
X
+f
n,i0S
[
λi
+
[t, x, φ∗S (t, x), φ∗−S (t, x)] Vx[S] (t, x)+
]
(3)
gi [t, x, φ∗S (t, x), φ∗−S (t, x)] = 0,
i =1
V [S] (T , x [S] (T )) =
n
h
i
X
[n,i S ]
λi 0 qi x [S] (T ) ,
i =1
where for all S ⊆ N maximum in LHS is a unique couple
{φ∗i (t, x), (t, x) ∈ [t0 , T ] × R, i ∈ N} ,
where φ∗i (t, x) , i ∈ N are continuous on [t0 , T ] × R functions, then the couple
{φ∗i (t, x), (t, x) ∈ [t0 , T ] × R, i ∈ N} , is SME (Γ(x0 , T − t0 )).
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
12/ 34
Strong equilibrium definition
Theorem
Example
Example
To apply the theorem in an example, consider some properties of
the following PDE:
∂V (t, x)
+η1
∂t
∂V (t, x)
∂x
2
+η2
∂V (t, x)
∂V (t, x)
x+ae bt
+r (t) = 0
∂x
∂x
(4)
V (T , x) = η3 x,
where a, b, η1 , η2 , η3 are known constants, b 6= η2 , r (t) –
continuously differentiable function on [t0 , T ].
Lemma 2.
Equation (4) has a unique solution V (t, x) on [t0 , T ] and:
∂V (t, x)
= η3 e η2 (T −t) .
∂x
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
13/ 34
Strong equilibrium definition
Theorem
Example
Example
Example.
Consider the game Γ(x0 , T − t0 ), where N = {1, 2, 3}, n = 3, with state
dynamics:
ẋ (t) = ax + b1 u1 + b2 u2 + b3 u3 , x(t0 ) = x0 .
(5)
Player 1 ∈ N tends to maximize the following functional:
ZT 3
J{1} =
−u12 − u22 − u32 + u1 x + u2 x + u3 x − x 2 + r [1] (t) dt + x(T ), (6)
4
t0
player 2 ∈ N :
ZT 1 2
2
2
2
[2]
J{2} =
u1 − u2 − u3 − u1 x + u2 x + u3 x − x + r (t) dt + x(T ), (7)
4
t0
player 3 ∈ N :
ZT 3
J{3} =
−3u12 + u22 − u32 + 3u1 x − u2 x + u3 x − x 2 + r [3] (t) dt + x(T ),
4
t0
(8)
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
14/ 34
Strong equilibrium definition
Theorem
Example
Example
where r [1] (t), r [2] (t), r [3] (t), t ∈ [t0 , T ] – continuously
differentiable functions.
In the game (5)-(8) strong equilibrium SME is found.
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
15/ 34
Strong equilibrium definition
Theorem
Example
Example
Consider coalition S = {1, 2, 3} = N and vector
N
[n,i N ] [n,i N ] [n,i N ]
λ[n,i0 ] = λ1 0 , λ2 0 , λ3 0 .
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
16/ 34
Strong equilibrium definition
Theorem
Example
Example
The expression (3) has the following form:
[N]
Vt (t, x) + max
(u1 ,u2 ,u3 )
[N]
(ax + b1 u1 + b2 u2 + b3 u3 ) Vx (t, x)+
3 2
[1]
−
+ u1 x + u2 x + u3 x − x + r (t) +
4
1 2
[n,i0N ]
2
2
2
[2]
+λ2
u1 − u2 − u3 − u1 x + u2 x + u3 x − x + r (t) +
4
3 2
[n,i0N ]
2
2
2
[3]
+λ3
−3u1 + u2 − u3 + 3u1 x − u2 x + u3 x − x + r (t)
= 0,
4
[n,i N ]
+λ1 0
n
−u12
u22
− u32
V [N](T , x [N] (T )) =
n
X
[n,i0N ] [N]
λi
x
(T ),
(9)
i =1
Performing the indicated maximization, we obtain:
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
17/ 34
Strong equilibrium definition
Theorem
Example
Example
φ∗1 (t, x) =
φ∗2 (t, x) =
2
φ∗3 (t, x) =
b1
[n,i0N ]
2 λ1
− λ2
[n,i0N ]
+ 3λ3
b2
[n,i N ]
λ1 0
2
[n,i0N ]
+
[n,i N ]
λ2 0
−
[n,i N ]
λ3 0
b3
[n,i N ]
λ1 0
+
[n,i N ]
λ2 0
N. A. Zenkevich and A. V. Zyatchin
+
x
[N]
Vx (t, x) + ,
2
x
[N]
Vx (t, x) + ,
2
[n,i N ]
λ3 0
(10)
x
[N]
Vx (t, x) + .
2
Strong Equilibrium in Differential Games
18/ 34
Strong equilibrium definition
Theorem
Example
Example
Upon substituting (10) into (9) we have:
b1 b2 b3
[N]
[N]
Vt (t, x) + a +
+
+
xVx (t, x)+
2
2
2

b12
b22

+
+
+ [n,i N ]
[n,i N ]
[n,i N ]
[n,i N ]
[n,i N ]
[n,i N ]
4 λ1 0 − λ2 0 + 3λ3 0
4 λ1 0 + λ2 0 − λ3 0

2
2
b3

[N]
+ V
(t,
x)
+
(11)

x
[n,i N ]
[n,i N ]
[n,i N ]
4 λ1 0 + λ2 0 + λ3 0
[n,i0N ] [1]
[n,i N ]
[n,i N ]
r (t) + λ2 0 r [2] (t) + λ3 0 r [3] (t) = 0,
[n,i N ]
[n,i N ]
[n,i N ]
V [N](T , x [N] (T )) = λ1 0 + λ2 0 + λ3 0 x [N] (T ).
+λ1
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
19/ 34
Strong equilibrium definition
Theorem
Example
Example
By lemma, equation (11) has a unique solution and:
n b1 b2 b3 o
a+ 2 + 2 + 2 (T −t)
[n,i N ]
[n,i N ]
[n,i N ]
[N]
Vx (t, x) = λ1 0 + λ2 0 + λ3 0 e
.
(12)
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
20/ 34
Strong equilibrium definition
Theorem
Example
Example
Subject to (12) from (10) we have
φ123
1 (t, x)
[n,i N ]
[n,i N ]
[n,i N ]
o
n
b 1 λ1 0 + λ2 0 + λ3 0
b
b
b
x
a+ 21 + 22 + 23 (T −t)
+ ,
= e
N
N
N
[n,i0 ]
[n,i0 ]
[n,i0 ]
2
2 λ1
− λ2
+ 3λ3
[n,i N ]
[n,i N ]
[n,i N ]
n
o
b 2 λ1 0 + λ2 0 + λ3 0
b
b
b
x
a+ 21 + 22 + 23 (T −t)
φ123
+ ,
e
2 (t, x) =
N
N
N
[n,i0 ]
[n,i0 ]
[n,i0 ]
2
2 λ1
+ λ2
− λ3
φ123
3 (t, x) =
b3
e
2
n
b
b
b
a+ 21 + 22 + 23
N. A. Zenkevich and A. V. Zyatchin
o
(T −t)
+
(13)
x
.
2
Strong Equilibrium in Differential Games
21/ 34
Strong equilibrium definition
Theorem
Example
Example
State dynamics has the following form:
b1 b2 b3
ẋ(t) = a +
+
+
x+
2
2
2
[n,i N ]
[n,i N ]
[n,i N ]
n
o
b12 λ1 0 + λ2 0 + λ3 0
b
b
b
a+ 21 + 22 + 23 (T −t)
e
+ +
[n,i N ]
[n,i N ]
[n,i N ]
2 λ1 0 − λ2 0 + 3λ3 0
[n,i N ]
[n,i N ]
[n,i N ]
n
o
b22 λ1 0 + λ2 0 + λ3 0
b
b
b
a+ 21 + 22 + 23 (T −t)
e
+ +
[n,i N ]
[n,i N ]
[n,i N ]
2 λ1 0 + λ2 0 − λ3 0
b2
+ 3e
2
n
a+
b1
b
b
+ 22 + 23
2
o
(T −t)
,
(14)
x(t0 ) = x0 .
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
22/ 34
Strong equilibrium definition
Theorem
Example
Example
Consider an expression (3) for the rest coalitions {1, 2}, {1, 3},
{2, 3}, {1}, {2}, {3}, when additional coalition players plays
according to strategies defined in (13).
Finally we obtain:
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
23/ 34
Strong equilibrium definition
Theorem
Example
Example
For N = {1, 2, 3} :
[n,i N ]
[n,i N ]
[n,i N ]
n
o
b1 λ1 0 + λ2 0 + λ3 0
b
b
b
a+ 21 + 22 + 23 (T −t) x
123
φ1 (t, x) = e
+ ,
[n,i N ]
[n,i N ]
[n,i N ]
2
2 λ1 0 − λ2 0 + 3λ3 0
[n,i N ]
[n,i N ]
[n,i N ]
n
o
b2 λ1 0 + λ2 0 + λ3 0
b
b
b
a+ 21 + 22 + 23 (T −t) x
123
e
φ2 (t, x) = + ,
[n,i N ]
[n,i N ]
[n,i N ]
2
2 λ1 0 + λ2 0 − λ3 0
φ123
3 (t, x)
b3
= e
2
n
a+
N. A. Zenkevich and A. V. Zyatchin
b1
b
b
+ 22 + 23
2
o
(T −t)
x
+ .
2
Strong Equilibrium in Differential Games
24/ 34
Strong equilibrium definition
Theorem
Example
Example
For S = {1, 2} :
φ12
1 (t, x)
{1,2}
[n,i
λ1 0
]
{1,2}
[n,i
λ2 0
]
n
o
+
b1
b
b
b
x
a+ 21 + 22 + 23 (T −t)
e
=
+ ,
{1,2}
{1,2}
2
[n,i
]
[n,i
]
2 λ1 0
− λ2 0
φ12
2 (t, x)
b2
= e
2
n
a+
N. A. Zenkevich and A. V. Zyatchin
b1
b
b
+ 22 + 23
2
o
(T −t)
x
+ .
2
Strong Equilibrium in Differential Games
25/ 34
Strong equilibrium definition
Theorem
Example
Example
For S = {1, 3} :
φ13
1 (t, x)
{1,3}
[n,i
λ1 0
]
{1,3}
[n,i
λ3 0
]
n
o
+
b1
b
b
b
x
a+ 21 + 22 + 23 (T −t)
e
=
+ ,
{1,3}
{1,3}
2
[n,i
]
[n,i
]
2 λ1 0
+ 3λ3 0
φ13
3 (t, x)
b3
= e
2
n
a+
N. A. Zenkevich and A. V. Zyatchin
b1
b
b
+ 22 + 23
2
o
(T −t)
x
+ .
2
Strong Equilibrium in Differential Games
26/ 34
Strong equilibrium definition
Theorem
Example
Example
For S = {2, 3} :
φ23
2 (t, x)
{2,3}
[n,i
λ2 0
]
{2,3}
[n,i
λ3 0
]
n
o
+
b2
b
b
b
x
a+ 21 + 22 + 23 (T −t)
e
=
+ ,
{2,3}
{2,3}
2
[n,i
]
[n,i
]
2 λ2 0
− λ3 0
φ23
3 (t, x)
b3
= e
2
n
a+
N. A. Zenkevich and A. V. Zyatchin
b1
b
b
+ 22 + 23
2
o
(T −t)
x
+ .
2
Strong Equilibrium in Differential Games
27/ 34
Strong equilibrium definition
Theorem
Example
Example
For S = {i } :
φii (t, x)
bi
= e
2
n
a+
b1
b
b
+ 22 + 23
2
o
(T −t)
x
+ ,
2
i = 1, 2, 3.
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
28/ 34
Strong equilibrium definition
Theorem
Example
Example
Consider the following vectors:
N
λ[n,i0 ] = (1, 0, 0),
{2,3}
λ[n,i0
]
= (0, 1, 0),
{1,2}
λ[n,i0
]
= (1, 0, 0),
{1}
λ[n,i0
{3}
λ[n,i0
]
]
N. A. Zenkevich and A. V. Zyatchin
= (1, 0, 0),
{1,3}
]
= (1, 0, 0),
{2}
]
= (0, 1, 0),
λ[n,i0
λ[n,i0
= (0, 0, 1).
Strong Equilibrium in Differential Games
(15)
29/ 34
Strong equilibrium definition
Theorem
Example
Example
Upon substituting (15) into expressions for strategies obtained, we
have:
12
φ123
1 (t, x) = φ1 (t, x) =
n
o
b1 a+ b21 + b22 + b23 (T −t)
1
= φ13
e
+
1 (t, x) = φ1 (t, x) =
2
12
φ123
2 (t, x) = φ2 (t, x) =
n
o
b2 a+ b21 + b22 + b23 (T −t)
2
= φ23
e
+
2 (t, x) = φ2 (t, x) =
2
13
φ123
3 (t, x) = φ3 (t, x) =
n
o
b3 a+ b21 + b22 + b23 (T −t)
3
= φ23
e
+
3 (t, x) = φ3 (t, x) =
2
N. A. Zenkevich and A. V. Zyatchin
x
,
2
x
,
2
(16)
x
,
2
Strong Equilibrium in Differential Games
30/ 34
Strong equilibrium definition
Theorem
Example
Example
and the dynamics for any coalition has the form:
b1 b2 b3
+
+
x+
ẋ(t) = a +
2
2
2
2
n b b b o
b1
b22 b32
a+ 21 + 22 + 23 (T −t)
+
+
+
e
,
2
2
2
x(t0 ) = x0 .
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
31/ 34
Strong equilibrium definition
Theorem
Example
Example
Therefore it is shown that for the couple
n
o
b1 a+ b21 + b22 + b23 (T −t)
e
+
2
n
o
b2 a+ b21 + b22 + b23 (T −t)
e
+
2
n
o
b3 a+ b21 + b22 + b23 (T −t)
e
+
2
{1,2}
{1,3}
x
2
x
2
x
2
{2,3}
there exists numbers i0N = 1, i0
= 1, i0
= 1, i0
= 2,
{1}
{2}
{3}
i0 = 1, i0 = 2, i0 = 3, therefore by theorem φ∗ (t, x) ∈ SME .
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
32/ 34
Strong equilibrium definition
Theorem
Example
Example
Example 2.
ẋ(t) = x + 2u1 + 4u2 + 6u3 ,
x(t0 ) = x0 .
J{1} [x0 , u 1 , u2 , u3 ] =
=
ZT h
t0
i
−u12 − u22 − u32 + 2u1 x + 4u2 x − 2u3 x − 6x 2 + r [1] (t) dt + x(T ),
J{2} [x0 , u 1 , u2 , u3 ] =
=
ZT h
t0
i
u12 − u22 − u32 − 2e 5(T −t) x + 4u2 x − 2u3 x − 6x 2 + r [2] (t) dt + x(T ),
J{3} [x0 , u 1 , u2 , u3 ] =
=
ZT h
t0
SME:
i
−u12 + u22 − 2u32 − 6e 5(T −t) x − 4u3 x − 5x 2 + r [3] (t) dt + 2x(T ),
φ∗ (t, x) = e 5(T −t) + x, 2e 5(T −t) + 2x, 3e 5(T −t) − x .
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
33/ 34
Strong equilibrium definition
Theorem
Example
Example
Thanks for attention!
N. A. Zenkevich and A. V. Zyatchin
Strong Equilibrium in Differential Games
34/ 34