Controlability of Stochastic Systems
Aurel R¼
aşcanu
"Alexandru Ioan Cuza" University and "Octav Mayer" Mathematics Institute of Romanian Academy,
ROMANIA
May 9-19, 2011
CIMPA-UNESCO Conference
Monastir, Tunisie
LECTURE 1
Aurel R¼
aşcanu
Controlability of Stochastic Systems
Object
In the …rst lecture we discuss the controllability of the stochastic system
dX (t ) = f (t, X (t ) , v (t ))dt + g (t, X (t ) , v (t ))dW (t )
(1)
where
(t, x , v ) 7! f (t, x , v ) : [0, T ]
(t, x , v ) 7! g (x , v ) : [0, T ]
H
H
U!H
U ! L 2 ( H0 ; H )
are continuous functions and there exists L > 0 such that for all x , y 2 H and
v 2U
(
(i )
jf (t, x , v ) f (t, y , v )j + jg (t, x , v ) g (t, y , v )j L jx y j ,
(ii )
jf (t, x , v )j + jg (t, x , v )j
L (1 + jx j + jv j) .
(2)
fW (t ) : t 0g is a H0 -valued Wiener process (H0 -valued Brownian motion)
with respect to a stochastic basis (Ω, F , P, fFt : t 0g) . We assume that
Ft = FtW
(the natural …ltration associated to W ).
Aurel R¼
aşcanu
Controlability of Stochastic Systems
A stochastic process Y : Ω
all almost t 2 [0, T ] :
[0, T ] ! H is adapted (or nonanticipative) if for
ω 7 ! Y (ω, t ) : Ω ! H
is Ft measurable
A process v 2 Λ2U (0, T ) = L2ad (Ω (0, T ) ; U) is admissible if it takes values
in a given subset U U. Denote by U the set of all admissible controls
Remark that for all x 2 H and for all v 2 U , the SDE (1) has a unique solution
2
X v 2 SH
[0, T ] = L2ad (Ω; C ([0, T ] ; H)) .
De…nitions
The stochastic system (1) is exactly terminal-controllable if for any
ξ 2 L2 (Ω, FT , P; H) there exists at least one admissible control u 2 U ,
such that the corresponding trajectory t 7 ! X u (ω, t ) satis…es the
terminal condition X u (ω, T ) = ξ (ω ), P a.s. ω 2 Ω.
The stochastic system (1) is exactly controllable if for any
ξ 2 L2 (Ω, FT , P; H) and for any x 2 H, there exists at least one
admissible control u 2 U , such that the corresponding trajectory
t 7 ! X u (ω, t ) satis…es
X u (ω, 0) = x
and
X u (ω, T ) = ξ (ω ) ,
Aurel R¼
aşcanu
P
a.s. ω 2 Ω.
Controlability of Stochastic Systems
Stochastic framework
(Ω, F , P, fFt gt 0 ) is a stochastic basis.
(H, j jH ) , H0 , j jH0 are real separable Hilbert spaces.
p
SH
[0, T ] Lpad (Ω; C ([0, T ] ; H)) , p 0, is the space of progressively
measurable continuous stochastic processes X : Ω [0, T ] ! H (i.e.
t 7 ! X (ω, t ) is continuous a.s. ω 2 Ω, and
(ω, s ) 7 ! X (ω, s ) : Ω [0, t ] ! H is Ft B[0,t ] , BH measurable for
all t 2 [0, T ]), such that
8
1
>
< E kX kpT p ^1 < ∞, if p > 0,
kX kS p [0,T ] =
H
>
:
E [1 ^ kX kT ] ,
if p = 0,
where
def
kX kT =
sup jXt j .
t 2[0,T ]
p
1, then (SH
[0, T ] , k kS p [0,T ] ) is a Banach space;
H
p
If 0 p < 1, then SH
[0, T ] , , is a complete metric space with the metric
ρ(Z1 , Z2 ) = kZ1 Z2 kS p [0,T ] .
If p
d
If H = Rd we shall denote
p
SH
[0, T ] = Sdp [0, T ]
Aurel R¼
aşcanu
Controlability of Stochastic Systems
Wiener process
The H0 -Wiener process ( cylindrical Brownian motion ) is a family
W = fWt (h ) : (t, h ) 2 [0, T ]
0 and h, k 2 H0 ,
such that for all t, s
H0 g
L0 (Ω, F , P)
N 0, t jh j2
(a )
W t (h )
(b )
E [Wt (h )Ws (k )] = (t ^ s )
(c )
FtW = σfWs (h ); s 2 [0, t ], h 2 H0 g _ NP
(b )
W t + δ (h )
L2 (H
hh, k iH0 ,
Ft
Wt (h ) is independent of Ft , for all δ > 0.
0 ; H) of Hilbert–Schmidt operators from H0 into H.
Remark
if H0 = R
then
k
L 2 ( H0 ; H )
if H0 = R and H = R
If fei ; i 2 I
d
then
H
L 2 ( H0 ; H )
Rd
N g is an orthonormal basis of H0 , then
βi = f βit = Wt (ei ); t 2 [0, T ]g, i 2 I ,
are independent real Wiener processes; if dim (H0 ) < ∞, then
Wt =
Aurel R¼
aşcanu
∑ βit ei
i 2I
.
Controlability of Stochastic Systems
k
.
Space of Itô’s integrands
ΛpL2 (H
(0, T ) = ΛpH H0 (0, T ) = Lpad Ω; L2 0, T ; L2 (H0 , H) ,
p 2 [0, ∞[, the space of progressively measurable processes
Z : Ω ]0, T [! L2 (H0 ; H) such that:
8 "
p # 1 ^1
Z T
>
p
>
2
>
> E
>
kZs k2HS ds
< ∞, if p > 0,
>
>
0
<
kZ k Λp =
>
"
1#
>
Z T
>
2
>
>
2 ds
>
,
if p = 0.
E
1
^
k
Z
k
>
s HS
:
0
0 ,H)
The space (ΛpL2 (H ,H) (0, T ) , k kΛp ), p 1, is a Banach space and
0
ΛpL2 (H ,H) (0, T ) , 0 p < 1, is a complete metric space with the metric
0
ρ (Z 1 , Z 2 ) = kZ 1
If H0 =
Rk
Z 2 k Λp .
and H = Rd then L2 (H0 , H) = Rd
ΛpL2 (H ,H)
0
(0, T ) =
Λpd k
k
and we shall denote
(0, T ) .
If H0 = R and H = Rd then L2 (H0 , H) = Rd and we shall denote
ΛpL2 (H
0 ,H)
Aurel R¼
a şcanu
(0, T ) = Λpd (0, T )
Controlability of Stochastic Systems
Itô’s integral
Let fei ; i 2 I
N g is an orthonormal basis of H0 . For any
Z 2 Λ2L2 (H ,H) (0, T ), the stochastic integral
0
It (Z ) =
Z t
0
def
Zs dWs =
∑
i 2I
Z t
0
Zs (ei )dWs (ei ),
p
The application I : ΛpL2 (H ,H) (0, T ) ! SH
[0, T ] is a linear continuous
0
operator and it has the following properties:
(a )
E [It (Z )] = 0,
(b )
, if p 2,
E j IT
=
1
kZ kpΛp E sup jIt (Z ) jp cp kZ kpΛp , if p > 0,
cp
t 2[0,T ]
(c )
(Z ) j2
if p
1,
kZ k2Λ2
( Burkholder-Davis-Gundy inequality )
(d )
for p
1, I (Z ) is a martingale i.e. EFs [It (Z )] = Is (Z ) , for all 0
Aurel R¼
aşcanu
Controlability of Stochastic Systems
s
t.
Itô’s formula
0 [0, T ] be of the form
Let X 2 SH
X (t ) = X 0 +
Z t
0
F (s ) ds +
Z t
0
G (s ) dW (s ) ,
8t
0, a.s.,
with F 2 L0ad Ω; L1 (0, T ; H) and G 2 L0ad Ω; L2 0, T ; L2 (H0 , H)
Theorem (Itô’s formula)
Let ϕ 2 C 1,2 (R+
H; R) and
1
def
00
A ϕ (t, x ) = F (t ) , ϕx0 (t, x ) + Tr G (t ) G (t ) ϕxx
(t, x ) .
2
Then for all t
0 :
ϕ (t, X (t )) =
ϕ (0, X0 ) +
+
Z t
0
Z t
0
∂ϕ
(s, X (s )) + A ϕ (s, X (s )) ds
∂t
h ϕx0 (s, X (s )) , G (s ) dW (s )i , P
Aurel R¼
aşcanu
Controlability of Stochastic Systems
a.s..
In particular
Z t
jX (t )j2 = jX0 j2 +
+2
0
Z t
0
Using the identity
hx , y i =
1
jx + y j2
2
2 hX (s ) , F (s )i + jG (s )j2 ds
hX (s ) , G (s ) dW (s )i , P
1
jx j2
2
a.s.
1
jy j2 , 8 x , y 2 H
2
:
Corollary
0 [0, T ] are Itô processes of the form
If X , Y 2 SH
X (t ) = X 0 +
Y (s ) = Y 0 +
then for all t
Z t
Z0 t
0
F (s ) ds +
E (s ) ds +
+
0
Z0 t
0
G (s ) dW (s ) , t
0, and
H (s ) dW (s ) , t
0,
0
hX (t ) , Y (t )i = hX0 , Y0 i +
Z t
Z t
Z t
0
Tr (G (s ) H (s )) ds +
[hF (s ) , Y (s )i + hX (s ) , E (s )i] ds
Z t
0
(Y (s ) G (s ) + X (s ) H (s )) dW (s ) , a.s.
Aurel R¼
aşcanu
Controlability of Stochastic Systems
Martingale Representation Theorem
Theorem
If 0 < T
∞ and ξ 2 Lp (Ω, FT , P; H), p > 1, then there exists a unique
p
Z 2 ΛL2 (H ,H) (0, T ) such that
0
ξ = Eξ +
Z T
0
Z (s ) dW (s )
In particular, if M is a H valued continuous p integrable martingale, then
there exists a unique Z 2 ΛpH H0 (0, T ) such that
M (t ) = EFt M (T ) =
Aurel R¼
aşcanu
Z t
0
Z (s ) dW (s ) .
Controlability of Stochastic Systems
(3)
Corollary
p
Let 0 < T < ∞, ξ 2 Lp (Ω, FT , P; H) and S 2 SH
[0, T ] , where p > 1 Then
p
p
there exists a unique pair (Y , Z ) 2 SH [0, T ] ΛL2 (H ,H) (0, T ), such that for
0
all t 2 [0, T ] :
Y t = ξ + S (T )
S (t )
Z T
t
Z (s ) dW (s ) , a.s. .
Proof.
By Martingale Representation Theorem there exists a unique
Z 2 ΛpL2 (H ,H) (0, T ) such that
0
ξ + S (T ) = E (ξ + S (T )) +
Z T
0
Z (s ) dW (s ) .
p
and the stochastic process Y 2 SH
[0, T ] is uniquely de…ned by
Y (t ) = E (ξ + S (T ))
Aurel R¼
aşcanu
S (t ) +
Z t
0
Z (s ) dW (s ) .
Controlability of Stochastic Systems
(4)
Controllability and BSDE
Consider the controllability problem g (t, x , v ) = h (t, x ) + v
(
dX (t ) = f (t, X (t ) , v (t )) dt + (h (t, X (t )) + v (t )) dW (t ) ,
X (T ) = ξ,
0
t
(5)
T.
The exact terminal-controllability means to …nd a pair of stochastic processes
(X , v ) solution of (5).
With the notation
F (t, x , z ) =
f (t, x , z
h (t, x ))
the Eq. (5) becomes
dX (t ) = F (t, X (t ) , Z (t )) dt
X (T ) = ξ,
0
t
Z (t ) dW (t ) ,
(6)
T.
and
v (t ) =
h (t, X (t )) + Z (t )
Hence in this case the exact terminal-controllability of (1) is equivalent to an
existence result for the BSDE (6) .
Aurel R¼
aşcanu
Controlability of Stochastic Systems
BSDE with Lipschitz conditions
Consider the backward stochastic di¤erential equation : P
t 2 [0, T ]
Y (t ) = ξ +
Z T
t
F (s, Y (s ) , Z (s )) ds
Z T
t
a.s., for all
Z (s ) dW (s ) ,
(7)
or formally
dY (t ) = F (t, Y (t ) , Z (t )) dt
Z (t ) dW (t )
under the assumptions
the function F ( , , y , z ) : Ω
( y , z ) 2 H L 2 ( H0 , H ) ,
[0, T ] ! H is P –measurable for every
there exist L 2 L1 (0, T ), ` 2 L2 (0, T ) such that
8
(I ) Lipschitz conditions:
>
>
> for all y , y 0 2 H, z, z 0 2 L2 (H , H) , d P dt a.e. :
>
>
0
>
>
>
(L y )
jF (t, y 0 , z ) F (t, y , z )j L (t ) jy 0 y j ,
>
<
jF (t, y , z 0 ) F (t, y , z )j ` (t ) j kz 0 z kHS ;
(L z )
>
>
>
II
Boundedness
condition:
(
)
>
>
>
Z T
p
>
>
>
:
E
<∞.
(B F )
jF (t, 0, 0)j dt
0
Aurel R¼
aşcanu
Controlability of Stochastic Systems
(8)
Theorem
Let p > 1 and ξ 2 Lp (Ω, FT , P; H) . Let the assumption (8) be satis…ed.
p
Then the BSDE (7) has a unique solution (Y , Z ) 2 SH
[0, T ] ΛpH H0 (0, T ) .
Proof.
p
Let K = SH
[0, T ] ΛpL2 (H ,H) (0, T ) . The solution of the equation (7) the
0
…xed point of the mapping
Γ : K ! K,
de…ned by
(Y , Z ) = Γ (X , U ) ,
where
Y (t ) = η +
Z T
t
F (r , X (r ) , U (r )) dr
Aurel R¼
aşcanu
Z T
t
Z (r ) dW (r ) , a.s. t 2 [0, T ] .
Controlability of Stochastic Systems
Continuation of the proof.
Let M 2 N and 0 = T0 < T1 <
γ
T
M
def
=
sup
T
0 <s t < M
Z sh
t
< TM = T , with Ti =
i
L (r ) + `2 (r ) dr ! 0,
iT
. Then
M
as M ! ∞ .
It is shown that Γ is a a strict contraction on the Banach space
p
SH
[TM 1 , T ] ΛpL2 (H ,H) (TM 1 , T ) with the norm
0
"
jjj(X , U )jjjM = E
p
sup
r 2[T M
1 ,T ]
jX r j + E
Z T
TM
2
1
jUr j dr
#
p/2 1/p
for M large enough. Consequently the equation (7) has a unique solution
p
(Y , Z ) 2 S H
[TM 0 1 , T ] ΛpL2 (H ,H) (TM 0 1 , T ) . The next step is to solve
0
the equation on the interval [TM 0 2 , TM 0 1 ] with the …nal value Y (TM 0
Repeating the same arguments, the proof is completed in M0 steps.
Aurel R¼
aşcanu
Controlability of Stochastic Systems
1) .
Exercise
Let Ai = T
T
,T
4i
1T
2 4i
α (t ) = 1A (t )
Then
1 for all t 2 [0, T ] :
2
1
(T
3
t)
Z T
3
1.
Z T
t) ;
ζ=
Z T
0
2
(T
3
α (s ) ds
8
(T
9
c )2 ds
( α (s )
If
i 2N A i .
1A c (t ) = 2 1A (t )
t
for all t 2 [0, T ] and c 2 R
t
S
, i = 0, 1, 2, 3, . . . , and A =
Let
t) ;
α (s ) dW (s )
then it is impossible to …nd x 2 R, b 2 Λ2 (0, T ) and σ 2 Λ2 (0, T ) , such
that
lim E jσ (t ) σ (T )j2 = 0
and
t !T
ζ=x+
Z T
0
b (s ) ds +
Aurel R¼
aşcanu
Z T
0
σ (s ) dW (s )
Controlability of Stochastic Systems
(9)
Proof.
We only prove the point 3. Assuming (9), then
Z T
[ α (s )
0
σ (s )] dW (s ) = x +
Z T
0
b (s ) ds
and therefore
Z t
0
σ (s )] dW (s ) = x +
[ α (s )
Z t
0
b (s ) ds + EFt
Z T
t
b (s ) ds
Hence
Z T
t
σ (s )] dW (s ) =
[ α (s )
Z T
t
EFt
b (s ) ds
Z T
t
b (s ) ds.
Then
8
(T
9
t)
E
Z T
2E
t
j α (s )
Z T
t
=E
Z T
t
Z T
j α (s )
( α (s )
σ (T )j2 ds
σ (s )j2 ds + 2E
σ (s )) dW (s )
Z
Aurel R¼
aşcanu T
Z T
t
2
j σ (s )
+2
Z T
t
Z
σ (T )j2 ds
E j σ (s )
2 of Stochastic
Controlability
T Systems
σ (T )j2 ds
Continuation of the proof.
Hence
8
(T
9
E
t)
(T
Z T
t
2
t)
Z T
t
Let δ > 0 such that for all jt
E j σ (t )
Then for jt
Tj
8
(T
9
+2
b (s ) ds
t
E j σ (s )
E b 2 (s ) ds + 2
Tj
σ (T )j2
Z T
Z T
t
σ (T )j2 ds
E j σ (s )
σ (T )j2 ds
δ,
1
9
Z T
and
t
E b 2 (s ) ds
1
.
9
δ:
t)
1
(T
9
t) +
2
(T
9
Aurel R¼
aşcanu
t)
CONTRADICTION !
Controlability of Stochastic Systems
Necessary condition of exactly terminal-controllability
Let
(
dX (t ) = f (t, X (t ) , v (t ))dt + g (t, X (t ) , v (t ))dW (t )
X (T ) = ξ
and the assumptions (2) be satis…ed.
(10)
Theorem
Assume that
lim E jg (t, x , a )
t !T
g (T , x , a )j2 = 0.
Then a necessary condition that the system (10) to be exactly
terminal-controlable is
for any a 2 U and p 2 H with jp j = 1, there exists at least one triple
(t, x , v ) 2 [0, T ] H U such that (g (t, x , v ) g (t, x , a )) p 6= 0
If
H = Rd ,
and
H0 = R,
U = Rk
g (t, x , v ) = g1 (t, x ) + G1 v ,
then the condition (11) is equivalent to
rank (G1 ) = d
and the condition is also su¢ Aurel
cient.
R¼
aşcanu
k
Controlability of Stochastic Systems
(11)
Proof.
Observe that
lim E jg (t, X (t ) , a )
t !T
g (T , X (T ) , a )j2 = 0.
If the condition (11) is false then there exist a 2 U, p 2 H, with jp j = 1, such
that for all (t, x , v ) 2 [0, T ] H U
g (t, x , a )) p = 0
(g (t, x , v )
Let X (T ) = ξ = ζp. Then
ζ = hp, X (T )i = hp, X (0)i +
+
Z T
0
Z T
0
hp, f (s, X (s ) , v (s ))i ds
hg (s, X (s ) , v (s ))p, dW (s )i
that is impossible.
If g (t, x , v ) = g1 (t, x ) + G1 v then the condition (11) becomes
rank (G1 ) = d
Aurel R¼
aşcanu
k.
Controlability of Stochastic Systems
Continuation of the proof.
The condition rank (G1 ) = d
k k matrix such that
k is su¢ cient then there exists M a invertible
G 1 M = [ Id
We set
M
1
v =
d , 0]
Z
θ
and the equation becomes
8
>
< dX (t ) = f (t, X (t ) , M Z (t ) )dt + [g1 (t, X (t )) + Z (t )] dW (t )
θ (t )
>
:
X (T ) = ξ
and setting θ = 0 the equation has a unique solution (X , Z ) .
RemarkIf d = k then the control v and the corresponding state X are unique.
Aurel R¼
aşcanu
Controlability of Stochastic Systems
Controllability of a linear stochastic systems
Let
dX (t ) = [F (t ) X (t ) + G (t ) v (t )] dt + (F1 (t ) X (t ) + G1 (t ) v (t )) dW (t )
(12)
where the coe¢ cients are measurable matrix-functions F (t ) , F1 (t ) 2 Rd d
and G (t ) , G1 (t ) 2 Rd k ;
jF (t )j + jF1 (t )j + jG (t )j + jG1 (t )j
L.
Theorem
The stochastic linear system (12) is exactly terminal-controllable if and only if
Rank G1 (t ) = d
By a simple linear transformation
v (t ) = M (t )
z (t )
+ K (t ) X (t )
u (t )
the system becomes
dX (t ) = [A (t ) X (t ) + A1 (t ) z (t ) + B (t ) u (t )] dt
where M is the invertible k
z (t ) dW (t )
k-matrix and
G 1 (t ) M (t ) = [ Id
Aurel R¼
aşcanu
d , 0]
Controlability of Stochastic Systems
Theorem
System (12) is exactly controllable if and only if
Rank E
Z T
0
Y (t ) B (t ) B (t ) Y (t ) dt = d
where Y (t ) is a Rd d valued stochastic process de…ned by
(
dY (t ) = Y (t )A (t )dt + Y (t )A1 (t )dW (t )]
Y (0 ) = Id
d
In particular for constant coe¢ cients: the system is exactly controllable if and
only if
Rank [B , AB , A1 B , AA1 B , A1 AB , . . .] = d
Aurel R¼
aşcanu
Controlability of Stochastic Systems
Liu, Feng & Peng, Shige : On Controllability for Stochastic Control
Systems when the Coe¢ cients is time-variant, J Syst Sci Complex, Vol.
23, 270–278, (2010) ;
Liu Yazeng & Peng, Shige : Determination of a controllable set for a
class of non-linear stochastic control systems, Optimal Control
Applications and Methods, Vol. 24 (3), 173–181, (2003)
Pardoux, Etienne & R¼
aşcanu, Aurel : SDEs, BSDEs and PDEs (book,
submitted October 2010);
Peng, Shige : Backward stochastic di¤erential equations and applications
to optimal control, Applied Mathematics and Optimization, 27(2),
125–144, (1993)
Aurel R¼
aşcanu
Controlability of Stochastic Systems
Thank you for your attention !
and
I invite you to ITN School and Conference in Iasi, Romania
Deterministic and Stochastic Controlled Systems
School: June 18-30, 2012
Conference: July 2-7, 2012
http://www.math.uaic.ro/~ITN2012/
Available Post-Doc Positions in ITN-Marie Curie Project of Iasi
http://www.math.uaic.ro/~ITN_Marie_Curie/recruitment.php
Iasi: 6 months.
Milano: 6 months.
Manchester: 6 months.
Marrakech: 6 months.
Aurel R¼
aşcanu
Controlability of Stochastic Systems