Applications of Optimal Control to Stabilization
Mythily Ramaswamy
TIFR Centre for Applicable Mathematics, Bangalore, India
CIMPA Pre-School,
I.I.T Bombay
22 June - July 4, 2015
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
1 / 44
Contents
1
Introduction
2
LQR Problem - Finite Horizon
3
LQR Problem - Infinite Horizon
4
Algorithm to solve Ricatti Equation
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
2 / 44
Introduction
Stabilization
Consider the equation in Y

y 0 (t) = Ay(t) + Bu(t),



y(0) = y0 ,
E



y0 ∈ Y, u ∈ U,
∀ t ≥ 0,
where
Y and U are two Hilbert spaces.
The unbounded operator (A, D(A)) is the infinitesimal generator of a
strongly continuous semigroup on Y , denoted by {etA }t≥0 .
The control operator B ∈ L(U ; Y ).
Suppose that {etA }t≥0 is unstable.
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
3 / 44
Introduction
Exponential stabilization
Aim Find a control u ∈ L2 (0, ∞; U ), in a feedback form
u(t) = Ky(t),
so that the closed loop system
y 0 (t) = (A + BK)y(t),
y(0) = y0 ,
is exponentially stable on Y , i.e.
ket(A+BK) k ≤ M e−λt ,
∀ t ≥ 0,
for some constants M > 0 and λ > 0.
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
4 / 44
Introduction
Definition
The pair (A, B) is said to be stabilizable, if there exists K ∈ L(Y ; U ) such
that {et(A+BK) }t≥0 is exponentially stable on Y .
Remark If system E is null controllable, (A, B) is stabilizable.
Theorem (Datko theorem)
Let {S(t)}t≥0 be a strongly continuous semigroup on Y . The semigroup
{S(t)}t≥0 is exponentially stable if and only if
Z ∞
kS(t)y0 k2 < ∞, ∀ y0 ∈ Y.
0
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
5 / 44
Introduction
Connection with optimal control problem
In view of the above theorem, to solve the stabilization problem we look
for the solution to the control problem P
obeys E, u ∈ L2 (0, ∞; U )},
Z
1 ∞
1 ∞
2
J(y, u) =
ky(t)kY dt +
ku(t)k2u dt.
2 0
2 0
inf{J(y, u),
(y, u)
Z
Qn Does there exist (ȳ, ū), the solution of the optimal problem P?
Qn Can the optimal control be found in the feedback form
ū(t) = K ȳ(t)?
Qn Is {et(A+BK) }t≥0 exponentially stable?
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
6 / 44
Introduction
Aim and outlines to prove it
The optimal control problem P is called LQR problem with infinite
horizon.
First, we need to solve LQR problem with finite horizon and find the
optimal control in a feedback form via the solution of a differential
Riccati equation.
Under a finite cost condition, we show that P admits unique solution
and obtain the optimal control in a feedback form via the solution of
a algebraic Riccati equation.
Finally, the connection between the optimal control of P obtained in
the feedback form and the stabilization of E is brought out.
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
7 / 44
LQR Problem - Finite Horizon
Consider the problem P (0, T, y0 )
inf{JT (y, u),
1
JT (y, u) =
2
obeys E, u ∈ L2 (0, T ; U )},
Z
1 T
2
ky(t)kY dt +
ku(t)k2U dt.
2 0
(y, u)
Z
0
T
This problem P (0, T, y0 ) admits a unique solution (ȳ, ū) characterized by
the system
ȳ 0 (t) = Aȳ(t) − BB ∗ p(t),
−p0 (t) = A∗ p(t) + ȳ(t),
ȳ(0) = y0 ,
p(T ) = 0,
ū(t) = −B ∗ p(t).
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
8 / 44
LQR Problem - Finite Horizon
To find ū in feedback form,
ū = K ȳ(t),
we study the family of problems P (s, T, ζ)
inf{Js,T (y, u),
1
Js,T (y, u) =
2
(Es,ζ ), u ∈ L2 (0, T ; U )},
Z
1 T
2
ky(t)kY dt +
ku(t)k2U dt,
2 s
(y, u)
Z
s
T
obeys
and
(Es,ζ )
Mythily Ramaswamy
y 0 (t) = Ay(t) + Bu(t),
Optimal Control - Stabilization
y(s) = ζ.
3rd July, 2015
9 / 44
LQR Problem - Finite Horizon
The solution (yζs , usζ ) to P (s, T, ζ) is characterized by
dyζs (t)
= Ayζs (t) − BB ∗ psζ (t), yζs (s) = ζ,
dt
dpsζ (t)
= A∗ psζ (t) + yζs (t), psζ (T ) = 0,
−
dt
usζ (t) = −B ∗ psζ (t).
Consider the mapping
P (s) : ζ 7−→ psζ (s),
Mythily Ramaswamy
∀ s ∈ [0, T ).
Optimal Control - Stabilization
3rd July, 2015
10 / 44
LQR Problem - Finite Horizon
The mapping is linear from Y into itself.
We can show it using the linearity
s
(yβζ
, psβζ1 +ζ2 , usβζ1 +ζ2 ) = β(yζs1 , psζ1 , usζ1 ) + (yζs2 , psζ2 , usζ2 ).
1 +ζ2
For all t ∈ [0, T ), P (t) = P (t)∗ ≥ 0.
With an integration by parts formula between the solution psζ to
−p0 (t) = A∗ p(t) + yζs (t),
p(T ) = 0,
and the solution yξs to
y 0 (t) = Ay(t) − BB ∗ psξ (t),
we obtain
Z
P (s)ζ, ξ
=
Y
T
yζs , yξs
Z
s
Y
+
T
yξs (s) = ξ,
B ∗ psζ , B ∗ psξ
s
U
= ζ, P (s)ξ ,
Y
for all ζ ∈ Y and all ξ ∈ Y .
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
11 / 44
LQR Problem - Finite Horizon
For all t ∈ [0, T ), P (t) ∈ L(Y ).
From the above identity, we write
1
P (s)ζ, ζ
= Js,T (yζs , usζ ).
2
Y
Consider the trajectory z(t) = e(t−s)A ζ for all t ≥ s and ζ ∈ D(A).
Then z ∈ C 1 ([s, T ]; D(A)) ∩ C([s, T ]; Y ) and
Js,T (z, 0) = kzk2Y ≤ M kζk2Y ,
∀ t ≥ s,
for some constant M > 0.
Since yζs , usζ is the solution of the optimal problem P (s, T, ζ), we have
Js,T (yζs , usζ ) ≤ Js,T (e(t−s)A ζ, 0).
Combining all the above inequalities, we obtain for all s ∈ [0, T ),
1
≤ M kζk2Y .
2 P (s)ζ, ζ
Y
Thus for all t ∈ [0, T )
1
1
kP (t) 2 kL(Y ) ≤ M 2 ,
Mythily Ramaswamy
kP (t)kL(Y ) ≤ M.
Optimal Control - Stabilization
3rd July, 2015
12 / 44
LQR Problem - Finite Horizon
For any t ∈ [s, T ), psζ (t) = P (t)yζs (t).
From the Dynamic programming principle (subsolutions of an optimal
solution of a problem are themselves the optimal solutions for their
corresponding subproblems), usζ (·) and utys (t) (·), the optimal controls
ζ
for the problems P (s, T, ζ) and P (t, T, yζs (t)) satisfy
usζ (τ ) = utys (t) (τ ),
ζ
for almost τ ∈ (t, T ).
From the above relation, we derive
psζ (t) = ptys (t) (t),
ζ
t ∈ [s, T ).
From the definition of map P (·), we obtain
ptys (t) (t) = P (t)yζs (t),
ζ
∀ t ∈ [0, T ).
From the above two relations, the claim follows.
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
13 / 44
LQR Problem - Finite Horizon
For all ζ ∈ Y and all ξ ∈ Y , t 7−→ P (t)ζ, ξ
t ∈ [0, T ).
Y
is continuous on
From the Duhamel formula and relation in above slide, for all
t ∈ (s, T ),
kyζs (t)kY
≤ ke
(t−s)A
Z
T
ζkY +
ke(t−τ )A BB ∗ P (τ )yζs (τ )kY dτ.
s
Then, for some constant C > 0,
kyζs (t)kY ≤ CkζkY ,
kpsζ (t)kY ≤ CkζkY .
Using these estimates, we can show
limh→0 kyζs+h − yζs kC([(s+h)∧s,T ];Y ) = 0,
limh→0 kpζs+h − psζ kC([(s+h)∧s,T ];Y ) = 0,
From that, we deduce t 7−→ P (t)ζ, ξ
is continuous on [0, T ) for all
Y
ζ ∈ Y and all ξ ∈ Y .
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
14 / 44
LQR Problem - Finite Horizon
P (·), the solution to a differential Riccati equation
Definition (Cs ([0, T ]; L(Y )))
We denote by Cs ([0, T ]; L(Y )) the space of mapping P from [s, T ] to
L(Y ) such that t 7−→ P (t)ζ belongs to C([0, T ]; L(Y )) for all ζ ∈ Y .
We can prove that the map P ∈ Cs ([0, T ]; L(Y )).
We show P (·) is the solution to the differential Riccati equation
(DRE)
P (t) = P (t)∗ ,
P (t) ≥ 0,
t ∈ [0, T ],
P 0 (t) + A∗ P (t) + P (t)A − P (t)BB ∗ P (t) + I = 0,
t ∈ [0, T ],
P (T ) = 0.
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
15 / 44
LQR Problem - Finite Horizon
Definition (Solution to (DRE))
A function P ∈ Cs ([0, T ]; L(Y )) is a solution to (DRE) on (0, T ) if and
only if, for every (ζ, ξ) ∈ D(A) × D(A) the function (P (·)ζ, ξ)Y belongs
to W 1,1 (0, T ) and satisfies
P (t) = P (t)∗ ,
P (t) ≥ 0,
t ∈ [0, T ],
d
dt (P (t)ζ, ξ)Y + (P (t)ζ, Aξ)Y + (P (t)Aζ, ξ)Y
−(P (t)BB ∗ ζ, ξ)Y + (ζ, ξ)Y = 0, t ∈ [0, T ],
(P (T )ζ, ξ)Y = 0.
Theorem
The map P defined above is a unique solution to (DRE) on (0, T ).
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
16 / 44
LQR Problem - Finite Horizon
d+ Proof We shall calculate
P (s)ζ, ξ , the right hand side
ds
Y
derivative of the map
s → P (s)ζ, ξ
Y
on [0, T ) for all (ζ, ξ) ∈ D(A) × D(A).
For that, consider
y 0 (t) = Ay(t) − BB ∗ p(t),
−p0 (t) = A∗ p(t) + y(t),
and
p(T ) = 0,
z 0 (t) = Az(t) − BB ∗ q(t),
−q 0 (t) = A∗ q(t) + z(t),
As our previous notations, (y, p) =
Mythily Ramaswamy
y(s) = ζ,
z(s) = ξ,
q(T ) = 0.
(yζs , psζ )
Optimal Control - Stabilization
and (z, q) = (yξs , psξ ).
3rd July, 2015
17 / 44
LQR Problem - Finite Horizon
Using the expressions
y(t) = e
(t−s)A
Z
t
ζ−
e(t−τ )A BB ∗ p(τ ) dτ,
s
z(t) = e(t−s)A ξ −
Z
t
e(t−τ )A BB ∗ q(τ ) dτ,
s
we can show
1
y(s + h) − y(s) − Aζ + BB ∗ p(s)kY = 0,
h
1
limh&0 k z(s + h) − z(s) − Aξ + BB ∗ q(s)kY = 0.
h
limh&0 k
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
18 / 44
LQR Problem - Finite Horizon
Using the equations satisfied by p and z and an integration by parts
formula, we derive
P (s + h)y(s + h), z(s + h) − P (s)y(s), z(s)
Y
Y
Z s ∗
∗
=
y(τ ), z(τ ) + B p(τ ), B q(τ ) dτ
s+h
Y
U
Using this identity and by a standared calculation, we can show that
d+
(P (t)ζ, ξ)Y + (P (t)ζ, Aξ)Y + (P (t)Aζ, ξ)Y
dt
−(P (t)BB ∗ ζ, ξ)Y + (ζ, ξ)Y = 0, t ∈ [0, T ],
(P (T )ζ, ξ)Y = 0.
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
19 / 44
LQR Problem - Finite Horizon
Since s → (P (s)ζ, ξ)Y is continuous on [0, T ) and
+
s → ddt (P (s)ζ, ξ)Y is bounded and continuous on [0, , T ), we can
affirm that s → (P (s)ζ, ξ)Y is of class C 1 on [0, T ). Hence P (·) is a
solution of DRE.
For uniqueness, let Q(·) be a solution DRE.
Claim P (t) = Q(t),
∀ t ∈ [0, T ].
Consider for s ∈ [0, T ) and ζ ∈ D(A),
z 0 (t) = Az(t) − BB ∗ Q(t)z(t),
t ≥ s,
z(s) = ζ.
Let {un }n∈N ∈ C 1 ([0, T ]; U ) such that
un → −B ∗ Qz,
in L2 (0, T ; U ).
{zn }n∈N satisfies
z 0 (t) = Az(t) − BB ∗ un (t),
Mythily Ramaswamy
t ≥ s,
Optimal Control - Stabilization
z(s) = ζ.
3rd July, 2015
20 / 44
LQR Problem - Finite Horizon
Set q(t) = Q(t)z(t) and qn (t) = Q(t)zn (t) for all n ∈ N. Then using
the terminal condition of DRE, q(T ) = Q(T ) = 0.
Using DRE, we can show
−qn0 (t) = A∗ qn (t) + zn (t) − Q(t)BB ∗ Q(t)zn (t) − Q(t)Bun (t)
and taking n → ∞,
−q 0 (t) = A∗ q(t) + z.
Thus (z, q) satisfies the problem P (s, T, ζ) and by the uniqueness of
the solution of optimality problem P (s, T, ζ), we get (z, q) = (yζs , psζ )
and hence
Q(t) = P (t), ∀ t ∈ [0, T ].
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
21 / 44
LQR Problem - Finite Horizon
Remark Setting Q(t) = P (T − t), where P (·) is the solution of DRE, we
can show that Q(·) satisfies
Q(t) = Q(t)∗ ,
Q(t) ≥ 0,
t ∈ [0, T ],
Q0 (t) = A∗ Q(t) + Q(t)A − Q(t)BB ∗ Q(t) + I,
t ∈ [0, T ],
Q(0) = 0.
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
22 / 44
LQR Problem - Infinite Horizon
Consider the optimal control problem P
obeys E, u ∈ L2 (0, ∞; U )},
Z
1 ∞
1 ∞
2
J(y, u) =
ky(t)kY dt +
ku(t)k2U dt.
2 0
2 0
inf{J(y, u),
(y, u)
Z
and
E
y 0 (t) = Ay(t) + Bu,
y(0) = y0 ,
Main steps
Give a suitable condition such that J(y, u) finite for some y and u.
Under this condition, problem P admits a unique solution.
The optimal control of P can be found in a feedback form via solving
an algebraic Riccati equation and it stabilizes E.
For that, we need to go from finite horizon to infinite horizon problem
taking T → ∞ and using Banach-Steinhaus theorem.
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
23 / 44
LQR Problem - Infinite Horizon
Finite cost condition(FCC) For every y0 ∈ Y , there exists a
uy0 ∈ L2 (0, ∞; U ) such that
J(y(y0 , uy0 ), uy0 ) < ∞.
If (A, B) is stabilizable, then the (F CC) is satisfied.
Conversely, if (F CC) is satisfied, then (A, B) is stabilizable.
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
24 / 44
LQR Problem - Infinite Horizon
Lemma
Suppose that (FCC) is satisfied. Then P admits a unique solution. This
solution (ȳ, ū) obeys
ū(t) = −B ∗ P ȳ(t),
∀ t ≥ 0,
where P is the minimal solution to ARE
P = P ∗ ≥ 0,
A∗ P + P A∗ − P BB ∗ P + I = 0.
Moreover,
J(ȳ, ū) =
Mythily Ramaswamy
1
P y0 , y0 .
2
Y
Optimal Control - Stabilization
3rd July, 2015
25 / 44
LQR Problem - Infinite Horizon
Definition (Solution to ARE )
An operator P ∈ L(Y ) is a solution to ARE if and only if
P = P ∗ ≥ 0,
(P ζ, Aξ)Y + (P Aζ, ξ)Y − (P BB ∗ ζ, ξ)Y + (ζ, ξ)Y = 0.
An operator P is a minimal solution to ARE, if it is a solution to ARE and
obeys
P ≤ Q,
for any solution Q to ARE.
Theorem
The ARE admits a unique minimal solution.
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
26 / 44
LQR Problem - Infinite Horizon
Proof
Consider the problem Q(s, T, ζ)
inf{I(s, T, ζ, u), | u ∈ L2 (0, T ; U )},
Z
Z
1 T s
1 T
I(s, T, ζ, u) =
kyζ,u (t)k2Y dt +
ku(t)k2U dt,
2 s
2 s
s is the solution to
and yζ,u
y 0 (t) = Ay(t) + Bu(t),
Mythily Ramaswamy
t ≥ s,
Optimal Control - Stabilization
y(s) = ζ.
3rd July, 2015
27 / 44
LQR Problem - Infinite Horizon
For each ζ ∈ Y , let uζ be the solution of Q(s, T, ζ).
Let Pmin be the solution to the differential Riccati equation
P = P ∗ ≥ 0,
P (0) = 0,
P 0 = A∗ P + P A − P BB ∗ P + I.
For every ζ ∈ Y , t 7→ (P (t)ζ, ζ)Y is nondecreasing.
Let 0 < T1 < T2 .
We have
1
(P (T1 )ζ, ζ)Y ,
2
1
inf I(0, T2 , ζ, u) = (P (T2 )ζ, ζ)Y .
2
inf I(0, T1 , ζ, u) =
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
28 / 44
LQR Problem - Infinite Horizon
inf I(0, T2 , ζ, u)
n
o
0
= inf I(0, T2 , ζ, u) + inf{I(T1 , T2 , yζ,u
, u)}
≥ inf I(0, T1 , ζ, u).
Hence t 7→ (P (t)ζ, ζ)Y is nondecreasing.
We also have
(P (t)ζ, ζ)Y ≤ 2I(0, t, ζ, uζ ) ≤ J(z(ζ, uζ ), uζ ) < ∞.
Thus limt→∞ (P (t)ζ, ζ)Y exist and finite for every ζ ∈ Y .
Note that
1
1
(P (t)ζ, ξ)Y = (P (t)(ζ + ξ), (ζ + ξ))Y − (P (t)(ζ − ξ), (ζ − ξ))Y .
4
4
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
29 / 44
LQR Problem - Infinite Horizon
Applying Banach-Steinhaus theorem to the family of operators
{(P (t)ζ, ·)Y }t≥0 for every ζ ∈ Y , we obtain supt≥0 |(P (t)ζ, ·)Y | < ∞.
Again applying Banach-Steinhaus theorem to the family of operators
{(P (t)·, ·)Y }t≥0 , we obtain supt≥0 |(P (t)·, ·)Y | < ∞.
∞ ∈ L(Y ) such that
Therefore, there exists Pmin
∞
limt→∞ (P (t)ζ, ζ)Y = (Pmin
ζ, ζ)Y .
∞ = (P ∞ )∗ ≥ 0.
Since P (t) = P (t)∗ ≥ 0 for all t ≥ 0, we have Pmin
min
For every ζ ∈ D(A),
d
dt (P (t)ζ, ζ)Y = (P (t)ζ, Aζ)Y
−(P (t)BB ∗ ζ, ζ)Y + (ζ, ζ)Y ,
Mythily Ramaswamy
Optimal Control - Stabilization
+ (P (t)Aζ, ζ)Y
t ∈ [0, T ].
3rd July, 2015
30 / 44
LQR Problem - Infinite Horizon
Since the mapping, t 7→ (P (t)ζ, ζ)Y , is of class C 1 , the right hand
side of the above equation admits a limit when t → ∞.
d
Thus limt→∞ (P (t)ζ, ζ)Y exists and we can show that this limit is
dt
necessarily zero.
∞ satisfies ARE.
Hence Pmin
∞ is a minimal solution, we suppose that P̂ is an another
To show Pmin
solution.
Note that P̂ is also a solution to the differential Riccati equation
P = P ∗ ≥ 0,
P (0) = P̂ ,
P 0 = A∗ P + P A − P BB ∗ P + I.
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
31 / 44
LQR Problem - Infinite Horizon
Since Pmin (0) ≤ P̂ (0), we have that Pmin (t) ≤ P̂ (t) = P̂ for all
t ≥ 0.
∞ ≤ P̂ .
Taking t → ∞, Pmin
Uniqueness of the minimal solution can be proved in a classical
manner.
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
32 / 44
LQR Problem - Infinite Horizon
Theorem
The unique solution (ȳ, ū) to P satisfies the feedback formula
∞
ū = −B ∗ Pmin
ȳ(t),
∀ t ≥ 0,
where ȳ satisfies
∞
ȳ 0 (t) = Aȳ(t) − BB ∗ Pmin
ȳ(t),
Moreover,
J(ȳ, ū) =
∀ t ≥ 0,
ȳ(0) = y0 .
1 ∞
Pmin y0 , y0 .
2
Y
Proof
Let ȳ be the solution to
∞
ȳ 0 (t) = Aȳ − BB ∗ Pmin
ȳ,
Mythily Ramaswamy
Optimal Control - Stabilization
ȳ = y0 .
3rd July, 2015
33 / 44
LQR Problem - Infinite Horizon
The solution to the problem
Z
n1 Z ∞
1 ∞
kyu (t)k2Y dt +
ku(t)k2u dt
20
2
0
o
∞ y (T ), y (T )
+ 12 Pmin
| u ∈ L2 (0, T ; U ) ,
u
u
Y
where yu satisfies
E
y 0 (t) = Ay(t) + Bu,
y(0) = y0 ,
is given by (ŷ, û) = (ŷ, −B ∗ P ŷ), where P solves
P = P ∗ ≥ 0,
∞ ,
P (T ) = Pmin
P 0 + A∗ P + P A∗ − P BB ∗ P + I = 0.
and ŷ solves
ŷ 0 (t) = Aŷ(t) − BP (t)ŷ(t),
ŷ(0) = y0 ,
∞ is the unique solution to the above DRE.
Pmin
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
34 / 44
LQR Problem - Infinite Horizon
From the previous part, we have
Z ∞
Z ∞
2
P (0)y0 , y0
=
kŷ(t)kY dt +
kû(t)k2u dt
Y
0
0
∞ ŷ(T ), ŷ(T )
+ Pmin
.
Y
Consequently, we have (ȳ, ū) = (ŷ, û) and for every T > 0,
Z ∞
Z ∞
∞
2
Pmin y0 , y0
=
kȳ(t)kY dt +
kū(t)k2u dt
Y
0
0
∞ ȳ(T ), ȳ(T )
+ Pmin
.
Y
Taking T → ∞, we obtain
∞
2J(ȳ, ū) ≤ Pmin
y0 , y0 .
Y
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
35 / 44
LQR Problem - Infinite Horizon
Considering the optimal problem
o
n1 Z ∞
kyu (t)k2Y + ku(t)k2u dt | u ∈ L2 (0, T ; U ),
2 0
we have
∞
Pmin
y0 , y0
Z
Y
≤
T
kȳ(t)k2Y + kū(t)k2u dt ≤ 2J(ȳ, ū).
0
and
∞
Pmin
y0 , y0
Z
Y
≤
T
kyu (t)k2Y + kū(t)k2u dt ≤ 2J(yu , u),
0
∞ is the unique solution to above
for all u ∈ L2 (0, ∞; U ) and Pmin
DRE.
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
36 / 44
LQR Problem - Infinite Horizon
Taking T → ∞,
and
∞
Pmin
y0 , y0
Z
Y
∞
Pmin
y0 , y0
Thus
∞
kȳ(t)k2Y + kū(t)k2u dt ≤ 2J(ȳ, ū).
≤
0
Y
≤ 2J(yu , u),
∞ y ,y
Pmin
0 0
Y
∀ u ∈ L2 (0, ∞; U ).
= 2J(ȳ, ū) = 2 inf J(y, u)
and (ȳ, ū) is the unique solution to P.
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
37 / 44
LQR Problem - Infinite Horizon
Lemma
If P is a solution to the ARE, the operator A − BB ∗ P with domain
D(A) is the generator of an exponentially stable semigroup on Y .
Proof
Let ζ ∈ Y and y be the solution to
y 0 = Ay − BB ∗ P y,
y(0) = ζ.
First suppose that ζ ∈ D(A) and {un }n∈N be a sequence in
C 1 ([0, ∞); U ) ∩ L2 (0, ∞; U ) converging to −B ∗ P y in L2 (0, ∞; U ).
Let yn be the solution to
yn0 = Ayn + Bun ,
Mythily Ramaswamy
yn (0) = ζ.
Optimal Control - Stabilization
3rd July, 2015
38 / 44
LQR Problem - Infinite Horizon
With the ARE, we get for all s ≥ 0
d
(P yn (s), yn (s))Y = 2(Ayn (s) + Bun (s), P yn (s))Y
ds
= −(yn (s), yn (s))Y + (B ∗ P yn (s), B ∗ P yn (s))U
+2(un (s), B ∗ P yn (s))U
= −(yn (s), yn (s))Y − (B ∗ P yn (s), B ∗ P yn (s))U
+2(B ∗ P yn (s), B ∗ P yn (s))U + 2(un (s), B ∗ P yn (s))U .
Integrating between 0 to t, for all t > 0, we obtain
Z t
(P yn (t), yn (t))Y +
kyn (s)k2Y + kB ∗ P yn (s)k2U ds
Z t 0
= (P ζ, ζ) + 2
kB ∗ P yn (s)k2U + (un (s), B ∗ P yn (s))U ds.
0
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
39 / 44
LQR Problem - Infinite Horizon
Now passing through the limit n → ∞, we get
Z t
(P y(t), y(t))Y +
ky(s)k2Y + kB ∗ P y(s)k2U ds = (P ζ, ζ).
0
By a density argument, the above identity is also true for any ζ ∈ Y .
Since P is a nonnegative operator, from above inequality we obtain
Z t
ky(s)k2Y + kB ∗ P y(s)k2U ds ≤
0
Z t
(P y(t), y(t))Y +
ky(s)k2Y + kB ∗ P y(s)k2U ds
0
= (P ζ, ζ).
Hence, taking t → ∞, we get
Z ∞
ky(s)k2Y + kB ∗ P y(s)k2U ds ≤ (P ζ, ζ),
0
and by Datko theorem, the lemma follows.
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
40 / 44
LQR Problem - Infinite Horizon
Lemma
Let P and Q be two solutions to ARE. Suppose that the operator
A − BB ∗ P with domain D(A), is the generator of an exponentially
semigroup in Y . Then P ≥ Q.
Proof
Since P and Q are solutions of ARE, we have
(P − Q)(A − BB ∗ P ) + (A − BB ∗ P )∗ (P − Q)
+(P − Q)BB ∗ (P − Q) = 0.
From this identity, for ζ ∈ Y , we deduce
d
t(A−BB ∗ P ) ζ, et(A−BB ∗ P ) ζ
(P
−
Q)e
dt
= −kB ∗ (P − Q)et(A−BB
Mythily Ramaswamy
∗P )
Optimal Control - Stabilization
Y
ζk2U .
3rd July, 2015
41 / 44
LQR Problem - Infinite Horizon
Integrating from 0 to T , for all T > 0, we obtain
∗
∗
(P − Q)ζ, ζ
= (P − Q)eT (A−BB P ) ζ, eT (A−BB P ) ζ
Y
ZY T
∗
t(A−BB ∗ P ) 2
+
kB (P − Q)e
ζkU dt
0
∗
∗
≥ (P − Q)eT (A−BB P ) ζ, eT (A−BB P ) ζ .
Y
Taking T → ∞ and using that A − BB ∗ P generates exponentially
stable semigroup in Y , we get
(P − Q)ζ, ζ
≥ 0, ∀ ζ ∈ D(A),
Y
that is P ≥ Q.
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
42 / 44
Algorithm to solve Ricatti Equation
Algorithm to solve Ricatti Equation
Numerical resolution of the finite dimensional Riccati equation
P = P ∗ ≥ 0,
A∗ P + P A − P BB ∗ P + C ∗ C = 0.
Hypothesis (A, B) is stabilizable and (A, C) is detectable.
Methods based on computation of eigenvalues of the matrix
A
−BB ∗
H=
CC ∗ −A∗
The spectrum of H is symmetric with respect to the origin and has no
eigenvalues with a zero real part.
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
43 / 44
Algorithm to solve Ricatti Equation
Algorithm
Algorithm
1. Calculate the eigenvalues and eigenvectors of H by QR- method
(writing the matrix as a product of an orthogonal matrix and an
upper triangular matrix).
2. Select the eigenvectors corresponding to the eigenvalues with with a
negative real part. Let V1 be the matrix whose columns correspond to
these vectors:
h V i
11
.
V1 =
V21
∗ P = V ∗ , to calculate P .
3. Solve V11
21
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
44 / 44
Algorithm to solve Ricatti Equation
V. Barbu, Mathematical Methods in Optimization of Differential
Systems, Kluwer Academic Publishers, 1994.
A. Bensoussan, G. Da Prato, M. Delfour, S. K. Mitter, Representation
and control of infinite dimensional systems. Second edition. Systems
and Control: Foundations & Applications. Birkhäuser Boston, Inc.,
Boston, MA, 2007.
Lasiecka and Triggiani, Differential and Algebraic Riccati Equations
with Applications to Boundary/Point control Problems,
Springer-Verlag, 1991
J-L Lions, Optimal Control of systems governed by Partial Differential
Equations, Springer, 1971.
Jean-Pierre Raymond, Optimal Control of PDEs, FICUS Course
Notes, 2001.
Jean-Pierre Raymond, Optimal control and stabilisation of flow related
models, FICUS Course Notes, 2001.
Mythily Ramaswamy
Optimal Control - Stabilization
3rd July, 2015
44 / 44