Optimal Control of Heat Equation
Mythily Ramaswamy
TIFR Centre for Applicable Mathematics, Bangalore, India
CIMPA Pre-School,
I.I.T Bombay
22 June - July 4, 2015
Optimal Control
2nd July, 2015
1 / 38
contents
1
2
3
Introduction
Optimal Control of ODE systems
Mathematical Framework
Constrained Optimization
Pontryagin Maximum Principle
Optimal Control of PDE
4
Tools for studying Evolutionary PDE
Sobolev Spaces involving time
Differential equation in Banach space
5
Weak Solution
6
Optimal control problem for Heat equation
Existence of a unique optimal control
Optimality conditions
7
Abstract result
Optimal Control
2nd July, 2015
2 / 38
Introduction
Optimal control problems
In many practical problems, a differential equation describes the evolution
of the state of the system.
Sometimes the dynamics involves some control parameters also.
Our aim is to find the best control which maximizes a certain payoff
criterion or cost functional.
Major Questions :
Does an optimal control exist?
How can we characterize such an optimal control?
How to build an optimal control?
Optimal Control
2nd July, 2015
3 / 38
Introduction
Optimal control problems
In many practical problems, a differential equation describes the evolution
of the state of the system.
Sometimes the dynamics involves some control parameters also.
Our aim is to find the best control which maximizes a certain payoff
criterion or cost functional.
Major Questions :
Does an optimal control exist?
How can we characterize such an optimal control?
How to build an optimal control?
Optimal Control
2nd July, 2015
3 / 38
Optimal Control of ODE systems
Moon Landing Problem
Question:
How to bring the spacecraft to land softly on the lunar surface using
minimum amount of fuel?
The system can be modelled by
v̇(t) = −g +
α(t)
m(t)
ḣ(t) = v(t)
ṁ(t) = −kα(t)
Here h(t) is the height ; v(t) is the velocity
m(t) is the mass of the spacecraft ; α(t) is the thrust
Optimal Control
2nd July, 2015
4 / 38
Optimal Control of ODE systems
Moon Landing Problem
Problem is to minimize the fuel or to maximize the remaining amount
when it lands.
Minimizing fuel is equivalent to maximising the mass when it lands.
Define
J(α(.)) = m(T ),
where T is the first time when v(T ) = 0, that is it lands on the surface.
Then the problem is to maximize J(α(.)) .
Other constraints are
h(t) ≥ 0,
m(t) ≥ 0,
Optimal Control
0 ≤ α(t) ≤ 1
2nd July, 2015
5 / 38
Optimal Control of ODE systems
Mathematical Framework
Mathematical Framework
Control problem:
Solve the minimization problem
T
Z
Min
L(t, x(t), u(t))dt + g(x(T ))
0
subject to
dx(t)
= f (t, x(t), u(t)),
dt
β(x(0), x(T )) = 0 = γ(x(0), x(T )) in Rd
Control u(t) is in Ω ⊂ Rm ,
J is cost functional ; L is running cost; g is the terminal cost.
Optimal Control
2nd July, 2015
6 / 38
Optimal Control of ODE systems
Constrained Optimization
Augmented Functional
Question:
Can we use Lagrange Multiplier idea in optimal control problems too?
Assuming g = 0, define augmented functional, with multipliers λ0 ≥ 0, µ, λ
˜ ẋ, u, p, λ0 , µ, λ) = λ0 J(x, u)
J(x,
Z
+
T
pT (f (x, u) − ẋ) + µT β(x(0), x(T )) + λT γ(x(0), x(T ))
0
At a minimum, vanishing of the derivative in each of the variable gives the
optimality conditions.
Then the integrand is :
L̃ = λ0 L(x, u) + pT (f (x, u) − ẋ)
Introduce the Control Hamiltonian
H(x, p) = λ0 L(x, u) + pT (f (x, u)
Optimal Control
2nd July, 2015
7 / 38
Optimal Control of ODE systems
Constrained Optimization
Optimality Conditions
Formally, calculus of variations applied to J˜ to characterize the extremal
gives :
d
L̃ẋ − L̃x = ṗ + Hx = 0
dt
This gives the equation for the adjoint vector p :
ṗ = −p(t)T Df (t, x∗ (t), u∗ (t)) − ∇x L(t, x∗ (t), u∗ (t))
with Transversality conditions, obtained by variations in the parameters
L̃ẋ |t=0 = ∇1 (µT β + λT γ)
L̃ẋ |t=T = ∇2 (µT β + λT γ)
Partial gradient with respect to the first group of variables, x(0) is ∇1 ; for
x(T ) is ∇2 .
Optimal Control
2nd July, 2015
8 / 38
Optimal Control of ODE systems
Pontryagin Maximum Principle
Pontryagin Principle
Theorem
Under the usual smoothness assumptions, let ( x∗ (.), u∗ (.) ) be the
optimal process.
Then there exist λ0 ≥ 0, λ, µ ∈ Rd and a vector function p(.)
satisfying the adjoint equation together with boundary conditions given by
transversality relations, such that for a.e. t ∈ [0, T ],
min H(t, x∗ (t), p∗ (t), u) = H(t, x∗ (t), p∗ (t), u∗ (t))
u
Optimal Control
2nd July, 2015
9 / 38
Optimal Control of PDE
Optimal control for elliptic equation : an example
Ω is a domain with boundary Γ.
y = Electrical potential and u = Current density,
Or
y = Temperature distribution and u = Thermal flux.
y satisfies
∂y
= u on Γ.
∂n
Problem: Minimize the distance between y and a given distribution yd
Z
|y − yd |2 .
−∆y + y = f
in
Ω,
Ω
The consumed energy is
Z
|u|2 .
Γ
Optimal Control
2nd July, 2015
10 / 38
Optimal Control of PDE
Control Problem:
Minimize
J(y, u) =
−∆y + y = f
u ∈ L2 (Γ),
yd ∈ L2 (Ω),
in
1
2
R
Ω,
Ω |y
− yd |2 +
∂y
∂n
= u on
β
2
2
Γ |u| ,
R
Γ,
f ∈ L2 (Ω),
β > 0.
Optimal Control
2nd July, 2015
11 / 38
Optimal Control of PDE
Optimal control for parabolic equation : an example
Cooling process in metallurgy:
Ω is a domain with boundary Γ.
T = Temperature in Ω, c(T ) = Specific heat capacity, ρ(T ) =
Density,
K(T ) = The conductivity of steel at the temperature T ,
R = a nonlinear function related to radiation law.
The problem is described by nonlinear heat equation of the form:
ρ(T )c(T ) ∂T
∂t = div(K(T )∇T )
K(T ) ∂T
∂n = R(T, u)
on
in
Ω × (0, tf ),
Γ × (0, tf )
Here u is the control variable.
Optimal Control
2nd July, 2015
12 / 38
Optimal Control of PDE
Control problem:
Minimize J(y, u) = β1
R
2
Ω |T (tf ) − T | + β2
ρ(T )c(T ) ∂T
∂t = div(K(T )∇T )
K(T ) ∂T
∂n = R(T, u)
β1 > 0,
on
in
R tf
0
|u|q ,
Ω × (0, tf ),
Γ × (0, tf ),
β2 > 0,
Here tf is the terminal time of the process ;
T is a desired profile of temperature;
exponent q is chosen in function of the radiation law R.
Optimal Control
2nd July, 2015
13 / 38
Tools for studying Evolutionary PDE
Sobolev Spaces involving time
Sobolev spaces involving time
Search for solutions u(x, t) in Sobolev spaces involving time.
Let Y be a Banach space.
Definition
The space
Lp (0, T ; Y )
consists of all strongly measurable functions u : [0, T ] → Y with
R
1
p
T
i) kukLp (0,T ;Y ) := 0 ku(t)kpY dt < ∞, for 1 ≤ p < ∞.
ii) kukL∞ (0,T ;Y ) := ess supt∈[0,T ] ku(t)kY < ∞.
Optimal Control
2nd July, 2015
14 / 38
Tools for studying Evolutionary PDE
Sobolev Spaces involving time
Definition
The space
C([0, T ]; Y )
consists of all continuous functions u : [0, T ] → Y with
kukC([0,T ];Y ) := max ku(t)kY < ∞.
t∈[0,T ]
Optimal Control
2nd July, 2015
15 / 38
Tools for studying Evolutionary PDE
Differential equation in Banach space
Differential equation in Banach space
Consider the equation
y 0 = Ay + f,
y(0) = y0 ,
with f ∈ C(R; Y ) and y0 ∈ Y , Y is a Banach space.
If A ∈ L(Y ) , then this equation admits a unique solution in C 1 (R; Y )
given by
Z t
tA
y(t) = e y0 +
e(t−s)A f (s) ds,
0
etA =
∞ n
X
t
n=0
n!
An ,
Optimal Control
t ∈ R.
2nd July, 2015
16 / 38
Tools for studying Evolutionary PDE
Differential equation in Banach space
When A ∈ L(Y ), the family (etA )t∈R satisfies:
i) e0A = I,
ii) etA ∈ L(Y ),
∀ t ∈ R,
iii) e(s+t)A = esA ◦ etA ,
iv) limt→0
ketA
∀ s ∈ R, ∀ t ∈ R,
− IkL(Y ) = 0,
v) Ay = limt→0
(etA y−y)
t
,
∀ y ∈ Y.
Qn : What about unbounded operator A ?
Optimal Control
2nd July, 2015
17 / 38
Tools for studying Evolutionary PDE
Differential equation in Banach space
One Dimensional Heat equation
The heat equation in (0, L) × (0, T )
y ∈ L2 (0, T ; H01 (0, L)) ∩ C([0, T ]; L2 (0, L)),
yt − yxx = 0
in
(0, L) × (0, T ),
y(0, t) = y(L, t) = 0
y(x, 0) = y0 (x)
in
in
(0, T ),
(0, L),
where T > 0, L > 0 and y0 ∈ L2 (0, L).
2
d
2
1
Here A = dx
2 is defined on a suitable subspace, D(A) = H ∩ H0 of
2
2
L (0, L)) into L (0, L)) but is unbounded .
Optimal Control
2nd July, 2015
18 / 38
Tools for studying Evolutionary PDE
Differential equation in Banach space
To find an expression for etA , introduce
r
kπx
2
sin
, ∀ k ∈ N,
φk (x) =
L
L
∀ x ∈ (0, L).
The family (φk )k∈N is an orthonormal basis of the Hilbert space L2 (0, L).
φk is an eigenfunction of the operator (A, D(A)):
φk ∈ D(A),
Aφk = λk φk ,
λk = −
k2 π2
.
L2
Search for a solution y in the form
y(x, t) =
∞
X
gk (t)φk (x).
k=1
Optimal Control
2nd July, 2015
19 / 38
Tools for studying Evolutionary PDE
Differential equation in Banach space
If
y0 (x) = y(x, 0) =
∞
X
gk (0)φk (x),
k=1
and if the PDE is satisfied in the sense of distributions in (0, L) × (0, T ),
then gk obeys
2 2
gk0 + kLπ2 gk = 0 in (0, T ),
gk (0) = y0k = (y0 , φk )L2 (0,L) .
We have
gk (t) = y0k e−
k2 π 2 t
L2
∀ t ∈ (0, T ).
,
The function y ∈ L2 (0, T ; H01 (0, L)) ∩ C([0, T ]; L2 (0, L)),
y(x, t) =
∞
X
y0k e−
k2 π 2 t
L2
φk (x),
∀ x ∈ (0, L),
t ∈ (0, T ),
k=1
is the solution of the heat equation.
Optimal Control
2nd July, 2015
20 / 38
Tools for studying Evolutionary PDE
Differential equation in Banach space
Remark : y is not defined for t < 0.
Let us define
S(t)y0 =
∞
X
k2 π 2 t
(y0 , φk )L2 (0,L) e− L2 φk (x),
∀ x ∈ (0, L),
t ∈ (0, T ),
k=1
Then we have
i) S(0) = I,
ii) S(t) ∈ L(L2 (0, L)),
∀ t > 0,
iii) S(s + t)y0 = S(s) ◦ S(t)y0 ,
∀ s ≥ 0, ∀ t ≥ 0,
iv) limt↓0 kS(t)y0 − y0 kL2 (0,L) = 0,
∀ y0 ∈ L2 (0, L),
∀ y0 ∈ L2 (0, L).
Optimal Control
2nd July, 2015
21 / 38
Weak Solution
Weak solutions to evolution equations
The differential equation is
y 0 = Ay + f,
y(0) = y0 ,
with f ∈ C(R; Y ) and y0 ∈ Y , Y a Banach space.
Definition (Weak solution)
A weak solution to the equation in Lp (0, T ; Y ), for 1 ≤ p < ∞, is a
function y ∈ Lp (0, T ; Y ) such that, for all z ∈ D(A∗ ), the mapping
t → hy(t), ziY,Y 0
belongs to W 1,p (0, T ) and obeys
d
dt hy(t), zi
= hy(t), A∗ zi + hf (t), zi,
∈ (0, T ),
hy(0), zi = hy0 , zi.
Optimal Control
2nd July, 2015
22 / 38
Weak Solution
Theorem (Existence and Uniqueness of Weak solution)
If y0 ∈ Y and if f ∈ Lp (0, T ; Y ), then the equation admits a unique weak
solution in Lp (0, T ; Y ).
Moreover, this solution belongs to C([0, T ]; Y ) and it satisfies
Z
t
S(t − s)f (s),
y(t) = S(t)y0 +
∀ t ∈ [0, T ].
0
From the expression for the solution
ky(t)kC([0,T ];Y ) ≤ C ky0 kY + kf kL1 (0,T ;Y ) ,
for some positive constant C.
Optimal Control
2nd July, 2015
23 / 38
Optimal control problem for Heat equation
Optimal control problem for Heat equation
Let Ω be a bounded domain in Rn , with a boundary Γ of class C 2 .
Let T > 0, set Q = Ω × (0, T ) and Σ = Γ × (0, T ).
The heat equation with a distributed control is
∂y
∂t
− ∆y = f + χω u
y=0
on
Σ,
in Q,
y(x, 0) = y0 (x),
in
Ω.
The optimal control problem is
inf J(y, u) | u ∈ L2 (ω × (0, T )) ,
for some suitable J.
Optimal Control
2nd July, 2015
24 / 38
Optimal control problem for Heat equation
Optimal control
Let us take
J(y, u) =
1
2
R
Q |y
− yd |2 +
1
2
R
Ω |y(T )
− yd (T )|2 +
β
2
R
ω×(0,T ) |u|
2.
Here β > 0 and yd ∈ C([0, T ]; L2 (Ω)) and (y, u) solves the controlled heat
equation .
Then we have estimate for the state variable:
kykC([0,T ];L2 (Ω)) ≤ C ky0 kL2 (Ω) + kf kL2 (Q) + kukL2 (ω×(0,T )) .
Optimal Control
2nd July, 2015
25 / 38
Optimal control problem for Heat equation
Existence of a unique optimal control
Existence of a unique optimal control
Set F (u) = J(y(u), u).
Let (un )n∈N be a minimizing sequence in L2 (ω × (0, T )),
lim F (un ) = inf u∈L2 (ω×(0,T )) F (u).
n→∞
Let yn be the solution corresponding to un . Suppose that (un )n∈N is
bounded in L2 (ω × (0, T )) and that
un * u,
weakly in L2 (ω × (0, T )).
Set y = y(u).
Optimal Control
2nd July, 2015
26 / 38
Optimal control problem for Heat equation
Existence of a unique optimal control
Show
yn * y,
weakly in L2 (Q),
yn (T ) * y(T ),
weakly in L2 (Ω).
Using the weakly lower semicontinuity of k · k2L2 (Q) , k · k2L2 (Ω) ,
k · k2L2 (ω×(0,T )) , show that u is a solution of the minimizing problem.
Uniqueness of the solution follows from the strict convexity of F .
Optimal Control
2nd July, 2015
27 / 38
Optimal control problem for Heat equation
Optimality conditions
Derivative of the state variable
Introduce
Equations satisfied by zλ = y(u + λv) − y(u) is
∂zλ
∂t
− ∆zλ = λχω v
zλ = 0
on
Σ,
in Q,
zλ (x, 0) = 0,
in
Ω.
From the estimate it follows that
kzλ kC([0,T ];L2 (Ω)) ≤ C|λ|kvkL2 (ω×(0,T )) .
This yields that as λ tends to zero,
y(u + λv) → y(u)
in C([0, T ]; L2 (Ω)).
Optimal Control
2nd July, 2015
28 / 38
Optimal control problem for Heat equation
Optimality conditions
Derivative of F
Recall that
F (u + λv) − F (u)
.
λ&0
λ
F 0 (u) = lim
By a classical calculation, we have
R
F 0 (u)v = Q (y(u) − yd )z(v)
R
R
+ Ω (y(u)(T ) − yd (T ))z(v)(T ) + β ω×(0,T ) uv,
where z satisfies
∂z
∂t
− ∆z = χω v
z=0
on
Σ,
in Q,
z(x, 0) = 0,
Optimal Control
in
Ω.
2nd July, 2015
29 / 38
Optimal control problem for Heat equation
Optimality conditions
0
Identification of F (u)
Our aim: To find q such that
Z
Z
Z
(y(u) − yd )z(v) + (y(u)(T ) − yd (T ))z(v)(T ) =
Q
Ω
qv
ω×(0,T )
Let p be any regular function defined in Q.
Using an integration by parts between z(v) and p we have
R
R
pv
=
ω×(0,T )
Q (zt − ∆z)p
R
R
R ∂z
= Q z(−pt − ∆p) + Ω z(T )p(T ) − Σ ∂n
p
Optimal Control
2nd July, 2015
30 / 38
Optimal control problem for Heat equation
Optimality conditions
Consider the adjoint problem
− ∂p
∂t − ∆p = y(u) − yd
p=0
on
Σ,
in Q,
p(x, T ) = (y(u) − yd )(T ),
Then we have
F 0 (u)v =
in
Ω.
Z
(p + βu)v,
ω×(0,T )
if the above equations are justified.
For that we need to study the existence and the regularity of the solution
of the adjoint problem.
Optimal Control
2nd July, 2015
31 / 38
Optimal control problem for Heat equation
Optimality conditions
Adjoint problem
Theorem
Let g ∈ L2 (Q) and pT ∈ L2 (Ω).
The terminal boundary value problem
− ∂p
∂t − ∆p = g
p=0
on
Σ,
in Q,
p(x, T ) = pT ,
in
Ω
is wellposed and p satisfies
kpkC([0,T ];L2 (Ω)) ≤ C kpT kL2 (Ω) + kgkL2 (Q) .
Optimal Control
2nd July, 2015
32 / 38
Optimal control problem for Heat equation
Optimality conditions
Integration by parts between z and p
Theorem
Suppose that φ ∈ L2 (Q), g ∈ L2 (Q), pT ∈ L2 (Ω).
Then the solution z of the equation
∂z
∂t
− ∆z = φ
z=0
on
in Q,
Σ,
z(x, 0) = 0,
in
Ω.
and the solution p of the adjoint equation satisfy the following formula
Z
Z
Z
φp =
zg +
z(T )pT .
Q
Q
Ω
Optimal Control
2nd July, 2015
33 / 38
Optimal control problem for Heat equation
Optimality conditions
Outline of the proof
If pT ∈ H01 (Ω), then z and p belong to
L2 (0, T ; D(A)) ∩ H 1 (0, T ; L2 (Ω)).
Green’s formula gives
Z
Z
−∆z(t)p(t)dx =
−∆p(t)z(t)dx,
Ω
Ω
for almost every t ∈ [0, T ], and
Z TZ
Z
Z TZ
∂z
∂p
p=−
z+
z(T )pT .
0
Ω ∂t
Ω
0
Ω ∂t
Thus IBP formula is established for pT ∈ H01 (Ω).
Then by density argument and using the estimate, we obtain the
equation for any pT ∈ L2 (Q).
Optimal Control
2nd July, 2015
34 / 38
Optimal control problem for Heat equation
Optimality conditions
Theorem
If (y, u) is the solution, then
1
u = − p|ω×(0,T ) ,
β
where p is the solution of the adjoint problem corresponding to y:
− ∂p
∂t − ∆p = y − yd
p=0
on
Σ,
in Q,
p(x, T ) = y(T ) − yd (T ),
in
Ω.
Next theorem is the converse of the above one.
Optimal Control
2nd July, 2015
35 / 38
Optimal control problem for Heat equation
Optimality conditions
Theorem
If a pair (e
y , pe) ∈ C([0, T ]; L2 (Ω)) × C([0, T ]; L2 (Ω)) obeys the system
∂ ye
∂t
− ∆e
y = f − β1 χω pe in Q,
ye = 0
on
Σ,
ye(x, 0) = y0 (x),
− ∂∂tpe − ∆e
p = ye − yd
pe = 0
on
Σ,
in
Ω,
in Q,
pe(x, T ) = ye(T ) − yd (T ),
in
Ω,
Then the pair (e
y , − β1 pe) is the optimal solution to the problem.
Optimal Control
2nd July, 2015
36 / 38
Abstract result
Abstract result
Consider the problem P (0, T, y0 )
(y, u) | u ∈ L2 (0, T ; U )},
Z
Z
1 T
1 T
2
JT (y, u) =
ky(t)kY dt +
ku(t)k2u dt.
2 0
2 0
inf{JT (y, u),
and (y, u) satisfies
y 0 (t) = Ay(t) + Bu(t),
∀ t ≥ 0,
y(0) = y0 ,
y0 ∈ Y,
u ∈ U,
where
Y and U are two Hilbert spaces.
The unbounded operator (A, D(A)) is the infinitesimal generator of a
strongly continuous semigroup on Y .
The control operator B ∈ L(U ; Y ).
Optimal Control
2nd July, 2015
37 / 38
Abstract result
Theorem
This problem P (0, T, y0 ) admits a unique solution (ȳ, ū) and (ȳ, ū)
satisfies the system
ȳ 0 (t) = Aȳ(t) − BB ∗ p(t),
−p0 (t) = A∗ p(t) + ȳ(t),
ū(t) =
ȳ(0) = y0 ,
p(T ) = 0,
−B ∗ p(t).
Optimal Control
2nd July, 2015
38 / 38
Abstract result
A. Bensoussan, G. Da Prato, M. Delfour, S. K. Mitter, Representation
and control of infinite dimensional systems. Second edition. Systems
and Control: Foundations & Applications. Birkhäuser Boston, Inc.,
Boston, MA, 2007.
L.C. Evans, An Introduction to Mathematical Optimal Control Theory,
Lecture Notes.
Fleming and Rishel, Deterministic and Stochastic Optimal Control,
Springer
J-L Lions, Optimal Control of systems governed by PDEs
Donald Kirk, Optimal Control Theory - An introduction, Dover
Publications.
Jean-Pierre Raymond, Optimal Control of PDEs, FICUS Course Notes.
Troltzsch, Optimal Control of Partial Differential Equations, Theory,
Methods and Applications, AMS Vol 112.
Optimal Control
2nd July, 2015
38 / 38