Theory and Applications of Optimal Control
Problems with Time-Delays
Helmut Maurer
University of Münster
Institute of Computational and Applied Mathematics
South Pacific Optimization Meeting (SPOM)
Newcastle, CARMA, 9–12 February 2013
Helmut Maurer
Optimal Control Problems with Time-Delays
EXPECT DELAYS
Helmut Maurer
Optimal Control Problems with Time-Delays
Challenges for delayed optimal control problems
Theory and Numerics for non-delayed optimal control problems
with control and state constraints are rather complete:
1
2
3
4
Necessary and sufficient conditions,
Stability and sensitivity analysis,
Numerical methods:
Boundary value methods, Discretization and NLP,
Semismooth Newton methods,
Real-time control techniques for perturbed extremals.
Challenge: establish similar theoretical and numerical methods
for delayed (retarded) optimal control problems.
Helmut Maurer
Optimal Control Problems with Time-Delays
Book on SSC for regular and bang-bang controls
Applications to Regular
and Bang-Bang Control
This book is devoted to the theory and applications of second-order necessary and
sufficient optimality conditions in the calculus of variations and optimal control. The
authors develop theory for a control problem with ordinary differential equations
subject to boundary conditions of both the equality and inequality type and for mixed
state-control constraints of the equality type. The book is distinctive in that
H. Maurer
N. P. Osmolovskii
Nikolai P. Osmolovskii is a Professor in the Department of Informatics and Applied
Mathematics, Moscow State Civil Engineering University; the Institute of Mathematics
and Physics, University of Siedlce, Poland; the Systems Research Institute, Polish
Academy of Science; the University of Technology and Humanities in Radom, Poland;
and of the Faculty of Mechanics and Mathematics, Moscow State
University. He was an Invited Professor in the Department of Applied
Mathematics, University of Bayreuth, Germany (2000), and at the
Centre de Mathématiques Appliquées, École Polytechnique, France
(2007). His fields of research are functional analysis, calculus of
variations, and optimal control theory. He has written fifty papers
and four monographs.
Helmut Maurer was a Professor of Applied Mathematics at the
Universität Münster, Germany (retired 2010) and has conducted
research in Austria, France, Poland, Australia, and the United
States. His fields of research in optimal control are control and state
constraints, numerical methods, second-order sufficient conditions,
sensitivity analysis, real-time control techniques, and various
applications in mechanics, mechatronics, physics, biomedical
and chemical engineering, and economics.
Applications to Regular and Bang-Bang Control
This book is suitable for researchers in calculus of variations and optimal control and
researchers and engineers in optimal control applications in mechanics; mechatronics;
physics; chemical, electrical, and biological engineering; and economics.
Second-Order Necessary and Sufficient Optimality
Conditions in Calculus of Variations and Optimal Control
• necessary and sufficient conditions are given in the form of no-gap conditions,
• the theory covers broken extremals where the control has finitely many points
of discontinuity, and
• a number of numerical examples in various application areas are fully solved.
Nikolai P. Osmolovskii
Helmut Maurer
For more information about SIAM books, journals,
conferences, memberships, or activities, contact:
Society for Industrial and Applied Mathematics
3600 Market Street, 6th Floor
Philadelphia, PA 19104-2688 USA
+1-215-382-9800 • Fax +1-215-386-7999
[email protected] • www.siam.org
Second-Order Necessary and Sufficient
Optimality Conditions in Calculus
of Variations and Optimal Control
Nikolai P. Osmolovskii
Helmut Maurer
DC24
ISBN 978-1-611972-35-1
DC24
DC24_Osmolovskii-Maurer_cover.indd 1
Helmut Maurer
10/2/2012 9:58:56 AM
Optimal Control Problems with Time-Delays
Overview
1
2
3
4
5
Formulation of optimal control problems with
state and control delays.
Example: Two-stage continuous stirred tank
reactor (CSTR).
Minimum Principle.
NLP methods: discretize and optimize.
Optimal control of the innate immune response.
Helmut Maurer
Optimal Control Problems with Time-Delays
Delayed Optimal Control Problem with State Constraints
State x(t) ∈ Rn , Control u(t) ∈ Rm , Delays dx , du ≥ 0.
Dynamics and Boundary Conditions
ẋ(t) = f (x(t), x(t − dx ), u(t), u(t − du )), a.e. t ∈ [0, tf ] ,
x(t) = x0 (t),
t ∈ [−dx , 0] ,
u(t) = u0 (t),
t ∈ [−du , 0),
ψ(x(tf )) = 0
Control and State Constraints
u(t) ∈ U ⊂ Rm , S(x(t)) ≤ 0,
t ∈ [0, tf ]
(S : Rn → Rk ) .
Minimize
Z
tf
f0 (x(t), x(t − dx ), u(t), u(t − du )) dt
J(u, x) = g (x(tf )) +
0
Helmut Maurer
Optimal Control Problems with Time-Delays
Two-Stage Continuous Stirred Tank Reactor (CSTR)
Time delays are caused by transport between the two tanks.
Helmut Maurer
Optimal Control Problems with Time-Delays
Two Stage CSTR
Dadebo S. and Luus R. Optimal control of time-delay systems
by dynamic programming, Optimal Control Applications and
Methods 13, pp. 29–41 (1992).
A chemical reaction A ⇒ B is processed in two tanks.
State and control variables:
Tank 1 :
Tank 2 :
Helmut Maurer
x1 (t)
:
(scaled) concentration
x2 (t)
:
(scaled) temperature
u1 (t)
:
temperature control
x3 (t)
:
(scaled) concentration
x4 (t)
:
(scaled) temperature
u2 (t)
:
temperature control
Optimal Control Problems with Time-Delays
Dynamics of the Two-Stage CSTR
Reaction term in Tank 1
:
R1 (x1 , x2 ) = (x1 + 0.5) exp
Reaction term in Tank 2
:
R2 (x3 , x4 ) = (x3 + 0.25) exp
25x2
1+x2
25x4
1+x4
Dynamics:
ẋ1 (t) = −0.5 − x1 (t) − R1 (t),
ẋ2 (t) = −(x2 (t) + 0.25) − u1 (t)(x2 (t) + 0.25) + R1 (t),
ẋ3 (t) = x1 (t − d) − x3 (t) − R2 (t) + 0.25,
ẋ4 (t) = x2 (t − d) − 2x4 (t) − u2 (t)(x4 (t) + 0.25) + R2 (t) − 0.25.
Initial conditions:
x1 (t) = 0.15, x2 (t) = −0.03, −d ≤ t ≤ 0,
x3 (0) = 0.1,
x4 (0) = 0.
Delays d = 0.1, d = 0.2, d = 0.4 in the state variables x1 , x2 .
Helmut Maurer
Optimal Control Problems with Time-Delays
Optimal control problem for the Two-Stage CSTR
Minimize
Rtf
( x12 + x22 + x32 + x42 + 0.1u12 + 0.1u22 ) dt
(tf = 2) .
0
Hamiltonian with yk (t) = xk (t − d), k = 1, 2 :
H(x, y , λ, u) = f0 (x, u) + λ1 ẋ1
+λ2 (−(x2 + 0.25) − u1 (x2 + 0.25) + R1 (x1 , x2 ) )
+λ3 (y1 − x3 − R2 (x3 , x4 ) + 0.25)
+λ4 (y2 − 2x4 − u2 (x4 + 0.25) + R2 (x3 , x4 ) + 0.25)
Advanced adjoint equations:
λ̇1 (t) = −Hx1 (t) − χ [ 0,tf −d ] λ3 (t + d),
λ̇2 (t) = −Hx2 (t) − χ [ 0,tf −d ] λ4 (t + d),
λ̇k (t) = −Hxk (t) (k = 3, 4).
The minimum condition yields Hu = 0 and thus
u1 = 5λ2 (x2 + 0.25),
Helmut Maurer
u2 = 5λ4 (x4 + 0.25).
Optimal Control Problems with Time-Delays
Two-Stage CSTR with free x(tf ) : x1 , x2 , x3 , x4
concentration x1
0.16
0.14
0.12
0.1
0.08
0.06
0.04
0.02
0
-0.02
temperature x2
d=0.1
d=0.2
d=0.4
d=0.1
d=0.2
d=0.4
0.02
0.01
0
-0.01
-0.02
-0.03
0
0.5
1
1.5
2
0
0.5
concentration x3
0.1
0.06
0.04
0.02
0
-0.02
-0.04
0
0.5
1
1.5
Delays
Helmut Maurer
1.5
2
temperature x4
0.045
0.04
0.035
0.03
0.025
0.02
0.015
0.01
0.005
0
d=0.1
d=0.2
d=0.4
0.08
1
2
d=0.1
d=0.2
d=0.4
0
0.5
1
1.5
d = 0.1, d = 0.2, d = 0.4.
Optimal Control Problems with Time-Delays
2
Two-Stage CSTR with free x(tf ) : u1 , u2 , λ1 , λ2
control u1
0.2
control u2
0.3
d=0.1
d=0.2
d=0.4
0.1
d=0.1
d=0.2
d=0.4
0.25
0
0.2
-0.1
0.15
-0.2
0.1
-0.3
0.05
-0.4
0
0
0.2 0.4 0.6 0.8
1
1.2 1.4 1.6 1.8
2
0
adjoint variable λ1
0.16
0.14
0.12
0.1
0.08
0.06
0.04
0.02
0
-0.02
0.2 0.4 0.6 0.8
1
1.2 1.4 1.6 1.8
2
adjoint variable λ2
d=0.1
d=0.2
d=0.4
d=0.1
d=0.2
d=0.4
0.1
0
-0.1
-0.2
-0.3
-0.4
0
0.2 0.4 0.6 0.8
1
1.2 1.4 1.6 1.8
Delays
Helmut Maurer
2
0
0.2 0.4 0.6 0.8
1
1.2 1.4 1.6 1.8
d = 0.1, d = 0.2, d = 0.4.
Optimal Control Problems with Time-Delays
2
Two-Stage CSTR with x(tf ) = 0 : x1 , x2 , x3 , x4
concentration x1
0.16
temperature x2
d=0.1
d=0.2
d=0.4
0.14
0.12
d=0.1
d=0.2
d=0.4
0.02
0.01
0.1
0
0.08
0.06
-0.01
0.04
-0.02
0.02
0
-0.03
0
0.5
1
1.5
2
0
0.5
concentration x3
0.1
1.5
2
temperature x4
0.03
d=0.1
d=0.2
d=0.4
0.08
1
d=0.1
d=0.2
d=0.4
0.025
0.06
0.02
0.04
0.015
0.02
0.01
0
0.005
-0.02
0
0
0.5
1
1.5
Delays
Helmut Maurer
2
0
0.5
1
1.5
d = 0.1, d = 0.2, d = 0.4.
Optimal Control Problems with Time-Delays
2
Two-Stage CSTR with x(tf ) = 0 : u1 , u2 , λ1 , λ2
control u1
control u2
0.3
d=0.1
d=0.2
d=0.4
0.2
0.1
d=0.1
d=0.2
d=0.4
0.25
0.2
0
0.15
-0.1
-0.2
0.1
-0.3
0.05
-0.4
0
0
0.2 0.4 0.6 0.8
1
1.2 1.4 1.6 1.8
2
0
adjoint variable λ1
0.18
1
1.2 1.4 1.6 1.8
2
adjoint variable λ2
0.2
d=0.1
d=0.2
d=0.4
0.16
0.2 0.4 0.6 0.8
d=0.1
d=0.2
d=0.4
0.1
0.14
0
0.12
-0.1
0.1
0.08
-0.2
0.06
-0.3
0.04
-0.4
0
0.2 0.4 0.6 0.8
1
1.2 1.4 1.6 1.8
Delays
Helmut Maurer
2
0
0.2 0.4 0.6 0.8
1
1.2 1.4 1.6 1.8
d = 0.1, d = 0.2, d = 0.4.
Optimal Control Problems with Time-Delays
2
Two-Stage CSTR with x(tf ) = 0 and x4 (t) ≤ 0.01
temperature x4
control u1
d=0.1
d=0.2
d=0.4
0.014
0.012
d=0.1
d=0.2
d=0.4
0.3
0.2
0.1
0.01
0
0.008
-0.1
0.006
-0.2
0.004
-0.3
0.002
-0.4
0
-0.5
0
0.5
1
1.5
2
0
0.2 0.4 0.6 0.8
multiplier µ for x4 <= 0.01
0.7
1.2 1.4 1.6 1.8
2
control u2
0.3
d=0.1
d=0.2
d=0.4
0.6
1
d=0.1
d=0.2
d=0.4
0.25
0.5
0.4
0.2
0.3
0.15
0.2
0.1
0.1
0
0.05
0
0.5
1
1.5
Delays
Helmut Maurer
2
0
0.2 0.4 0.6 0.8
1
1.2 1.4 1.6 1.8
d = 0.1, d = 0.2, d = 0.4.
Optimal Control Problems with Time-Delays
2
Delayed Optimal Control Problem with State Constraints
State x(t) ∈ Rn , Control u(t) ∈ Rm , Delays dx , du ≥ 0.
Dynamics and Boundary Conditions
ẋ(t) = f (x(t), x(t − dx ), u(t), u(t − du )), a.e. t ∈ [0, tf ] ,
x(t) = x0 (t),
t ∈ [−dx , 0] ,
u(t) = u0 (t),
t ∈ [−du , 0),
ψ(x(tf )) = 0
Control and State Constraints
u(t) ∈ U ⊂ Rm , S(x(t)) ≤ 0,
t ∈ [0, tf ]
(S : Rn → Rk ) .
Minimize
Z
tf
f0 (x(t), x(t − dx ), u(t), u(t − du )) dt
J(u, x) = g (x(tf )) +
0
Helmut Maurer
Optimal Control Problems with Time-Delays
Literature on optimal control with time-delays
State delays and pure control constraints:
Kharatishvili (1961), Oguztöreli (1966), Banks (1968),
Halanay (1968), Soliman, Ray (1970, chemical engineering),
Warga (1972, abstract theory, optimization in Banach spaces),
Guinn (1976, transform delayed problems to standard problems),
Colonius, Hinrichsen (1978), Clarke, Wolenski (1991),
Dadebo, Luus (1992), Mordukhovich, Wang (2003–).
State delays and pure state constraints: Angell, Kirsch (1990).
State and control delays and mixed control–state constraints:
Göllmann, Kern, Maurer (OCAM 2009),
Göllmann, Maurer: ”Theory and applications of optimal control
problems with multiple time-delays,” to appear in JIMO 2013.
Helmut Maurer
Optimal Control Problems with Time-Delays
Optimal control problems with state constraints
Use the transformation method of Guinn (1976) and transform an
optimal control problem with delays and state constraints to a
standard non–delayed optimal control problem with state
constraints. Then apply the
necessary conditions for non-delayed problems:
Jacobson, Lele, Speyer (1975): KKT conditions in Banach
spaces.
Maurer (1979) : Regularity of multipliers for state constraints.
Hartl, Sethi, Thomsen (SIAM Review 1995): Survey on
Maximum Principles.
Vinter (2000): (Nonsmooth) Optimal Control
Helmut Maurer
Optimal Control Problems with Time-Delays
Hamiltonian
Hamiltonian (Pontryagin) Function
H(x, y , λ, u, v ) :=λ0 f0 (t, x, y , u, v ) + λf (t, x, y , u, v )
y variable with y (t) = x(t − dx )
v variable with v (t) = u(t − du )
λ ∈ Rn , λ0 ∈ R adjoint (costate) variable
Let (u, x) ∈ L∞ ([0, tf ], Rm ) × W 1,∞ ([0, tf ], Rn ) be a
locally optimal pair of functions. Then there exist
an adjoint function λ ∈ BV([0, tf ], Rn ) and λ0 ≥ 0,
a multiplier ρ ∈ Rq (associated with terminal conditions),
a multiplier function (measure) µ ∈ BV([0, tf ], Rk ),
such that the following conditions are satisfied for a.e. t ∈ [0, tf ] :
Helmut Maurer
Optimal Control Problems with Time-Delays
Minimum Principle
(i) Advanced adjoint equation and transversality condition:
λ(t) =
Rtf
t
( Hx (s) + χ[0,tf −dx ] (t) Hy (s + dx ) ) ds +
+ (λ0 g + ρψ)x (x(tf ))
Rtf
Sx (x(s)) dµ(s)
t
( if S(x(tf )) < 0 ),
where Hx (t) and Hy (t + dx ) denote evaluations along the optimal
trajectory and χ[0,tf −dx ] is the characteristic function.
(ii) Minimum Condition:
H(t) + χ[0,tf −du ] (t) H(t + du )
= min w ∈U [ H(x(t), y (t), λ(t), w , v (t))
+ χ[0,tf −du ] (t) H(t + du )H(x(t + du ), y (t), λ(t + du ), u(t + du ), w ) ]
(iii) Multiplier condition and complementarity condition:
Ztf
dµ(t) ≥ 0,
S(x(t)) dµ(t) = 0
0
Helmut Maurer
Optimal Control Problems with Time-Delays
Minimum Principle
(i) Advanced adjoint equation and transversality condition:
λ(t) =
Rtf
t
( Hx (s) + χ[0,tf −dx ] (t) Hy (s + dx ) ) ds +
+ (λ0 g + ρψ)x (x(tf ))
Rtf
Sx (x(s)) dµ(s)
t
( if S(x(tf )) < 0 ),
where Hx (t) and Hy (t + dx ) denote evaluations along the optimal
trajectory and χ[0,tf −dx ] is the characteristic function.
(ii) Minimum Condition:
H(t) + χ[0,tf −du ] (t) H(t + du )
= min w ∈U [ H(x(t), y (t), λ(t), w , v (t))
+ χ[0,tf −du ] (t) H(t + du )H(x(t + du ), y (t), λ(t + du ), u(t + du ), w ) ]
(iii) Multiplier condition and complementarity condition:
Ztf
dµ(t) ≥ 0,
S(x(t)) dµ(t) = 0
0
Helmut Maurer
Optimal Control Problems with Time-Delays
Minimum Principle
(i) Advanced adjoint equation and transversality condition:
λ(t) =
Rtf
t
( Hx (s) + χ[0,tf −dx ] (t) Hy (s + dx ) ) ds +
+ (λ0 g + ρψ)x (x(tf ))
Rtf
Sx (x(s)) dµ(s)
t
( if S(x(tf )) < 0 ),
where Hx (t) and Hy (t + dx ) denote evaluations along the optimal
trajectory and χ[0,tf −dx ] is the characteristic function.
(ii) Minimum Condition:
H(t) + χ[0,tf −du ] (t) H(t + du )
= min w ∈U [ H(x(t), y (t), λ(t), w , v (t))
+ χ[0,tf −du ] (t) H(t + du )H(x(t + du ), y (t), λ(t + du ), u(t + du ), w ) ]
(iii) Multiplier condition and complementarity condition:
Ztf
dµ(t) ≥ 0,
S(x(t)) dµ(t) = 0
0
Helmut Maurer
Optimal Control Problems with Time-Delays
Regularity conditions for dµ(t) = η(t)dt if du = 0
Boundary arc
:
S(x(t)) = 0
for
t1 ≤ t ≤ t2 .
Assumption
:
u(t) ∈ int(U)
for
t1 < t < t2 .
Under certain regularity conditions we have dµ(t) = η(t) dt
with a smooth multiplier η(t) for all t1 < t < t2 .
Adjoint equation and jump conditions
λ̇(t) = −Hx (t) − χ[0,tf −dx ] Hy (t + dx ) − η(t)Sx (x(t))
λ(tk +) = λ(tk −) − νk Sx (x(tk )) ,
νk ≥ 0
at each contact or junction time tk , νk = µ(tk +) − µ(tk −)
Minimum condition on the boundary
Hu (t) = 0 .
This condition allows to compute the multiplier η = η(x, λ).
Helmut Maurer
Optimal Control Problems with Time-Delays
Model of the immune response
Dynamic model of the immune response:
Asachenko A, Marchuk G, Mohler R, Zuev S,
Disease Dynamics, Birkhäuser, Boston, 1994.
Optimal control:
Stengel RF, Ghigliazza R, Kulkarni N, Laplace O,
Optimal control of innate immune response,
Optimal Control Applications and Methods 23, 91–104 (2002),
Lisa Poppe, Julia Meskauskas: Diploma theses, Universität
Münster (2006,2008).
L. Göllmann, H. Maurer: Optimal control problems with multiple
time-delays, to appear in JIMO 2013.
Helmut Maurer
Optimal Control Problems with Time-Delays
Innate Immune Response: state and control variables
State variables:
x1 (t)
:
x2 (t)
:
x3 (t)
:
x4 (t)
:
concentration of pathogen
(=concentration of associated antigen)
concentration of plasma cells,
which are carriers and producers of antibodies
concentration of antibodies, which kill the pathogen
(=concentration of immunoglobulins)
relative characteristic of a damaged organ
( 0 = healthy, 1 = dead )
Control variables:
u1 (t)
u2 (t)
u3 (t)
u4 (t)
Helmut Maurer
:
:
:
:
pathogen killer
plasma cell enhancer
antibody enhancer
organ healing factor
Optimal Control Problems with Time-Delays
Innate Immune Response: state and control variables
State variables:
x1 (t)
:
x2 (t)
:
x3 (t)
:
x4 (t)
:
concentration of pathogen
(=concentration of associated antigen)
concentration of plasma cells,
which are carriers and producers of antibodies
concentration of antibodies, which kill the pathogen
(=concentration of immunoglobulins)
relative characteristic of a damaged organ
( 0 = healthy, 1 = dead )
Control variables:
u1 (t)
u2 (t)
u3 (t)
u4 (t)
Helmut Maurer
:
:
:
:
pathogen killer
plasma cell enhancer
antibody enhancer
organ healing factor
Optimal Control Problems with Time-Delays
Generic dynamical model of the immune response
ẋ1 (t) = (1 − x3 (t))x1 (t) − u1 (t),
ẋ2 (t) = 3A(x4 (t))x1 (t − d)x3 (t − d) − (x2 (t) − 2) + u2 (t) ,
ẋ3 (t) = x2 (t) − (1.5 + 0.5x1 (t))x3 (t) + u3 (t) ,
ẋ4 (t) = x1 (t) − x4 (t) − u4 (t) .
Immune deficiency function triggered by target organ damage
cos(πx4 ) , 0 ≤ x4 ≤ 0.5
A(x4 ) =
.
0
0.5 ≤ x4
For 0.5 ≤ x4 (t) the production of plasma cells stops.
State delay d ≥ 0 in variables x1 and x3
Initial conditions (d = 0) : x2 (0) = 2, x3 (0) = 4/3, x4 (0) = 0
Case 1 :
Case 2 :
Case 3 :
Helmut Maurer
x1 (0) = 1.5,
x1 (0) = 2.0,
x1 (0) = 3.0,
decay, requires no therapy (control)
slower decay, requires no therapy
diverges without control (lethal case)
Optimal Control Problems with Time-Delays
Generic dynamical model of the immune response
ẋ1 (t) = (1 − x3 (t))x1 (t) − u1 (t),
ẋ2 (t) = 3A(x4 (t))x1 (t − d)x3 (t − d) − (x2 (t) − 2) + u2 (t) ,
ẋ3 (t) = x2 (t) − (1.5 + 0.5x1 (t))x3 (t) + u3 (t) ,
ẋ4 (t) = x1 (t) − x4 (t) − u4 (t) .
Immune deficiency function triggered by target organ damage
cos(πx4 ) , 0 ≤ x4 ≤ 0.5
A(x4 ) =
.
0
0.5 ≤ x4
For 0.5 ≤ x4 (t) the production of plasma cells stops.
State delay d ≥ 0 in variables x1 and x3
Initial conditions (d = 0) : x2 (0) = 2, x3 (0) = 4/3, x4 (0) = 0
Case 1 :
Case 2 :
Case 3 :
Helmut Maurer
x1 (0) = 1.5,
x1 (0) = 2.0,
x1 (0) = 3.0,
decay, requires no therapy (control)
slower decay, requires no therapy
diverges without control (lethal case)
Optimal Control Problems with Time-Delays
Generic dynamical model of the immune response
ẋ1 (t) = (1 − x3 (t))x1 (t) − u1 (t),
ẋ2 (t) = 3A(x4 (t))x1 (t − d)x3 (t − d) − (x2 (t) − 2) + u2 (t) ,
ẋ3 (t) = x2 (t) − (1.5 + 0.5x1 (t))x3 (t) + u3 (t) ,
ẋ4 (t) = x1 (t) − x4 (t) − u4 (t) .
Immune deficiency function triggered by target organ damage
cos(πx4 ) , 0 ≤ x4 ≤ 0.5
A(x4 ) =
.
0
0.5 ≤ x4
For 0.5 ≤ x4 (t) the production of plasma cells stops.
State delay d ≥ 0 in variables x1 and x3
Initial conditions (d = 0) : x2 (0) = 2, x3 (0) = 4/3, x4 (0) = 0
Case 1 :
Case 2 :
Case 3 :
Helmut Maurer
x1 (0) = 1.5,
x1 (0) = 2.0,
x1 (0) = 3.0,
decay, requires no therapy (control)
slower decay, requires no therapy
diverges without control (lethal case)
Optimal Control Problems with Time-Delays
Generic dynamical model of the immune response
ẋ1 (t) = (1 − x3 (t))x1 (t) − u1 (t),
ẋ2 (t) = 3A(x4 (t))x1 (t − d)x3 (t − d) − (x2 (t) − 2) + u2 (t) ,
ẋ3 (t) = x2 (t) − (1.5 + 0.5x1 (t))x3 (t) + u3 (t) ,
ẋ4 (t) = x1 (t) − x4 (t) − u4 (t) .
Immune deficiency function triggered by target organ damage
cos(πx4 ) , 0 ≤ x4 ≤ 0.5
A(x4 ) =
.
0
0.5 ≤ x4
For 0.5 ≤ x4 (t) the production of plasma cells stops.
State delay d ≥ 0 in variables x1 and x3
Initial conditions (d = 0) : x2 (0) = 2, x3 (0) = 4/3, x4 (0) = 0
Case 1 :
Case 2 :
Case 3 :
Helmut Maurer
x1 (0) = 1.5,
x1 (0) = 2.0,
x1 (0) = 3.0,
decay, requires no therapy (control)
slower decay, requires no therapy
diverges without control (lethal case)
Optimal Control Problems with Time-Delays
Optimal control model: cost functional
State x = (x1 , x2 , x3 , x4 ) ∈ R4 , Control u = (u1 , u2 , u3 , u4 ) ∈ R4
L2 -functional quadratic in control: Stengel et al.
Minimize J2 (x, u) = x1 (tf )2 + x4 (tf )2
Rtf
+ ( x12 + x42 + u12 + u22 + u32 + u42 ) dt
0
L1 -functional linear in control
Minimize J1 (x, u) = x1 (tf )2 + x4 (tf )2
Rtf
+ ( x12 + x42 + u1 + u2 + u3 + u4 ) dt
0
Control constraints: 0 ≤ ui (t) ≤ umax , i = 1, .., 4
Final time: tf = 10
Helmut Maurer
Optimal Control Problems with Time-Delays
Optimal control model: cost functional
State x = (x1 , x2 , x3 , x4 ) ∈ R4 , Control u = (u1 , u2 , u3 , u4 ) ∈ R4
L2 -functional quadratic in control: Stengel et al.
Minimize J2 (x, u) = x1 (tf )2 + x4 (tf )2
Rtf
+ ( x12 + x42 + u12 + u22 + u32 + u42 ) dt
0
L1 -functional linear in control
Minimize J1 (x, u) = x1 (tf )2 + x4 (tf )2
Rtf
+ ( x12 + x42 + u1 + u2 + u3 + u4 ) dt
0
Control constraints: 0 ≤ ui (t) ≤ umax , i = 1, .., 4
Final time: tf = 10
Helmut Maurer
Optimal Control Problems with Time-Delays
Optimal control model: cost functional
State x = (x1 , x2 , x3 , x4 ) ∈ R4 , Control u = (u1 , u2 , u3 , u4 ) ∈ R4
L2 -functional quadratic in control: Stengel et al.
Minimize J2 (x, u) = x1 (tf )2 + x4 (tf )2
Rtf
+ ( x12 + x42 + u12 + u22 + u32 + u42 ) dt
0
L1 -functional linear in control
Minimize J1 (x, u) = x1 (tf )2 + x4 (tf )2
Rtf
+ ( x12 + x42 + u1 + u2 + u3 + u4 ) dt
0
Control constraints: 0 ≤ ui (t) ≤ umax , i = 1, .., 4
Final time: tf = 10
Helmut Maurer
Optimal Control Problems with Time-Delays
L2 –functional, d = 0 : optimal state and control variables
State variables x1 , x2 , x3 , x4 and optimal controls u1 , u2 , u3 , u4 :
second-order sufficient conditions via matrix Riccati equation
Helmut Maurer
Optimal Control Problems with Time-Delays
L2 –functional, d = 0 : state constraint x4 (t) ≤ 0.2
State and control variables for state constraint x4 (t) ≤ 0.2 .
Boundary arc x4 (t) ≡ 0.2 for t1 = 0.398 ≤ t ≤ t2 = 1.35
Helmut Maurer
Optimal Control Problems with Time-Delays
L2 –functional, multiplier η(t) for constraint x4 (t) ≤ 0.2
Compute multiplier η as function of (x, λ) :
η(x, λ) = λ2 3π sin(πx4 )x1 x3 − λ1 + 2λ4 − 2x3 x1 + 2x4
Scaled multiplier 0.1 η(t) and boundary arc x4 (t) = 0.2
Helmut Maurer
Optimal Control Problems with Time-Delays
L2 –functional, delay d > 0 , constraint x4 (t) ≤ α
Dynamics with state delay d > 0
ẋ1 (t) = (1 − x3 (t))x1 (t) − u1 (t),
ẋ2 (t) = 3 cos(πx4 ) x1 (t − d)x3 (t − d) − (x2 (t) − 2) + u2 (t) ,
ẋ3 (t) = x2 (t) − (1.5 + 0.5x1 (t))x3 (t) + u3 (t) ,
ẋ4 (t) = x1 (t) − x4 (t) − u4 (t)
x4 (t) ≤ α ≤ 0.5
Initial conditions
x1 (t) = 0 ,
−d ≤ t < 0, x1 (0) = 3,
x3 (t) = 4/3 , −d ≤ t ≤ 0,
x2 (0) = 2,
x4 (0) = 0.
Helmut Maurer
Optimal Control Problems with Time-Delays
L2 –functional : delay d = 1 and x4 (t) ≤ 0.2
State variables for d = 0 and d = 1
Helmut Maurer
Optimal Control Problems with Time-Delays
L2 –functional : delay d = 1 and x4 (t) ≤ 0.2
Optimal controls for d = 0 and d = 1
Helmut Maurer
Optimal Control Problems with Time-Delays
L2 –functional, d = 1 : multiplier η(t) for x4 (t) ≤ 0.2
Compute multiplier η as function of (x, λ) :
η(x, y , λ) = λ2 3π sin(πx4 )y1 y3 − λ1 + 2λ4 − 2x3 x1 + 2x4
Scaled multiplier 0.1 η(t) and boundary arc x4 (t) = 0.2 ;
η(t) is discontinuous at t = d = 1
Helmut Maurer
Optimal Control Problems with Time-Delays
L1 –functional and state delay d ≥ 0
Minimize
J1 (x, u) = p11 x1 (tf )2 + p44 x4 (tf )2
Rtf
+ ( p11 x12 + p44 x42 + q1 u1 + q2 u2 + q3 u3 + q4 u4 ) dt
0
Dynamics with delay d and control constraints
ẋ1 (t) = (1 − x3 (t))x1 (t) − u1 (t),
ẋ2 (t) = 3A(x4 (t))x1 (t − r )x3 (t − r ) − (x2 (t) − 2) + u2 (t) ,
ẋ3 (t) = x2 (t) − (1.5 + 0.5x1 (t))x3 (t) + u3 (t) ,
ẋ4 (t) = x1 (t) − x4 (t) − u4 (t) ,
0 ≤ ui (t) ≤ umax ,
Helmut Maurer
0 ≤ t ≤ tf
(i = 1, .., 4)
Optimal Control Problems with Time-Delays
L1 –functional : non-delayed problem d = 0 : umax = 2
Helmut Maurer
Optimal Control Problems with Time-Delays
L1 –functional : delayed problem d = 1 : umax = 2
Helmut Maurer
Optimal Control Problems with Time-Delays
L1 –functional : non-delayed, time–optimal control for
x1 (tf ) = x4 (tf ) = 0, x3 (tf ) = 4/3
umax = 1: minimal time tf = 2.2151, singular arc for u4 (t)
Helmut Maurer
Optimal Control Problems with Time-Delays
Further applications and future work
1
Optimal oil extraction and exploration : state delay (Bruns,
Maurer, Semmler)
2
Biomedical applications: optimal protocols in cancer
treatment and immunology
3
Vintage control problems
4
Delayed control problems with free final time
5
Optimal control problems with state-dependent delays
6
Verifiable sufficient conditions
Helmut Maurer
Optimal Control Problems with Time-Delays
Thank you for your attention !
Helmut Maurer
Optimal Control Problems with Time-Delays