Theory and Applications of Optimal Control
Problems with Time Delays
Helmut Maurer
University of Münster
Applied Mathematics: Institute of Analysis and Numerics
Université Pierre et Marie Curie, Paris, March 10, 2017
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
What can you expect from this talk ?
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Challenges for Optimal Control Problems with Delays
Theory and Numerics for non-delayed optimal control
problems with control and state constraints are well developed:
1
2
3
4
Necessary and sufficient conditions,
Stability and sensitivity analysis,
Numerical methods: Boundary value methods,
Discretization and NLP, Semismooth Newton methods,
Real-time control techniques for perturbed extremals.
CHALLENGE: Establish similar theoretical and numerical
methods for delayed (retarded) optimal control problems.
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Overview
1
2
3
4
5
6
7
Case Study: Combination Therapies for Cancer
(with Ledzewicz, Schättler, Klamka, Swierniak)
Optimal Control Problems with Time Delays in State and
Control Variables
Minimum Principle for State-Constrained Control
Problems
Numerical Treatment: Discretize and Optimize
(with L. Göllmann)
A Non-Convex Academic Example with a State Constraint
Case Study:Two-stage Continuous Stirred Tank Reactor
(CSTR)
Case Study: Optimal Control of a Tuberculosis Model
with Time Delays (with C. Silva, D.F. Torres)
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Combination Therapies of Cancer
Tumour Anti-Angiogenesis: J. Folkman (1972) et al.
State and control variables:
p : primary tumour volume [mm3 ]
q : carrying capacity, or endothelial support [mm3 ]
u : anti-angiogenic agent
v : chemotoxic agent
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Combination Therapies of Cancer: Literature
U. Ledzewicz and H. Schaettler: Antiangiogenic therapy in
cancer treatment as an optimal control problem, SIAM Journal on
Control and Optimization, 46, (2007), 1052–1079. (Monotherapy)
Hahnfeldt et al model with Gompertzian Growth:
U. Ledzewicz, H. Maurer, and H. Schättler, Optimal and
suboptimal protocols for a mathematical model for tumor
antiangiogenesis in combination with chemotherapy, Mathematical
Biosciences 22, pp. 13–26 (2009).
Ergun et al model with Gompertzian Growth:
U. Ledzewicz, H. Maurer, and H. Schaettler, On optimal
delivery of combination therapy for tumors, Mathematical Biosciences
and Engineering, 8, (2011), 307–323.
Hahnfeldt et al model with Logistic Growth:
J. Klamka, H. Maurer and A. Swierniak: Local Controllability
and Optimal Control for a Model of Combined Anticancer therapy with
Control Delays, Math. Biosc. Eng. 14(1), 195–216 (2017).
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Optimal Control Problem
p : tumor volume,
u : anti-angiogenic control,
y : total amount of u,
q : carrying capacity,
v : chemotoxic control,
z : total amount of v .
Dynamics of the Hahnfeldt et al model
ṗ(t) = G (p(t), q(t)) − ϕ p(t) v (t),
q̇(t) = b p(t) − q(t) (d p(t)2/3 + µ + γ u(t) + η v (t))
ẏ (t) = u(t),
ż(t) = v (t).
Initial conditions: p(0) = p0 , q(0) = q0 , y (0) = 0, z(0) = 0.
Growth functions
Gompertzian Growth : G (p, q) = −ξ p ln(p/q)
Logistic Growth
: G (p, q) = ξ p (1 − p/q)
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Control problem and parameters
Control problem
Minimize final tumor volume p(T )
subject to the dynamic constraints, the control constraints
0 ≤ u(t) ≤ umax ,
0 ≤ v (t) ≤ vmax ,
and the constraints on the total amount of drugs
y (T ) ≤ ymax ,
z(T ) ≤ zmax .
PARAMETERS (obtained from mice):
ξ = 0.084, b = 5.85, d = 0.00873, γ = 0.15,
ϕ = 0.2,
η = 0.05, µ = 0.02.
BOUNDS: umax = 75,
Helmut Maurer
ymax = 300,
vmax = 2,
zmax = 10.
Theory and Applications of Optimal Control Problems with Tim
Monotherapy : only anti-angiogenic control u
Ledzewicz, Schättler (2007):
Gompertzian Growth G (p, q) = −ξpln(p/q) , free terminal time T .
Compute singular control in feedback form:
q
− (µ + d p 2/3 ) .
u = using (p, q) = γ1 ξ ln pq + b qp + 23 ξ db p1/3
control u
tumor p and vasculature q
80
70
60
50
40
30
20
10
0
p
q
14000
11000
8000
5000
2000
0
1
2
3
4
5
time t (days)
6
0
1
2
3
4
5
time t (days)
6
Optimal control is bang-singular-bang. Sufficient conditions by
synthesis analysis or switching time optimization.
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Approximation of bang-singular-bang control u
Gompertzian Growth and free terminal time.

 t1 = 0.07386,
for 0 ≤ t < t1 
 umax
u = 46.08
uc
for t1 ≤ t ≤ t2
u(t) =
, c

 t2 = 6.463
0
for t2 < t ≤ T
T = 6.615
control u
control u
80
70
60
50
40
30
20
10
0
80
70
60
50
40
30
20
10
0
0
1
2
3
4
5
time t (days)
p(T ) = 8533
6
0
1
2
3
4
5
time t (days)
6
p(T ) = 8541
SSC hold for the approximative control w.r.t. z = (t1 , t2 , uc ).
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Combination therapy: anti-angiogenic u, chemotherapy v
Ledzewicz, M., Schättler (2009): Gompertzian Growth, T is free.
Compute singular control in feedback form:
u = using (p, q, v ) = γ1 ξ ln qp + b qp + 23 ξ
d q
b p 1/3
− (µ + d p 2/3 )
+ ϕ−η
γ v.
Solution for ymax = 300 and zmax = 10 (total amount of drugs):
control u
control v
80
70
60
50
40
30
20
10
0
tumor p and vasculature q
15000
2
9000
1
6000
0.5
3000
0
0
1
2
3
time t (days)
4
5
p
q
12000
1.5
0
0
1
2
3
4
time t (days)
5
0
1
2
3
4
5
time t (days)
Surprising fact: chemotherapy v always starts later (pruning).
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Combination Therapy with Logistic Growth
Klamka, Maurer, Swierniak (2015):
Logistic Growth G (p, q) = ξp(1 − p/q), T is free.
Controls u and v are bang-bang: there are no singular arcs !
Chemotherapy control : v (t) ≡ 2 for terminal time T = 5.
control u and switching function φu
φu
100
80
60
40
20
0
-20
tumor p and vasculature q
16000
p
q
12000
8000
4000
0
0
1
2
3
time t (days)
4
5
0
1
2
3
4
time t (days)
5
SSC hold for bang-bang controls !
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Dynamics with control delays
p
u
v
y
:
:
:
:
tumor volume,
anti-angiogenic control,
chemotoxic control,
total amount of u,
q : carrying capacity,
delay du = 10.6,
delay dv = 1.84,
z : total amount of v .
Dynamics of the Hahnfeldt et al model
ṗ(t) = G (p(t), q(t)) − ϕ p(t) v (t − dv ),
q̇(t) = b p(t) − q(t) (d p(t)2/3 + µ + γ1 u(t) + γ2 u(t − du )
+η v (t)) ),
ẏ (t) = u(t),
( 0 ≤ u(t) ≤ umax , y (T ) ≤ ymax )
ż(t) = v (t),
( 0 ≤ v (t) ≤ vmax , z(T ) ≤ zmax )
Initial conditions: p(0) = p0 , q(0) = q0 , y (0) = 0, z(0) = 0.
Objective
Minimize final tumor volume p(T )
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Combination therapy with control delays
Klamka, Maurer, Swierniak (2015) : Logistic Growth, T = 16 fixed.
Solution for
umax = 40, ymax = 300 and vmax = 2, zmax = 10 ,
Delays
du = 10.6, dv = 1.84
control u and switching function φu
φu
50
40
30
20
10
0
-10
-20
0
2
4
6
8
10 12 14 16
time t (days)
control v and switching function φv
3
2.5
2
1.5
1
0.5
0
-0.5
φv
tumor p and vasculature q
20000
p
q
15000
10000
5000
0
0
2
4
6
8
10 12 14 16
0
2
time t (days)
4
6
8 10 12 14 16
time t (days)
Necessary conditions are satisfied (extremal solution),
but we cannot check sufficient conditions.
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Delayed Optimal Control Problem with State Constraints
State x(t) ∈ Rn , Control u(t) ∈ Rm , Delays dx , du ≥ 0.
Dynamics and Boundary Conditions
ẋ(t) = f (x(t), x(t − dx ), u(t), u(t − du )), a.e. t ∈ [0, tf ] ,
x(t) = x0 (t),
t ∈ [−dx , 0] ,
u(t) = u0 (t),
t ∈ [−du , 0),
ψ(x(tf )) = 0q
(0 ≤ q ≤ n).
Control and State Constraints
umin ≤ u(t) ≤ umax , S(x(t)) ≤ 0,
∀ t ∈ [0, tf ]
(S : Rn → Rk ) .
Minimize
Z
tf
f0 (x(t), x(t − dx ), u(t), u(t − du )) dt
J(u, x) = g (x(tf )) +
0
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Literature on optimal control with time-delays
Time delays in state variables and pure control constraints:
Kharatishvili (1961), Oguztöreli (1966), Banks (1968),
Halanay (1968), Soliman, Ray (1970, chemical engineering),
Warga (1968,1972): optimization in Banach spaces),
Guinn (1976) : transform delayed problems to standard problems),
Colonius, Hinrichsen (1978), Clarke, Wolenski (1991),
Dadebo, Luus (1992), Mordukhovich, Wang (2003–).
Time delays in state variables and pure state constraints:
Angell, Kirsch (1990).
State and control delays and mixed control–state constraints:
Göllmann, Maurer (OCAM 2009, JIMO 2014),
Time delays in state and control variables and state constraints:
Vinter (2016) : Maximum Principle for a general problem
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Method of steps : Warga (1968) and Guinn (1976)
Transform an optimal control problem with delays to a standard
non–delayed optimal control problem: requires commensurability.
Then apply the necessary conditions for non-delayed problems:
Jacobson, Lele, Speyer (1975): KKT conditions in Banach
spaces.
Maurer (1979) : Regularity of multipliers for state constraints.
Hartl, Sethi, Thomsen (SIAM Review 1995): Survey on
Maximum Principles.
Vinter (2000): (Nonsmooth) Optimal Control.
Applications to mixed control-state constraints:
single delays: Göllmann, Kern, Maurer (OCAM 2009),
multiple delays: Göllmann, Maurer (JIMO 2014)
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Hamiltonian
Hamiltonian (Pontryagin) Function
H(x, y , λ, u, v ) :=λ0 f0 (t, x, y , u, v ) + λf (t, x, y , u, v )
y variable with y (t) = x(t − dx )
v variable with v (t) = u(t − du )
λ ∈ Rn , λ0 ∈ R adjoint (costate) variable
Let (u, x) ∈ L∞ ([0, tf ], Rm ) × W 1,∞ ([0, tf ], Rn ) be a
locally optimal pair of functions. Then there exist
an adjoint function λ ∈ BV([0, tf ], Rn ) and λ0 ≥ 0,
a multiplier ρ ∈ Rq (associated with terminal conditions),
a multiplier function (measure) µ ∈ BV([0, tf ], Rk ),
such that the following conditions are satisfied for a.e. t ∈ [0, tf ] :
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Minimum Principle
(i) Advanced adjoint equation and transversality condition:
λ(t) =
Rtf
t
( Hx (s) + χ[0,tf −dx ] (s) Hy (s + dx ) ) ds +
+ (λ0 g + ρψ)x (x(tf ))
Rtf
Sx (x(s)) dµ(s)
t
( if S(x(tf )) < 0 ),
where Hx (t) and Hy (t + dx ) denote evaluations along the optimal
trajectory and χ[0,tf −dx ] is the characteristic function. (ii)
Minimum Condition for U = [ umin , umax ] :
H(t) + χ[0,tf −du ] (t) H(t + du )
= min w ∈U [ H(x(t), y (t), λ(t), w , v (t))
+ χ[0,tf −du ] (t) H(x(t + du ), y (t), λ(t + du ), u(t + du ), w ) ].
(iii) Multiplier condition and complementarity condition:
Ztf
dµ(t) ≥ 0,
S(x(t)) dµ(t) = 0.
0
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Regularity conditions for dµ(t) = η(t)dt if du = 0
Boundary arc
:
S(x(t)) = 0
for
t1 ≤ t ≤ t2 .
Assumption
:
u(t) ∈ int(U)
for
t1 < t < t2 .
Under certain regularity conditions we have dµ(t) = η(t) dt
with a continuous multiplier η(t) for all t1 < t < t2 .
Adjoint equation and jump conditions
λ̇(t) = −Hx (t) − χ[0,tf −dx ] (t) Hy (t + dx ) − η(t)Sx (x(t))
λ(tk +) = λ(tk −) − νk Sx (x(tk )) ,
νk ≥ 0,
at each contact or junction time tk , νk = µ(tk +) − µ(tk −).
Minimum condition: no control constraints and delays
Hu (t) = 0 .
This condition allows to compute the multiplier η = η(x, λ).
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Discretization and NLP
Consider MAYER problem with cost functional J(u, x) = g (x(tf )) .
Commensurability Assumption: There exists a stepsize h > 0 and
integers k, l, N ∈ N with
dx = k · h,
Grid points
ti := i · h
du = l · h,
tf = N · h .
(i = 0, 1, . . . , N),
tN = tf .
Approximation at grid points:
u(ti ) ≈ ui ∈ Rm ,
x(ti ) ≈ xi ∈ Rn (i = 0, . . . , N).
For simplicity EULER method:
xi+1 = xi + h · f (ti , xi , xi−k , ui , ui−l ),
x−i := x0 (−ih)
Helmut Maurer
(i = 0, .., k),
i = 0, 1, ..., N − 1,
u−i := u0 (−ih)
(i = 1, .., l).
Theory and Applications of Optimal Control Problems with Tim
Large-scale NLP Problem
Include mixed control-state constraint C (x(t), u(t)) ≤ 0.
Minimize
J(u, x) = g (xN )
subject to
xi+1 − xi − h · f (ti , xi , xi−k , ui , ui−l ) = 0,
i = 0, .., N − 1,
ψ(xN ) = 0,
x−i := x0 (−ih)
(i = 0, .., k),
C (xi , ui ) ≤ 0,
i = 0, .., N,
S(xi ) ≤ 0,
i = 0, .., N,
u−i := u0 (−ih)
(i = 1, .., l).
Optimization Variable:
z := (u0 , x1 , u1 , x2 , ..., uN−1 , xN ) ∈ RN(m+n)
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Multipliers and NLP-Solvers
Approximations:
Adjoint variable : λ(ti ) ≈ λi
Multiplier
, multiplier for ODE,
: η(ti ) ≈ ηi /h , multiplier for S(xi ) ≤ 0.
AMPL : Programming language (Fourer, Gay, Kernighan)
IPOPT: Interior point method (A. Wächter et al.)
LOQO: Interior point method (B. Vanderbei et al.
WORHP : SQP–method (C. Büskens, M. Gerdts)
Other NLP solvers embedded in AMPL : cf. NEOS server.
Special feature: solvers provide LAGRANGE-multipliers.
BOCOP : F. Bonnans, P. Martinon.
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Academic Example: state delays
state x(t) ∈ R, control u(t) ∈ R, delay d ≥ 0
Dynamics and Boundary Conditions
ẋ(t) = x(t − d)2 − u(t),
x(t) = x0 (t) = 1,
t ∈ [0, 2],
t ∈ [−d, 0],
x(2) = 1
Control and State Constraints
x(t) ≥ α,
i.e., S(x(t)) = −x(t) + α ≤ 0,
t ∈ [0, 2]
Minimize
Z
J(u, x) =
2
(x(t)2 + u(t)2 ) dt
0
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Optimal solutions without state constraints
Z
2
min
(x(t)2 +u(t)2 ) dt
s.t. ẋ(t) = x(t − d)2 −u(t), x0 (t) ≡ 1, x(2) = 1
0
optimal state and control for
delays
d = 0.0,
d = 0.1,
d = 0.2,
state x
1
0.95
0.9
0.85
0.8
0.75
0.7
0.65
0.6
0.55
0.5
d = 0.5,
control u
2.5
d=0
d=0.1
d=0.2
d=0.5
2
1.5
d=0
d=0.1
d=0.2
d=0.5
1
0.5
0
-0.5
0
Helmut Maurer
0.5
1
1.5
2
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Theory and Applications of Optimal Control Problems with Tim
Optimal solutions with state constraint x(t) ≥ α = 0.7
Optimal state and control for
delays
d = 0.0,
d = 0.1,
d = 0.2,
state x
1
control u
2.5
d=0
d=0.1
d=0.2
d=0.5
0.95
0.9
d = 0.5
d=0
d=0.1
d=0.2
d=0.5
2
1.5
0.85
1
0.8
0.5
0.75
0.7
0
0.65
-0.5
0
0.5
1
1.5
2
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Optimal controls u(t) are continuous !
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Minimum Principle
Augmented Hamiltonian: y (t) = x(t − d)
H(x, y , λ, η, u) = u 2 + x 2 + λ(y 2 − u) + η(−x + α)
Adjoint equation
λ̇(t) = −Hx (t) − χ[0,2−d] Hy (t + d)
−2x(t) − 2λ(t + d)x(t) + η(t) , 0 ≤ t ≤ 2 − d
=
−2x(t) + η(t) ,
2−d ≤t ≤2
Minimum condition
Hu (t) = 0
Helmut Maurer
⇒ u(t) = λ(t)/2
Theory and Applications of Optimal Control Problems with Tim
Boundary arc x(t) = α = 0.7 for t1 ≤ t ≤ t2
x(t) ≡ α
⇒ x(t − d)2 = u(t) = λ(t)/2
⇒ ẋ(t) = 0
Computation of multiplier η(t) by differentiation
η(t) = 2(2x(t −d)(x(t −2d)2 −λ(t −d)/2)+x(t)+λ(t +d)x(t))
delays d = 0.0,
d = 0.1,
d = 0.2,
1
0.9
d = 1.0
3
d=0
d=0.1
d=0.2
d=0.5
0.95
d = 0.5,
multiplier η
state x
2.5
2
0.85
1.5
0.8
1
0.75
0.5
0.7
0
0.65
0
Helmut Maurer
0.5
1
1.5
2
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Theory and Applications of Optimal Control Problems with Tim
Two-Stage Continuous Stirred Tank Reactor (CSTR)
Time delays are caused by transport between the two tanks.
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Two Stage CSTR
Dadebo S., Luus R. Optimal control of time-delay systems by
dynamic programming, Optimal Control Applications and Methods
13, pp. 29–41 (1992).
A chemical reaction A ⇒ B is processed in two tanks.
State and control variables:
Tank 1 :
Tank 2 :
x1 (t)
:
(scaled) concentration
x2 (t)
:
(scaled) temperature
u1 (t)
:
temperature control
x3 (t)
:
(scaled) concentration
x4 (t)
:
(scaled) temperature
u2 (t)
:
temperature control
State variables denote deviations from equilibrium.
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Dynamics of the Two-Stage CSTR
Reaction term in Tank 1
:
R1 (x1 , x2 ) = (x1 + 0.5) exp
Reaction term in Tank 2
:
R2 (x3 , x4 ) = (x3 + 0.25) exp
25x2
1+x2
25x4
1+x4
Dynamics:
ẋ1 (t) = −0.5 − x1 (t) − R1 (t),
ẋ2 (t) = −(x2 (t) + 0.25) − u1 (t)(x2 (t) + 0.25) + R1 (t),
ẋ3 (t) = x1 (t − d) − x3 (t) − R2 (t) + 0.25,
ẋ4 (t) = x2 (t − d) − 2x4 (t) − u2 (t)(x4 (t) + 0.25) + R2 (t) − 0.25.
Initial conditions:
x1 (t) = 0.15, x2 (t) = −0.03, −d ≤ t ≤ 0,
x3 (0) = 0.1,
x4 (0) = 0.
Delays d = 0.1, d = 0.2, d = 0.4 in the state variables x1 , x2 .
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Optimal control problem for the Two-Stage CSTR
Minimize
Rtf
( x12 + x22 + x32 + x42 + 0.1u12 + 0.1u22 ) dt
(tf = 2) .
0
Hamiltonian with yk (t) = xk (t − d), k = 1, 2 :
H(x, y , λ, u) = f0 (x, u) + λ1 ẋ1
+λ2 (−(x2 + 0.25) − u1 (x2 + 0.25) + R1 (x1 , x2 ) )
+λ3 (y1 − x3 − R2 (x3 , x4 ) + 0.25)
+λ4 (y2 − 2x4 − u2 (x4 + 0.25) + R2 (x3 , x4 ) + 0.25)
Advanced adjoint
λ̇1 (t)
λ̇2 (t)
λ̇k (t)
equations:
= −Hx1 (t) − χ [ 0,tf −d ] (t) λ3 (t + d),
= −Hx2 (t) − χ [ 0,tf −d ] (t) λ4 (t + d),
= −Hxk (t) (k = 3, 4).
The minimum condition yields Hu = 0 and thus
u1 = 5λ2 (x2 + 0.25),
Helmut Maurer
u2 = 5λ4 (x4 + 0.25).
Theory and Applications of Optimal Control Problems with Tim
Two-Stage CSTR with free x(tf ) : x1 , x2 , x3 , x4
concentration x1
0.16
0.14
0.12
0.1
0.08
0.06
0.04
0.02
0
-0.02
temperature x2
d=0.1
d=0.2
d=0.4
d=0.1
d=0.2
d=0.4
0.02
0.01
0
-0.01
-0.02
-0.03
0
0.5
1
1.5
2
0
0.5
concentration x3
0.1
0.06
0.02
0
-0.02
-0.04
0.5
1
1.5
Delays
Helmut Maurer
2
d=0.1
d=0.2
d=0.4
0.045
0.04
0.035
0.03
0.025
0.02
0.015
0.01
0.005
0
0.04
0
1.5
temperature x4
d=0.1
d=0.2
d=0.4
0.08
1
2
0
0.5
1
1.5
2
d = 0.1, d = 0.2, d = 0.4.
Theory and Applications of Optimal Control Problems with Tim
Two-Stage CSTR with free x(tf ) : u1 , u2 , λ2 , λ4
control u1
control u2
0.3
d=0.1
d=0.2
d=0.4
0.2
0.1
d=0.1
d=0.2
d=0.4
0.25
0.2
0
0.15
-0.1
-0.2
0.1
-0.3
0.05
-0.4
0
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
adjoint variable λ2
0.3
adjoint variable λ4
0.2
0.18
0.16
0.14
0.12
0.1
0.08
0.06
0.04
0.02
0
d=0.1
d=0.2
d=0.4
0.2
0.1
0
-0.1
-0.2
-0.3
-0.4
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Delays
Helmut Maurer
d=0.1
d=0.2
d=0.4
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
d = 0.1, d = 0.2, d = 0.4.
Theory and Applications of Optimal Control Problems with Tim
Two-Stage CSTR with x(tf ) = 0 : x1 , x2 , x3 , x4
concentration x1
0.16
0.14
0.12
0.1
0.08
0.06
0.04
0.02
0
temperature x2
0.02
d=0.1
d=0.2
d=0.4
d=0.1
d=0.2
d=0.4
0.01
0
-0.01
-0.02
-0.03
0
0.5
1
1.5
2
0
0.5
concentration x3
0.1
1.5
2
temperature x4
0.035
d=0.1
d=0.2
d=0.4
0.08
1
d=0.1
d=0.2
d=0.4
0.03
0.025
0.06
0.02
0.04
0.015
0.02
0.01
0
0.005
-0.02
0
0
0.5
1
1.5
Delays
Helmut Maurer
2
0
0.5
1
1.5
2
d = 0.1, d = 0.2, d = 0.4.
Theory and Applications of Optimal Control Problems with Tim
Two-Stage CSTR with x(tf ) = 0 : u1 , u2 , λ3 , λ4
control u1
control u2
0.35
d=0.1
d=0.2
d=0.4
0.2
0.1
d=0.1
d=0.2
d=0.4
0.3
0.25
0
0.2
-0.1
0.15
-0.2
0.1
-0.3
0.05
-0.4
0
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
adjoint variable λ3
0.15
adjoint variable λ4
0.25
d=0.1
d=0.2
d=0.4
0.1
d=0.1
d=0.2
d=0.4
0.2
0.05
0.15
0
0.1
-0.05
0.05
-0.1
0
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Delays
Helmut Maurer
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
d = 0.1, d = 0.2, d = 0.4.
Theory and Applications of Optimal Control Problems with Tim
Two-Stage CSTR with x(tf ) = 0 and x4 (t) ≤ 0.01
temperature x4
0.016
0.014
0.012
0.01
0.008
0.006
0.004
0.002
0
control u1
0.3
0.2
0.1
0
-0.1
-0.2
-0.3
-0.4
-0.5
d=0.1
d=0.2
d=0.4
0
0.5
1
1.5
2
d=0.1
d=0.2
d=0.4
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
multiplier η for x4 <= 0.01
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
control u2
0.35
d=0.1
d=0.2
d=0.4
d=0.1
d=0.2
d=0.4
0.3
0.25
0.2
0.15
0.1
0.05
0
0.5
1
1.5
Delays
Helmut Maurer
2
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
d = 0.1, d = 0.2, d = 0.4.
Theory and Applications of Optimal Control Problems with Tim
Two-Stage CSTR: time-optimal with x(tf ) = 0
d = 0 : tf = 1.8725496
control u1 and (scaled) switching function φ1
control u1
0.4
0.3
0.2
0.1
0
-0.1
-0.2
-0.3
-0.4
-0.5
-0.6
0.2
0.15
0.1
0.05
0
-0.05
-0.1
-0.15
-0.2
0
0.5
1
1.5
2
u1
φ1
0
0.5
1
0.4
0.3
0.2
0.1
0
-0.1
-0.2
-0.3
-0.4
0.2
0.15
0.1
0.05
0
-0.05
-0.1
-0.15
-0.2
0
0.5
1
1.5
2
control u1 and (scaled) switching function φ1
control u1
1.5
2
u1
φ1
0
0.5
1
1.5
2
d = 0.4 : tf = 2.0549
Boccia, Falugi, Maurer, Vinter : CDC 2013 .
Vinter (2016) : Maximum Principle in general form.
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Compartment Model for Tuberculosis
C.J. Silva and D.F.M. Torres, Optimal control strategies for
Tuberculosis treatment: a case study in Angola, Numerical Algebra,
Control and Optimization, 2 (2012), 601–617.
C.J. Silva, H. Maurer, and D.F.M. Torres, Optimal control of a
Tuberculosis model with state and control delays, Mathematical
Biosciences and Engineering 14(1), pp. 321–337 (2017).
5 state variables and delay in I (t):
S
L1
I
L2
R
N
:
:
:
:
:
:
Susceptible individuals
early latent individuals, recently infected (less than two years)
infectious individuals, who have active TB
persistent latent individuals
recovered individuals
total population N = S + L1 + I + L2 + R, assumed constant
dI
:
delay in I , represents delay in diagnosis
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Dynamical TB model with state and control delays
Control variables and delays:
u1
: effort on early detection and treatment
of recently infected individuals L1 ,
du1 : delay on the diagnosis of latent TB
and commencement of latent TB treatment,
u2
: chemotherapy or post-exposure vaccine
to persistent latent individuals (L2 ),
du2 : delay in the prophylactic treatment of persistent latent L2 .
Dynamical model : R = N − S − L1 − I − L2
Ṡ(t) = µN −
L̇1 (t) =
β
N I (t)S(t)
− µS(t),
β
N I (t)(S(t) + σL2 (t) + σR R(t))
−(δ + τ1 + 1 u1 (t − du1 ) + µ)L1 (t),
İ (t) = φδL1 (t) + ωL2 (t) + ωR R(t) − τ0 I (t − dI ) − µI (t),
L̇2 (t) = (1 − φ)δL1 (t) − σ Nβ I (t)L2 (t)
−(ω + 2 u2 (t − du2 ) + τ2 + µ)L2 (t).
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Optimal control problem with state and control delays
Initial functions in view of delays (N = 30000) :
S(0) = 76 N/120, L1 (0) = 36 N/120, L2 (0) = 2 N/120,
I (t) = 5 N/120
∀ − dI ≤ t ≤ 0,
R(0) = N/120,
uk (t) = 0
∀ − duk ≤ t < 0, k = 1, 2.
Control constraints
0 ≤ u1 (t) ≤ u1max = 1,
0 ≤ u2 (t) ≤ u2max = 1
for 0 ≤ t ≤ T .
Control problem: x = (S, L1 , I , L2 ) ∈ R4 , u = (u1 , u2 ) ∈ R2
Minimize L1 objective or L2 objective
RT
J1 (x, u) = 0 ( I (t) + L2 (t) + W1 u1 (t) + W2 u2 (t) ) dt ,
RT
J2 (x, u) = 0 ( I (t) + L2 (t) + W1 u1 (t)2 + W2 u2 (t)2 ) dt
(weights W1 , W2 > 0)
subject to the dynamic and control constraints.
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Parameters in the TB Model
Symbol
β
µ
δ
φ
ω
ωR
σ
σR
τ0
τ1
τ2
N
T
1
2
Description
Transmission coefficient
Death and birth rate
Rate at which individuals leave L1
Proportion of individuals going to I
Reactivation rate for persistent latent infections
Reactivation rate for treated individuals
Factor reducing the risk of infection as a result of
aquired immunity to a previous infection for L2
Rate of exogenous reinfection of treated patients
Rate of recovery under treatment of active TB
Rate of recovery under treatment of L1
Rate of recovery under treatment of L2
Total population
Total simulation duration
Efficacy of treatment of early latent L1
Efficacy of treatment of persistent latent L2
Value
100
1/70 yr −1
12 yr −1
0.05
0.0002 yr −1
0.00002 yr −1
0.25
0.25
2 yr −1
2 yr −1
1 yr −1
30, 000
5 yr
0.5
0.5
Tabelle: Parameter values.
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Optimal controls of the non-delayed control problems
Adjoint variable
λ = (λS , λL1 , λI , λL2 )
Switching functions φk (t) = Huk [t] = −Wk − k λLk (t) Lk (t) (k = 1, 2).
Control law for Maximum Principle:
1 ,
if φk (t) > 0
uk (t) =
, k = 1, 2.
0 ,
if φk (t) < 0
control u1 and switching function φ1
2
u1
φ1
1.5
control u2 and switching function φ2
3
2.5
u2
φ2
2
1
1.5
0.5
0.5
1
0
0
-0.5
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
Weights W1 = W2 = 50 : Optimal controls are bang-bang.
SSC are satisfied, since SSC hold for the switching time
optimization problem and the strict bang-bang property holds.
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
State Variables in the Non-Delayed Control Problem
Optimal state variables for weights W1 = W2 = 50 :
susceptible S and recovered R
infectious I
1600
1400
1200
1000
800
600
400
200
0
S
R
35000
30000
25000
20000
15000
10000
5000
0
0
1
2
3
4
5
0
1
early latent L1
9000
8000
7000
6000
5000
4000
3000
2000
1000
0
3
4
5
4
5
14000
12000
10000
8000
6000
4000
2000
0
0
Helmut Maurer
2
persistent latent L2
1
2
3
4
5
0
1
2
3
Theory and Applications of Optimal Control Problems with Tim
Optimal controls : comparison of L1 and L2 objectives
Objectives:
J1 (x, u) =
RT
J2 (x, u) =
RT
0
0
( I (t) + L2 (t) + W1 u1 (t) + W2 u2 (t) ) dt,
( I (t) + L2 (t) + W1 u1 (t)2 + W2 u2 (t)2 ) dt
L2 functional : optimal controls are continuous.
control u1 for J1 and J2 objective
1.4
1.2
1
0.8
0.6
0.4
0.2
0
J1
J2
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
control u2 for J1 and J2 objective
1.4
1.2
1
0.8
0.6
0.4
0.2
0
J1
J2
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
Weights W1 = W2 = 50
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Optimal controls for delays dI = 0.1, du1 = du2 = 0.2
Switching functions for k = 1, 2
)
(
−Wk − k λL1 (t + duk )L1 (t + duk ) for 0 ≤ t ≤ T − duk
φk (t) =
−Wk
for T − duk ≤ t ≤ T
Optimal controls u1 and u2 are bang-bang with one switch.
The strict bang-bang property holds.
control u1 and (scaled) switching function φ
1.2
u1
1
φ1
0.8
0.6
0.4
0.2
0
-0.2
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
susceptibles S and recovered R
0
1
control u2 and (scaled) switching function φ
3
2.5
2
1.5
1
0.5
0
-0.5
u2
φ2
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
Helmut Maurer
2
3
infectious individuals I
1600
1400
1200
1000
800
600
400
200
0
S
R
35000
30000
25000
20000
15000
10000
5000
0
4
5
0
1
early latent L1
2
3
4
5
4
5
early latent L2
9000
8000
7000
6000
5000
4000
3000
2000
1000
0
14000
12000
10000
8000
6000
4000
2000
0
0
1
2
3
4
5
0
1
2
3
Theory and Applications of Optimal Control Problems with Tim
Sensitivity of optimal control w.r.t. parameter β
control u1
1.4
1.2
1
0.8
0.6
0.4
0.2
0
β=50
β=150
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
control u2
1.4
1.2
1
0.8
0.6
0.4
0.2
0
β=150
β=150
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
Comparison of extremal controls for β = 50 and β = 150 :
L1 objective, W1 = W2 = 150, delays dI = 0.1, du1 = du2 = 0.2.
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim
Trahison des Images ?
Ceci n’était pas une conférence.
Merci pour votre attention !
Helmut Maurer
Theory and Applications of Optimal Control Problems with Tim