Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Theory and Applications of Constrained Optimal
Control Problems with Delays
PART 1 : Mixed Control–State Constraints
Helmut Maurer1 , Laurenz Göllmann
1 Institut
2 Department
2
für Numerische und Angewandte Mathematik,
Universität Münster, Germany
of Mechanical Engineering, University of Applied Sciences, Steinfurt,
Germany
ANOC Spring School and Workshop,
ENSTA Paris, April 23–27, 2012
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Overview of Part 1
Constrained Optimal Control Problems with Delays
Discretization and NLP methods (”Discretize and Optimize”)
Illustrative Example
Minimum Principle for Delayed Optimal Control Problems
with Mixed Control–State Constraints
Optimal Exploitation of a Renewable Resource with
Investment Delay
Optimal Control of Continuous Stirred Tank Reactors (CSTR)
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Constrained Optimal Control Problems with Delays
State variable x ∈ Rn , Control variable u ∈ Rm .
Dynamics and Boundary Conditions
ẋ(t) = f (t, x(t), x(t − dx ), u(t), u(t − du )), a.e. t ∈ [0, tf ],
x(t) = x0 (t),
t ∈ [−dx , 0)
(state delay dx ≥ 0 ),
u(t) = u0 (t),
t ∈ [−du , 0)
(control delay du ≥ 0 ),
ψ(x(tf )) = 0
Mixed control-state constraints and pure state constraints
C (x(t), u(t)) ≤ 0,
S(x(t)) ≤ 0,
t ∈ [0, tf ]
Minimize
Z
tf
f0 (t, x(t), x(t − dx ), u(t), u(t − du )) dt
J(u, x) = g (x(tf )) +
0
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Discretization and NLP
For simplicity consider a MAYER-type problem with cost functional
J(u, x) = g (x(tf ))
Assume: there exists a stepsize h > 0 and integers k, l, N ∈ N with
dx = k · h,
du = l · h,
tf = N · h .
For simplicity: EULER discretization at grid points
ti := i · h,
i = 0, 1, . . . , N .
Approximation at grid points:
u(ti ) ≈ ui ∈ Rm (i = 0, . . . , N),
Helmut Maurer, Laurenz Göllmann
x(ti ) ≈ xi ∈ Rn (i = 0, . . . , N)
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Discretization and NLP
For simplicity consider a MAYER-type problem with cost functional
J(u, x) = g (x(tf ))
Assume: there exists a stepsize h > 0 and integers k, l, N ∈ N with
dx = k · h,
du = l · h,
tf = N · h .
For simplicity: EULER discretization at grid points
ti := i · h,
i = 0, 1, . . . , N .
Approximation at grid points:
u(ti ) ≈ ui ∈ Rm (i = 0, . . . , N),
Helmut Maurer, Laurenz Göllmann
x(ti ) ≈ xi ∈ Rn (i = 0, . . . , N)
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Discretization and NLP
For simplicity consider a MAYER-type problem with cost functional
J(u, x) = g (x(tf ))
Assume: there exists a stepsize h > 0 and integers k, l, N ∈ N with
dx = k · h,
du = l · h,
tf = N · h .
For simplicity: EULER discretization at grid points
ti := i · h,
i = 0, 1, . . . , N .
Approximation at grid points:
u(ti ) ≈ ui ∈ Rm (i = 0, . . . , N),
Helmut Maurer, Laurenz Göllmann
x(ti ) ≈ xi ∈ Rn (i = 0, . . . , N)
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Discretization and NLP
For simplicity consider a MAYER-type problem with cost functional
J(u, x) = g (x(tf ))
Assume: there exists a stepsize h > 0 and integers k, l, N ∈ N with
dx = k · h,
du = l · h,
tf = N · h .
For simplicity: EULER discretization at grid points
ti := i · h,
i = 0, 1, . . . , N .
Approximation at grid points:
u(ti ) ≈ ui ∈ Rm (i = 0, . . . , N),
Helmut Maurer, Laurenz Göllmann
x(ti ) ≈ xi ∈ Rn (i = 0, . . . , N)
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Large-scale NLP Problem
Minimize
J(u, x) = g (xN )
subject to
xi+1 − xi − h · f (ti , xi , xi−k , ui , ui−l ) = 0,
i = 0, .., N − 1,
ψ(xN ) = 0,
x−i := x0 (−ih)
(i = 0, .., k),
C (xi , ui ) ≤ 0,
i = 0, .., N,
S(xi ) ≤ 0,
i = 0, .., N,
u−i := u0 (−ih)
(i = 1, .., l).
Optimization Variable:
z := (u0 , x1 , u1 , x2 , ..., uN−1 , xN ) ∈ RN(m+n)
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Large-scale NLP Problem
Minimize
J(u, x) = g (xN )
subject to
xi+1 − xi − h · f (ti , xi , xi−k , ui , ui−l ) = 0,
i = 0, .., N − 1,
ψ(xN ) = 0,
x−i := x0 (−ih)
(i = 0, .., k),
C (xi , ui ) ≤ 0,
i = 0, .., N,
S(xi ) ≤ 0,
i = 0, .., N,
u−i := u0 (−ih)
(i = 1, .., l).
Optimization Variable:
z := (u0 , x1 , u1 , x2 , ..., uN−1 , xN ) ∈ RN(m+n)
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Application of NLP-Solvers
AMPL : Programming language (Fourer, Gay, Kernighan)
IPOPT: Interior point method (A. Wächter et al.)
LOQO: Interior point method (B. Vanderbei et al.
WORHP : SQP–method (C. Büskens, M. Gerdts)
Other NLP solvers embedded in AMPL : cf. NEOS server
Special feature: solvers provide LAGRANGE-multipliers
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
A non-convex control problem with state delays
state x(t) ∈ R, control u(t) ∈ R, delay d ≥ 0
Dynamics and Boundary Conditions
ẋ(t) = x(t − d)2 − u(t),
x(t) = x0 (t) = 1,
t ∈ [0, 2],
t ∈ [−d, 0],
x(2) = 1
Control and State Constraints
x(t) ≥ α,
i.e., S(x(t)) = −x(t) + α ≤ 0,
t ∈ [0, 2]
Minimize
Z
2
J(u, x) =
(x(t)2 + u(t)2 ) dt
0
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Optimal solutions without state constraints
Z
2
min
(x(t)2 +u(t)2 ) dt
s.t. ẋ(t) = x(t − d)2 −u(t), x0 (t) ≡ 1, x(2) = 1
0
optimal state and control for
delays
d = 0.0,
d = 0.1,
d = 0.2,
state x
1
0.95
0.9
0.85
0.8
0.75
0.7
0.65
0.6
0.55
0.5
d = 0.5,
control u
2.5
d=0
d=0.1
d=0.2
d=0.5
2
1.5
d=0
d=0.1
d=0.2
d=0.5
1
0.5
0
-0.5
0
0.5
1
1.5
2
Helmut Maurer, Laurenz Göllmann
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Optimal solutions with state constraint x(t) ≥ α = 0.7
Optimal state and control for
delays
d = 0.0,
d = 0.1,
d = 0.2,
state x
1
control u
2.5
d=0
d=0.1
d=0.2
d=0.5
0.95
0.9
d = 0.5
2
1.5
d=0
d=0.1
d=0.2
d=0.5
0.85
1
0.8
0.5
0.75
0
0.7
0.65
-0.5
0
0.5
1
1.5
2
Helmut Maurer, Laurenz Göllmann
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Minimum Principle
Augmented Hamiltonian: y (t) = x(t − d)
H(x, y , λ, η, u) = u 2 + x 2 + λ(y 2 − u) + η(−x + α)
Adjoint equation
λ̇(t) = −Hx (t) − χ[0,2−d] Hy (t + d)
−2x(t) − 2λ(t + d)x(t) + η(t) , 0 ≤ t ≤ 2 − d
=
−2x(t) + η(t) ,
2−d ≤t ≤2
Minimum condition
Hu (t) = 0
Helmut Maurer, Laurenz Göllmann
⇒ u(t) = λ(t)/2
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Boundary arc x(t) = α = 0.7 for t1 ≤ t ≤ t2
x(t) ≡ α
⇒ x(t − d)2 = u(t) = λ(t)/2
⇒ ẋ(t) = 0
Computation of multiplier η(t) by differentiation
η(t) = 2(2x(t −d)(x(t −2d)2 −λ(t −d)/2)+x(t)+λ(t +d)x(t))
delays d = 0.0,
d = 0.1,
d = 0.2,
state x
1
0.9
d = 1.0
3
d=0
d=0.1
d=0.2
d=0.5
0.95
d = 0.5,
multiplier for state constraint x(t) >= alpha
2.5
2
0.85
1.5
0.8
1
0.75
0.5
0.7
0.65
0
0
0.5
1
1.5
2
Helmut Maurer, Laurenz Göllmann
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Optimal Control Problems with Control-State Constraints
State variable x ∈ Rn , Control variable u ∈ Rm .
Dynamics and Boundary Conditions
ẋ(t) = f (t, x(t), x(t − dx ), u(t), u(t − du )), a.e. t ∈ [0, tf ],
x(t) = x0 (t),
t ∈ [−dx , 0)
(state delay dx ≥ 0 ),
u(t) = u0 (t),
t ∈ [−du , 0)
(control delay du ≥ 0 ),
ψ(x(tf )) = 0
Mixed control-state constraints and pure state constraints
C (x(t), u(t)) ≤ 0,
t ∈ [0, tf ]
Minimize
Z
tf
f0 (t, x(t), x(t − dx ), u(t), u(t − du )) dt
J(u, x) = g (x(tf )) +
0
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
References for Minimum/Maximum Principle
State delays and pure control constraints:
Kharatishvili (1961), Oguztöreli (1966), Banks (1968),
Halanay (1968), Soliman, Ray (1970), Colonius, Hinrichsen (1978),
Clarke, Wolenski (1991), Dadebo, Luus (1992),
Mordukhovich, Wang (2003–)
State delays and pure state constraints:
Angell, Kirsch (1990)
State and control delays and mixed control–state constraints:
L. Göllmann, D. Kern, H. Maurer. Optimal control problems with
delays in state and control and mixed control-state constraints.
Optimal Control Applications and Methods 30, 341–365 (2009).
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Hamiltonian for mixed control-state constraints
(Augmented) Hamiltonian or Pontryagin Function
H(t, x, y , u, v , λ, µ) := λ0 f0 (t, x, y , u, v ) + λf (t, x, y , u, v )
+ µC (t, x, u)
y represents delayed state
v represents delayed control
λ ∈ Rn , λ0 ∈ R adjoint (costate) variables
µ ∈ Rk multiplier for control-state constraint C (x, u) ≤ 0
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Pontryagin (Type) Minimum Principle
Let (u, x) ∈ L∞ ([0, tf ], Rm ) × W 1,∞ ([0, tf ], Rn ) be a
locally optimal pair of functions.
Regularity assumption:
rank
∂Cj (x(t),u(t))
∂u
j∈J0 (t)
= # J0 (t) ,
J0 (t) := { j ∈ {1, .., p} | Cj (x(t), u(t)) = 0 }
Then there exist
an adjoint function λ ∈ W 1,∞ ([0, tf ], Rn ) and λ0 ≥ 0 ,
a multiplier function µ ∈ L∞ ([0, tf ], Rk ),
and a multiplier ρ ∈ Rq
such that the following conditions are satisfied for a.a. t ∈ [0, tf ] :
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Pontryagin Type Minimum Principle
(i) Advanced adjoint ODE and transversality condition:
λ̇(t)
= −Hx (t) − χ[0,tf −dx ] (t)Hy (t + dx ) ,
λ(T ) = (λ0 g + ρψ)x (x(tf )) ,
where Hx (t) and Hy (t + dx ) denote evaluations along the optimal
trajectory and χ[0,t−dx ] is the characteristic function.
(ii) Local Minimum Condition for augmented Hamiltonian:
0 = Hu (t) + χ[0,T −du ] (t)Hv (t + du )
(iii) Non-negativity of multiplier and complementarity condition:
µ(t) ≥ 0
and µi (t) Ci (x(t), u(t)) = 0,
Helmut Maurer, Laurenz Göllmann
i = 1, . . . , k.
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Pontryagin Type Minimum Principle
(i) Advanced adjoint ODE and transversality condition:
λ̇(t)
= −Hx (t) − χ[0,tf −dx ] (t)Hy (t + dx ) ,
λ(T ) = (λ0 g + ρψ)x (x(tf )) ,
where Hx (t) and Hy (t + dx ) denote evaluations along the optimal
trajectory and χ[0,t−dx ] is the characteristic function.
(ii) Local Minimum Condition for augmented Hamiltonian:
0 = Hu (t) + χ[0,T −du ] (t)Hv (t + du )
(iii) Non-negativity of multiplier and complementarity condition:
µ(t) ≥ 0
and µi (t) Ci (x(t), u(t)) = 0,
Helmut Maurer, Laurenz Göllmann
i = 1, . . . , k.
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Pontryagin Type Minimum Principle
(i) Advanced adjoint ODE and transversality condition:
λ̇(t)
= −Hx (t) − χ[0,tf −dx ] (t)Hy (t + dx ) ,
λ(T ) = (λ0 g + ρψ)x (x(tf )) ,
where Hx (t) and Hy (t + dx ) denote evaluations along the optimal
trajectory and χ[0,t−dx ] is the characteristic function.
(ii) Local Minimum Condition for augmented Hamiltonian:
0 = Hu (t) + χ[0,T −du ] (t)Hv (t + du )
(iii) Non-negativity of multiplier and complementarity condition:
µ(t) ≥ 0
and µi (t) Ci (x(t), u(t)) = 0,
Helmut Maurer, Laurenz Göllmann
i = 1, . . . , k.
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Proof ideas: ”delay chain arguments”
Apply Kuhn–Tucker conditions for optimization problems in
Banach–spaces.
Simple proof: Rationality assumption for delays:
k
dx
= ∈ Q with
du
l
k, l ∈ N
coprime integers .
This implies
dx = k · hmax ,
du = l · hmax ,
hmax :=
du
l
maximal stepsize
Then augment the control system by the number of intervals
of length hmax in [0, tf ] and use continuity of state variables.
Apply the standard Minimum Principle to the augmented
control system and translate the conditions back to the
retarded control problem.
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Optimal Fishing, Clark, Clarke, Munro
Colin W. Clark, Frank H. Clarke, Gordon R. Munro:
The optimal exploitation of renewable resource stock: problem of
irreversible investment, Econometric 47, pp. 25–47 (1979).
State variables and control variables:
x(t)
:
population biomass at time t ∈ [0, tf ] ,
renewable resource, e.g., fish,
K (t)
:
amount of capital invested in the fishery,
e.g., number of ”standardized” fishing vessels available,
E (t)
:
fishing effort (control), h(t) = E (t)x(t) is harvest rate ,
I (t)
:
investment rate (control),
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Optimal Fishing
Dynamics in [0, tf ] ( parameters a = 1, b = 5, γ = 0.03 )
ẋ(t) = a · x(t) · (1 − x(t)/b) − E (t) · x(t) ,
K̇ (t) = I (t − d) − γ · K (t) ,
x(0) = x0 ,
K (0) = K0 .
Mixed Control-State Constraint and Control Constraint
0 ≤ E (t) ≤ K (t) ,
0 ≤ I (t) ≤ Imax ,
t ∈ [0, tf ],
Maximize benefit ( parameters r = 0.05, cE = 2, cI = 1.1 )
Z
tf
exp(−r · t)( p · E (t) · x(t) − cE · E (t) − cI · I (t) ) dt
J(u, x) =
0
Parameters in Clark et al.: Imax = ∞ , γ = 0 , d = 0.
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Optimal Fishing : Necessary Conditions
Controls E and I appear linearly in the Hamiltonian.
Switching functions
σE (t) = exp(−r t)(px(t) − cE ) − λx x,
− exp(−r t)cI + λK (t + d) , 0 ≤ t < tf − d
σI (t) =
− exp(−r t)cI
, tf − d ≤ t ≤ tf
Optimal controls maximizing the Hamiltonian


for σE (t) > 0
 K (t)

0
for σE (t) < 0
E (t) =


singular for σE (t) = 0 on Is ⊂ [0, tf ]
Imax for σI (t) > 0
I (t) =
0
for σI (t) < 0
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Optimal Fishing: x0 = 1.5, K0 = 0.1, Imax = 0.1
fishing rate E, investment I, capital K
E
I
K
1
E, K, I
0.8
0.6
0.4
0.2
0
0
2
4
6
time t (years)
Helmut Maurer, Laurenz Göllmann
8
10
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Optimal Fishing: x0 = 1.5, K0 = 0.1, Imax = 0.1
Switching functions σE (t) and σI (t)
Investment I and switching function σI
I
σI
0.3
0.2
0.1
I, σI
E, K, σE
rate E, capital K, switching function σE
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
-0.1
-0.2
-0.3
0
-0.1
-0.2
0
2
4
6
time t (years)
8
10
Helmut Maurer, Laurenz Göllmann
0
1
2
3
4 5 6 7
time t (years)
8
9 10
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Optimal Fishing: d = 0.5, x0 = 1.5, K0 = 0.1, Imax = 0.1
fishing rate E, investment I, capital K
E
I
K
1
E, K, I
0.8
0.6
0.4
0.2
0
0
2
4
6
time t (years)
Helmut Maurer, Laurenz Göllmann
8
10
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Optimal Fishing: d = 0.5, x0 = 1.5, K0 = 0.1, Imax = 0.1
Switching functions σE (t) and σI (t)
I , σI
0
2
4
6
time t (years)
8
10
Helmut Maurer, Laurenz Göllmann
Investment I and function σI
E, K, σE
rate E, capital K, switching function σE
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
-0.1
-0.2
0.25
0.2
0.15
0.1
0.05
0
-0.05
-0.1
0
1
2
3
4 5 6 7
time t (years)
8
9 10
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Optimal Fishing: x0 = 1.5, K0 = 0.5, Imax = 0.1
fishing rate E, investment I, capital K
E
I
K
1
E, K, I
0.8
0.6
0.4
0.2
0
0
2
4
6
time t (years)
Helmut Maurer, Laurenz Göllmann
8
10
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Optimal Fishing: d = 0.5, x0 = 1.5, K0 = 0.5, Imax = 0.1
fishing rate E, investment I, capital K
E
I
K
1
E, K, I
0.8
0.6
0.4
0.2
0
0
2
4
6
time t (years)
Helmut Maurer, Laurenz Göllmann
8
10
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Optimal Fishing: d = 0.5, Impulsive control Imax = 2
fishing rate E, investment I, capital K
3
E
I
K
2.5
E, K, I
2
1.5
1
0.5
0
0
2
4
6
time t (years)
Helmut Maurer, Laurenz Göllmann
8
10
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Continuous Stirred Chemical Tank Reactor (CSTR)
Seinfeld S.H.: Optimal control of a continuous stirred tank
reactor with transportation lag, Intern. Journal of Control 10,
29–39 (1969).
Ray W.H. and Soliman M.A.: The optimal control of
processes containing pure time delays – I, necessary conditions for
an optimum, Chemical Engineering Science 25, 1911–1925 (1970).
Soliman M.A. and Ray W.H.: Optimal control of multivariable
systems with pure time delays, Automatica 7, 681–689 (1971).
Dadebo S. and Luus R. Optimal control of time-delay systems
by dynamic programming, Optimal Control Applications and
Methods 13, pp. 29–41 (1992).
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Continuous Stirred Chemical Tank Reactor (CSTR)
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
CSTR Model
State variables: (deviations from a desired final state)
x1 (t) : concentration of product
x2 (t) : concentration of catalyst,
x3 (t) : temperature inside vessel
Control variables:
u1 (t)
u2 (t)
:
:
temperature control by heat exchanger
reactant inlet valve,
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
CSTR Model
System Dynamics for t ∈ [0, 0.2] :
ẋ1 (t) = −x1 (t) − R(x1 (t), x2 (t), x3 (t)),
ẋ2 (t) = −x2 (t) + 0.9u2 (t − du ) + 0.1u2 (t),
ẋ3 (t) = −2x3 (t) + 0.25R(t) − 1.05u1 (t)x3 (t − dx ),
25x3
R(x1 , x2 , x3 ) := (1 + x1 )(1 + x2 ) exp
.
1 + x3
Boundary conditions and control bounds:
x3 (t) = −0.02,
u2 (t) = 1,
t ∈ [−dx , 0),
dx = 0.015
t ∈ [−du , 0),
du = 0.02
∗
x(0) = (0.49, −0.0002, −0.02) ,
x(0.2) = (0, 0, 0)∗ ,
|u1 (t)| ≤ 500,
Helmut Maurer, Laurenz Göllmann
t ∈ [0, 0.2].
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
CSTR Model
System Dynamics for t ∈ [0, 0.2] :
ẋ1 (t) = −x1 (t) − R(x1 (t), x2 (t), x3 (t)),
ẋ2 (t) = −x2 (t) + 0.9u2 (t − du ) + 0.1u2 (t),
ẋ3 (t) = −2x3 (t) + 0.25R(t) − 1.05u1 (t)x3 (t − dx ),
25x3
R(x1 , x2 , x3 ) := (1 + x1 )(1 + x2 ) exp
.
1 + x3
Boundary conditions and control bounds:
x3 (t) = −0.02,
u2 (t) = 1,
t ∈ [−dx , 0),
dx = 0.015
t ∈ [−du , 0),
du = 0.02
∗
x(0) = (0.49, −0.0002, −0.02) ,
x(0.2) = (0, 0, 0)∗ ,
|u1 (t)| ≤ 500,
Helmut Maurer, Laurenz Göllmann
t ∈ [0, 0.2].
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
CSTR Model
Objective functional: quadratic in control u2 :
Z
Minimize J(u, x) =
0.2
(kx(t)k22 + 0.01u2 (t)2 )dt
0
Numerical solution: discretize and optimize
Delays: dx = 0.015, du = 0.02
EULER discretization: N = 16, 000 meshpoints
Performance Index computed: J(x, u) = 0.1197054
CPU Time: 63, 932 CPU seconds (!)
Solver used: IPOPT with AMPL
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
CSTR Model
Objective functional: quadratic in control u2 :
Z
Minimize J(u, x) =
0.2
(kx(t)k22 + 0.01u2 (t)2 )dt
0
Numerical solution: discretize and optimize
Delays: dx = 0.015, du = 0.02
EULER discretization: N = 16, 000 meshpoints
Performance Index computed: J(x, u) = 0.1197054
CPU Time: 63, 932 CPU seconds (!)
Solver used: IPOPT with AMPL
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
CSTR: Numerical Results - State and Adjoint Variables
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
CSTR: Numerical Results - State and Adjoint Variables
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
CSTR: Numerical Results - Control
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
CSTR: Numerical Results - Control
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
CSTR: Numerical Results - Control
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
CSTR: Numerical Results - Control
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Continuous Stirred Chemical Tank Reactor (CSTR)
Seinfeld S.H.: Optimal control of a continuous stirred tank
reactor with transportation lag, Intern. Journal of Control 10,
29–39 (1969).
Dadebo S. and Luus R. Optimal control of time-delay systems
by dynamic programming, Optimal Control Applications and
Methods 13, pp. 29–41 (1992).
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
CSTR (Seinfeld): Delay in mixed control-state constraint
normalized concentration of product, 0 ≤ t ≤ 4,
normalized temperature,
temperature control
25x2
Dynamics: Reaction term R(x2 ) = exp 1+x
:
2
ẋ1 (t) = −x1 (t) − (1 + x1 (t))k R(x2 (t)) − 1 , k = 1, 2,
ẋ2 (t) = −2x2 (t) + 0.25 (1 + x1 (t))k R(x2 (t)) − 1
x1 (t)
x2 (t)
u(t)
:
:
:
−u(t)x2 (t − dx )(x2 (t) + 0.125).
Mixed control–state constraint with state delay:
−1 ≤ u(t)x2 (t − 0.1) ≤ 1
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
CSTR (Seinfeld): Delay in mixed control-state constraint
normalized concentration of product, 0 ≤ t ≤ 4,
normalized temperature,
temperature control
25x2
Dynamics: Reaction term R(x2 ) = exp 1+x
:
2
ẋ1 (t) = −x1 (t) − (1 + x1 (t))k R(x2 (t)) − 1 , k = 1, 2,
ẋ2 (t) = −2x2 (t) + 0.25 (1 + x1 (t))k R(x2 (t)) − 1
x1 (t)
x2 (t)
u(t)
:
:
:
−u(t)x2 (t − dx )(x2 (t) + 0.125).
Mixed control–state constraint with state delay:
−1 ≤ u(t)x2 (t − 0.1) ≤ 1
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
CSTR : Delay in mixed control-state constraint
Initial conditions: Delay dx = 0.1
x1 (0) = −0.82298,
x2 (t) =
0.10286,
t ∈ [−0.1, 0],
Cost functional: Minimize
Z4
J(x, u) =
x1 (t)2 dt
0
Switching function : σ(t) = λ2 (t)x2 (t − dx )(x2 (t) + 0.125) .
−sign (σ(t)), if σ(t) 6= 0
u(t)x2 (t − dx ) =
singular,
if σ(t) = 0
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
CSTR : Numerical solution
x1
x2
0
-0.1
-0.2
-0.3
-0.4
-0.5
-0.6
-0.7
-0.8
-0.9
0.12
0.1
0.08
0.06
0.04
0.02
0
-0.02
0
0.5
1
1.5
2
2.5
3
3.5
4
0
0.5
1
1.5
2
2.5
3
3.5
4
3.5
4
constraint u(t)*x2(t-d)
u
100
50
0
-50
-100
-150
-200
-250
-300
-350
1
0.5
0
-0.5
-1
0
0.5
1
1.5
2
2.5
3
3.5
4
0
0.5
1
1.5
2
2.5
3
Constraint u(t)x2 (t − 0.1) is bang–singular !
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Two Stage CSTR
Dadebo S. and Luus R. Optimal control of time-delay systems
by dynamic programming, Optimal Control Applications and
Methods 13, pp. 29–41 (1992).
A chemical reaction A ⇒ B is processed in two tanks.
State and control variables:
Tank 1 :
Tank 2 :
x1 (t)
:
(scaled) concentration
x2 (t)
:
(scaled) temperature
u1 (t)
:
temperature control
x3 (t)
:
(scaled) concentration
x4 (t)
:
(scaled) temperature
u2 (t)
:
temperature control
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Dynamics of the Two-Stage CSTR
Reaction term in Tank 1
:
R1 (x1 , x2 ) = (x1 + 0.5) exp
Reaction term in Tank 2
:
R2 (x3 , x4 ) = (x3 + 0.25) exp
25x2
1+x2
25x4
1+x4
Dynamics:
ẋ1 (t) = −0.5 − x1 (t) − R1 (t),
ẋ2 (t) = −(x2 (t) + 0.25) − u1 (t)(x2 (t) + 0.25) + R1 (t),
ẋ3 (t) = x1 (t − d) − x3 (t) − R2 (t) + 0.25,
ẋ4 (t) = x2 (t − d) − 2x4 (t) − u2 (t)(x4 (t) + 0.25) + R2 (t) − 0.25.
Initial conditions:
x1 (t) = 0.15, x2 (t) = −0.03, −d ≤ t ≤ 0,
x3 (0) = 0.1,
x4 (0) = 0.
Delays d = 0.1, d = 0.2, d = 0.4 .
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Optimal control problem for the Two-Stage CSTR
Minimize
Rtf
( x12 + x22 + x32 + x42 + 0.1u12 + 0.1u22 ) dt
(tf = 2) .
0
Hamiltonian with yk (t) = xk (t − d), k = 1, 2 :
H(x, y , λ, u) = f0 (x, u) + λ1 ẋ1
+λ2 (−(x2 + 0.25) − u1 (x2 + 0.25) + R1 (x1 , x2 ) )
+λ3 (y1 − x3 − R2 (x3 , x4 ) + 0.25)
+λ4 (y2 − 2x4 − u2 (x4 + 0.25) + R2 (x3 , x4 ) + 0.25)
Adjoint equations:
λ̇1 (t) = −Hx1 (t) − χ [ 0,tf −d ] λ3 (t + d),
λ̇2 (t) = −Hx2 (t) − χ [ 0,tf −d ] λ4 (t + d),
λ̇k (t) = −Hxk (t) (k = 3, 4).
The minimum condition yields Hu = 0 and thus
u1 = 5λ2 (x2 + 0.25),
Helmut Maurer, Laurenz Göllmann
u2 = 5λ4 (x4 + 0.25).
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Two-Stage CSTR with free x(tf ) : x1 , x2 , x3 , x4
concentration x1
0.16
0.14
0.12
0.1
0.08
0.06
0.04
0.02
0
-0.02
temperature x2
d=0.1
d=0.2
d=0.4
d=0.1
d=0.2
d=0.4
0.02
0.01
0
-0.01
-0.02
-0.03
0
0.5
1
1.5
2
0
0.5
concentration x3
0.1
0.06
0.04
0.02
0
-0.02
-0.04
0
0.5
1
1.5
2
temperature x4
0.045
0.04
0.035
0.03
0.025
0.02
0.015
0.01
0.005
0
d=0.1
d=0.2
d=0.4
0.08
1
1.5
Delays
2
d=0.1
d=0.2
d=0.4
0
0.5
1
1.5
2
d = 0.1, d = 0.2, d = 0.4.
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Two-Stage CSTR with free x(tf ) : u1 , u2 , λ1 , λ2
control u1
0.2
control u2
0.3
d=0.1
d=0.2
d=0.4
0.1
d=0.1
d=0.2
d=0.4
0.25
0
0.2
-0.1
0.15
-0.2
0.1
-0.3
0.05
-0.4
0
0
0.2 0.4 0.6 0.8
1
1.2 1.4 1.6 1.8
2
0
adjoint variable λ1
0.16
0.14
0.12
0.1
0.08
0.06
0.04
0.02
0
-0.02
0.2 0.4 0.6 0.8
1
1.2 1.4 1.6 1.8
2
adjoint variable λ2
d=0.1
d=0.2
d=0.4
d=0.1
d=0.2
d=0.4
0.1
0
-0.1
-0.2
-0.3
-0.4
0
0.2 0.4 0.6 0.8
1
1.2 1.4 1.6 1.8
Delays
2
0
0.2 0.4 0.6 0.8
1
1.2 1.4 1.6 1.8
2
d = 0.1, d = 0.2, d = 0.4.
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Two-Stage CSTR with x(tf ) = 0 : x1 , x2 , x3 , x4
concentration x1
0.16
temperature x2
d=0.1
d=0.2
d=0.4
0.14
0.12
d=0.1
d=0.2
d=0.4
0.02
0.01
0.1
0
0.08
0.06
-0.01
0.04
-0.02
0.02
0
-0.03
0
0.5
1
1.5
2
0
0.5
concentration x3
0.1
1.5
2
temperature x4
0.03
d=0.1
d=0.2
d=0.4
0.08
1
d=0.1
d=0.2
d=0.4
0.025
0.06
0.02
0.04
0.015
0.02
0.01
0
0.005
-0.02
0
0
0.5
1
1.5
Delays
2
0
0.5
1
1.5
2
d = 0.1, d = 0.2, d = 0.4.
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Two-Stage CSTR with x(tf ) = 0 : u1 , u2 , λ1 , λ2
control u1
control u2
0.3
d=0.1
d=0.2
d=0.4
0.2
0.1
d=0.1
d=0.2
d=0.4
0.25
0.2
0
0.15
-0.1
-0.2
0.1
-0.3
0.05
-0.4
0
0
0.2 0.4 0.6 0.8
1
1.2 1.4 1.6 1.8
2
0
adjoint variable λ1
0.18
1
1.2 1.4 1.6 1.8
2
adjoint variable λ2
0.2
d=0.1
d=0.2
d=0.4
0.16
0.2 0.4 0.6 0.8
d=0.1
d=0.2
d=0.4
0.1
0.14
0
0.12
-0.1
0.1
0.08
-0.2
0.06
-0.3
0.04
-0.4
0
0.2 0.4 0.6 0.8
1
1.2 1.4 1.6 1.8
Delays
2
0
0.2 0.4 0.6 0.8
1
1.2 1.4 1.6 1.8
2
d = 0.1, d = 0.2, d = 0.4.
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble
Example: Optimal Exploitation of Renewable Resources
Optimal Control of a Chemical Tank Reactor (CSTR)
Two-Stage CSTR with x(tf ) = 0 and x4 (t) ≤ 0.01
temperature x4
control u1
d=0.1
d=0.2
d=0.4
0.014
0.012
d=0.1
d=0.2
d=0.4
0.3
0.2
0.1
0.01
0
0.008
-0.1
0.006
-0.2
0.004
-0.3
0.002
-0.4
0
-0.5
0
0.5
1
1.5
2
0
0.2 0.4 0.6 0.8
multiplier µ for x4 <= 0.01
0.7
1.2 1.4 1.6 1.8
2
control u2
0.3
d=0.1
d=0.2
d=0.4
0.6
1
d=0.1
d=0.2
d=0.4
0.25
0.5
0.4
0.2
0.3
0.15
0.2
0.1
0.1
0
0.05
0
0.5
1
1.5
Delays
2
0
0.2 0.4 0.6 0.8
1
1.2 1.4 1.6 1.8
2
d = 0.1, d = 0.2, d = 0.4.
Helmut Maurer, Laurenz Göllmann
Theory and Applications of Constrained Optimal Control Proble