Numerical Optimal Control
– Part 2: Discretization techniques, structure exploitation,
calculation of gradients –
SADCO Summer School and Workshop on Optimal and Model Predictive Control
– OMPC 2013, Bayreuth –
Matthias Gerdts
Institute of Mathematics and Applied Computing
Department of Aerospace Engineering
Universität der Bundeswehr München (UniBw M)
[email protected]
http://www.unibw.de/lrt1/gerdts
Fotos: http://de.wikipedia.org/wiki/München
Magnus Manske (Panorama), Luidger (Theatinerkirche), Kurmis (Chin. Turm), Arad Mojtahedi (Olympiapark), Max-k (Deutsches Museum), Oliver Raupach (Friedensengel), Andreas Praefcke (Nationaltheater)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Schedule and Contents
Time
9:00 - 10:30
Topic
Introduction, overview, examples, indirect method
10:30 - 11:00
Coffee break
11:00 - 12:30
Discretization techniques, structure exploitation, calculation of gradients, extensions: sensitivity analysis, mixedinteger optimal control
12:30 - 14:00
Lunch break
14:00 - 15:30
Function space methods: Gradient and Newton type
methods
15:30 - 16:00
Coffee break
16:00 - 17:30
Numerical experiments
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Indirect, Direct, and Function Space Methods
Optimal Control Problem
Indirect method
I
I
I
based on necessary
optimality conditions
(minimum principle)
Direct discretization method
I
leads to a boundary
value problem (BVP)
I
BVP needs to be
solved numerically by,
e.g., multiple shooting
methods
I
based on
discretization, e.g.
collocation or direct
shooting
leads to finite
dimensional
optimization problem
(NLP)
NLP needs to be
solved numerically by,
e.g., SQP
Function space methods
I
work in Banach spaces
I
generalizations of
gradient method,
Lagrange-Newton
method, SQP
method,...
I
discretization only at
subproblem level
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Contents
Direct Discretization Methods
Full Discretization
Reduced Discretization and Shooting Methods
Adjoint Estimation
Convergence
Examples
Mixed-Integer Optimal Control
Variable Time Transformation
Parametric Sensitivity Analysis and Realtime-Optimization
Parametric Optimal Control Problems
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Contents
Direct Discretization Methods
Full Discretization
Reduced Discretization and Shooting Methods
Adjoint Estimation
Convergence
Examples
Mixed-Integer Optimal Control
Variable Time Transformation
Parametric Sensitivity Analysis and Realtime-Optimization
Parametric Optimal Control Problems
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Optimal Control Problem
Optimal Control Problem (OCP)
Minimize
ϕ(x(t0 ), x(tf ))
(t0 , tf fixed)
subject to the differential equation
x 0 (t) = f (t, x(t), u(t)),
t0 ≤ t ≤ tf ,
the control and state constraints
c(t, x(t), u(t)) ≤ 0,
t0 ≤ t ≤ tf ,
and the boundary conditions
ψ(x(t0 ), x(tf )) = 0.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Direct Discretization Idea
infinite dimensional optimal control problem OCP with state x and control u
⇓
discretization xh , uh
⇓
finite dimensional optimization problem (NLP)
Minimize
J(xh , uh )
subject to
G(xh , uh ) ≤ 0, H(xh , uh ) = 0
Remarks:
I
NLP can be large-scale and sparse or small and dense, depending on the type of
discretization
I
many different approaches for discretization exist: full discretization, collocation,
pseudospectral methods, direct multiple shooting methods,....
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Direct Discretization Idea
x1
xh
xM
(BDF, RK)
uh
(B-Splines)
u1
uN
control grid
t0
tN
state grid
t̄0
t̄M
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
State Discretization
Throughout we use one-step methods for the discretization of the differential equation
x 0 (t) = f (t, x(t), u(t))
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
State Discretization
Throughout we use one-step methods for the discretization of the differential equation
x 0 (t) = f (t, x(t), u(t))
on a grid (equidistant for simplicity)
Gh := {ti := t0 + ih; | i = 0, 1, . . . , N},
h=
tf − t0
, N ∈N
N
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
State Discretization
Throughout we use one-step methods for the discretization of the differential equation
x 0 (t) = f (t, x(t), u(t))
on a grid (equidistant for simplicity)
Gh := {ti := t0 + ih; | i = 0, 1, . . . , N},
h=
tf − t0
, N ∈N
N
One-step Methods
The one-step method is defined by the recursion
xi+1 = xi + hΦ(ti , xi , uh , h),
i = 0, 1, . . . , N − 1,
and provides approximations xi = xh (ti ) ≈ x(ti ) at ti ∈ Gh .
The function Φ is called increment function. It depends on uh , which represents a
control parameterization to be discussed later.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
State Discretization
Throughout we use one-step methods for the discretization of the differential equation
x 0 (t) = f (t, x(t), u(t))
on a grid (equidistant for simplicity)
Gh := {ti := t0 + ih; | i = 0, 1, . . . , N},
h=
tf − t0
, N ∈N
N
One-step Methods
The one-step method is defined by the recursion
xi+1 = xi + hΦ(ti , xi , uh , h),
i = 0, 1, . . . , N − 1,
and provides approximations xi = xh (ti ) ≈ x(ti ) at ti ∈ Gh .
The function Φ is called increment function. It depends on uh , which represents a
control parameterization to be discussed later.
Remark:
I The control approximation u needs to be defined. There are different ways to do
h
it; all of them lead to different discretization schemes for OCP.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
State Discretization
Examples (One-step methods)
I
Euler method:
xi+1 = xi + hf (ti , xi , ui ),
Φ(t, x, u, h) = f (t, x, u)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
State Discretization
Examples (One-step methods)
I
Euler method:
xi+1 = xi + hf (ti , xi , ui ),
I
Φ(t, x, u, h) = f (t, x, u)
Heun’s method:
k1
=
f (ti , xi , ?)
k2
=
xi+1
=
f (ti + h, xi + hk1 , ?)
h
xi + (k1 + k2 )
2
)
“stage derivatives”
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
State Discretization
Examples (One-step methods)
I
Euler method:
xi+1 = xi + hf (ti , xi , ui ),
I
Φ(t, x, u, h) = f (t, x, u)
Heun’s method:
k1
=
f (ti , xi , ui )
k2
=
xi+1
=
f (ti + h, xi + hk1 , ui )
h
xi + (k1 + k2 )
2
)
“stage derivatives”
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
State Discretization
Examples (One-step methods)
I
Euler method:
xi+1 = xi + hf (ti , xi , ui ),
I
Φ(t, x, u, h) = f (t, x, u)
Heun’s method:
k1
=
k2
=
xi+1
=
Φ(t, x, u, h)
=
f (ti , xi , ui )
)
“stage derivatives”
f (ti + h, xi + hk1 , ui )
h
xi + (k1 + k2 )
2
1
(f (t, x, u) + f (t + h, x + hf (t, x, u), u))
2
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
State Discretization
Examples (One-step methods)
I
Euler method:
xi+1 = xi + hf (ti , xi , ui ),
I
Φ(t, x, u, h) = f (t, x, u)
Heun’s method:
k1
=
f (ti , xi , ui )
k2
=
xi+1
=
f (ti + h, xi + hk1 , ui+1 )
h
xi + (k1 + k2 )
2
)
“stage derivatives”
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
State Discretization
Examples (One-step methods)
I
Euler method:
xi+1 = xi + hf (ti , xi , ui ),
I
Φ(t, x, u, h) = f (t, x, u)
Heun’s method:
k1
=
k2
=
xi+1
=
Φ(t, x, ui , ui+1 , h)
=
f (ti , xi , ui )
)
“stage derivatives”
f (ti + h, xi + hk1 , ui+1 )
h
xi + (k1 + k2 )
2
1
(f (t, x, ui ) + f (t + h, x + hf (t, x, ui ), ui+1 ))
2
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
State Discretization
Examples (One-step methods)
I
Modified Euler method:
k1
=
k2
=
xi+1
=
f (ti , xi , ?)
h
h
f (ti + , xi + k1 , ?)
2
2
xi + hk2
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
State Discretization
Examples (One-step methods)
I
Modified Euler method:
k1
=
k2
=
xi+1
=
f (ti , xi , ui )
h
h
f (ti + , xi + k1 , ui )
2
2
xi + hk2
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
State Discretization
Examples (One-step methods)
I
Modified Euler method:
k1
=
k2
=
xi+1
=
Φ(t, x, u, h)
=
f (ti , xi , ui )
h
h
f (ti + , xi + k1 , ui )
2
2
xi + hk2
h
h
f (t + , x + f (t, x, u), u)
2
2
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
State Discretization
Examples (One-step methods)
I
Modified Euler method:
k1
=
k2
=
xi+1
=
f (ti , xi , ui )
h
h
ui + ui+1
f (ti + , xi + k1 ,
)
2
2
2
xi + hk2
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
State Discretization
Examples (One-step methods)
I
Modified Euler method:
k1
=
k2
=
xi+1
=
Φ(t, x, ui , ui+1 , h)
=
f (ti , xi , ui )
h
h
ui + ui+1
f (ti + , xi + k1 ,
)
2
2
2
xi + hk2
h
ui + ui+1
h
f (t + , x + f (t, x, ui ),
)
2
2
2
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
State Discretization
Examples (One-step methods)
I
Modified Euler method:
k1
=
k2
=
xi+1
=
f (ti , xi , ui )
h
h
f (ti + , xi + k1 , ui+1/2 )
2
2
xi + hk2
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
State Discretization
Examples (One-step methods)
I
Modified Euler method:
k1
=
k2
=
xi+1
=
Φ(t, x, ui , ui+1/2 , h)
=
f (ti , xi , ui )
h
h
f (ti + , xi + k1 , ui+1/2 )
2
2
xi + hk2
h
h
f (t + , x + f (t, x, ui ), ui+1/2 )
2
2
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
State Discretization
Examples (One-step methods)
I
I
Modified Euler method:
k1
=
k2
=
xi+1
=
Φ(t, x, ui , ui+1/2 , h)
=
f (ti , xi , ui )
h
h
f (ti + , xi + k1 , ui+1/2 )
2
2
xi + hk2
h
h
f (t + , x + f (t, x, ui ), ui+1/2 )
2
2
In general: Runge-Kutta methods (explicit or implicit)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Collocation
Collocation idea
Approximate the solution x of the initial value problem
0
x (t) = f (t, x(t), u(t)),
x(ti ) = xi ,
in [ti , ti+1 ] by a polynomial p : [ti , ti+1 ] → Rnx of degree s.
Construction:
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Collocation
Collocation idea
Approximate the solution x of the initial value problem
0
x (t) = f (t, x(t), u(t)),
x(ti ) = xi ,
in [ti , ti+1 ] by a polynomial p : [ti , ti+1 ] → Rnx of degree s.
Construction:
I
collocation points ti ≤ τ1 < τ2 < · · · τs ≤ ti+1
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Collocation
Collocation idea
Approximate the solution x of the initial value problem
0
x (t) = f (t, x(t), u(t)),
x(ti ) = xi ,
in [ti , ti+1 ] by a polynomial p : [ti , ti+1 ] → Rnx of degree s.
Construction:
I
collocation points ti ≤ τ1 < τ2 < · · · τs ≤ ti+1
I
collocation conditions:
p(ti ) = xi ,
0
p (τk ) = f (τk , p(τk ), u(τk )),
k = 1, . . . , s
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Collocation
Collocation idea
Approximate the solution x of the initial value problem
0
x (t) = f (t, x(t), u(t)),
x(ti ) = xi ,
in [ti , ti+1 ] by a polynomial p : [ti , ti+1 ] → Rnx of degree s.
Construction:
I
collocation points ti ≤ τ1 < τ2 < · · · τs ≤ ti+1
I
collocation conditions:
p(ti ) = xi ,
I
define xi+1 := p(ti+1 )
0
p (τk ) = f (τk , p(τk ), u(τk )),
k = 1, . . . , s
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Collocation
Collocation idea
Approximate the solution x of the initial value problem
0
x (t) = f (t, x(t), u(t)),
x(ti ) = xi ,
in [ti , ti+1 ] by a polynomial p : [ti , ti+1 ] → Rnx of degree s.
Construction:
I
collocation points ti ≤ τ1 < τ2 < · · · τs ≤ ti+1
I
collocation conditions:
p(ti ) = xi ,
I
0
p (τk ) = f (τk , p(τk ), u(τk )),
k = 1, . . . , s
define xi+1 := p(ti+1 )
Example
For s = 2, τ1 = ti , τ2 = ti+1 , the collocation idea yields
xi+1 = xi +
This is the implicit trapezoidal rule!
h
(f (ti , xi , ui ) + f (ti+1 , xi+1 , ui+1 )) .
2
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Collocation
Collocation idea
Approximate the solution x of the initial value problem
0
x (t) = f (t, x(t), u(t)),
x(ti ) = xi ,
in [ti , ti+1 ] by a polynomial p : [ti , ti+1 ] → Rnx of degree s.
Construction:
I
collocation points ti ≤ τ1 < τ2 < · · · τs ≤ ti+1
I
collocation conditions:
p(ti ) = xi ,
I
0
p (τk ) = f (τk , p(τk ), u(τk )),
k = 1, . . . , s
define xi+1 := p(ti+1 )
Example
For s = 2, τ1 = ti , τ2 = ti+1 , the collocation idea yields
xi+1 = xi +
h
(f (ti , xi , ui ) + f (ti+1 , xi+1 , ui+1 )) .
2
This is the implicit trapezoidal rule!
In general: Every collocation method corresponds to an implicit Runge-Kutta method.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Discretization of Optimal Control Problems
Grid: (equidistant for simplicity)
Gh := {t0 + ih; | i = 0, 1, . . . , N},
h=
tf − t0
, N ∈N
N
Control parameterization (approximation of u on Gh )
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Discretization of Optimal Control Problems
Grid: (equidistant for simplicity)
Gh := {t0 + ih; | i = 0, 1, . . . , N},
h=
tf − t0
, N ∈N
N
Control parameterization (approximation of u on Gh )
I
piecewise constant or continuous and piecewise linear approximation (local support):
ui ≈ u(ti )
(ti ∈ Gh )
=⇒
uh = (u0 , u1 , . . . , uN )
>
(N+1)nu
∈R
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Discretization of Optimal Control Problems
Grid: (equidistant for simplicity)
Gh := {t0 + ih; | i = 0, 1, . . . , N},
h=
tf − t0
, N ∈N
N
Control parameterization (approximation of u on Gh )
I
piecewise constant or continuous and piecewise linear approximation (local support):
ui ≈ u(ti )
I
(ti ∈ Gh )
=⇒
uh = (u0 , u1 , . . . , uN )
>
(N+1)nu
∈R
interpolating cubic spline (non-local support, twice continuously differentiable)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Discretization of Optimal Control Problems
Grid: (equidistant for simplicity)
Gh := {t0 + ih; | i = 0, 1, . . . , N},
h=
tf − t0
, N ∈N
N
Control parameterization (approximation of u on Gh )
I
piecewise constant or continuous and piecewise linear approximation (local support):
ui ≈ u(ti )
(ti ∈ Gh )
=⇒
uh = (u0 , u1 , . . . , uN )
>
(N+1)nu
∈R
I
interpolating cubic spline (non-local support, twice continuously differentiable)
I
polynomials (non-local support, smooth) −→ pseudospectral methods
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Discretization of Optimal Control Problems
Grid: (equidistant for simplicity)
Gh := {t0 + ih; | i = 0, 1, . . . , N},
h=
tf − t0
, N ∈N
N
Control parameterization (approximation of u on Gh )
I
piecewise constant or continuous and piecewise linear approximation (local support):
ui ≈ u(ti )
(ti ∈ Gh )
=⇒
uh = (u0 , u1 , . . . , uN )
>
(N+1)nu
∈R
I
interpolating cubic spline (non-local support, twice continuously differentiable)
I
polynomials (non-local support, smooth) −→ pseudospectral methods
I
B-spline approximations of order k (local support, arbitrary smoothness possible):
N+k−1
uh (t) = uh (t; w) :=
X
k
wi Bi (t),
w = (w1 , . . . , wM )
>
∈R
Mnu
i=1
Bik : basis functions (elementary B-splines); w: control parameterization
, M := N + k − 1
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Discretization of Optimal Control Problems
Example (Elementary B-splines: Basis functions)
Elementary B-splines of orders k = 2, 3, 4: ([t0 , tf ] = [0, 1], N = 5, equidistant grid)
1
1
1
0.8
0.8
0.8
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
0
0
0.2
0.4
0.6
0.8
1
0
0.2
0.4
0.6
0.8
1
0.2
0.4
0.6
Elementary B-splines of order k = 1 are piecewise constant basis functions.
0.8
1
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Discretization of Optimal Control Problems
Definition (elementary B-Spline of order k )
Let k ∈ N. Define the auxiliary grid
k
Gh := {τi | i = 1, . . . , N + 2k − 1}
with auxiliary grid points
τi :=



 t0 ,
if 1 ≤ i ≤ k,


 t ,
N
if N + k ≤ i ≤ N + 2k − 1.
ti−k ,
if k + 1 ≤ i ≤ N + k − 1,
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Discretization of Optimal Control Problems
Definition (elementary B-Spline of order k )
Let k ∈ N. Define the auxiliary grid
k
Gh := {τi | i = 1, . . . , N + 2k − 1}
with auxiliary grid points
τi :=
The elementary B-splines



 t0 ,
if 1 ≤ i ≤ k,


 t ,
N
if N + k ≤ i ≤ N + 2k − 1.
ti−k ,
Bik (·)
(
1
Bi (t)
k
Bi (t)
:=
:=
if k + 1 ≤ i ≤ N + k − 1,
of order k for i = 1, . . . , N + k − 1 are defined by the recursion
1,
if τi ≤ t < τi+1
0,
otherwise
(“piecewise constant”)
t − τi
τi+k − t
k−1
k −1
B
(t) +
B
(t)
τi+k −1 − τi i
τi+k − τi+1 i+1
convention: 0/0 = 0 whenever auxiliary grid points coincide.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Discretization of Optimal Control Problems
Example
I
For k = 2 we obtain the continuous and piecewise linear functions

t−τi
if τi ≤ t < τi+1 ,

τi+1 −τi ,




τi+2 −t
2
Bi (t) =
if τi+1 ≤ t < τi+2 ,
τi+2 −τi+1 ,





0,
otherwise.
I
For k = 3 we obtain the continuously differentiable function

(t−τi )2

,


 (τi+2 −τi )(τi+1 −τi )



(t−τ )(τ
−t)
(τi+3 −t)(t−τi+1 )


,
 (τ −τi )(τi+2 −τ ) + (τ −τ
i+2
i
i+2
i+1
i+3
i+1 )(τi+2 −τi+1 )
3
Bi (t) =

2

(τ
−t)

 (τ −τ i+3)(τ −τ ) ,


i+3
i+1
i+3
i+2




0,
if t ∈ [τi , τi+1 ),
if t ∈ [τi+1 , τi+2 ),
if t ∈ [τi+2 , τi+3 ),
otherwise.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Discretization of Optimal Control Problems
Remarks:
I
The elementary B-splines Bik (·), i = 1, . . . , N + k − 1, restricted to the intervals
[tj , tj+1 ], j = 0, . . . , N − 1, are polynomials of degree at most k − 1.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Discretization of Optimal Control Problems
Remarks:
I
The elementary B-splines Bik (·), i = 1, . . . , N + k − 1, restricted to the intervals
[tj , tj+1 ], j = 0, . . . , N − 1, are polynomials of degree at most k − 1.
I
elementary B-splines are bounded and sum up to one
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Discretization of Optimal Control Problems
Remarks:
I
The elementary B-splines Bik (·), i = 1, . . . , N + k − 1, restricted to the intervals
[tj , tj+1 ], j = 0, . . . , N − 1, are polynomials of degree at most k − 1.
I
elementary B-splines are bounded and sum up to one
I
For k ≥ 2 the B-splines Bik (·) are k − 2 times continuously differentiable.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Discretization of Optimal Control Problems
Remarks:
I
The elementary B-splines Bik (·), i = 1, . . . , N + k − 1, restricted to the intervals
[tj , tj+1 ], j = 0, . . . , N − 1, are polynomials of degree at most k − 1.
I
elementary B-splines are bounded and sum up to one
I
For k ≥ 2 the B-splines Bik (·) are k − 2 times continuously differentiable.
I
For k ≥ 3 the derivative obeys the recursion
k −1
k −1
d k
k −1
k −1
B (t) =
B
(t) −
B
(t).
dt i
τi+k−1 − τi i
τi+k − τi+1 i+1
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Discretization of Optimal Control Problems
Remarks:
I
The elementary B-splines Bik (·), i = 1, . . . , N + k − 1, restricted to the intervals
[tj , tj+1 ], j = 0, . . . , N − 1, are polynomials of degree at most k − 1.
I
elementary B-splines are bounded and sum up to one
I
For k ≥ 2 the B-splines Bik (·) are k − 2 times continuously differentiable.
I
For k ≥ 3 the derivative obeys the recursion
k −1
k −1
d k
k −1
k −1
B (t) =
B
(t) −
B
(t).
dt i
τi+k−1 − τi i
τi+k − τi+1 i+1
I
elementary B-splines possess local support supp(Bik ) ⊂ [τi , τi+k ] with
(
Bik (t)
> 0,
if t ∈ (τi , τi+k ),
= 0,
otherwise,
for k > 1.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Discretization of Optimal Control Problems
Remarks:
I
The elementary B-splines Bik (·), i = 1, . . . , N + k − 1, restricted to the intervals
[tj , tj+1 ], j = 0, . . . , N − 1, are polynomials of degree at most k − 1.
I
elementary B-splines are bounded and sum up to one
I
For k ≥ 2 the B-splines Bik (·) are k − 2 times continuously differentiable.
I
For k ≥ 3 the derivative obeys the recursion
k −1
k −1
d k
k −1
k −1
B (t) =
B
(t) −
B
(t).
dt i
τi+k−1 − τi i
τi+k − τi+1 i+1
I
elementary B-splines possess local support supp(Bik ) ⊂ [τi , τi+k ] with
(
Bik (t)
I
> 0,
if t ∈ (τi , τi+k ),
= 0,
otherwise,
for k > 1.
most often k = 1 (piecewise constant) or k = 2 (continuous, piecewise linear) are
used in discretization of optimal control problems
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Contents
Direct Discretization Methods
Full Discretization
Reduced Discretization and Shooting Methods
Adjoint Estimation
Convergence
Examples
Mixed-Integer Optimal Control
Variable Time Transformation
Parametric Sensitivity Analysis and Realtime-Optimization
Parametric Optimal Control Problems
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Full Discretization of OCP
State discretization
For a given control parameterization uh (·; w) approximations xi ≈ x(ti ), ti ∈ Gh , are
obtained by a one-step method
xi+1 = xi + hΦ(ti , xi , w, h),
i = 0, 1, . . . , N − 1.
This yields a state approximation
xh = (x0 , x1 , . . . , xN )> ∈ R(N+1)nx
Example (Explicit Euler method)
xi+1 = xi + hf (ti , xi , ui ),
i = 0, 1, . . . , N − 1.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Full Discretization of OCP
Discretization of the optimal control problem with Mayer-type objective function and
B-spline control parameterization of order k yields:
Fully discretized optimal control problem (D-OCP)
Find xh = (x0 , . . . , xN )> ∈ R(N+1)nx and w = (w1 , . . . , wM )> ∈ RMnu such that
ϕ(x0 , xN )
becomes minimal subject to the discretized dynamic constraints
xi + hΦ(ti , xi , w, h) − xi+1 = 0,
i = 0, 1, . . . , N − 1,
the discretized control and state constraints
c(ti , xi , uh (ti ; w)) ≤ 0,
i = 0, 1, . . . , N,
and the boundary conditions
ψ(x0 , xN ) = 0.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Full Discretization of OCP
Remark
The constraints
xi + hΦ(ti , xi , w, h) − xi+1 = 0,
i = 0, 1, . . . , N − 1,
can be understood in two ways, if Φ defines a Runge-Kutta method with stages kj , j = 1, . . . , s:
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Full Discretization of OCP
Remark
The constraints
xi + hΦ(ti , xi , w, h) − xi+1 = 0,
i = 0, 1, . . . , N − 1,
can be understood in two ways, if Φ defines a Runge-Kutta method with stages kj , j = 1, . . . , s:
I
The stage equations kj = . . . are not explicitly added as equality constraints in D-OCP, e.g.
for Heun’s method one would only add the constraints
xi +
h
(f (ti , xi , ui ) + f (ti+1 , xi + hf (ti , xi , ui ), ui )) − xi+1 = 0
2
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Full Discretization of OCP
Remark
The constraints
xi + hΦ(ti , xi , w, h) − xi+1 = 0,
i = 0, 1, . . . , N − 1,
can be understood in two ways, if Φ defines a Runge-Kutta method with stages kj , j = 1, . . . , s:
I
The stage equations kj = . . . are not explicitly added as equality constraints in D-OCP, e.g.
for Heun’s method one would only add the constraints
xi +
I
h
(f (ti , xi , ui ) + f (ti+1 , xi + hf (ti , xi , ui ), ui )) − xi+1 = 0
2
The stage equations kj = . . . are explicitly added as equality constraints in D-OCP, e.g. for
Heun’s method one would add the constraints
k1 − f (ti , xi , ui )
=
0
k2 − f (ti+1 , xi + hk1 , ui )
=
0
h
(k1 + k2 ) − xi+1
2
=
0
xi +
This version is typically implemented for implicit Runge-Kutta methods.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Full Discretization of OCP
The fully discretized optimal control problem is a standard nonlinear optimization
problem, which is large-scale but exhibits a sparse structure:
Nonlinear Optimization Problem (NLP)
Minimize
J(z) := ϕ(x0 , xN )
w.r.t. z = (xh , w)> subject to the constraints
H(z) = 0,
G(z) ≤ 0,
where

x0 + hΦ(t0 , x0 , w, h) − x1


.

.
.
H(z) := 

 x
+
hΦ(t
,
xN−1 , w, h) − xN
N−1
N−1

ψ(x0 , xN )





,





G(z) := 

c(t0 , x0 , uh (t0 ; w))
.
.
.
c(tN , xN , uh (tN ; w))



.

Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Sparsity Structures in D-OCP
0
J (z)
=
ϕ0x
0
ϕ0x
N
0
,
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Sparsity Structures in D-OCP
0
J (z)
=

0
H (z)
=







ϕ0x
ϕ0x
0
M0
0
N
.
..
hΦ0w [t0 ]
.
.
.
.
MN−1
0
,
−Id
..
ψx0
−Id
ψx0
N




,

0
hΦw [tN−1 ] 

0
Mj := Id + hΦ0x [tj ]
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Sparsity Structures in D-OCP
0
J (z)
=

0
H (z)
=







ϕ0x
ϕ0x
0
M0
0
N
0
G (z)
=




,
hΦ0w [t0 ]
.
.
.
−Id
..
.
..
.
MN−1
−Id
ψx0
cx0 [t0 ]
0
cu0 [t0 ]uh,w
(t0 ; w)
.
.
.
N
..
.
cx0 [tN ]




,

0
hΦw [tN−1 ] 

0
ψx0
0

0
(tN ; w)
cu0 [tN ]uh,w



,

Mj := Id + hΦ0x [tj ]
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Sparsity Structures in D-OCP
0
J (z)
=

0
H (z)
=







ϕ0x
ϕ0x
0
M0
0
N
0
G (z)
0
uh,w (tj ; w)
=
=




,
hΦ0w [t0 ]
.
.
.
−Id
..
.
..
.
MN−1
−Id
ψx0
cx0 [t0 ]
0
cu0 [t0 ]uh,w
(t0 ; w)
.
.
.
N
..
.
cx0 [tN ]
B1k (tj ) · Id
B2k (tj ) · Id
0
(tN ; w)
cu0 [tN ]uh,w
···




,

0
hΦw [tN−1 ] 

0
ψx0
0

Mj := Id + hΦ0x [tj ]



,

k
(tj ) · Id
BM
Observation:
I
0 (t ; w) and in Φ0 [t ] vanish
Bik have local support =⇒ most entries in uh,w
j
w j
I
H 0 (z) and G0 (z) have a large-scale and sparse structure.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Sparsity Structures in the Full Discretization Approach
Lagrange function:
L(z, λ, µ)
=
ϕ(x0 , xN ) + σ > ψ(x0 , xN )
+
N−1
X
λ>
i+1 (xi + hΦ(ti , xi , w, h) − xi+1 ) +
i=0
N
X
µ>
i c(ti , xi , uh (ti ; w))
i=0
Hessian matrix:

L00
 x0 ,x0



L00
zz (z, λ, µ) = 
 L00
 xN ,x0
L00
w,x0
L00
x0 ,xN
..
.
···
L00
x0 ,w
..
.
L00
xN ,xN
L00
xN ,w
L00
w,xN
L00
w,w








> and L00
00
Note: The blocks L00
xj ,w = (Lw,xj )
w,w are sparse matrices if B-splines are used.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Full Euler Discretization I
B-splines of order k = 1 (piecewise constant) and k = 2 (continuous and piecewise
linear) satisfy
uj−1 := u(tj−1 ; w) =
N+k
−1
X
wi Bik (tj−1 ) = wj
(j = 1, . . . , N + k − 1)
i=1
Hence:
The function value of u at tj and the coefficient wj+1 coincide.
This will be exploited in the following examples.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Full Euler Discretization II
Example (D-OCP using explicit Euler)
Find z = (xh , uh )> with
xh
=
(x0 , . . . , xN )> ∈ R(N+1)nx
uh
=
(u0 , . . . , uN )> ∈ R(N+1)nu
such that
J(z) = ϕ(x0 , xN )
becomes minimal subject to the constraints
xi + hf (ti , xi , ui ) − xi+1 = 0,
c(ti , xi , ui ) ≤ 0,
ψ(x0 , xN ) = 0.
i = 0, 1, . . . , N − 1,
i = 0, 1, . . . , N,
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Full Euler Discretization III
Sparsity: (Φ = f )

H 0 (z)
=








G0 (z)
=




M0
−Id
..
.
..
..
.
MN−1
ψx0N
cx0 [t0 ]
cu0 [t0 ]
0
cx0 [tN ]
where Mj = Id + hfx0 [tj ], j = 0, . . . , N − 1
···



,

0
hfu [tN−1 ] 

0

..
.
.
−Id
ψx00
..

hfu0 [t0 ]
.
cu0 [tN ]


,

Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Full Euler Discretization IV
Lagrange function:
L(z, λ, µ)
=
ϕ(x0 , xN ) + σ > ψ(x0 , xN )
+
N−1
X
λ>
i+1 (xi + hf (ti , xi , ui , h) − xi+1 ) +
i=0
N
X
µ>
i c(ti , xi , ui )
i=0
Hessian matrix:

L00
x0 ,x0




 00
 Lx ,x0
00
N
Lzz (z, λ, µ) = 
 00
 Lu0 ,x0




L00
x0 ,xN
..
L00
x0 ,u0

..
.
.
L00
xN ,uN
L00
xN ,xN
L00
u0 ,u0
..
..
.
L00
uN ,xN
.
L00
uN ,uN













Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Full Trapezoidal Discretization
Example (D-OCP using trapezoidal rule (Collocation))
Find z = (xh , uh )> with
xh
=
(x0 , . . . , xN )> ∈ R(N+1)nx
uh
=
(u0 , . . . , uN )> ∈ R(N+1)nu
such that
J(z) = ϕ(x0 , xN )
becomes minimal subject to the constraints
xi +
h
(f (ti , xi , ui ) + f (ti+1 , xi+1 , ui+1 )) − xi+1 = 0,
2
c(ti , xi , ui ) ≤ 0,
ψ(x0 , xN ) = 0.
i = 0, 1, . . . , N − 1,
i = 0, 1, . . . , N,
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Full Hermite-Simpson Discretization (Collocation) I
Example (Hermite-Simpson, Collocation)
The Hermite-Simpson rule reads
xi+1 = xi +
h
6
fi + 4fi+ 1 + fi+1
,
i = 0, 1, . . . , N − 1,
2
where
fi
:=
fi+ 1
:=
xi+ 1
:=
2
2
f (ti , xi , ui ),
fi+1 := f (ti+1 , xi+1 , ui+1 ),
h
f (ti + , xi+ 1 , ui+ 1 ),
2
2
2
1
h
(xi + xi+1 ) + (fi − fi+1 ) .
2
8
Herein, we need to specify what ui+ 1 is supposed to be. Several choices are possible, e.g.
2
I
if a continuous and piecewise linear control approximation is chosen, then
ui+ 1 = 12 (ui + ui+1 );
I
ui+ 1 can be introduced as an additional optimization variable without specifying any relations
2
2
to ui and ui+1 .
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Full Hermite-Simpson Discretization (Collocation) II
Caution!
See what happens if you apply the modified Euler method with additional optimization variables
uk + 1 at the midpoints tk + h2 to the following problem:
2
Minimize
1
2
Z
1
2
2
u(t) + 2x(t) dt
0
subject to the constraints
0
x (t) =
1
x(t) + u(t),
2
x(0) = 1.
The optimal solution is
x̂(t) =
2 exp(3t) + exp(3)
,
exp(3t/2)(2 + exp(3))
û(t) =
2(exp(3t) − exp(3))
.
exp(3t/2)(2 + exp(3))
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Numerical Solution of D-OCP
It remains to solve the large-scale and sparse NLP by. e.g.
I
sequential-quadratic programming (SQP), e.g. WORHP or SNOPT
I
interior-point methods, e.g. IPOPT or KNITRO
I
any other software package suitable for large-scale nonlinear programs
Caution: Designing a code for large-scale nonlinear programming is non-trivial,
because
I
sparse Hessian and Jacobian approximations are expensive to compute in
general (graph coloring problem)
I
regularization techniques for singular or indefinite Hessians are required
I
one cannot use standard BFGS updates since the update leads to large-scale and
dense matrices (fill-in!)
I
linear equation solvers are needed to solve linear equations with large-scale
saddle point matrices (KKT matrices) of type
L00
zz
A>
A
−S
!
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Grid Refinement
Approaches:
[1]
I
Refinement based on the local discretization error of the state dynamics and local
refinement at junction points of active/inactive state constraints, see [1, 2]
I
Refinement taking into account the discretization error in the optimality system
including adjoint equations, see [3]
Betts, J. T. and Huffman, W. P.
Mesh Refinement in Direct Transcription Methods for Optimal Control.
Optimal Control Applications and Methods, 19; 1–21, 1998.
[2]
C. Büskens.
Optimierungsmethoden und Sensitivitätsanalyse für optimale Steuerprozesse mit Steuer- und Zustandsbeschränkungen.
PhD thesis, Fachbereich Mathematik, Westfälische Wilhems-Universität Münster, Münster, Germany, 1998.
[3]
J. Laurent-Varin, F. Bonnans, N. Berend, C. Talbot, and M. Haddou.
On the refinement of discretization for optimal control problems.
IFAC Symposium on Automatic Control in Aerospace, St. Petersburg, 2004.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Pseudospectral Methods
Approach:
I
global approximation of control and state by Legendre or Chebyshev polynomials
I
direct discretization
I
sparse nonlinear programming solver
Advantages:
I
exponential (or spectral) rate of convergence (faster than polynomial)
I
good accuracy already with coarse grids
Disadvantages:
I
[1]
oscillations for non-differentiable trajectories
G. Elnagar, M. A. Kazemi, and M. Razzaghi.
The Pseudospectral Legendre Method for Discretizing Optimal Control Problems.
IEEE Transactions on Automatic Control, 40:1793–1796, 1995.
[2]
J. Vlassenbroeck and R. Van Doreen.
A Chebyshev Technique for Solving Nonlinear Optimal Control Problems.
IEEE Transactions on Automatic Control, 33:333–340, 1988.
[3]
F. Fahroo and I.M. Ross.
Direct Trajectory Optimization by a Chebyshev Pseudospectral Method.
Journal of Guidance Control and Dynamics, 25, 2002.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Contents
Direct Discretization Methods
Full Discretization
Reduced Discretization and Shooting Methods
Adjoint Estimation
Convergence
Examples
Mixed-Integer Optimal Control
Variable Time Transformation
Parametric Sensitivity Analysis and Realtime-Optimization
Parametric Optimal Control Problems
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Reduced Discretization (Direct Single Shooting)
Fully discretized optimal control problem
Minimize
ϕ(x0 , xN )
w.r.t. xh = (x0 , . . . , xN )> ∈ R(N+1)nx and w = (w1 , . . . , wM )> ∈ RMnu subject to
xi + hΦ(ti , xi , w, h) − xi+1
=
0,
i = 0, 1, . . . , N − 1,
c(ti , xi , uh (ti ; w))
≤
0,
i = 0, 1, . . . , N,
ψ(x0 , xN )
=
0.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Reduced Discretization (Direct Single Shooting)
Reduction of size by solving the discretized differential equations:
x0
=: X0 (x0 , w),
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Reduced Discretization (Direct Single Shooting)
Reduction of size by solving the discretized differential equations:
x0
=: X0 (x0 , w),
x1 = x0 + hΦ(t0 , x0 , w, h)
=: X1 (x0 , w),
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Reduced Discretization (Direct Single Shooting)
Reduction of size by solving the discretized differential equations:
x0
=: X0 (x0 , w),
x1 = x0 + hΦ(t0 , x0 , w, h)
=: X1 (x0 , w),
x2 = x1 + hΦ(t1 , x1 , w, h)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Reduced Discretization (Direct Single Shooting)
Reduction of size by solving the discretized differential equations:
x0
=: X0 (x0 , w),
x1 = x0 + hΦ(t0 , x0 , w, h)
=: X1 (x0 , w),
x2 = x1 + hΦ(t1 , x1 , w, h)
= X1 (x0 , w) + hΦ(t1 , X1 (x0 , w), w, h)
..
.
=: X2 (x0 , w),
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Reduced Discretization (Direct Single Shooting)
Reduction of size by solving the discretized differential equations:
x0
=: X0 (x0 , w),
x1 = x0 + hΦ(t0 , x0 , w, h)
=: X1 (x0 , w),
x2 = x1 + hΦ(t1 , x1 , w, h)
= X1 (x0 , w) + hΦ(t1 , X1 (x0 , w), w, h)
..
.
xN = xN−1 + hΦ(tN−1 , xN−1 , w, h)
=: X2 (x0 , w),
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Reduced Discretization (Direct Single Shooting)
Reduction of size by solving the discretized differential equations:
x0
=: X0 (x0 , w),
x1 = x0 + hΦ(t0 , x0 , w, h)
=: X1 (x0 , w),
x2 = x1 + hΦ(t1 , x1 , w, h)
= X1 (x0 , w) + hΦ(t1 , X1 (x0 , w), w, h)
=: X2 (x0 , w),
..
.
xN = xN−1 + hΦ(tN−1 , xN−1 , w, h)
= XN−1 (x0 , w) + hΦ(tN−1 , XN−1 (x0 , w), w, hN−1 )
=: XN (x0 , w).
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Reduced Discretization (Direct Single Shooting)
Reduction of size by solving the discretized differential equations:
x0
=: X0 (x0 , w),
x1 = x0 + hΦ(t0 , x0 , w, h)
=: X1 (x0 , w),
x2 = x1 + hΦ(t1 , x1 , w, h)
= X1 (x0 , w) + hΦ(t1 , X1 (x0 , w), w, h)
=: X2 (x0 , w),
..
.
xN = xN−1 + hΦ(tN−1 , xN−1 , w, h)
= XN−1 (x0 , w) + hΦ(tN−1 , XN−1 (x0 , w), w, hN−1 )
Remarks:
=: XN (x0 , w).
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Reduced Discretization (Direct Single Shooting)
Reduction of size by solving the discretized differential equations:
x0
=: X0 (x0 , w),
x1 = x0 + hΦ(t0 , x0 , w, h)
=: X1 (x0 , w),
x2 = x1 + hΦ(t1 , x1 , w, h)
= X1 (x0 , w) + hΦ(t1 , X1 (x0 , w), w, h)
=: X2 (x0 , w),
..
.
xN = xN−1 + hΦ(tN−1 , xN−1 , w, h)
= XN−1 (x0 , w) + hΦ(tN−1 , XN−1 (x0 , w), w, hN−1 )
=: XN (x0 , w).
Remarks:
I
The state trajectory is fully determined by the initial value x0 and the control
parameterization w (direct shooting idea).
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Reduced Discretization (Direct Single Shooting)
Reduction of size by solving the discretized differential equations:
x0
=: X0 (x0 , w),
x1 = x0 + hΦ(t0 , x0 , w, h)
=: X1 (x0 , w),
x2 = x1 + hΦ(t1 , x1 , w, h)
= X1 (x0 , w) + hΦ(t1 , X1 (x0 , w), w, h)
=: X2 (x0 , w),
..
.
xN = xN−1 + hΦ(tN−1 , xN−1 , w, h)
= XN−1 (x0 , w) + hΦ(tN−1 , XN−1 (x0 , w), w, hN−1 )
=: XN (x0 , w).
Remarks:
I
The state trajectory is fully determined by the initial value x0 and the control
parameterization w (direct shooting idea).
I
The above procedure is nothing else than solving the initial value problem
x 0 (t) = f (t, x(t), uh (t; w)),
x(t0 ) = x0
with the one-step method with increment function Φ.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Reduced Discretization (Direct Single Shooting)
Reduced Discretization (RD-OCP)
Minimize
ϕ(x0 , XN (x0 , w))
with respect to x0 ∈
Rnx
and w ∈ RMnu subject to the constraints
ψ(x0 , XN (x0 , w)) = 0,
c(tj , Xj (x0 , w), uh (tj ; w)) ≤ 0,
j = 0, 1, . . . , N.
Remarks:
I
much smaller than D-OCP (fewer optimization variables and fewer constraints)
I
but: more nonlinear than D-OCP
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Reduced Discretization (Direct Single Shooting)
The reduced discretization is again a finite dimensional nonlinear optimization problem,
but of reduced size:
Reduced Nonlinear Optimization Problem (R-NLP)
Minimize
J(z) := ϕ(x0 , XN (x0 , w))
w.r.t. z = (x0 , w)> ∈ Rnx +Mnu subject to the constraints
H(z) = 0,
G(z) ≤ 0,
where

H(z) := ψ(x0 , XN (x0 , w)),
c(t0 , x0 , uh (t0 ; w))

 c(t1 , X1 (x0 , w), uh (t1 ; w))

G(z) := 
.

.

.

c(tN , XN (x0 , w), uh (tN ; w))




.



Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Reduced Discretization (Direct Single Shooting)
Derivatives: (required in gradient based optimization method)
0
0
0
ϕ
·
X
J 0 (z) =
ϕ0x0 + ϕ0xf · XN,x
x
N,w
f
0

0 (t ; w)
cx0 [t0 ]
cx0 [t0 ] + cu0 [t0 ] · uh,w
0


 0
0
0
0
0
0
cx [t1 ] · X1,w + cu [t1 ] · uh,w (t1 ; w)
 cx [t1 ] · X1,x0


0
G (z) = 
..
..


.
.


 0
0
0 (t ; w)
0
cx0 [tN ] · XN,w
+ cu0 [tN ] · uh,w
cx [tN ] · XN,x
N
0
H 0 (z)
=
0
0
0
ψx00 + ψx0f · XN,x
ψxf · XN,w
0
How to compute sensitivities
0
Xi,x
(x0 , w),
0
0
Xi,w
(x0 , w),
i = 1, . . . , N ?













Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Computation of Derivatives
Different approaches exist:
(a) The sensitivity differential equation approach is advantageous if the number of
constraints is (much) larger than the number of variables in the discretized
problem.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Computation of Derivatives
Different approaches exist:
(a) The sensitivity differential equation approach is advantageous if the number of
constraints is (much) larger than the number of variables in the discretized
problem.
(b) The adjoint equation approach is preferable if the number of constraints is less
than the number of variables in the discretized problem.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Computation of Derivatives
Different approaches exist:
(a) The sensitivity differential equation approach is advantageous if the number of
constraints is (much) larger than the number of variables in the discretized
problem.
(b) The adjoint equation approach is preferable if the number of constraints is less
than the number of variables in the discretized problem.
(c) A powerful tool for the evaluation of derivatives is algorithmic differentiation, see
www.autodiff.org. This approach assumes that the evaluation of a function is
performed by a F ORTRAN or C procedure. Algorithmic differentiation means that
the complete procedure is differentiated step by step using, roughly speaking,
chain and product rules. The result is again a F ORTRAN or C procedure that
provides the derivative of the function.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Computation of Derivatives
Different approaches exist:
(a) The sensitivity differential equation approach is advantageous if the number of
constraints is (much) larger than the number of variables in the discretized
problem.
(b) The adjoint equation approach is preferable if the number of constraints is less
than the number of variables in the discretized problem.
(c) A powerful tool for the evaluation of derivatives is algorithmic differentiation, see
www.autodiff.org. This approach assumes that the evaluation of a function is
performed by a F ORTRAN or C procedure. Algorithmic differentiation means that
the complete procedure is differentiated step by step using, roughly speaking,
chain and product rules. The result is again a F ORTRAN or C procedure that
provides the derivative of the function.
(d) The approximation by finite differences is straightforward, but has the drawback of
being computationally expensive and often suffers from low accuracy.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Computation of Derivatives by Sensitivity Analysis
Given: one-step method
X0 (x0 , w)
=
x0 ,
Xi+1 (x0 , w)
=
Xi (x0 , w) + hΦ(ti , Xi (x0 , w), w, h),
i = 0, 1, . . . , N − 1.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Computation of Derivatives by Sensitivity Analysis
Given: one-step method
X0 (x0 , w)
=
x0 ,
Xi+1 (x0 , w)
=
Xi (x0 , w) + hΦ(ti , Xi (x0 , w), w, h),
Goal: Compute sensitivities
0 (x , w)
Si := Xi0 (x0 , w) =
Xi,x
0
0
0 (x , w)
Xi,w
0
i = 0, 1, . . . , N − 1.
,
i = 0, 1, . . . , N.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Computation of Derivatives by Sensitivity Analysis
Given: one-step method
X0 (x0 , w)
=
x0 ,
Xi+1 (x0 , w)
=
Xi (x0 , w) + hΦ(ti , Xi (x0 , w), w, h),
Goal: Compute sensitivities
0 (x , w)
Si := Xi0 (x0 , w) =
Xi,x
0
0
0 (x , w)
Xi,w
0
i = 0, 1, . . . , N − 1.
,
i = 0, 1, . . . , N.
Idea: Differentiate the one-step method step-by-step with respect to x0 and w.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Computation of Derivatives by Sensitivity Analysis
Sensitivity Computation (Internal Numerical Differentiation, IND)
Init:
S0 =
Id
0
∈ Rnx ×(nx +Mnu ) .
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Computation of Derivatives by Sensitivity Analysis
Sensitivity Computation (Internal Numerical Differentiation, IND)
Init:
S0 =
Id
0
∈ Rnx ×(nx +Mnu ) .
For i = 0, 1, . . . , N − 1 compute
Si+1 = Si + h Φ0x [ti ] · Si + Φ0w [ti ] ·
∂w
∂(x0 , w)
,
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Computation of Derivatives by Sensitivity Analysis
Sensitivity Computation (Internal Numerical Differentiation, IND)
Init:
S0 =
Id
0
∈ Rnx ×(nx +Mnu ) .
For i = 0, 1, . . . , N − 1 compute
Si+1 = Si + h Φ0x [ti ] · Si + Φ0w [ti ] ·
where
∂w
=
0
∂(x0 , w)
Id
∂w
∂(x0 , w)
∈ RMnu ×(nx +Mnu ) .
,
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Computation of Derivatives by Sensitivity Analysis
Sensitivity Computation (Internal Numerical Differentiation, IND)
Init:
S0 =
Id
0
∈ Rnx ×(nx +Mnu ) .
For i = 0, 1, . . . , N − 1 compute
Si+1 = Si + h Φ0x [ti ] · Si + Φ0w [ti ] ·
where
∂w
=
0
∂(x0 , w)
Id
∂w
∂(x0 , w)
,
∈ RMnu ×(nx +Mnu ) .
Note: The dimension of Si depends on the dimension nx + Mnu of the optimization
vector (x0 , w)> , but not on the number of constraints of the reduced optimization
problem. Hence, this approach is useful, if the number of constraints exceeds the
number of optimization variables.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Computation of Derivatives by Sensitivity Analysis
Explicit Euler method:
i = 0, 1, . . . , N − 1.
Xi+1 (x0 , w) = Xi (x0 , w) + hf (ti , Xi (x0 , w), ui ),
Recall: uj−1 = wj , j = 1, . . . , N, i.e. w = (u0 , u1 , . . . , uN−1 )>
Sensitivity Computation (Internal Numerical Differentiation, IND)
Init:
S0 =
Id
0
∈ Rnx ×(nx +Nnu ) .
For i = 0, 1, . . . , N − 1 compute
Si+1 = Si + h
where
∂ui
=
0
∂(x0 , w)
fx0 [ti ] · Si + fu0 [ti ] ·
0
···
0
Id
∂ui
∂(x0 , w)
0
···
,
0
.
↑ i-th nu × nu -block
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
IND and Sensitivity Differential Equation
Discretization of the sensitivity differential equation
0
S 0 (t) = fx0 [t] · S(t) + fu0 [t] · uh,(x
(t; w),
0 ,w)
∈ Rnx ×(nx +Nnu )
S(t0 ) =
Id 0
with the explicit Euler method yields the internal numerical differentiation formulae
∂ui
Si+1 = Si + h fx0 [ti ] · Si + fu0 [ti ] ·
,
i = 0, 1, . . . , N − 1,
∂(x0 , w)
∈ Rnx ×(nx +Nnu )
S0 =
Id 0
Conclusion:
I
The IND approach for the calculation of sensitivities coincides with the
discretization of the sensitivity differential equation, if the same one-step method
and control approximation are used.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Gradient Computation by Adjoint Equation
Given:
I
One-step discretization scheme (z = (x0 , w)> ):
X0 (z) = x0 ,
Xi+1 (z) = Xi (z) + hΦ(ti , Xi (z), w, h),
i = 0, 1, . . . , N − 1,
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Gradient Computation by Adjoint Equation
Given:
I
One-step discretization scheme (z = (x0 , w)> ):
X0 (z) = x0 ,
Xi+1 (z) = Xi (z) + hΦ(ti , Xi (z), w, h),
I
i = 0, 1, . . . , N − 1,
A function Γ of type
Γ(z) := γ(x0 , XN (z), w).
This can be the objective function of the reduced optimal control problem or the
boundary conditions or any of the discretized state constraints (with N replaced
by i).
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Gradient Computation by Adjoint Equation
Given:
I
One-step discretization scheme (z = (x0 , w)> ):
X0 (z) = x0 ,
Xi+1 (z) = Xi (z) + hΦ(ti , Xi (z), w, h),
I
i = 0, 1, . . . , N − 1,
A function Γ of type
Γ(z) := γ(x0 , XN (z), w).
This can be the objective function of the reduced optimal control problem or the
boundary conditions or any of the discretized state constraints (with N replaced
by i).
Goals:
I
Compute gradient of Γ with respect to z.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Gradient Computation by Adjoint Equation
Given:
I
One-step discretization scheme (z = (x0 , w)> ):
X0 (z) = x0 ,
Xi+1 (z) = Xi (z) + hΦ(ti , Xi (z), w, h),
I
i = 0, 1, . . . , N − 1,
A function Γ of type
Γ(z) := γ(x0 , XN (z), w).
This can be the objective function of the reduced optimal control problem or the
boundary conditions or any of the discretized state constraints (with N replaced
by i).
Goals:
I
Compute gradient of Γ with respect to z.
I
Avoid the costly computation of the sensitivity matrices Si , i = 0, . . . , N, with IND.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Gradient Computation by Adjoint Equation
Define the auxiliary functional Γa using multipliers λi , i = 1, . . . , N:
Γa (z) := Γ(z) +
N−1
X
i=0
λ>
i+1 (Xi+1 (z) − Xi (z) − hΦ(ti , Xi (z), w, h))
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Gradient Computation by Adjoint Equation
Define the auxiliary functional Γa using multipliers λi , i = 1, . . . , N:
Γa (z) := Γ(z) +
N−1
X
λ>
i+1 (Xi+1 (z) − Xi (z) − hΦ(ti , Xi (z), w, h))
i=0
Differentiating Γa with respect to z leads to the expression
> 0
0
>
· SN + γw0
Γ0a (z) =
γx00 − λ>
1 − hλ1 Φx [t0 ] · S0 + γxN + λN
+
N−1
X
i=1
N−1
X
∂w
0
>
>
0
λ>
hλ>
.
i − λi+1 − hλi+1 Φx [ti ] · Si −
i+1 Φw [ti ] ·
∂z
i=0
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Gradient Computation by Adjoint Equation
Define the auxiliary functional Γa using multipliers λi , i = 1, . . . , N:
Γa (z) := Γ(z) +
N−1
X
λ>
i+1 (Xi+1 (z) − Xi (z) − hΦ(ti , Xi (z), w, h))
i=0
Differentiating Γa with respect to z leads to the expression
> 0
0
>
· SN + γw0
Γ0a (z) =
γx00 − λ>
1 − hλ1 Φx [t0 ] · S0 + γxN + λN
+
N−1
X
i=1
Xi0 (z)
N−1
X
∂w
0
>
>
0
λ>
hλ>
.
i − λi+1 − hλi+1 Φx [ti ] · Si −
i+1 Φw [ti ] ·
∂z
i=0
The terms Si =
are just the sensitivities in the sensitivity equation approach
which we alertdo not (!) want to compute here.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Gradient Computation by Adjoint Equation
Define the auxiliary functional Γa using multipliers λi , i = 1, . . . , N:
Γa (z) := Γ(z) +
N−1
X
λ>
i+1 (Xi+1 (z) − Xi (z) − hΦ(ti , Xi (z), w, h))
i=0
Differentiating Γa with respect to z leads to the expression
> 0
0
>
· SN + γw0
Γ0a (z) =
γx00 − λ>
1 − hλ1 Φx [t0 ] · S0 + γxN + λN
+
N−1
X
i=1
N−1
X
∂w
0
>
>
0
λ>
hλ>
.
i − λi+1 − hλi+1 Φx [ti ] · Si −
i+1 Φw [ti ] ·
∂z
Xi0 (z)
i=0
The terms Si =
are just the sensitivities in the sensitivity equation approach
which we alertdo not (!) want to compute here.
Idea: Choose λi such that the red terms involving Si vanish.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Gradient Computation by Adjoint Equation
Discrete adjoint equation: (to be solved backwards in time)
0
>
>
λ>
i − λi+1 − hλi+1 Φx [ti ] = 0,
i = 0, . . . , N − 1
Transversality condition: (terminal condition at t = tN )
0
λ>
N = −γxN (x0 , XN (z), w)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Gradient Computation by Adjoint Equation
Discrete adjoint equation: (to be solved backwards in time)
0
>
>
λ>
i − λi+1 − hλi+1 Φx [ti ] = 0,
i = 0, . . . , N − 1
Transversality condition: (terminal condition at t = tN )
0
λ>
N = −γxN (x0 , XN (z), w)
Then:
N−1
X
∂w
0
Γ0a (z) = γx00 − λ>
,
· S0 + γw0 −
hλ>
0
i+1 Φw [ti ] ·
∂z
i=0
where S0 =
Id 0 .
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Gradient Computation by Adjoint Equation
Discrete adjoint equation: (to be solved backwards in time)
0
>
>
λ>
i − λi+1 − hλi+1 Φx [ti ] = 0,
i = 0, . . . , N − 1
Transversality condition: (terminal condition at t = tN )
0
λ>
N = −γxN (x0 , XN (z), w)
Then:
N−1
X
∂w
0
Γ0a (z) = γx00 − λ>
,
· S0 + γw0 −
hλ>
0
i+1 Φw [ti ] ·
∂z
i=0
where S0 =
Id 0 .
What is the relation between Γ0a and Γ0 ?
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Gradient Computation by Adjoint Equation
Theorem
It holds
N−1
X
∂w
0
· S0 + γw0 −
hλ>
Γ0 (z) = Γ0a (z) = γx00 − λ>
.
0
i+1 Φw [ti ] ·
∂z
i=0
Notes:
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Gradient Computation by Adjoint Equation
Theorem
It holds
N−1
X
∂w
0
· S0 + γw0 −
hλ>
Γ0 (z) = Γ0a (z) = γx00 − λ>
.
0
i+1 Φw [ti ] ·
∂z
i=0
Notes:
I
With Γ0 (z) = Γ0a (z) we found a formula for the gradient of Γ.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Gradient Computation by Adjoint Equation
Theorem
It holds
N−1
X
∂w
0
· S0 + γw0 −
hλ>
Γ0 (z) = Γ0a (z) = γx00 − λ>
.
0
i+1 Φw [ti ] ·
∂z
i=0
Notes:
I
I
With Γ0 (z) = Γ0a (z) we found a formula for the gradient of Γ.
The size of the adjoint equation does not depend on the dimension of w and in
particular not on the number N that defines the grid. But an individual adjoint
equation has to be solved for every function whose gradient is required. For the
reduced optimal control problem these functions are the objective function, the
boundary conditions, and the discretized state constraints.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Gradient Computation by Adjoint Equation
Example
We compare the CPU times for the emergency landing maneuver without dynamic
pressure constraint for the sensitivity equation approach and the adjoint equation
approach for different values of N:
250
0.8
CPU time [s]
CPU time per iteration [s]
0.7
200
sensitivity eq.
adjoint eq.
150
100
50
sensitivity eq.
adjoint eq.
0.6
0.5
0.4
0.3
0.2
0.1
0
0
0
50
100
150
200
N
250
300
350
0
50
100
150
200
N
250
300
350
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Contents
Direct Discretization Methods
Full Discretization
Reduced Discretization and Shooting Methods
Adjoint Estimation
Convergence
Examples
Mixed-Integer Optimal Control
Variable Time Transformation
Parametric Sensitivity Analysis and Realtime-Optimization
Parametric Optimal Control Problems
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Optimal Control Problem and its Euler Discretization
Goal:
I Derive adjoint estimates from the optimal solution of D-OCP
Approach:
I Compare necessary conditions for OCP and D-OCP
OCP
D-OCP
Find an absolutely continuous state vector x
and an essentially bounded control vector u
such that
ϕ(x(t0 ), x(tf ))
Find xh = (x0 , . . . , xN )> and
uh = (u0 , . . . , uN−1 )> such that
becomes minimal subject to the differential
equation
becomes minimal subject to the discretized
dynamic constraints
0
x (t) = f (t, x(t), u(t))
a.e. in [t0 , tf ],
the control-state constraints
c(t, x(t), u(t)) ≤ 0
a.e. in [t0 , tf ],
the pure state constraints
s(t, x(t)) ≤ 0
ϕ(x0 , xN )
xi +hi f (ti , xi , ui )−xi+1 = 0,
i = 0, 1, . . . , N−1,
the discretized control-state constraints
c(ti , xi , ui ) ≤ 0,
i = 0, 1, . . . , N − 1,
the discretized pure constraints
in [t0 , tf ],
and the boundary conditions
ψ(x(t0 ), x(tf )) = 0.
s(ti , xi ) ≤ 0,
i = 0, 1, . . . , N,
and the boundary conditions
ψ(x0 , xN ) = 0.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Optimal Control Problem
>
>
Augmented Hamilton function: Ĥ(t, x, u, λ, η) := λ f (t, x, u) + η c(t, x, u)
Local Minimum Principle
I
`0 ≥ 0, (`0 , σ, λ, η) 6= 0
I
Adjoint differential equation:
Z t
Z
f
0
>
λ(t) = λ(tf ) +
Ĥx (τ, x̂(τ ), û(τ ), λ(τ ), η(τ )) dt +
t
I
0
>
sx (τ, x̂(τ )) dµ(τ )
Transversality conditions:
>
λ(t0 )
I
tf
t
0
> 0
= − `0 ϕx + σ ψx ,
0
λ(tf )
0
>
0
>
0
f
= `0 ϕx + σ ψx
f
Stationarity of augmented Hamilton function: Almost everywhere we have
0
Ĥu (t, x̂(t), û(t), λ(t), η(t)) = 0
I
Complementarity conditions:
>
η(t) ≥ 0,
Z
tf
t0
>
s(t, x̂(t)) dµ(t) = 0,
η(t) c(t, x̂(t), û(t)) = 0
Z
tf
t0
>
h(t) dµ(t) ≥ 0
n
∀0 ≤ h ∈ C(I, R s )
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Approximation of Adjoints
Theorem (Discrete Minimum Principle)
Assumptions:
I
ϕ, f0 , f , c, s, ψ are continuously differentiable
I
(x̂h , ûh ) is a local minimum of D-OCP
Then there exist multipliers (`0 , κ, λ, ζ, ν) 6= 0 with:
I
`0 ≥ 0
I
Discrete adjoint equations: For i = 0, . . . , N − 1 we have
λi
=
λN +
N−1
X
j=i
I
hj Ĥ0x
ζj
tj , x̂j , ûj , λj+1 ,
hj
!>
+
N−1
X
sx0 (tj , x̂j )> νj
j=i
Discrete transversality conditions:
λ0 = − `0 ϕ0x0 (x̂0 , x̂N )> + ψx00 (x̂0 , x̂N )> κ
λN
=
`0 ϕ0xf (x̂0 , x̂N )> + ψx0f (x̂0 , x̂N )> κ + sx0 (tN , x̂N )> νN
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Approximation of Adjoints
Theorem (Discrete Minimum Principle (continued))
I
Discrete stationarity conditions: For i = 0, . . . , N − 1 we have
ζi
Ĥ0u ti , x̂i , ûi , λi+1 ,
=0
hi
I
Discrete complementarity conditions:
0 ≤ ζi ,
ζi> c(ti , x̂i , ûi ) = 0,
0 ≤ νi ,
νi> s(ti , x̂i )
= 0,
i = 0, . . . , N − 1
i = 0, . . . , N
Proof: evaluation of Fritz John conditions from nonlinear programming
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Approximation of Adjoints – Comparison with Minimum Principle
Adjoints:
I Discrete:
N−1
λi
=
λN +
X
0
hj Ĥx
tj , x̂j , ûj , λj+1 ,
j=i
λN
Interpretation:
=
0
`0 ϕx (x̂0 , x̂N )
f
>
0
f
>
ζj
hj
N−1
>
+
X
0
0
>
sx (tj , x̂j ) νj
j=i
>
+ ψx (x̂0 , x̂N ) κ + sx (tN , x̂N ) νN
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Approximation of Adjoints – Comparison with Minimum Principle
Adjoints:
I Discrete:
N−1
λi
=
λN +
X
0
hj Ĥx
tj , x̂j , ûj , λj+1 ,
j=i
λN
I
=
0
`0 ϕx (x̂0 , x̂N )
>
f
0
f
ζj
hj
N−1
>
+
X
0
>
sx (tj , x̂j ) νj
j=i
>
0
>
+ ψx (x̂0 , x̂N ) κ + sx (tN , x̂N ) νN
Continuous:
tf
Z
λ(t)
=
λ(tf ) +
t
λ(tf )
Interpretation:
=
0
0
`0 ϕx (x(t0 ), x(tf ))
f
>
Ĥx (t, x̂(t), û(t), λ(t), η(t)) dt +
>
0
f
>
+ ψx (x(t0 ), x(tf )) σ
tf
Z
t
0
>
sx (t, x̂(t)) dµ(t)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Approximation of Adjoints – Comparison with Minimum Principle
Adjoints:
I Discrete:
N−1
λi
=
λN +
X
0
hj Ĥx
tj , x̂j , ûj , λj+1 ,
j=i
λN
I
=
0
`0 ϕx (x̂0 , x̂N )
>
f
0
f
N−1
>
+
X
0
>
sx (tj , x̂j ) νj
j=i
>
0
>
+ ψx (x̂0 , x̂N ) κ + sx (tN , x̂N ) νN
Continuous:
tf
Z
λ(t)
=
λ(tf ) +
t
λ(tf )
=
0
>
Ĥx (t, x̂(t), û(t), λ(t), η(t)) dt +
0
`0 ϕx (x(t0 ), x(tf ))
f
>
κ ≈ σ,
λi ≈ λ(ti ),
ζi
hi
0
f
>
+ ψx (x(t0 ), x(tf )) σ
Interpretation:
I
ζj
hj
≈ η(ti )
tf
Z
t
0
>
sx (t, x̂(t)) dµ(t)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Approximation of Adjoints – Comparison with Minimum Principle
Adjoints:
I Discrete:
N−1
λi
=
λN +
X
0
hj Ĥx
tj , x̂j , ûj , λj+1 ,
j=i
λN
I
=
0
`0 ϕx (x̂0 , x̂N )
>
f
0
f
ζj
hj
N−1
>
+
X
0
>
sx (tj , x̂j ) νj
j=i
>
0
>
+ ψx (x̂0 , x̂N ) κ + sx (tN , x̂N ) νN
Continuous:
tf
Z
λ(t)
=
λ(tf ) +
t
λ(tf )
=
0
>
Ĥx (t, x̂(t), û(t), λ(t), η(t)) dt +
0
`0 ϕx (x(t0 ), x(tf ))
f
>
0
f
Interpretation:
I
κ ≈ σ,
λi ≈ λ(ti ),
I
νi ≈ µ(ti+1 ) − µ(ti ),
ζi
hi
>
+ ψx (x(t0 ), x(tf )) σ
≈ η(ti )
i = 0, . . . , N − 1
tf
Z
t
0
>
sx (t, x̂(t)) dµ(t)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Approximation of Adjoints – Comparison with Minimum Principle
Adjoints:
I Discrete:
N−1
λi
=
λN +
X
0
hj Ĥx
tj , x̂j , ûj , λj+1 ,
j=i
λN
I
=
0
`0 ϕx (x̂0 , x̂N )
>
f
0
f
+
X
0
>
sx (tj , x̂j ) νj
j=i
>
0
>
+ ψx (x̂0 , x̂N ) κ + sx (tN , x̂N ) νN
Continuous:
tf
Z
λ(t)
=
λ(tf ) +
t
λ(tf )
=
0
>
Ĥx (t, x̂(t), û(t), λ(t), η(t)) dt +
0
`0 ϕx (x(t0 ), x(tf ))
f
>
0
f
>
+ ψx (x(t0 ), x(tf )) σ
Interpretation:
λi ≈ λ(ti ),
ζi
hi
I
κ ≈ σ,
I
νi ≈ µ(ti+1 ) − µ(ti ),
i = 0, . . . , N − 1
νN is interpreted as jump height at t = tf , i.e.
I
N−1
>
ζj
hj
≈ η(ti )
−
νN ≈ µ(tf ) − µ(tf ).
Note that µ can jump at tf .
tf
Z
t
0
>
sx (t, x̂(t)) dµ(t)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Approximation of Adjoints – Comparison with Minimum Principle
Adjoints:
I Discrete:
N−1
λi
=
λN +
X
0
hj Ĥx
tj , x̂j , ûj , λj+1 ,
j=i
λN
I
=
0
`0 ϕx (x̂0 , x̂N )
>
f
0
f
N−1
>
ζj
hj
+
X
0
>
sx (tj , x̂j ) νj
j=i
>
0
>
+ ψx (x̂0 , x̂N ) κ + sx (tN , x̂N ) νN
Continuous:
tf
Z
λ(t)
=
λ(tf ) +
t
λ(tf )
=
0
0
`0 ϕx (x(t0 ), x(tf ))
f
>
0
f
tf
Z
>
Ĥx (t, x̂(t), û(t), λ(t), η(t)) dt +
t
0
>
sx (t, x̂(t)) dµ(t)
>
+ ψx (x(t0 ), x(tf )) σ
Interpretation:
κ ≈ σ,
I
νi ≈ µ(ti+1 ) − µ(ti ),
i = 0, . . . , N − 1
νN is interpreted as jump height at t = tf , i.e.
I
λi ≈ λ(ti ),
ζi
hi
I
≈ η(ti )
−
νN ≈ µ(tf ) − µ(tf ).
Note that µ can jump at tf .
I
−
λN is interpreted as the left-sided limit of λ at tf , i.e. λN ≈ λ(tf ). Note that λ may jump at tf .
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Approximation of Adjoints – Comparison with Minimum Principle
Complementarity conditions:
I Discrete:
0 ≤ νi
I
Continuous:
Z tf
s(t, x̂(t))> dµ(t)
⊥
−s(ti , x̂i ) ≥ 0,
Z
tf
0,
=
i = 0, . . . , N
h(t)> dµ(t) ≥ 0
∀0 ≤ h ∈ C(I, Rns )
t0
t0
Interpretation implies:
I for h = (h , . . . , h )> ≥ 0 the condition
0
N
0 ≤ νi ≈ µ(ti+1 ) − µ(ti )
“monotonicity of µ”
implies
0≤
N
X
hi> νi ≈
i=0
I
tf
Z
h(t)> dµ(t)
t0
Summing up νi> s(ti , x̂i ) = 0 yields
0=
N
X
i=0
νi> s(ti , x̂i ) ≈
Z
tf
t0
s(t, x̂(t))> dµ(t)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Contents
Direct Discretization Methods
Full Discretization
Reduced Discretization and Shooting Methods
Adjoint Estimation
Convergence
Examples
Mixed-Integer Optimal Control
Variable Time Transformation
Parametric Sensitivity Analysis and Realtime-Optimization
Parametric Optimal Control Problems
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Convergence for Nonlinear Problems
I
order O(h) for mixed control-state constraints and Euler discretization, see [3]
Assumptions: u∗ is continuous, ...
I
order O(h) for mixed control-state and pure state constraints and Euler
discretization, see [1]
Assumptions: u∗ ∈ W 1,∞ , ...
I
second order O(h2 ), see [2]
Assumptions: u∗ ∈ W 1,∞ and u∗0 is of bounded variation, ...
I
order O(hp ) for certain Runge-Kutta methods, see [4]
(p−1)
Assumptions: u∗
is of bounded variation, ...
[1]
A. L. Dontchev, W. W. Hager, and K. Malanowski.
Error Bounds for Euler Approximation of a State and Control Constrained Optimal Control Problem.
Numerical Functional Analysis and Optimization, 21(5 & 6):653–682, 2000.
[2]
A. L. Dontchev, W. W. Hager, and V. M. Veliov.
Second-Order Runge-Kutta Approximations in Control Constrained Optimal Control.
SIAM Journal on Numerical Analysis, 38(1):202–226, 2000.
[3]
K. Malanowski, C. Büskens, and H. Maurer.
Convergence of Approximations to Nonlinear Optimal Control Problems.
In Anthony Fiacco, editor, Mathematical programming with data perturbations, volume 195, pages 253–284. Dekker. Lecture Notes in
Pure and Applied Mathematics, 1997.
[4]
W. W. Hager.
Runge-Kutta methods in optimal control and the transformed adjoint system.
Numerische Mathematik, 87(2):247–282, 2000.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Convergence for Linear and Linear-Quadratic Problems
I
I
Linear control problems, Euler discretization, see [3]
order O(h) in L1 -norm for u
Assumptions: bang-bang control, ...
Linear control problems, Euler discretization, see [1]
(i) order O(h) in L1 -norm for u
√
(ii) order O( h) in L2 -norm for u
I
order O(h) in L∞ -norm for x
Assumptions: no singular arcs, finite number of switches, structural stability, ...
Linear-quadratic control problems, Euler discretization, see [2]
(i) order O(h) in L1 -norm for u
(ii) order O(h) in L∞ -norm for x
Assumptions: no singular arcs, finite number of switches, structural stability, ...
[1]
W. Alt, R. Baier, M. Gerdts, and F. Lempio,
Approximation of linear control problems with bang-bang solutions,
Optimization: A Journal of Mathematical Programming and Operations Research, (2011), DOI 10.1080/02331934.2011.568619.
[2]
W. Alt, R. Baier, M. Gerdts, and F. Lempio,
Error bounds for Euler approximation of linear-quadratic control problems with bang-bang solutions,
Numerical Algebra, Control and Optimization, 2 (2012), 547–570.
[3]
V. M. Veliov,
Error analysis of discrete approximations to bang-bang optimal control problems: the linear case,
Control Cybern., 34 (2005), 967–982.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Contents
Direct Discretization Methods
Full Discretization
Reduced Discretization and Shooting Methods
Adjoint Estimation
Convergence
Examples
Mixed-Integer Optimal Control
Variable Time Transformation
Parametric Sensitivity Analysis and Realtime-Optimization
Parametric Optimal Control Problems
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Virtual Test-Drive: Double-Lane-Change Maneuver
Boundary: piecewise polynomials
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Virtual Test-Drive: Double-Lane-Change Maneuver
δ
Minimize
tf
Z
tf + α
2
u(t) dt
0
l
subject to
0
=
v (t) cos ψ(t),
0
=
v (t) sin ψ(t),
0
ψ (t)
=
0
=
v (t)
tan δ(t)
`
u2 (t)
0
=
u1 (t)
x (t)
y (t)
v (t)
δ (t)
δ
ψ
(x,y)
Notation:
and
δ
steering angle
δ(t) ∈ [−30, 30]
in [degree]
v
velocity
v (t) ∈ [0, 6]
in [m/s]
ψ
yaw angle
`
distance from rear axle
to front axle
(x, y)
midpoint position of rear
axle of car
u1 (t) ∈ [−30, 30]
in [degree/s]
u2 (t) ∈ [−1, 1]
in [m/s ]
+ initial conditions & track bounds
2
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Virtual Test-Drive: Double-Lane-Change Maneuver
State 2 vs State 1
State 3 vs time
3.5
state 3
3
y
2.5
2
1.5
1
0.5
0
0
2
4
6
8
10
12
State 4 vs time
1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
-1
-1.2
14
0.2
3.5
3
-0.2
0
0.4
0.6
Control 2 vs time
0.8
1
0.4
0.6
0.8
1
0
0
adjoint 2
0
-0.5
-0.15
-0.2
-0.25
-0.3
-1.5
0.2
0.4
0.6
0.8
1
-0.35
0
0.2
0.4
t
Adjoint 3 vs time
1
0
0.2
0.4
0.4
0.6
t
0.8
1
0.6
0.8
1
0
0.4
0.05
8.34
0.04
8.32
0.03
8.3
0.02
8.28
0.01
0
t
0.8
1
1
8.26
8.24
8.22
8.18
-0.04
0.6
0.8
8.2
-0.03
0.4
0.6
t
-0.02
0.2
1
Parameter 1 vs time
-0.01
0
0.2
t
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
-1
-1.2
-1.4
-1.6
-1.8
0.8
0.1
0.05
0
-0.05
-0.1
-0.15
-0.2
-0.25
-0.3
-0.35
-0.4
Adjoint 5 vs time
adjoint 5
adjoint 4
0.2
0.8
Adjoint 4 vs time
0.07
0.06
0.05
0.04
0.03
0.02
0.01
0
-0.01
-0.02
-0.03
-0.04
0
0.6
t
parameter 1
0
0.6
t
-1
-0.6
0.4
Adjoint 2 vs time
-0.05
-0.2
-0.4
0.2
Adjoint 1 vs time
adjoint 1
control 2
control 1
0.2
t
0.2
adjoint 3
-0.3
0
-0.1
0
0
-0.1
0.5
0.2
Control 1 vs time
1
2
1
t
0.5
0.1
2.5
1.5
0
0.4
0.3
4
x
0.6
State 5 vs time
4.5
state 5
4
state 4
4.5
8.16
0
0.2
0.4
0.6
t
0.8
1
0
0.2
0.4
0.6
t
0.8
1
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
A Coupled ODE-PDE Optimal Control Problem
Truck with a water tank: horizontal control force u
dW
dT
L
y
B(x) + h(t,x)
h
yT
B(x)
x
dT
d
mT : mass of truck, mW : mass of water tank, dT : distance truck, dW : distance water tank, B(x):
bottom elevation of tank, h(t, x): water level relative to bottom at time t and position x, v (t, x):
horizontal water velocity at time t and position x, L: length of water tank, c : spring force constant,
k: damper force constant
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
A Coupled ODE-PDE Optimal Control Problem
Coupled ODE-PDE system: (Ω := (0, T ) × (0, L))
(
0
mT dT00
= u − F (dT , dW , dT0 , dW
),
dT (0) = 0, dT0 (0) = (dT0 )0 , dT (T ) = d
ODE
00
0
0
0
T
0
mW dW
= F (dT , dW , dT , dW ) + mW aW (t), dW (0) = dW
, dW (T ) = dW
, dW
(0) =


h + (hv )x
= 0,
(t, x) ∈ Ω
t
Saint-Venant
0
 vt + 12 v 2 + gh
= −gBx − m1 F (dT , dW , dT0 , dW
)
W
x


h(0,
x)
=
h
(x),
x ∈ [0, L]
0





v (0, x) = v0 (x),
x ∈ [0, L]



v (t, 0) = v (t, L) = 0,
t ∈ [0, T ]
i.c./b.c.


0 
) , t ∈ [0, T ]
hx (t, 0) = −Bx (0) − g·m1 F (dT , dW , dT0 , dW


W
t



0 
hx (t, L) = −Bx (L) − g·m1 F (dT , dW , dT0 , dW
) , t ∈ [0, T ]
W
t
spring/damper force and total acceleration of water tank:
0
0
F (dT , dW , dT , dW )
=
aW (t)
=
=
0
0
c(dT − dW + d̄) + k(dT − dW )
Z L
1
vt (t, x)dx
L 0
g
1
g
0
0
− (B(L) − B(0)) −
F (dT , dW , dT , dW ) − (h(t, L) − h(t, 0))
L
mW
L
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
A Coupled ODE-PDE Optimal Control Problem
Approximation: Lax-Friedrich scheme (1st order in time, 2nd order in space, explicit)
n
n
hi ≈ h(tn , xi ),
vi ≈ v (tn , xi )
on equidistant grid (obey CFL condition!)
(tn , xi )
:=
(n∆t, i∆x)
(n = 0, . . . , N, i = 0, . . . , M, ∆t = T /N, ∆x = L/M)
Discretized Saint-Venant equations: For i = 0, . . . , M − 1 and n = 0, . . . , N − 1,
1
1 n
1 n n
n+1
n
n
n
hi
−
hi+1 + hi−1
+
hi+1 vi+1 − hi−1 vi−1 = 0,
∆t
2
2∆x
1
1 n 2
1 n
1 n 2
1
n
n
n
n+1
+
vi
−
vi+1 + vi−1
(vi+1 ) + ghi+1 − (vi−1 ) − ghi−1
∆t
2
2∆x
2
2
1
0
0
= −gBx (xi ) −
F (dT (tn ), dW (tn ), dT (tn ), dW (tn ))
mW
+ boundary and initial conditions
Optimal control problem (braking maneuver)
Minimize
0
2
0
2
α0 T + α1 (dT (T ) + dW (T ) )
(T free)
subject to the discretized Saint-Venant equations, truck dynamics, and control bounds.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
A Coupled ODE-PDE Optimal Control Problem
Results:
Trolley & Saint-Venant Equations
Trolley & Saint-Venant Equations
2.5
2.5
2
2
1.5
h [m]
1.5
v [m]
1
1
0.5
0.5
0
2.5
2
1.5
1
0.5
0
-0.5
-1
-1.5
-2
-2.5
2.5
2
1.5
1
0.5
0
-0.5
-1
-1.5
-2
-2.5
0
4
3.5
4
3
2.5
2
x [m]
1.5
1
0.5
0 0
4
2
6
10
8
time [s]
12
3.5
x [m]
3
2.5
2
Trolley & Saint-Venant Equations
1.5
1
0.5
0 0
2
6
4
10
8
time [s]
12
Trolley & Saint-Venant Equations
5000
10000
5000
0
0
-5000
F [N]
u [N]
-5000
-10000
-10000
-15000
-15000
-20000
-20000
-25000
-25000
0
2
4
6
time [s]
8
10
12
0
2
4
6
time [s]
8
10
12
(α0 = 0.1, α1 = 100, dT (T ) = 100, dW (T ) = 95, d 0 (0) = 10, d 0 (0) = 10, N = 600, M = 20, L = 4, mT = 2000,
T
W
mW = 4000, h0 = 1, c = 40000, k = 10000, g = 9.81)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Contents
Direct Discretization Methods
Full Discretization
Reduced Discretization and Shooting Methods
Adjoint Estimation
Convergence
Examples
Mixed-Integer Optimal Control
Variable Time Transformation
Parametric Sensitivity Analysis and Realtime-Optimization
Parametric Optimal Control Problems
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Optimal Control Problem
Mixed-Integer OCP
Minimize
s.t.
ϕ(x(t0 ), x(tf ))
ẋ(t) − f (t, x(t), u(t), v (t))
=
0
s(t, x(t))
≤
0
ψ(x(t0 ), x(tf ))
=
0
u(t)
∈
U
v (t)
∈
V
Notation
state:
x ∈ W 1,∞ ([t0 , tf ], Rnx )
controls:
u ∈ L∞ ([t0 , tf ], Rnu ), v ∈ L∞ ([t0 , tf ], Rnv )
U ⊆ Rnu convex, closed, V = {v1 , . . . , vM } discrete
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Solution Approaches
Indirect approach:
exploit necessary optimality conditions (minimum principle)
Direct discretization approaches:
(a) variable time transformation [Lee, Teo, Jennings, Rehbock, G., ...]
(b) Branch & Bound [von Stryk, G., ...]
(c) sum-up-rounding strategy [Sager]
(d) stochastic/heuristic optimization [Schlüter/G.,...]
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Contents
Direct Discretization Methods
Full Discretization
Reduced Discretization and Shooting Methods
Adjoint Estimation
Convergence
Examples
Mixed-Integer Optimal Control
Variable Time Transformation
Parametric Sensitivity Analysis and Realtime-Optimization
Parametric Optimal Control Problems
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Discretization
Main grid
Gh : ti = t0 + ih,
i = 0, . . . , N, h =
tf − t0
N
Minor grid
Gh,M : τi,j = ti + j
h
,
M
j = 0, . . . , M, i = 0, . . . , N − 1
M = number of discrete values in V = {v1 , . . . , vM }
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Idea
Replace the discrete control v by a fixed and piecewise constant function on the minor
grid according to
vGh,M (τ ) = vj
for τ ∈ [τi,j−1 , τi,j ], i = 0, . . . , N − 1, j = 1, . . . , M
vM
.
..
v1
ti−1
τi−1,j
ti
τi,j
ti+1
τi+1 , j
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Idea: Variable Time Transformation
Variable time transformation:
τ
Z
t = t(τ ),
w(s)ds,
t(τ ) := t0 +
τ ∈ [t0 , tf ]
t0
and
tf
Z
tf − t0 =
w(s)ds
t0
Remark:
I
w is the speed of running through [t0 , tf ]:
dt
= w(τ ),
dτ
I
τ ∈ [t0 , tf ]
w(τ ) = 0 in [τi,j , τi,j+1 ) ⇒ [t(τi,j ), t(τi,j+1 )] shrinks to {t(τi,j )}
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Time Transformation
w(τ )
ti+2
t(τ )
ti+1
ti
ti−1
ti−1
τi−1,j
ti
τi,j
ti+1
τi+1 , j
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
New Control
Consider w as new control subject to the restrictions:
I
w(τ ) ≥ 0 for all τ (→ no running back in time)
I
w(τ ) piecewise constant on the minor grid Gh,M
I
Major grid points are invariant under time transformation:
ti+1
Z
w(τ )dτ = ti+1 − ti = h,
i = 0, . . . , N − 1
ti
Control set:





W := w ∈ L∞ ([t0 , tf ], R)




w(τ ) ≥ 0,
w piecewise constant on Gh,M ,
Z
ti+1
w(τ )dτ = ti+1 − ti ∀i
ti









Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Backtransformation
w(τ ):
vGh,M (τ ):
Corresponding control v (s) = vGh,M (t −1 (s)):
ti−1
ti
t(τi,1 )
= t(τi,2 )
ti+1
t(τi+1,1 )
= t(τi+1,2 )
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Transformed Optimal Control Problem
TOCP
Minimize
s.t.
ϕ(x(t0 ), x(tf ))
ẋ(τ ) − w(τ )f (τ, x(τ ), u(τ ), vGh,M (τ ))
=
0
s(τ, x(τ ))
≤
0
ψ(x(t0 ), x(tf ))
=
0
u(τ )
∈
U
w
∈
W
Remarks:
I
If w(τ ) ≡ 0 in [τi,j , τi,j+1 ] then x remains constant therein!
I
vGh,M is the fixed function defined before!
I
TOCP has only ‘continuous’ controls, no discrete controls anymore!
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Virtual Test-Drives: Optimal Control Problem
Simulation of Test-Drives:
I
model of automobile
→ differential equations
I
model of test-course
→ state constraints, boundary conditions
I
model of driver
→ optimal control problem
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Single-Track Model
Usage: basic investigation of lateral dynamics (up to 0.4g), controller design
lf
eSP
lr
vf
αf
δ
1
0
0
1
0
1
v
β
1
0
0
1
(x, y )
αr
vr
Flr
Fsr
Fsf
ψ
Flf
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Equations of Motion
ẋ
=
v cos(ψ − β)
ẏ
=
v̇
=
β̇
=
v sin(ψ − β)
h
(Flr − FAx ) cos β + Flf cos(δ + β) − Fsr − FAy sin β
i
−Fsf sin(δ + β) /m
h
wz − (Flr − FAx ) sin β + Flf sin(δ + β)
i
+ Fsr − FAy cos β + Fsf cos(δ + β) /(m · v )
ψ̇
=
ẇ z
=
wz
h
i
Fsf · lf · cos δ − Fsr · lr − FAy · eSP + Flf · lf · sin δ /Izz
δ̇
=
wδ
Lateral tire forces Fsf , Fsr : ‘magic formula’ [Pacejka’93]
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Equations of Motion: Details
Longitudinal tire forces (rear wheel drive):
Flf = −FBf − FRf ,
Flr =
Mwheel (φ, µ)
− FBr − FRr
R
(FRf , FRr rolling resistances front/rear)
Power train torque: [Neculau’92]
Mwheel (φ, µ) = ig (µ) · it · Mmot (φ, µ)
(ig : gear transmission, it : motor torque transmission,
Mmot : motor torque)
Controls:
wδ
∈
[wmin , wmax ]
:
steering angle velocity
FBf , FBr
∈
[Fmin , Fmax ]
:
braking forces front/rear
φ
∈
[0, 1]
:
gas pedal position
µ
∈
{1, 2, 3, 4, 5}
:
gear
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Result: 20 grid points
steering angle velocity wδ :
gear shift µ:
0.03
5
0.02
4
0.01
0
3
-0.01
2
-0.02
-0.03
1
0
0.2
0.4
0.6
0.8
1
0
0.2
0.4
Braking force: FB (t) ≡ 0, Gas pedal position: φ(t) ≡ 1
Complete enumeration (1 [s] to solve DOCP): ≈ 3 · 106 years
Branch & Bound: 23 m 52 s, objective value: 6.781922
Transformation: 2 m 1.154 s, objective value: 6.774669
0.6
0.8
1
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Result: 40 grid points
Steering angle velocity wδ :
Gear shift µ:
0.06
0.05
5
0.04
0.03
4
0.02
0.01
3
0
-0.01
2
-0.02
-0.03
1
-0.04
-0.05
0
0.2
0.4
0.6
0.8
1
0
0.2
0.4
Braking force: FB (t) ≡ 0, Gas pedal position: φ(t) ≡ 1
Complete enumeration (1 [s] to solve DOCP): ≈ 3 · 1020 years
Branch & Bound: 232 h 25 m 31 s, objective value: 6.791374
Transformation: 9 m 39.664 s, objective value: 6.787982
0.6
0.8
1
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Result: 80 grid points
Steering angle velocity wδ :
Gear shift µ:
0.08
5
0.06
0.04
4
0.02
3
0
2
-0.02
-0.04
-0.06
1
0
0.2
0.4
0.6
0.8
1
0
0.2
Braking force: FB (t) ≡ 0, Gas pedal position: φ(t) ≡ 1
65 m 3.496 s, objective value: 6.795366
0.4
0.6
0.8
1
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
F8 Aircraft
Minimize T subject to
x10
=
−0.877x1 + x3 − 0.088x1 x3 + 0.47x12 − 0.019x22
−x12 x3 + 3.846x13 − 0.215v + 0.28x12 v + 0.47x1 v 2 + 0.63v 3
x20
x30
=
x3
=
−4.208x1 − 0.396x3 − 0.47x12 − 3.564x13 − 20.967v
+6.265x12 v + 46x1 v 2 + 61.4v 3
v
∈
{−0.05236, 0.05236}
x(0) = (0.4655, 0, 0)> , x(T ) = (0, 0, 0)>
Source: http://mintoc.de by Sebastian Sager
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
F8 Aircraft (N = 500, T = 5.728674)
state 1
state 2
0.5
0.2
0.45
0.1
0.4
0.35
0
0.3
-0.1
0.25
0.2
-0.2
0.15
-0.3
0.1
0.05
-0.4
0
-0.05
-0.5
0
0.2
0.4
0.6
normalized time
0.8
1
0
0.2
state 3
0.4
0.6
normalized time
0.8
1
0.8
1
control v (normalized values)
0.6
2
0.4
1.8
0.2
1.6
0
1.4
-0.2
1.2
-0.4
-0.6
1
0
0.2
0.4
0.6
normalized time
0.8
1
0
0.2
0.4
0.6
normalized time
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Lotka-Volterra Fishing Problem
Minimize
12
Z
(x1 (t) − 1)2 + (x2 (t) − 1)2 dt
0
subject to
x10 (t)
=
x1 (t) − x1 (t)x2 (t) − 0.4x1 (t)v (t)
x20 (t)
=
−x2 (t) + x1 (t)x2 (t) − 0.2x2 (t)v (t)
v (t)
∈
{0, 1}
x(0)
=
(0.5, 0.7)>
Source: http://mintoc.de by Sebastian Sager
Background: optimal fishing strategy on a fixed time horizon to bring the biomasses of
both predator as prey fish to a prescribed steady state
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Lotka-Volterra Fishing Problem – Solutions
Relaxed Solution:
No switching costs:
Lotka-Volterra Fishing Problem (relaxed)
1.8
Lotka-Volterra Fishing Problem
1.8
control v1
state x1
state x2
1.6
control v1
state x1
state x2
1.6
1.4
1.4
1.2
1.2
1
1
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0
-0.2
0
0
2
4
6
time t
8
10
12
0
2
4
6
time t
With Switching costs:
Lotka-Volterra Fishing Problem
1.8
control v1
statex1
state x2
1.6
1.4
1.2
1
0.8
0.6
0.4
0.2
0
0
2
4
6
time t
8
10
12
8
10
12
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
References
[1]
M. Gerdts.
Solving Mixed-Integer Optimal Control Problems by Branch&Bound: A Case Study from Automobile Test-Driving with Gear Shift.
Optimal Control, Applications and Methods, 26(1):1–18, 2005.
[2]
M. Gerdts.
A variable time transformation method for mixed-integer optimal control problems.
Optimal Control, Applications and Methods, 27(3):169–182, 2006.
[3]
S. Sager.
Numerical methods for mixed-integer optimal control problems.
PhD thesis, Naturwissenschaftlich-Mathematische Gesamtfakultät, Universität Heidelberg, Heidelberg, Germany, 2006.
[4]
S. Sager.
MIOCP benchmark site. http://mintoc.de
[5]
S. Sager, H. G. Bock, and G. Reinelt.
Direct methods with maximal lower bound for mixed-integer optimal control problems.
Mathematical Programming (A), 118(1):109–149, 2009.
[6]
H. W. J. Lee, K. L. Teo, V. Rehbock, and L. S. Jennings.
Control parametrization enhancing technique for optimal discrete-valued control problems.
Automatica, 35(8):1401–1407, 1999.
[7]
A. Siburian.
Numerical Methods for Robust, Singular and Discrete Valued Optimal Control Problems.
PhD thesis, Curtin University of Technology, Perth, Australia, 2004.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Contents
Direct Discretization Methods
Full Discretization
Reduced Discretization and Shooting Methods
Adjoint Estimation
Convergence
Examples
Mixed-Integer Optimal Control
Variable Time Transformation
Parametric Sensitivity Analysis and Realtime-Optimization
Parametric Optimal Control Problems
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Parametric Optimization
NLP(p)
Minimize
s.t.
f (z, p)
gi (z, p)
=
0,
i = 1, . . . , nE ,
gi (z, p)
≤
0,
i = nE + 1, . . . , ng
Notation:
I
f , gi : Rnz × Rnp −→ R, i = 1, . . . , ng , twice continuously differentiable
I
parameter p ∈ Rnp (no optimization variable!)
I
Active inequalities: I(z, p) := {i ∈ {nE + 1, . . . , ng } | gi (z, p) = 0}
I
Active set: A(z, p) := {1, . . . , nE } ∪ I(z, p)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Parametric Optimization
Definition
z ∗ strongly regular local minimum of NLP(p0 ) iff
(a) z ∗ is feasible.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Parametric Optimization
Definition
z ∗ strongly regular local minimum of NLP(p0 ) iff
(a) z ∗ is feasible.
(b) z ∗ fulfills the linear independence constraint qualification (LICQ).
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Parametric Optimization
Definition
z ∗ strongly regular local minimum of NLP(p0 ) iff
(a) z ∗ is feasible.
(b) z ∗ fulfills the linear independence constraint qualification (LICQ).
(c) The KKT conditions hold at (z ∗ , λ∗ ) with Lagrange multiplier λ∗ .
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Parametric Optimization
Definition
z ∗ strongly regular local minimum of NLP(p0 ) iff
(a) z ∗ is feasible.
(b) z ∗ fulfills the linear independence constraint qualification (LICQ).
(c) The KKT conditions hold at (z ∗ , λ∗ ) with Lagrange multiplier λ∗ .
(d) The strict complementarity condition holds:
∗
λ∗
i − gi (z , p0 ) > 0
for all i = nE + 1, . . . , ng .
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Parametric Optimization
Definition
z ∗ strongly regular local minimum of NLP(p0 ) iff
(a) z ∗ is feasible.
(b) z ∗ fulfills the linear independence constraint qualification (LICQ).
(c) The KKT conditions hold at (z ∗ , λ∗ ) with Lagrange multiplier λ∗ .
(d) The strict complementarity condition holds:
∗
λ∗
i − gi (z , p0 ) > 0
for all i = nE + 1, . . . , ng .
(e) Second-order sufficient condition:
∗
∗
L00
zz (z , λ , p0 )(d, d) > 0
for all 0 6= d ∈ TC (z ∗ , p0 ) with the critical cone

g 0 (z, p)(d) ≤


i,z

0
TC (z, p) := d ∈ Rnz gi,z
(z, p)(d) =



0
g (z, p)(d) =
i,z

0, i ∈ I(z, p), λi = 0, 


.
0, i ∈ I(z, p), λi > 0,



0, i ∈ {1, . . . , n }
E
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Parametric Optimization
Sensitivity Theorem [Fiacco’83]
Let z ∗ be a strongly regular local minimum of (NLP(p0 )) for nominal parameter p0 .
Then there exist neighborhoods B (p0 ) and Bδ (z ∗ , λ∗ ) with:
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Parametric Optimization
Sensitivity Theorem [Fiacco’83]
Let z ∗ be a strongly regular local minimum of (NLP(p0 )) for nominal parameter p0 .
Then there exist neighborhoods B (p0 ) and Bδ (z ∗ , λ∗ ) with:
I
NLP(p) has unique strongly regular local minimum
(z(p), λ(p)) ∈ Bδ (z ∗ , λ∗ )
for each p ∈ B (p0 ).
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Parametric Optimization
Sensitivity Theorem [Fiacco’83]
Let z ∗ be a strongly regular local minimum of (NLP(p0 )) for nominal parameter p0 .
Then there exist neighborhoods B (p0 ) and Bδ (z ∗ , λ∗ ) with:
I
NLP(p) has unique strongly regular local minimum
(z(p), λ(p)) ∈ Bδ (z ∗ , λ∗ )
for each p ∈ B (p0 ).
I
z(p), λ(p) are continuously differentiable w.r.t. p.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Parametric Optimization
Sensitivity Theorem [Fiacco’83]
Let z ∗ be a strongly regular local minimum of (NLP(p0 )) for nominal parameter p0 .
Then there exist neighborhoods B (p0 ) and Bδ (z ∗ , λ∗ ) with:
I
NLP(p) has unique strongly regular local minimum
(z(p), λ(p)) ∈ Bδ (z ∗ , λ∗ )
for each p ∈ B (p0 ).
I
z(p), λ(p) are continuously differentiable w.r.t. p.
I
The active set remains unchanged for each p ∈ B (p0 ):
A(z ∗ , p0 ) = A(z(p), p).
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Parametric Optimization
Sensitivity Theorem [Fiacco’83]
Let z ∗ be a strongly regular local minimum of (NLP(p0 )) for nominal parameter p0 .
Then there exist neighborhoods B (p0 ) and Bδ (z ∗ , λ∗ ) with:
I
NLP(p) has unique strongly regular local minimum
(z(p), λ(p)) ∈ Bδ (z ∗ , λ∗ )
for each p ∈ B (p0 ).
I
z(p), λ(p) are continuously differentiable w.r.t. p.
I
The active set remains unchanged for each p ∈ B (p0 ):
A(z ∗ , p0 ) = A(z(p), p).
Proof: implicit function theorem + stability of LICQ, critical cone and second-order
sufficient condition
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Applications
I
dependence of consistent initial values on parameters in DAEs
I
computation of gradients for bilevel optimization problems, e.g. Stackelberg
games, path planning under safety requirements (airraces)
I
parametric sensitivity analysis of reachable sets
I
model-predictive control: updates without re-optimization
see Vryan Palma’s talk Wednesday, September 11, 9:40 - 10:05
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Realtime Optimization [Büskens, Maurer]
Real-Time Approximation for NLP(p)
Given: (offline)
I
nominal parameter p0
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Realtime Optimization [Büskens, Maurer]
Real-Time Approximation for NLP(p)
Given: (offline)
I
nominal parameter p0
I
strongly regular local minimum z ∗
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Realtime Optimization [Büskens, Maurer]
Real-Time Approximation for NLP(p)
Given: (offline)
I
nominal parameter p0
I
strongly regular local minimum z ∗
I
sensitivity differentials
dz
(p0 )
dp
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Realtime Optimization [Büskens, Maurer]
Real-Time Approximation for NLP(p)
Given: (offline)
I
nominal parameter p0
I
strongly regular local minimum z ∗
I
sensitivity differentials
dz
(p0 )
dp
Taylor approximation/real-time approximation: (online)
For a perturbed parameter p 6= p0 compute
z̃(p) := z(p0 ) +
and use z̃(p) as an approximation of z(p).
dz
(p0 )(p − p0 )
dp
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Realtime Optimization [Büskens, Maurer]
Real-Time Approximation for NLP(p)
Given: (offline)
I
nominal parameter p0
I
strongly regular local minimum z ∗
I
sensitivity differentials
dz
(p0 )
dp
Taylor approximation/real-time approximation: (online)
For a perturbed parameter p 6= p0 compute
z̃(p) := z(p0 ) +
and use z̃(p) as an approximation of z(p).
Limitations:
dz
(p0 )(p − p0 )
dp
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Realtime Optimization [Büskens, Maurer]
Real-Time Approximation for NLP(p)
Given: (offline)
I
nominal parameter p0
I
strongly regular local minimum z ∗
I
sensitivity differentials
dz
(p0 )
dp
Taylor approximation/real-time approximation: (online)
For a perturbed parameter p 6= p0 compute
z̃(p) := z(p0 ) +
dz
(p0 )(p − p0 )
dp
and use z̃(p) as an approximation of z(p).
Limitations:
I
approximation error kz(p) − z̃(p)k = o(kp − p0 k)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Realtime Optimization [Büskens, Maurer]
Real-Time Approximation for NLP(p)
Given: (offline)
I
nominal parameter p0
I
strongly regular local minimum z ∗
I
sensitivity differentials
dz
(p0 )
dp
Taylor approximation/real-time approximation: (online)
For a perturbed parameter p 6= p0 compute
z̃(p) := z(p0 ) +
dz
(p0 )(p − p0 )
dp
and use z̃(p) as an approximation of z(p).
Limitations:
I
approximation error kz(p) − z̃(p)k = o(kp − p0 k)
I
constraint violation due to linearization (projection required if feasibility is an issue)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Realtime Optimization [Büskens, Maurer]
Real-Time Approximation for NLP(p)
Given: (offline)
I
nominal parameter p0
I
strongly regular local minimum z ∗
I
sensitivity differentials
dz
(p0 )
dp
Taylor approximation/real-time approximation: (online)
For a perturbed parameter p 6= p0 compute
z̃(p) := z(p0 ) +
dz
(p0 )(p − p0 )
dp
and use z̃(p) as an approximation of z(p).
Limitations:
I
approximation error kz(p) − z̃(p)k = o(kp − p0 k)
I
constraint violation due to linearization (projection required if feasibility is an issue)
I
linearization only justified in neighborhood B (p0 ), same active set
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Corrector Iteration for Feasibility [Büskens]
NLP(p)
Minimize
f (z, p)
s.t.
g(z, p) = 0
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Corrector Iteration for Feasibility [Büskens]
NLP(p)
Minimize
f (z, p)
s.t.
g(z, p) = 0
Realtime approximation:
z̃ [0] (p) := z̃(p) = z(p0 ) +
dz
(p0 )(p − p0 )
dp
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Corrector Iteration for Feasibility [Büskens]
NLP(p)
Minimize
f (z, p)
s.t.
g(z, p) = 0
Realtime approximation:
z̃ [0] (p) := z̃(p) = z(p0 ) +
dz
(p0 )(p − p0 )
dp
Introducing z̃(p) into constraints yields constraint violation
ε[0] := g(z̃(p), p)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Corrector Iteration for Feasibility [Büskens]
NLP(p)
Minimize
f (z, p)
s.t.
g(z, p) = 0
Realtime approximation:
z̃ [0] (p) := z̃(p) = z(p0 ) +
dz
(p0 )(p − p0 )
dp
Introducing z̃(p) into constraints yields constraint violation
ε[0] := g(z̃(p), p)
Idea of corrector iteration:
I perform sensitivity analysis w.r.t. ε for nominal parameter ε = 0
0
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Corrector Iteration for Feasibility [Büskens]
NLP(p)
Minimize
f (z, p)
s.t.
g(z, p) = 0
Realtime approximation:
z̃ [0] (p) := z̃(p) = z(p0 ) +
dz
(p0 )(p − p0 )
dp
Introducing z̃(p) into constraints yields constraint violation
ε[0] := g(z̃(p), p)
Idea of corrector iteration:
I perform sensitivity analysis w.r.t. ε for nominal parameter ε = 0
0
I apply correction (fixed point iteration):
z̃ [i+1] (p) := z̃ [i] (p) −
(note the minus sign!)
dz
(p0 , ε0 )ε[i] ,
dε
ε[i] := g(z̃ [i] (p), p)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Contents
Direct Discretization Methods
Full Discretization
Reduced Discretization and Shooting Methods
Adjoint Estimation
Convergence
Examples
Mixed-Integer Optimal Control
Variable Time Transformation
Parametric Sensitivity Analysis and Realtime-Optimization
Parametric Optimal Control Problems
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Optimal Control Problem
OCP
Minimize
s.t.
ϕ(x(t0 ), x(tf ), p)
x 0 (t) − f (x(t), u(t), p)
=
0
a.e. in [t0 , tf ]
c(x(t), u(t), p)
≤
0
a.e. in [t0 , tf ]
ψ(x(t0 ), x(tf ), p)
=
0
Notation
state:
x ∈ W 1,∞ ([t0 , tf ], IRnx )
control:
u ∈ L∞ ([t0 , tf ], IRnu )
parameter:
p (not an optimization variable!)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Realtime Optimization [Büskens, Maurer]
Real-Time Approximation for DOCP(p)
Given: (offline)
I
nominal parameter p0
I
strongly regular local minimum z ∗ of DOCP(p0 )
I
sensitivity differentials
dz
(p0 )
dp
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Realtime Optimization [Büskens, Maurer]
Real-Time Approximation for DOCP(p)
Given: (offline)
I
nominal parameter p0
I
strongly regular local minimum z ∗ of DOCP(p0 )
I
sensitivity differentials
dz
(p0 )
dp
Taylor approximation/real-time approximation: (online)
For a perturbed parameter p 6= p0 compute
x̃1 (p)
:=
ũi (p)
:=
dx1
(p0 )(p − p0 ),
dp
dui
ui (p0 ) +
(p0 )(p − p0 ).
dp
x1 (p) +
and use x̃1 (p) as an approximation of the initial value x1 (p) and ũi (p) as an
approximate optimal control for the parameter p.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Real-time optimal control: Application to DOCP(p)
Open-loop:
nominal solution
sensitivities
∆p
measurement:
parameter p
real-timecontrol
u(t, p)
x(t)
ẋ = f (x, u, p)
Open-loop real-time update formula (p: perturbation of p0 ):
(→ online)
ui (p) ≈ ui (p0 ) +
dui
(p0 ) · (p − p0 )
dp
(i = 0, . . . , N)
real-time control approximation: piecewise linear/constant with ui (p)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Example: Pendulum
Optimal control problem
Minimize
4.5
Z
2
u(t) dt
0
subject to
and
ẋ1 (t)
=
x3 (t) − 2x1 (t)y2 (t)
ẋ2 (t)
=
x4 (t) − 2x2 (t)y2 (t)
mẋ3 (t)
=
mẋ4 (t)
=
0
=
x1 (t) + x2 (t) − `
0
=
x1 (t)x3 (t) + x2 (t)x4 (t)
1
x1 (0) = −x2 (0) = √ ,
2
u(t)x2 (t)
`
u(t)x1 (t)
−mg − 2x2 (t)y1 (t) −
`
− 2x1 (t)y1 (t) +
2
2
2
x3 (0) = x4 (0) = 0,
and
−1 ≤ u(t) ≤ 1
x1 (4.5) = x3 (4.5) = 0
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Example: Pendulum
Nominal control and sensitivities du/dm and du/dg (nominal values m = 1, g = 9.81)
3
1.5
du/dm
u
2
1
1
0.5
0
0
-1
-0.5
-2
-1
-3
du/dg
u
-1.5
0
0.5
1
1.5
2
2.5
time [s]
3
3.5
4
4.5
0
0.5
1
1.5
2
2.5
time [s]
(smallest eigenvalue of reduced Hessian ≈ 0.0217 for N = 400)
3
3.5
4
4.5
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Full Car Model
BMW 1800/2000 [von Heydenaber’80]
ẋ(t)
=
f (x(t), y(t), u(t))
0
=
g(x(t), y(t), u(t))
Notation
state
:
x(t) ∈ IR37 , y(t) ∈ IR4
control
:
u(t) (steering angle velocity)
DAE
:
index 1, semi-explicit, gy0 non-sing.,
piecewise defined dynamics
(wheel: ground contact yes/np)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Real-time: BMW (10 %, 30 % perturbation)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Real-time optimal control: Numerical Results
Objective:
linear combination of final time and steering effort → min
Initial velocity:
25 [m/s]


p0 =

Real-time parameter:
offset 3.5 [m]
(→ state constraints)
centre of gravity 0.56 [m]
(→ car’s dynamics)
Nominal control:
Sensitivity differentials u:
1.5
1.4
1.2
1
1
0.8
0.5
0.6
0.4
0
0.2
-0.5
-0.2
0
-0.4
-1
0
0.2
0.4
0.6
0.8
1
-0.6
0
0.2
Sensitivity differentials tf :
dtf (p0 )
=
dp
0.4683 · 10−1
!
0.2574 · 10−1
Accuracy: Optimality/feasibility 10−11 , DSRTSE (10−11 diff, 10−4 alg), N = 81, M = 1
0.4
0.6
0.8
1
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Real-time optimal control: Numerical Results
Errors
Pert
objective
bound. cond. #1/#2
state constr.
control
structure
%
abs./rel.
9.9 · 10−1 /3.6 · 10−3
2.6 · 10−3 /9.6 · 10−6
2.4 · 10−3 /8.9 · 10−6
abs.
1.1 · 10−1 /4.5 · 10−3
1.5 · 10−4 /1.4 · 10−5
1.3 · 10−4 /1.5 · 10−5
max. abs.
2.9 · 10−1
max. abs./abs. L2
1.9 · 10−1 /7.6 · 10−2
1.7 · 10−3 /5.6 · 10−4
of control
diff.∗)
eq.
5.8 · 10−2 /2.1 · 10−4
1.9 · 10−1 /7.1 · 10−4
5.6 · 10−1 /2.0 · 10−3
2.0 · 10−3 /4.2 · 10−4
2.0 · 10−3 /1.8 · 10−3
4.1 · 10−2 /7.4 · 10−3
2.9 · 10−2
1.2 · 10−1
1.3 · 10−3 /4.3 · 10−4
4.3 · 10−2 /1.7 · 10−2
6.0 · 10−2 /2.6 · 10−2
4.6 · 10−1
1.7 · 10−1 /7.5 · 10−2
eq.
+5
+1
−1
−5
−10
−20
1.1 · 10−3
1.1 · 10−3
eq.
eq.
eq.
∗): Nominal control (left) and optimal control for perturbation of 5% (right)
1.5
2
1.5
1
1
0.5
0.5
0
0
-0.5
-0.5
-1
-1
-1.5
0
0.2
0.4
0.6
0.8
1
0
0.2
0.4
0.6
0.8
1
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Emergency Landing Manoeuvre in Realtime
I
Scenario: propulsion system breakdown
I
Goal: maximization of range w.r.t. current position
I
Controls: lift coefficient, angle of bank
I
no thrust available; no fuel consumption (constant mass)
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
References
[1] A. V. Fiacco.
Introduction to Sensitivity and Stability Analysis in Nonlinear Programming, volume 165 of Mathematics in Science and Engineering.
Academic Press, New York, 1983.
[2] J. F. Bonnans and A. Shapiro.
Perturbation Analysis of Optimization Problems.
Springer Series in Operations Research. Springer, New York, 2000.
[3] K. Malanowski and H. Maurer.
Sensitivity analysis for parametric control problems with control-state constraints.
Computational Optimization and Applications, 5(3):253–283, 1996.
[4] K. Malanowski and H. Maurer.
Sensitivity analysis for optimal control problems subject to higher order state constraints.
Annals of Operations Research, 101:43–73, 2001.
[5] H. Maurer and D. Augustin.
Sensitivity Analysis and Real-Time Control of Parametric Optimal Control Problems Using Boundary Value Methods.
In M. Grötschel, S. O. Krumke, and J. Rambau, editors, Online Optimization of Large Scale Systems, pages 17–55. Springer, 2001.
[6] H. Maurer and H. J. Pesch.
Solution differentiability for parametric nonlinear control problems with control-state constraints.
Journal of Optimization Theory and Applications, 86(2):285–309, 1995.
[7] C. Büskens.
Real-Time Solutions for Perturbed Optimal Control Problems by a Mixed Open- and Closed-Loop Strategy.
In M. Grötschel, S. O. Krumke, and J. Rambau, editors, Online Optimization of Large Scale Systems, pages 105–116. Springer, 2001.
[8] C. Büskens and H. Maurer.
Sensitivity Analysis and Real-Time Control of Parametric Optimal Control Problems Using Nonlinear Programming Methods.
In M. Grötschel, S. O. Krumke, and J. Rambau, editors, Online Optimization of Large Scale Systems, pages 56–68. Springer, 2001.
[9] H. J. Pesch.
Numerical computation of neighboring optimum feedback control schemes in real-time.
Applied Mathematics and Optimization, 5:231–252, 1979.
[10] H. J. Pesch.
Real-time computation of feedback controls for constrained optimal control problems. i, ii.
Optimal Control Applications and Methods, 10(2):129–145, 147–171, 1989.
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
References
J. T. Betts.
Practical methods for optimal control using nonlinear programming, volume 3 of
Advances in Design and Control.
SIAM, Philadelphia, 2001.
M. Gerdts.
Optimal Control of ODEs and DAEs.
DeGruyter, Berlin, 2011
Numerical Optimal Control – Part 2: Discretization techniques, structure exploitation, calculation of gradients –
Matthias Gerdts
Thanks for your Attention!
Questions?
Further information:
[email protected]
www.unibw.de/lrt1/gerdts
www.optimal-control.de
Fotos: http://de.wikipedia.org/wiki/München
Magnus Manske (Panorama), Luidger (Theatinerkirche), Kurmis (Chin. Turm), Arad Mojtahedi (Olympiapark), Max-k (Deutsches Museum), Oliver Raupach (Friedensengel), Andreas Praefcke (Nationaltheater)