CALCULUS OF VARIATIONS AND OPTIMIZATION METHODS
Part II. Optimization methods
11. Optimization control problems. Vector case
We considered optimization control problems with unique state function and unique control function. However there
exist the optimization problems with many states and controls. We will extend our previous results to this case. The
optimization problem for the flight of the missile will be considered as an example.
11.1. Problem statement
We extend the previous results to the vector case. Consider control system described by the
differential equations
(11.1)
x  f (t , u, x), t  (0, T )
with the initial conditions
х(0) = х0 ,
(11.2)
where x  x(t )   x1 (t ),..., xn (t )  is the state vector-function, u  u(t )   u1 (t ),..., ur (t )  is vector
control, f is the given vector-function, the vector x0   x01 ,..., x0n  is the given initial state. For all
control u we can determine the solution x of the problem (11.1), (11.2). The control u is chosen
from the set
U  u u (t )  G (t ), t  [0, t ] ,
where G (t ) is the given closed subset of the r-dimensional Euclid space. Consider the value
T
I   g  t , u (t ), x(t )  dt  h  x(T )  ,
0
where g and h are given functions.
Problem 11.1. Find the function u = u(t) from the set U, which minimize the value I.
We obtain Problem 10.2 for n  1, r  1, G (t )  [a(t ), b(t )] .
11.2. Maximum principle
Determine Lagrange function
T
n
L(u, x, р)  I (u, x)    рi (t )  xi (t )  f i  t , u (t ), x(t )   dt ,
0 i 1
where p  p(t )   p1 (t ),..., pn (t )  are Lagrange multipliers. If the function x is the solution of the
equation (11.3), then the value L is equal to I. Determine the function
n
H (t , u, x, p)   pi f i (t , u, x)  g (t , u, x).
(10.3)
i 1
Then we have the equality
 n

L(u, x, р)     рi (t ) xi (t )  Н  t , u (t ), x(t )  р(t )  dt  h  х(Т )  .

0  i 1
Suppose u is a solution of our problem (the optimal control). So we have then inequality
I = I(v,y) – I(u,x)  0 vU,
(11.4)
T
where х and у are the solutions of the problem (11.1), (11.2) for the controls u and v. Then previous
inequality can be transformed to
L = L(v,y,р) – L(u,x,р)  0 vU, р
(11.5)
Find the value
T
T
n
L    рi (t )  уi (t )  xi (t ) dt   Hdt  h,
0 i 1
0
where
H = H(t,v,y,р) – H(t,u,x,р), h = h[y(T)] – h[x(T)].
Suppose the functions of the problem statement are smooth enough. Using Taylor formula we get
n
h[ y (T )]  h[ x(T )  x(T )]  h[ x(T )]   hxi [ x(T )]xi (T )  1 ,
i 1
where x  y  x , hxi  h / dxi , and 1 is the high term with respect х(T). Then we have
n
H (t , v, y, p)  H (t , v, x  x, p)  H (t , v, x, p)   H xi (t , v, x, p) xi  2 
i 1
n
 H (t , v, x, p)   H xi (t , u, x, p) xi  2  3 ,
i 1
where H xi  H / xi , 2 is the high term with respect х,
n
3   [ H xi (t , v, x, p)  H xi (t , u, x, p)]xi .
i 1
So the inequality (13.5) can be transformed to
n
n


р
(
t
)

x
(
t
)
dt


H

Н

x
dt

hх i xi (T )    0 v U , р, (11.6)

i
i
хi
i
0 
0  u 
i 1
i 1
i 1

T
T
n
where
uH = H(t,v,x,р) – H(t,u,x,р),
T
  1   2  3  dt.
0
Using the integration by parts we get
T
T
T
 р (t )x (t )dt  р (Т )x (Т )  р (0)x(0)   р(t )x (t )dt  р (Т )x (Т )   р (t )x (t )dt,
i
i
i
i
i
i
0
i
0
i
i
i
0
because of the equalities х(0) = у(0) = х0. So we obtain the inequality
T
T
0
0 i 1
n
n
   u Hdt     Н хi  рi  xi dt    hxi  рi (T )  xi (T )    0 v  U , р.
(11.7)
i 1
Using the arbitrariness of the vector-function p chose it such that this inequality transforms to
the easy one. Suppose p is the solution of the equation
(11.8)
р   Н х , t  (0, T )
with final condition
р(T) = -hx ,
(11.9)
where H x   H x1 ,..., H xn  , hx   hx1 ,..., hxn  . The system (13.8), (13.9) is called the adjoint system.
It contains n differential equations as the state system (11.1). Then the inequality (13.7) is
transformed to
T
   u Hdt    0 v  U .
0
(11.10)
The left side of this inequality is the sum of two values. If the control v is close enough to the
optimal control v, then the high term  has the high order of the infinitesimality. Then the sign of
the sign is determine by the first term. So we obtain the inequality
T
   u Hdt  0 v  U .
0
We can prove that the sign of the integral is determine by the sign of the value under the integral.
So we get the inequality
u H  0 v U .
It can be transform to
(11.11)
H t , u (t ), x(t ), р(t )  max H t , v, x(t ), р(t )  , t  (0, T ).
v[ a ( t ),b ( t )]
This equality is called the maximum principle.
Theorem 11.1. The solution of the Problem 11.1 satisfies the maximum principle.
Hence we have the system (11.1), (11.2), (11.8), (11.9), (11.12) for finding unknown vectorfunctions u, х, р. The maximum principle is conditional problem of the maximization of the known
function H with respect to r variables (controls). The obtained system can be solved approximately
by iterative method as the corresponding optimality condition for Problem 10.2.
11.3. Example
Consider the system
 x1  x1  sin x2  u,

 x2  x2  cos x1  u
(10.13)
with the initial conditions
x1 (0)  x10 , x2 (0)  x20 .
The set of the admissible controls is determine by the inequalities
0  u (t )  1.
We have the problem of minimization the value
(10.14)
1
I    x1  ( x2 )3  u 2  dt  x1 (1).
0
Determine the functions
H  p1  x1  sin x2  u   p2  x2  cos x1  u    x1  ( x2 )3  u 2  ,
h  x1 (1).
Find the derivatives
H x1  p1  p2 sin x1  1, H x 2  p1 cos x2  p2  3( x2 ) 2 ,
hx1  1, hx 2  0.
The adjoint system (11.8), (11.9) is transformed to
 p1   p1  p2 sin x1  1,

2
 x2   p1 cos x2  p2  3( x2 )
p1 (1)  1, p2 (1)  0.
We have the maximum principle
 p1  p2  u  u 2  max  p1  p2  v  v 2  .
(11.15)
(11.16)
0v 1
Determine the derivative of the function H with respect to the control. It is equal to zero. So
we find
u   p1  p2  .
Then we find the solution of the maximum principle
if p1 (t )  p2 (t )  0,
0,

(11.17)
u   p1 (t )  p2 (t ), if 0  p1 (t )  p2 (t )  1,
1,
if p1 (t )  p2 (t )  1.

The system of the optimality conditions (11.13) – (11.17) can be solved with using of the iterative
method.
11.4. Maximization of the length of the missile flight
We return to Problem 10.1. This is the problem of the maximization of the length of the missile
flight. We have the system, described by the differential equations
 x(t ) = a cos u (t )
,
(11.18)

 y(t ) = a sin u (t )  g
with the initial conditions
 x(0) = 0 , x (0) = 0
.
(11.19)

 y (0) = 0 , y (0) = 0
The state functions x and y are coordinates of the missile here, the constant a is the acceleration of
the driving force (the parameter of the system), g is the gravitational acceleration, and the angle u
= u(t) is the control. We would like to choose the control such that the length of the flight
x (T )
L = x(T) +
(11.20)
y (T ) + y (T ) 2  2 g y (T )
g
will be maximal.
Transform this problem to Problem 11.1. Denote
х1  х , х2  у , х3  х , х4  у .
Determine the functions
f1 = х3, f2 = x4, f3 = a cos u, f4 = a sin u – g.
Then the system (11.18) is transformed to the state equations (11.1). The initial conditions (11.19)
are transformed to (11.2) with initial state x0 = 0. In our problem we have n  4, r  1. For the
transformation the maximizing value to the standard form we determine the functions
g (t , u, x)  0, h( x)  x1  x3  x4 + ( x4 )2  2 x3  / g.


Determine the function


4
H  t , u, x, p  =  g (t , u, x)   pi f i (t , u, x)  p1 x3 +p2 x4 + p3a cos u  p4 (a sin u – g ).
i 1
Then we determine the adjoint equations
 p1  0
p  0
 2

 p3  - p1 .
 p4  - p2
Add the corresponding final conditions
(11.21)
 p1 (T )  1

x3 (T )
 p2 (T ) 
2

 x4 (T ) +2 gx2 (T )


2
 p (T )  x4 (T )   x4 (T )  + 2 gx2 (T )
.
 3
g




x4 (T )

 p (T )  x3 (T ) 1 


2
 4
g 

x
(
T
)
+2
gx
(
T
)


4
2


(11.22)
Now we have the problem of maximization. So the optimal control is the point of the minimum
of the function H. Determine the function
F(u) = a (p3 cos u + p4 sin u).
It includes only the terms of the function H, which depends from the control. So we get the problem
of the minimization of the function F with respect to the control u. Using the stationary condition
we obtain
F'(u) = a (p4 cos u – p3 sin u ) = 0.
This is the algebraic equation with respect to u. So we find
tg u (t ) 
р4 (t )
, t  (0, T ).
р3 (t )
(11.23)
We can find the control if we will know the functions p3 and p4. It depends from the functions
p1 and p2 because of the first and second equalities (11.21). By third and fourth equalities (11.21)
the functions p1 and p2 are constants. Using final conditions (11.22) we find
x3 (T )
р1 (t )  1, р2 (t ) 
.
2
x
(
T
)

2
g
x
(
T
)
4 
2
Solve the problem (11.21), (11.22); we get
р3 (t )  р3 (T )  (T  t ) р1  g 1  x4 (T ) 

 x4 (T )
2
 2 gx2 (T )   (T  t ).

Then we find

x3 (T ) 
р4 (t )  р4 (T )  (T  t ) р2 
1 
g 




  (T  t )
2
 x4 (T ) + 2 gx2 (T ) 
x4 (T )



2
 x4 (T ) + 2 gx2 (T ) 
 x4 (T )
2
x3 (T )
+ 2 gx2 (T )  x4 (T )
g
x3 (T )
 x4 (T )
2
 2 gx2 (T )



 (T  t )  .

Find now the value
р4 (t )

р3 (t )
x3 (T )
 x4 (T )
2
, t  (0,T ).
(11.24)
+ 2 gx2 (T )
Using formulas (11.23), (11.24) we conclude that the optimal control u is constant. It depends from
the final values of the state functions x2, x3, x4. We solve the problem (11.18), (11.19) with constant
control. We find
t2
x2 (t )   a sin u  g  , x3 (t )  ta cos u, x4 (t )  t  a sin u  g  .
2
Using the equalities (11.23), (11.24), we find
sin u
a cos u

.
2
2
cos u
a sin u  ag sin u
It can be transform to the equality
g sin3u – 2a sin2u + a = 0.
Denote
v = sin u, b = a/g.
We obtain the cubic equation
(v) = v3 – 2bv2 + b = 0.
(11.25)
Hence the optimal control is determine only by the driving acceleration.
Determine the properties of the function . Find its derivative
'(v) = 3v2 – 4bv.
So this function has two local extremum. It has the local maximum at the point v = 0 and the local
minimum at the point v = 4/3 b (see Figure 11.1). Find the values
  0   b  0,   4 / 3b  
4b  16 b 2
1 
3 
9

  0.


b
4b/3
v
Figure 11.1. The function  = (b).
So the equation (11.25) has three solutions. One of them is negative, and third is greater than 4/3b
(see Figure 11.1). Note that in our problem the acceleration a is greater than g because the missile
need to flight. So the third solution is greater than 1. However it is impossible the parameter v is
the sinus. If is negative we do not have the flight to the increase of the coordinate x. Then the value
v is the point from unit interval. Find
(0) = b > 0 , (1) = 1 – b < 0.
Therefore the equation (11.25) has the unique solution v*=v*(а) from unit interval. So the solution
of our problem is
u(t) = arcsin v*, t (0,Т).
Outcome





Necessary conditions of the optimality for the vector optimization control problem consist
of the state system, the adjoint system, and Pontryagin’s maximum principle.
The adjoint system consists of the adjoint systems of differentiation equations and the final
conditions; the quantity of these equations is equal to the quantity of the state functions.
Maximum principle is a problem of the maximization of some function H of many
variables; its quantity is equal to the quantity if the control functions.
Necessary conditions of the optimality for the optimization control problem can be solved
by iterative method.
The problem of the length maximization for the flight of the missile can be considered as
an application of this theory.
Task 11. Optimization control problem for a system of differential equations
Consider the control system described by the differential equations
 x1  f1 ( x1 , x2 , u ),

 x2  f 2 ( x1 , x2 , u )
with initial conditions
x1 (0)  x10 , x2 (0)  x20 .
The set of the admissible control is described by the inequalities
a  u (t )  b.
We have the problem of the minimization of the value
T
I   g ( x1 , x2 , u )dt  h  x1 (T ), x2 (T )  .
0
Table of the parameters.
variants
1
f1 ( x, y, u)
x2  y 2  u
2
x 2  y  2u
3
 x 2  y  2u
4
f2 ( x, y, u)
x2  y 2  u
x  y u
g ( x, y , u )
h ( x, y )
x2  y 2  u 2
x2  y 2 / 2
x2  y 2  u 2
x2 / 2  y 2
y 2  x2  u 2
x2  y 2 / 2
2x 2  y 2  u 2
x2 / 2  y 2
2x 2  y 2  u
x2  2 y 2  u 2
x3
6
 x2  y 2  u / 2
x  y  u
x  y u
 x  y3  u
x  y u
x 2  y  2u
x2  y 4  u 2 / 2
 y2
7
x3  y 2  u
x  2y  u
y 4  x2  u 2 / 2
x2  y 2
8
 x2  y3  u
x  2y  u
x4  y 2  u 2 / 2
9
x2  y 2  u / 4
x  y u
y 2  x2
x4  y 2  u 2 / 2
y 2  x2 / 2
10
x2  y 2  u
x  y u
x2  y 2  u
2x 2  y 2  u 2
x2  y 2
x 2  y  2u
x2  2 y 2  u 2
y 2  x2
5
11
 x 2  y  2u
x2  y 4  u 2 / 2
- x3
13
 x  y3  u
x  y u
 x2  y 2  u / 2
y 4  x2  u 2 / 2
14
2x 2  y 2  u
x  y  u
 y2
x4  y 2  u 2 / 2
x2  y 2
15
x 2  y  2u
x  y u
x4  y 2  u 2 / 2
y 2  x2
16
x  2y  u
x3  y 2  u
x2  y 2  u 2
y 2  x2 / 2
17
x  2y  u
2x 2  y 2  u
x2  y 2  u 2
18
x  y u
x2 / 2  y 2
x 2  y  2u
y 2  x2  u 2
x3
12
Steps of the task.
1. Write the concrete problem statement.
2. Determine the function Н.
3. Determine the adjoint system.
4. Determine the maximum principle.
5. Find the control from the maximum principle.
6. Write the iterative method for solving the conditions of the optimality.
Next step
We considered the optimization control problems with free final states. However there exist practical optimization
control problems with fixed final state. We will try to extend these results to this case.