CALCULUS OF VARIATIONS AND OPTIMIZATION METHODS
Part II. Optimization methods
Lecture 12. Optimization control problem with fixed final state
We considered optimization control problems with free final state. However there exist the problems with fixed final
case. We would like to shift the system from a given initial state to the given final step with minimization some
functional in this case. We will determine the necessary conditions of the optimality for these problems. The time
optimal problem will be considered as an example. The shooting method will be used as a practical algorithm for
solving necessary conditions of the optimality.
12.1. Problem Statement
Consider optimization control problems with fixed final state. We have the system described by
differential equation
(12.1)
x  f (t , u, x), t  (0, T )
with initial condition
х(0) = х0.
(12.2)
Besides it is given the final state
х(Т) = х1.
(12.3)
The set of the admissible controls
U  u a(t )  u (t )  b(t ), t  (0, T ).
We have the integral
T
I   g  t , u (t ), x(t )  dt.
0
The smooth enough functions f and g, the functions a and b, and the numbers T, х0 and х1 are
given here.
Problem 12.1. Minimize the integral I on the set U, where x satisfies the equality (12.1) –
(12.3).
12.2. Optimality conditions
We will use the known technique for the analysis of the given problem. Determine Lagrange
function
T
L(u, x, р)  I (u )   р(t )  x(t )  f  t , u (t ), x(t )   dt.
0
It is equal to I if the function x is the solution of the equation (12.1). Let u is a solution of the
Problem 12.1, and x is the corresponding state function. We get the inequality
I = I(v) – I(u)  0 vU.
Then we have
L = L(v,y,р) – L(u,x,р)  0 vU, р.
where y is the solution of the system (12.1) – (12.3) for the control v. Find the value
T
T
0
0
L   р(t )  у (t )  x(t )  dt   Hdt  0,
where
H(t,u,x,р) = р f(t,u,x) – g(t,u,x),
H = H(t,v,y,р) – H(t,u,x,р).
Using the standard technique we get the inequality
T
T
0
0
(12.4)
 р(t )x(t )dt     H  Н x  dt    0 v U , р,
х
u
where
uH = H(t,v,x,р) – H(t,u,x,р),
and  is the high term. After the integration by parts we obtain
T
T
T
0
0
0
 р(t )x(t )dt  р(Т )x(Т )  р(0)x(0)   р(t )x(t )dt    р(t )x(t )dt ,
because of the conditions (12.2) and (12.3). Hence we get
T
T
0
0
   u Hdt    Н х  р  xdt    0 v  U , р.
Let the function p be the solution of the adjoint equation
р   Н х , t  (0, T ) .
So we have the inequality
(12.5)
T
   u Hdt    0 v  U ,
0
Using the standard technique we get the maximum principle
H t , u (t ), x(t ), р(t )  max H t , v, x(t ), р(t )  , t  (0, T )
v[ a ( t ),b ( t )]
(12.6)
Theorem 12.1. The solution u of Problem 12.1 satisfies the maximum principle (12.6), where
the functions x and p are the solutions of the system (12.1), (12.2), (12.3), (12.5).
Question: The adjoint equation does not have any additional conditions. What is its sense?
Note the difference between this result and the optimality conditions for the optimization
control problem with free final state. We had before state equation with initial condition and
adjoint equation with final condition. Now we have initial and final conditions for the state
equation, and we do not have any conditions for the adjoint equation. However we have two
differential equations with two boundary conditions. Indeed we can analyze the maximum
principle (12.6) as at the optimization problem with free final state. Hence we determine a
dependence of its solution from the state and adjoint functions
(12.7)
u(t )    t , x(t ), p(t )  ,
where Ф is a concrete function. Then we put it to the state and adjoint equations. We get the
system of two differential equations with respect to the state and adjoint functions with two
boundary conditions. So our optimality conditions have the sense in principle.
Table 12.1. The algorithm of solving for optimization problem with fixed final state.
step
action
remark
1
Definition of the concrete values
of known functions and parameters
The concrete problem is
transformed to the standard form.
2
Definition of the function H.
Using the formula (12.4).
3
Definition of the adjoint equation.
Using the formula (12.5).
4
Solving the maximum principle.
Determination of the dependence
(12.6) of the control from the state
and adjoint functions from (12.6)
5
Solving the state and adjoint equations.
Finding of the general solution of
the system (12.1), (12.5) after the
substitution here the solution of the
maximum principle.
Using the boundary conditions
(12.2), (12.3) for finding the state
and adjoint functions.
6
Finding the state and adjoint functions.
7
Finding the solution of the necessary conditions
of the optimality.
Substitution the state and adjoint
functions to the formula of the
control from the step 4.
8
Analysis of the result.
This control can be optimal,
but it can be non-optimal too.
12.3. Practical solution of the necessary conditions of the optimality
We obtain the system of the optimality conditions (12.1), (12.2), (12.3), (12.5), (12.6). Then we
have the necessity to have the constructive method of its solving.
Question: Could we use the known iterative method for solving this system?
We cannot to use the known iterative method for solving this system because we do not have any
conditions for the adjoint equation, and we have two conditions for the state equation. So we will
use other algorithm. It is based on the shooting method.
Determine the fictitious initial state for the adjoint equation
p(0)   ,
(12.8)
where the number  is unknown now. We get system (12.1), (12.2), (12.3), (12.5), (12.6), (12.8)
with respect to three unknown functions and parameter . Then the system (12.1), (12.2), (12.5),
(12.8) is Cauchy problem with respect to the state and adjoint functions. Its solutions depend
from  of course. Now the equality (12.3) can be interpreted as an equation with respect to 
F ( )  0,
(12.9)


1
where F ( )  x (T )  x , and x is the solution of the system (12.1), (12.2), (12.3), (12.5),
(12.6), (12.8) for the concrete value .
We have the algebraic equation (12.9) with respect to the number . It can be solved, for
example by iterative method
 k 1   k  k F ( k ),
(12.10)
where  k is a parameter of the algorithm. So we have the following algorithm. At first we
choose an initial approximation of the control u0 and 0 of parameter. If the control uk and
parameter k are given, then the functions x  xk and p  pk are determined from Cauchy
problem (12.1), (12.2), (12.5), (12.8) with known value u  uk . Then we find the next iteration
uk 1 by the formula
uk 1 (t )    t , xk (t ), pk (t ) 
because of the equality (12.7). The value  k 1 is determined by the formula
 k 1   k   k  xk (T )  x1  .
If the sequence {uk } converges, then its limit is the solution of the necessary conditions of the
optimality.
12.4. Example
Consider the system
x  u, t  (0,1),
х(0)  0 .
(12.11)
(12.12)
It is given the set of the admissible controls
U  u u (t )  1, t  (0,1)
and the integral
I
1
u
2
1
2
 x 2  dt.
0
We have the problem of the minimization the value I on the set U with additional final condition
х (1)  1 .
(12.13)
Determine the function
H = р u – (u2 + x2)/2.
We obtain the adjoint equation
р  х, t  (0,1).
(12.14)
Find the control from the maximum principle
 -1, if р(t )  -1,

(12.15)
u (t )   р(t ), if -1  р(t )  1,
 1, if р(t )  1.

So we have the system (12.11) – (12.15) with respect to three unknown functions u, x, and p.
Determine the algorithm of solving this system.
1. Choose the sequence { k } and initial approximations u0 and 0.
2. For the known values uk and k find the solution of Cauchy problem
xk  uk , р  хk , t  (0,1);
хk (0)  0, pk (0)   .
3. Find next iteration of the control by the formula
 -1, if рk (t )  -1,

uk 1 (t )   рk (t ), if -1  рk (t )  1,
 1, if р (t )  1.
k

4. Find next iteration of the parameter by the formula
k 1  k  k  xk (1) 1.
12.5. Vector case
Consider control system described by the differential equations
x  f (t , u, x), t  (0, T )
(12.16)
with the initial conditions
х(0) = х0
(12.17)
and final conditions
х(Т) = х1,
(12.18)
where x  x(t )   x1 (t ),..., xn (t )  is the state vector-function, u  u(t )   u1 (t ),..., ur (t )  is vector
control, f is the given vector-function, the vector x 0   x10 ,..., xn0  is the given initial state,
x1   x11 ,..., x1n  is the given final state. The control u is chosen from the set
U  u u (t )  G (t ), t  [0, t ] ,
where G (t ) is the given closed subset of the r-dimensional Euclid space. Consider the value
T
I   g  t , u (t ), x(t )  dt ,
0
where g is a given function.
Problem 12.2. Find the function u = u(t) from the set U, which minimize the value I.
We can solve this problem with using the standard technique. Determine the function
n
H (t , u, x, p)   pi f i (t , u, x)  g (t , u, x).
(12.19)
i 1
Then the optimal control satisfies the maximum principle
H t , u (t ), x(t ), р(t )  max H t , v, x(t ), р(t )  , t  (0, T ),
v[ a ( t ),b ( t )]
(12.20)
where p  p(t )   p1 (t ),..., pn (t )  is adjoint vector-function. It satisfies the adjoint system
р   Н х , t  (0, T ),
(12.21)
where H x   H / x1 ,..., H / xn  . The system (2.16), (2.17), (2.18), (12.20), (2.21) can be
solved with using shooting method.
Question: How we can solve the problem, where the part of the final states is fixed?
Consider the system
xi  fi (t , u, x1, x2 ), i  1,2, t  (0, T )
with the initial conditions
xi (0)  xi0 , i  1,2
and final condition
x1 (T )  x11.
We have the minimizing functional
T
I   g  t , u(t ), x1 (t ), x2 (t )  dt
0
and the set of admissible controls
U  u a(t )  u(t )  b(t ), 0  t  T .
We determine the function
H (t , u, x1, x2 , p1, p2 )  p1 f1(t , u, x1, x2 )  p2 f 2 (t , u, x1, x2 )  g (t , u, x1, x2 ).
The adjoint system consists of the equations
pi  H / xi , i  1,2
with unique final condition
p2 (T )  0.
12.6. Time optimization problem
We consider the movement of the body by a force. This phenomenon is described by the
Newton’s law
mх  F .
It can be transform to
х  u,
where the acceleration u is the control. We know the initial state a and initial velocity v of the
body. So we have the initial conditions
х(0)  a, х(0)  v.
We would like to move the body to the given final state at the final time T and to stop it here. We
choose the origin of coordinate as the final state. Then we have the final conditions
х(Т )  0, х(Т )  0,
Our control u = u(t) satisfies the inequality
| u(t) |  1.
We would like to choose the admissible control such that the time of the stopping T will be
minimal.
We transform our problem to the standard form. Determine the state functions
х1  х, х2  х
Then we have the system
х1  х2 , х2  и
(12.22)
with initial conditions
х1 (0)  а, х2 (0)  v
(12.23)
and final conditions
х1 (Т )  0, х2 (Т )  0.
(12.24)
The minimizing functional can be transformed to the integral
Т
I   dt.
0
We determine the function H by the formula
H  1  p1 x2  p2u.
The adjoint system is described by the differential equations
H
H
p1  
 0, p2  
  p1 .
(12.25)
x1
x2
Find the maximum of the function H with respect to the control on the interval [-1,1]. This
function is linear. So maximum can be obtained on the boundary this interval only. It depends
from the sign of the function p2. So the solution of the maximum principle is
 1, p2 (t )  0,
u (t )  
(12.26)
1, p2 (t )  0.
We have the system (12.22) – (12.26) for finding five unknown functions. There are the
control, two state functions, and two Lagrange multipliers. Find p1 (t )  с1 from the first equation
(12.16), where с1 is constant. Then we obtain general solution of the second equation (12.26)
p2 (t )  с1t  с2 , where с2 is constant. So the function p2 is linear. This function can have the
constant sign, or it can change the sign one time only. Therefore we can have four different
cases:
 1, t   ,
 1, t   ,
i) u (t )  1, ii) u(t )  1, iii) u (t )  
iv) u (t )  
 1, t   .
1, t   ,
Consider these cases.
If u (t )  1, then we find the solution of the system (12.22), (12.23)
х1 (t )  a  vt  t 2 / 2, х2 (t )  v  t
Put it to the equalities (12.24); we get
a  vТ  Т 2 / 2  0, v  Т  0.
2
Then we find v  Т , a  Т / 2. This result can be applicable, if the value v is negative, and
a  v2 / 2.
If u(t )  1, then we find the solution of the system (12.22), (12.23)
х1 (t )  a  vt  t 2 / 2, х2 (t )  v  t
Put it to the equalities (12.24); we get
a  vТ  Т 2 / 2  0, v  Т  0.
2
Then we find v  Т , a  Т / 2. This result can be applicable, if the value v is positive, and
a  v2 / 2.
For the third case we have
for t   . Then we find
х1 (t )  a  vt  t 2 / 2, х2 (t )  v  t
х1 ( )  a  v   2 / 2, х2 ( )  v  
(2.27)
The second equation (12.22) for t   has the general solution х2 (t )  с  t. Using the second
equality (2.27) we find the constant с  v  2 . So we have
х2 (t )  v  2  t.
2
Then the first equation (12.22) for t   has the general solution х1 (t )  с  (v  2 )t  t / 2.
Using the first equality (2.27) we find the constant с  а  2 . So we get
х1 (t )  а   2  (v  2 )t  t 2 / 2.
Using the equality (2.24) we obtain
а   2  (v  2 )Т  Т 2 / 2  0, v  2  Т  0.
We have v  2  Т from second equality. So the first equality can be transformed to
а  2  Т 2 / 2  0. Then we find the moment of the control switching   Т 2 / 2  a . The time
of the movement T satisfies the equality
T  v  2 Т 2 / 2  a.
So we find the final time
T  2v2  4a  v.
This result is applicable, if a  v2 / 2.
For the fourth case we have
for t   . Then we find
х1 (t )  a  vt  t 2 / 2, х2 (t )  v  t
х1 ( )  a  v  2 / 2, х2 ( )  v 
(2.28)
The second equation (12.22) for t   has the general solution х2 (t )  с  t. Using the second
equality (2.28) we find the constant с  v  2 . So we have
х2 (t )  v  2  t.
2
Then the first equation (12.22) for t   has the general solution х1 (t )  с  (v  2 )t  t / 2.
Using the first equality (2.28) we find the constant с  а  2 . So we get
х1 (t )  а   2  (v  2 )t  t 2 / 2.
Using the equality (2.24) we obtain
а   2  (v  2 )Т  Т 2 / 2  0, v  2  Т  0.
We have v  2  Т from second equality. So the first equality can be transformed to
а   2  Т 2 / 2  0. Then we find the moment of the control switching   Т 2 / 2  a . The time
of the movement satisfies the equality
T  v  2 Т 2 / 2  a.
So we find the final time
T  2v2  4a  v.
This result is applicable, if a  v2 / 2.

Outcome


The maximum principle is applicable for the optimization control problems with fixed
final state.
Necessary conditions of the optimality for these problems consist of the state equation
with two boundary conditions, the adjoint equation without any boundary conditions, and
maximum principle.



Necessary conditions of the optimality for these problems can be solved with using
shooting method.
These results are applicable for the vector case.
The time optimal problem can be considered as an application of this theory.
Task. Optimization control problem for a systems with fixed final time
Consider the control system described by the differential equations
 x1  f1 ( x1 , x2 , u ),

 x2  f 2 ( x1 , x2 , u )
with initial conditions
x1 (0)  x10 , x2 (0)  x20
and final condition
x1 (T )  xT
(*)
x2 (T )  xT
(**)
or
The set of the admissible control is described by the inequalities
a  u (t )  b.
We have the problem of the minimization of the value
T
I   g ( x1 , x2 , u )dt .
0
Table of the parameters.
variants
1
f1 ( x, y, u)
f 2 ( x, y, u)
g ( x, y , u )
final condition
x  y u
x  y u
y  x u /2
(*)
2
x 2  y  2u
x 2  y  2u
x4  y 2  u 2 / 2
(**)
3
 x  y  2u
 x  y  2u
x  y u /2
(*)
4
x  y  u / 2
x  y  u
x  y u
2x  y  u
(**)
x2  2 y 2  u 2
(*)
6
x  y  u / 2
x  y  u
x  y u
x  y u /2
(**)
7
x  y u
x  y u
y  x u /2
(*)
8
 x2  y3  u
2x  y  u
x4  y 2  u 2 / 2
(**)
9
2
x  y u/4
x  y  2u
x  y u /2
(*)
10
x2  y 2  u
x  y u
x2  y 2  u
x  y u
x2  y 2  u 2
(**)
x  y u
2
(*)
x  y  u
x  y u
y  x u
(**)
13
x  y  u
x  y u
2
x2  y 2  u 2
(*)
14
2x  y  u
2x  y  u
x  y u
2
(**)
15
x  y  2u
x  2y  u
x  y  2u
x  2y  u
y  x u
2
(*)
x  2y  u
x  y u
x  2y  u
x  y u
5
11
12
16
17
18
2
2
2
2
2
3
2
2
3
2
2
2
2
2
2
2
2
3
2
2
2
2
3
2
2
2
4
2
4
2
2
2
2
2
2
4
4
2
2
4
2
2
2
2
2
2
2
2
2
2
2
2
2x 2  y 2  u 2
(**)
x  2y  u
(*)
2
2
2
x2  y 4  u 2 / 2
(**)
Steps of the task.
1. Write the concrete problem statement.
2. Determine the function Н.
3. Determine the adjoint system.
4. Determine the maximum principle.
5. Find the control from the maximum principle.
6. Write the iterative method for solving the conditions of the optimality.
Next step
We consider the methods of solving extremum problems, which is based on the necessary conditions of the
extremum. However these methods transform the initial problem to another problem. There is the system of
optimality conditions. Then we used iterative methods for solving this system. Our next step is the analysis of the
direct iterative methods of solving extremum problems without using conditions of the extremum.
Reference
1. Алексеев В.М., Тихомиров В.М., Фомин С.В. Оптимальное управление. – М., Наука, 1979. –
С. 84-93; 103-107.
2. Miersemann E. Calculus of Variations. Lecture Notes. – Leipzig, 2012. – P. 170-174.