International Mathematical Forum, 2, 2007, no. 30, 1477 - 1498
Classical and Relaxed Optimization Methods
for Optimal Control Problems
I. Chryssoverghi, J. Coletsos and B. Kokkinis
Department of Mathematics, School of Appl. Mathematics and Physics
National Technical University of Athens
Zografou Campus, 15780 Athens, Greece
[email protected]
Abstract. We consider an optimal control problem for systems governed by nonlinear ordinary
differential equations, with control and state constraints, including pointwise state constraints. The
problem is formulated in the classical and in the relaxed form. Various necessary/sufficient conditions
for optimality are first given for both problems. For the numerical solution of these problems, we then
propose a penalized gradient projection method generating classical controls, and a penalized
conditional descent method generating relaxed controls. Using also relaxation theory, we study the
behavior in the limit of sequences constructed by these methods. Finally, numerical examples are
given.
Keywords: Optimal control, nonlinear systems, state constraints, gradient
projection method, conditional descent method, penalty method, relaxed controls.
Mathematics Subject Classification: 49M
1 Introduction
We consider an optimal control problem for systems governed by nonlinear ordinary
differential equations, with control and state constraints, including pointwise state
constraints. The problem is formulated in the classical form, and also in the relaxed
form using Young measures. Various necessary/sufficient conditions for optimality
are first given for both problems. For the numerical solution of these problems, we
then propose a penalized gradient projection method generating classical controls, and
a penalized conditional descent method generating relaxed controls. Under
appropriate assumptions, we prove that relaxed (resp. strong classical) limits of
subsequences (resp. sequences) constructed by the classical method are admissible
and weakly extremal relaxed (resp. classical) for the relaxed (resp. classical) problem,
and that relaxed limits of subsequences of controls constructed by the relaxed method
are admissible and strongly extremal for the relaxed problem. Finally, several
numerical examples are given. For classical and relaxed optimization and
1478
I. Chryssoverghi, J. Coletsos and B. Kokkinis
approximation methods applied to optimal control problems, see e.g. [2-9], [11-12],
[14], and the references therein.
2 Classical and relaxed optimal control problems
Consider the following optimal control problem. The state equation is given by
y '(t ) = f (t , y (t ), w(t )) , t ∈ I := [0, T ] , y (0) = y 0 ,
where y (t ) ∈ IR d , the constraints on the control w are w(t ) ∈ U , for t ∈ I , where U
is a compact subset of IR d ' , the constraints on the state y := yw are
T
G1 ( w) = g1 ( y (T )) + ∫ g1 (t , y (t ), w(t ))dt = 0 ,
0
T
G2 ( w) = g 2 ( y (T )) + ∫ g 2 (t , y (t ), w(t ))dt ≤ 0 ,
0
G3 ( w)( s ) = g3 ( s, y ( s )) ≤ 0 , for s ∈ I ,
where the vector functions gl , gl take values in IR ml , l = 1, 2 , and g3 in IR m3 , and the
cost functional is
T
G0 ( w) = g 0 ( y (T )) + ∫ g 0 (t , y (t ), w(t ))dt .
0
Defining the set of classical controls
W = {w : I → U w measurable} ⊂ L2 ( I , IR d ' ) ,
the classical optimal control problem is to minimize G0 ( w) subject to w ∈ W and to
the above state constraints.
It is well known that, even if the set U is convex, the classical problem may
have no solutions. The existence of such a solution is usually proved under strong,
often unrealistic for nonlinear systems, convexity assumptions (such as the Cesari
property). Reformulated in the so-called relaxed form, the problem is convexified in
some sense and has a solution in a larger space under weaker assumptions.
Next, we define the set of relaxed controls (Young measures; for the relevant
theory, see [13], [10]) by
R = {r : I → M 1 (U ) r weakly measurable} ⊂ L∞w ( I , M (U )) ≡ L1 ( I , C (U )) * ,
where M (U ) (resp. M 1 (U ) ) is the set of Radon (resp. probability) measures on U .
The set W (resp. R ) is endowed with the relative strong (resp. weak star) topology,
and R is convex, metrizable and compact. If each classical control w(⋅) is identified
with its associated Dirac relaxed control r (⋅) := δ w(⋅) , then W may be also considered
as a subset of R , and W is thus dense in R . For a given φ ∈ L1 ( I ; C (U ; IR n )) (or
equivalently φ ∈ B( I ,U ; IR n ) , where B is the set of Caratheodory functions in the
sense of Warga [13]) and r ∈ L∞w ( I , M (U )) (in particular, for r ∈ R ), we shall use for
simplicity the notation
φ ( x, r (t )) := ∫ φ (t , u )r (t )(du ),
U
and φ (t , r (t )) is thus linear (under convex combinations, for r ∈ R ) in r . A sequence
(rk ) converges to r ∈ R in R if and only if
Optimal Control Problems
1479
lim ∫ φ (t , rk (t ))dt = ∫ φ (t , r (t ))dt ,
k →∞ I
I
for every φ ∈ L ( I ; C (U ; IR )) , or φ ∈ B( I , U ; IR n ) , or φ ∈ C ( I × U ; IR n ) .
The relaxed optimal control problem is then defined by replacing w by r
(with the above notation) and W by R in the classical problem.
1
n
p
p
We define the norms x = ( ∑ xi2 )1/ 2 , x 1 = ∑ xi , x
i =1
and denote by ⋅
L2
,
⋅
L1
, ⋅
L∞
, ⋅
i =1
L∞
∞
= max xi , in IR p ,
i =1,..., p
the corresponding usual norms in L2 ( I ) p ,
L1 ( I ) p , L∞ ( I ) p , C ( I ) p , respectively. We denote by M ( I ) ≡ C ( I ) * the set of finite
regular measures on I , and by ⋅
*
the norm in M ( I ) defined by μ * = ∫ μ (dt )
I
(with μ = μ if μ is positive). The order relations between vectors, functions or
vector functions, are defined componentwise and/or pointwise.
We suppose that the function f is defined on I × IR d × U , measurable for
y , u fixed, continuous for t fixed, and satisfies
f (t , y, u ) ≤ ψ (t )(1 + y ), for every (t , y, u ) ∈ I × IR d × U , with ψ ∈ L1 ( I ) ,
f (t , y1 , u ) − f (t , y2 , u ) ≤ L y1 − y2 , for every (t , y1 , y2 , u ) ∈ I × IR 2 d × U .
The following result is standard (see [13]).
Theorem 2.1 For every relaxed (or classical, as W ⊂ R ) control r ∈ R , the state
equation has a unique absolutely continuous solution y := yr . Moreover, there exists a
constant b such that yr ∞ ≤ b for every r ∈ R .
Let B denote the closed ball in IR d with center 0 and radius b (see Theorem
2.1). We suppose now in addition that the functions gl , l = 0,1, 2 , are defined on
I × B × U , measurable for fixed y , u , continuous for fixed t , and satisfy
g l (t , y , u ) ≤ ζ l (t ), for every (t , y, u ) ∈ I × B × U ,
with ζ l ∈ L1 ( I ) , the function g3 is continuous on I × B , and that the functions gl are
continuous on B . The results of the following theorem are proved in [13].
Theorem 2.2 The mappings y → yw , from L2 to C ( I ) d , y → yr , from R to C ( I ) d ,
and Gl : W or R → R ml , l = 0,1, 2 , G3 : W or R → C ( I ) m3 , are continuous. If the relaxed
problem is feasible, then it has a solution.
Note that in the classical problem we have y '(t ) ∈ f (t , y (t ),U ) (velocity set),
while in the relaxed problem y '(t ) ∈ co(f (t , y (t ),U )) . Since W ⊂ R , we have in
general
cR := min G0 ( r ) ≤
inf G0 ( w) := cW ,
constraints on r
constraints on w
where the equality holds, in particular, if there are no state constraints, as W is dense
in R . Since usually approximation methods slightly violate the state constraints,
approximating an optimal relaxed control by a relaxed or a classical control, hence the
1480
I. Chryssoverghi, J. Coletsos and B. Kokkinis
relaxed optimal cost cR , is not a drawback in practice (see [13], p. 248). Note also
that approximating sequences of classical controls may converge to relaxed ones.
In order to state the necessary conditions for optimality, we suppose in
addition that the functions f , gl , f y , f u , gly , glu , l = 0,1, 2 , are defined on I × B '× U ' ,
where B ' (resp. U ' ) is a open set containing B (resp. U ), measurable on I for fixed
( y, u ) ∈ B × U , continuous on B × U for fixed t ∈ I , and satisfy
f y (t , y, u ) ≤ ξ (t ) ,
f u (t , y, u ) ≤ η (t ) ,
gly (t , y , u ) ≤ ζ l1 (t ),
g lu (t , y , u ) ≤ ζ l 2 (t ),
for every (t , y, u ) ∈ I × B × U , with ξ ,η , ζ l1 , ζ l 2 ∈ L1 ( I ) , and that the functions gly are
defined on B ' and continuous on B .
The two following theorems can be proved by using the techniques of [13].
Theorem 2.3 (i) If U is convex, the directional derivative of the mapping
Gl : W → R ml , for l = 0,1, 2 , is given by
G ( w + α ( w '− w)) − Gl ( w)
DGl ( w, w '− w) = lim+ l
α
α →0
T
= ∫ [ zl (t ) f u (t , y (t ), w(t )) + glu (t , y (t ), w(t ))][ w '(t ) − w(t )]dt , for w, w ' ∈ W ,
0
where y := yw , and the adjoint state zl := zl w , a row vector function ( l = 0 ), or a
matrix function ( l = 1, 2 ), is the solution of the classical linear adjoint equation
zl '(t ) = − zl (t ) f y (t , y (t ), w(t )) − gly (t , y (t ), w(t )) , t ∈ I ,
zl (T ) = gly ( y (T )) , with y := yw ,
the controls being regarded here as classical. The directional derivative of
G3 : W → C ( I ) m3 is given by the matrix function
DG3 ( w, w '− w)( s)
s
= g 3 y ( s, y ( s )) Z ( s ) −1 ∫ Z (t ) f u (t , y (t ), w(t ))[ w '(t ) − w(t )]dt , s ∈ I ,
0
where the matrix function Z := Z w satisfies the fundamental matrix equation
Z '(t ) = − Z (t ) f y (t , y (t ), w(t )) , t ∈ I ,
Z (T ) = E ( E identity matrix).
(ii) The directional derivative of the mapping Gl : R → IR ml , for l = 0,1, 2 , is given by
G (r + α (r '− r )) − Gl (r )
DGl (r , r '− r ) := lim+ l
α →0
α
= ∫ [ zl (t ) f (t , y (t ), r '(t ) − r (t )) + gl (t , y (t ), r '(t ) − r (t ))]dt , for r, r ' ∈ R ,
I
where y = yr , and the relaxed adjoint zl := zl r is the solution of the relaxed linear
adjoint equation
zl '(t ) = − zl (t ) f y (t , y (t ), r (t )) − gly (t , y (t ), r (t )) , t ∈ I ,
zl (T ) = g ly ( y (T )) , with y := yr .
The directional derivative of G3 : R → C ( I ) m3 is given by
1481
Optimal Control Problems
s
DG3 ( r, r '− r )( s ) = g3 y ( s, y ( s )) Z ( s ) −1 ∫ Z (t ) f u (t , y (t ), r '(t ) − r (t ))dt , s ∈ I ,
0
where Z := Z r is defined as in (i), but with w replaced by r .
(iii) The following mappings are continuous
z a zw , from W to C ( I ) d , z a zr , from R to C ( I ) d ,
( w, w ') a DGl ( w, w '− w) , l = 0,1, 2,3 , from W × W to IR ml , l = 0,1, 2 , C ( I ) m3 ,
( r, r ') a DGl ( r, r '− r ) ), l = 0,1, 2,3 , from R × R to IR ml , l = 0,1, 2 , C ( I ) m3 .
In the notations of DG , it is understood, depending on the arguments, w or
r , that the directional derivative is taken in the corresponding space, W or R , on
which G is defined. The following theorem gives various necessary conditions for
optimality (the weak relaxed minimum principle is proved similarly to [7]).
Theorem 2.4 (i) We suppose that U is convex. If w ∈ W is optimal for the classical
problem, then w is weakly extremal classical, i.e. there exist multipliers
λ0 ∈ IR , λ1 ∈ IR m1 , λ2 ∈ IR m2 , λ3 ∈ [C ( I ) m3 ]* ≡ M ( I ) m3 ,
with λ0 ≥ 0 , λ2 ≥ 0 , λ3 ≥ 0 ,
2
∑
l =0
m3
λl + λ3 * = 1 , where λ3 * = ∑ λ j 3
j =1
M1
,
such that
2
∑ λ DG (w, w '− w) + ∫
l
l =0
l
2
T
0
λ3 (ds ) DG3 ( w, w '− w)( s)
= ∑ λl ∫ [ zl (t ) f u (t , y (t ), w(t )) + glu (t , y (t ), w(t ))][ w '(t ) − w(t )]dt
T
0
l =0
T
s
+ ∫ λ3 ( ds ) g 3 y ( s, y ( s )) Z ( s ) −1 ∫ Z (t ) f u (t , y (t ), w(t ))[ w '(t ) − w(t )]dt
0
0
2
= ∫ {∑ λl [ zl (t ) f u (t , y (t ), w(t )) + glu (t , y (t ), w(t ))]
T
0
+
l =0
( ∫ λ (ds) g
T
t
3
3y
)
( s, y ( s )) Z ( s ) −1 Z (t ) fu (t , y (t ), w(t ))}[ w '(t ) − w(t )]dt ≥ 0 ,
for every w ' ∈ W ,
and
λ2G2 ( w) = 0 ,
∫
T
0
λ3 ( ds )G3 ( w)( s ) = 0 (classical transversality conditions).
The above inequality condition is equivalent to the pointwise weak classical minimum
principle
2
{∑ ( λl [ zl (t ) f u (t , y (t ), w(t )) + glu (t , y (t ), w(t ))]
l =0
T
+[ ∫ λ3 ( ds ) g 3 y ( s, y ( s )) Z ( s ) −1 ]Z (t ) f u (t , y (t ), w(t ))} w(t )
t
2
= min{{∑ λl [ zl (t ) f u (t , y (t ), w(t )) + glu (t , y (t ), w(t ))]
u∈U
T
l =0
+[ ∫ λ3 ( ds ) g 3 y ( s, y ( s )) Z ( s ) −1 ]Z (t ) f u (t , y (t ), w(t ))} u} , for a.a. t ∈ I .
t
1482
I. Chryssoverghi, J. Coletsos and B. Kokkinis
(ii) If r ∈ R is optimal for either the relaxed or the classical problem, then r is
strongly extremal relaxed, i.e. there exist multipliers as in (i), such that
2
∑ λ DG (r, r '− r ) + ∫
l
l
0
l =0
2
T
λ3 (ds ) DG3 ( r, r '− r )( s )
= ∑ λl ∫ [ zl (t ) f (t , y (t ), r '(t ) − r (t )) + gl (t , y (t ), r '(t ) − r (t ))]dt
T
0
l =0
T
s
+ ∫ λ3 ( ds ) g 3 y ( s, y ( s )) Z ( s ) −1 ∫ Z (t ) f (t , y (t ), r '(t ) − r (t ))dt
0
0
2
= ∫ {∑ λl [ zl (t ) f (t , y (t ), r '(t ) − r (t )) + gl (t , y (t ), r '(t ) − r (t ))]
T
0
l =0
T
+[ ∫ λ3 ( ds ) g 3 y ( s, y ( s )) Z ( s ) −1 ]Z (t ) f (t , y (t ), r '(t ) − r (t ))}dt ≥ 0 ,
t
for every r ' ∈ R,
and
λ2G2 (r ) = 0 ,
∫
T
0
λ3 ( ds )G3 ( r )( s ) = 0 (relaxed transversality conditions).
The above inequality condition is equivalent to the pointwise strong relaxed minimum
principle
2
∑ λ [ z (t ) f (t, y (t ), r(t )) + g (t, y(t ), r(t ))]
l
l
l
l =0
T
+[ ∫ λ3 ( ds ) g 3 y ( s, y ( s )) Z ( s ) −1 ]Z (t ) f (t , y (t ), r (t ))
t
2
= min{∑ λl [ zl (t ) f (t , y (t ), u ) + gl (t , y (t ), u )]
u∈U
l =0
T
+[ ∫ λ3 ( ds ) g 3 y ( s, y ( s )) Z ( s ) −1 ]Z (t ) f (t , y (t ), u )} , for a.a. t ∈ I .
t
If in addition U is convex, then this minimum principle implies the pointwise weak
relaxed minimum principle
2
{∑ λl [ zl (t ) f u (t , y (t ), r (t )) + glu (t , y (t ), r (t ))]
l =0
T
+[ ∫ λ3 ( ds ) g 3 y ( s, y ( s )) Z ( s ) −1 ]Z (t ) f u (t , y (t ), r (t ))} r (t )
t
2
= min{{∑ λl [ zl (t ) f u (t , y (t ), r (t )) + glu (t , y (t ), r (t ))]
φ
l =0
T
+[ ∫ λ3 ( ds ) g 3 y ( s, y ( s )) Z ( s ) −1 ]Z (t ) f u (t , y (t ), r (t ))} φ (t , r (t ))} , for a.a. t ∈ I ,
t
where the minimum is taken over the set B( I ,U ;U ) of Caratheodory functions (see
[13]) φ : I × U → U , which in turn implies the global weak relaxed condition
2
∫ {∑ λ [ z (t ) f (t, y(t ), r(t )) + g
T
0
l
l
u
lu
(t , y (t ), r (t ))]
l =0
T
+[ ∫ λ3 ( ds ) g 3 y ( s, y ( s )) Z ( s ) −1 ]Z (t ) f u (t , y (t ), r (t ))}[φ (t , r (t )) − r (t )]dt ≥ 0 ,
t
for every φ ∈ B( I ,U ;U ) .
A control r satisfying this condition and the above relaxed transversality conditions
is called weakly extremal relaxed.
1483
Optimal Control Problems
The following theorem gives sufficient conditions for optimality.
Theorem 2.5 With the derivatives in u omitted (resp. included) in our last
assumptions, we suppose in addition that the data are such that G0 , G2 , G3 are convex
and that G1 is affine. If r ∈ R (resp. w ∈ W , with U convex) is admissible and
strongly extremal relaxed (resp. weakly extremal classical) for the relaxed (resp.
classical) problem, with λ0 > 0 , then r is optimal for this problem.
Proof. (Relaxed case, the classical case is similar) The assumptions imply that the
functional
2
G (r ) := ∑ λl Gl (r ) + ∫ λ3 (ds )G3 (r )( s )
l =0
I
is convex. The necessary inequality condition of Theorem 2.4 is then satisfied if and
only if r minimizes G on R . Suppose now that r does not minimize G0 , in which
case there exists r ' ∈ R satisfying the state constraints and such that G0 (r ') < G0 (r ) .
Using the state constraints and the relaxed transversality conditions, we obtain
2
G (r ') := ∑ λl Gl (r ') + ∫ λ3 (ds )G3 (r ')( s )
l =0
I
2
≤ λ0G0 (r ') < λ0G0 (r ) = ∑ λl Gl (r ) + ∫ λ3 (ds )G3 (r )( s ) = G (r ) ,
l =0
I
i.e. r does not minimize G , a contradiction.
3 Classical and relaxed optimization methods
Let ( M lm ) , l = 1, 2,3 , be nonnegative increasing sequences such that M lm → ∞ as
m → ∞ , γ > 0 , b, c ∈ (0,1) , and ( β k ) , (ζ k ) positive sequences, with ( β m )
decreasing and converging to zero, and ζ k ≤ 1 .
Define first the penalized functionals on W
m1
m2
2
1
G m ( w) := G0 ( w) + {M 1m ∑ G1 j ( w) + M 2m ∑ [max(0, G2 j ( w))]2
2
j =1
j =1
m3
+ M 3m ∑ ∫ [max(0, G3 j ( w)( s))]2 ds} .
j =1
I
It can be easily shown that the directional derivative of G m is given by
DG m ( w, w '− w) = DG0 ( w, w '− w)
m1
m2
j =1
j =1
+ M 1m ∑ G1 j ( w) DG1 j ( w, w '− w) + M 2m ∑ max(0, G2 j ( w)) DG2 j ( w, w '− w)
m3
+ M 3m ∑ ∫ max(0, G3 j ( w)( s )) DG3nj ( w, w '− w)( s )ds .
j =1
I
The classical penalized gradient projection method is described by the following
Algorithm, where U is assumed to be convex.
1484
I. Chryssoverghi, J. Coletsos and B. Kokkinis
Algorithm 1
Step 1. Set k := 0 , m := 1 , and choose an initial control w01 ∈ W .
Step 2. Find vkm ∈ W such that
ek := DG m ( wkm , vkm − wkm ) + (γ / 2) vkm − wkm
2
L2
2
m
k L2
= min[ DG m ( wkm , v '− wkm ) + (γ / 2) v '− w
v '∈W
],
and set d k := DG m ( wkm , vkm − wkm ) .
Step 3. If d k ≤ β m , set wm := wkm , v m := vkm , em := ek , d m := d k , m := m + 1 , and then
go to Step 2.
Step 4. (Modified Armijo step search) Find the lowest integer value s (positive or
not), say s , such that α = c sζ k ∈ (0,1] and α satisfies the inequality
G m ( wkm + α k (vkm − wkm )) − G m ( wkm ) ≤ α k bd k ,
and then set α := c s ζ k .
Step 5. Set wkm+1 := wkm + α k (vkm − wkm ) , k := k + 1 , and go to Step 2.
One can see by “completing the square” that Step 2 amounts to finding the
projection vkl of the function
m2
ukm (t ) := wkm (t ) − (1/ γ ){∑ λlm [ zlkm (t ) fu (t , ykm (t ), wkm (t )) + glu (t , ykm (t ), wkm (t ))]
+
( ∫ λ (ds) g
T
t
m
3
l =0
3y
)
( s, ykm ( s )) Z ( s ) −1 Z (t ) f u (t , ykm (t ), wkm (t ))} ,
onto W , which in turn reduces to finding the corresponding pointwise projection onto
U for a.a. t ∈ I . On the other hand, by the definition of the directional derivative and
since b, c ∈ (0,1) , clearly the Armijo step α k in Step 4 can be found for every k . The
parameter γ is chosen experimentally to yield a good rate of convergence.
A (strongly or weakly, classical or relaxed) extremal control is called
abnormal if there exist multipliers as in the corresponding optimality conditions, with
λ0 = 0 . A control is admissible and abnormal extremal in rather exceptional situations
(see [13]).
With wm as defined in Step 3, define the sequences of multipliers
λ1m := M 1mG1 ( wm ), λ2m := M 2mmax(0, G2 ( wm )),
λ3m ( s ) := M 3mmax(0, G3 ( wm )( s )) = M 3mmax(0, g3 ( s, y m ( s ))) ,
where max denotes a vector of maximum values.
Theorem 3.1 We suppose here that U is convex.
(i) In the presence of state constraints, if the whole sequence ( wkm ( k ) ) generated by
Algorithm 1 converges to some w ∈ W in L2 strongly and the sequences (λlm ) ,
l = 1, 2,3 , (λ3m ) in L1 ( I )m3 , are bounded, then w is admissible and weakly extremal
classical for the classical problem. In the absence of state constraints, if a subsequence
1485
Optimal Control Problems
( wk ) k∈K (no index m ) converges to some w ∈ W in L2 strongly, then w is weakly
extremal classical for the classical problem.
(ii) In the presence of state constraints, if a subsequence ( wm ) m∈M (regarded as a
sequence in R ) of the sequence generated by Algorithm 1 in Step 3 converges to
some r in R and the sequences ( λlnm ) , l = 1, 2,3 , ( λ3nm ) in L1 ( I )m3 , are bounded,
then r is admissible and weakly extremal relaxed for the relaxed problem. In the
absence of state constraints, if a subsequence ( wk ) k∈K (no index m ) converges to
some r in R , then r is weakly extremal relaxed for the relaxed problem.
(iii) In any of the convergence cases (i) or (ii) with state constraints, suppose that the
classical, or relaxed, problem has no admissible, abnormal extremal, controls. If the
limit control is admissible, then the sequences of multipliers are bounded, and this
control is extremal as above.
Proof. (State constraints present) We shall first show that m → ∞ in the Algorithm.
Suppose on the contrary that m remains constant after a finite number of iterations in
k , and so we drop here the index m . Consider case (i). Let us show that then
d k → 0 . Since wk → w in L2 strongly, using also Proposition 2.1 in [1], we can
deduce that uk → u in L2 strongly, where
2
u (t ) := w(t ) − (1/ γ ){∑ λl [ zl (t ) fu (t , y (t ), w(t )) + glu (t , y (t ), w(t ))]
+
( ∫ λ (ds) g
l =0
T
t
3
3y
)
( s, y ( s )) Z ( s ) −1 Z (t ) fu (t , y (t ), w(t ))} .
Since vk is the projection of uk onto W , we have also vk → v in L2 strongly, where
v is the projection of u onto W . Clearly, by Step 2, d k ≤ ek ≤ 0 for every k , hence,
by Theorem 2.3
2
e := lim ek = DG ( w, v − w) + (γ / 2) v − w ≤ 0,
k →∞
d := lim d k = DG ( w, v − w) ≤ lim ek = e ≤ 0.
k →∞
k →∞
Suppose that d < 0 . The function Φ(α ) := G ( w + α (v − w)) is continuous on [0,1] ,
and since the directional derivative DG ( w, v − w) is linear in v − w , Φ is
differentiable on (0,1) and has derivative Φ '(α ) = DG ( w + α (v − w), v − w). By the
mean value theorem, for each α ∈ (0,1] there exists α '(α ) ∈ (0, α ) such that
G ( wk + α (vk − wk )) − G ( wk ) = α DG ( wk + α '(α )(vk − wk ), vk − wk ) .
Therefore, by Theorem 2.3
G ( wk + α ( vk − wk )) − G ( wk ) = α ( d + ε kα ), for α ∈ [0,1] ,
where ε kα → 0 as k → ∞ , k ∈ K , and α → 0+ . Now, we have d k = d + ηk , where
ηk → 0 as k → ∞ , and since b ∈ (0,1)
d + ε kα ≤ b(d + ηk ) = bd k , for α ∈ [0, α ] , k ≥ k ,
for some α > 0 and k . Hence
G ( wk + α (vk − wk )) − G ( wk ) ≤ α bd k , for α ∈ [0, α ] , k ≥ k .
It follows from the choice of the Armijo step α k in Step 4 that α k ≥ cα , for k ≥ k .
Hence
1486
I. Chryssoverghi, J. Coletsos and B. Kokkinis
G ( wk +1 ) − G ( wk ) = G ( wk + α k (vk − wk )) − G ( wk ) ≤ α k bd k ≤ cα bd k ≤ cα bd / 2,
for k ≥ k . This contradicts the fact that G ( wk ) → G ( w) , by Theorem 2.2. Therefore,
we must have d = e = 0 , d k → 0 , and ek → 0 . But Step 3 then implies that m → ∞ ,
which is a contradiction. Therefore, m → ∞ in Algorithm 1.
Let us show now that d k → 0 in case (ii). Since R is compact, let ( wk ) k∈K ,
(vk ) k∈K be subsequences, regarded as sequences in R , of the sequences generated by
Algorithm 1 that converge to some r% ∈ R , r ' ∈ R , respectively. Then, by the
continuity of the relaxed state and adjoint operators (Theorems 2.2-3), using
Proposition 2.1 in [1], and since d k ≤ ek ≤ 0 , we have
2
e := lim ek = lim [ DG ( wk , vk − wk ) + (γ / 2) vk − wk ]
k →∞ , k ∈K
k →∞ , k∈K
= d + (γ / 2) ∫ [r '( x) − r% ( x)]2 dt ≤ 0 ,
Ω
d = lim d k = lim DG ( wk , vk − wk ) ≤ lim ek = e ≤ 0 ,
k →∞ , k∈K
k →∞ , k∈K
k →∞ , k∈K
with
2
d := ∫ {∑ λl [ zl (t ) fu (t , y (t ), r% (t )) + glu (t , y (t ), r% (t ))]
T
0
+
l =0
( ∫ λ (ds) g
T
t
3
3y
)
( s, y ( s )) Z ( s ) −1 Z (t ) fu (t , y (t ), r% (t ))}[r '(t ) − r% (t )]dt ,
where λ0 := 1 , y := yr% , z := zr% . Note that d k is, but d is not, a directional derivative in
this case. Suppose that d < 0 . Since we have also, for α '(α ) ∈ (0, α )
lim + DG ( wk + α '(α )(vk − wk ), vk − wk ) = d ,
k →∞ , k∈K , α → 0
we get as above
G ( wk +1 ) − G ( wk ) ≤ cα bd / 2, for k ≥ k , k ∈ K ,
for some k . Since the whole sequence (G ( wk )) is non-increasing by Steps 4 and 5, it
follows that G ( wk ) → −∞ as k → ∞ , k ∈ K , which leads to a contradiction, as
above. Therefore, d = e = 0 , d k → 0 , ek → 0 , k ∈ K , and this holds also for the
whole sequences by the uniqueness of the limit 0. We conclude as above that l → ∞
in Algorithm 1.
(i) Suppose that the sequences (λlm ) are bounded and, up to subsequences, that
λlm → λl , l = 1, 2 , and λ3m → λ3 in M ( I )m weak star. By Theorem 2.2, since
3
wm → w in L2 strongly, we have
0 = lim
m →∞
0 = lim
m →∞
0 = lim
m →∞
λ1m
M
m
1
λ2m
M
m
2
λ3m
M
= lim G1 ( wm ) = G1 ( w),
m →∞
= lim[max(0, G1 ( wm ))] = max(0, G2 ( w)),
L1
m
3
m →∞
1
m →∞ M m
3
= lim
= ∫ max(0, G3 ( w)( s ))ds ,
I
∫λ
I
m
3
( s )ds = lim ∫ max(0, G3 ( wm )( s ))ds
m →∞ I
1487
Optimal Control Problems
which show that w is admissible. Now, let any w ' ∈ W and let ( w ' ∈ W ) be a
subsequence converging to w ' in L2 strongly. By Steps 2 and 3, we have
2
DG m ( wm , w '− wm ) + (γ / 2) w '− wm
L2
= DG0 ( wm , w '− wm ) + λ1m DG1 ( wm , w '− wm ) + λ2m DG2 ( wm , w '− wm )
+ ∫ λ3m ( s ) DG3 ( wm , w '− wm )( s )ds + (γ / 2) w '− wm
I
2
L2
≥ dm .
By Theorem 2.3, the derivatives DGl , l = 0,1, 2,3 ( DG3 with values in C ( I ) m3 ) are
continuous and λ3m → λ3 in M ( I ) m3 weak star. Since also wm → w in L2 strongly,
we can pass to the limit in the above inequality and obtain
DG0 ( w, w '− w) + λ1 DG1 ( w, w '− w) + λ2 DG2 ( w, w '− w)
T
2
0
L2
+ ∫ λ3 (ds ) DG3 ( w, w '− w)( s ) + (γ / 2) w '− w
≥ 0,
which holds for every w ' ∈ W . Replacing w ' by w + θ ( w '− w) , θ ∈ (0,1] , dividing
by θ , and taking the limit as θ → 0 , we obtain the same inequality, but without the
term containing the norm. On the other hand, by the definition of λ2m and Theorem
2.2, if G2 j ( w) < 0 , for some index j ∈ {1,..., m2 } , then λ2mj = 0 for m sufficiently
large, hence λ2 j = 0 , which shows that λ2G2 ( w) = 0 . Now, since w is admissible, for
each j = 1,..., m3 fixed, we have G3 j ( w) ≤ 0 , i.e.
g3 j ( s, y ( s )) ≤ 0 , s ∈ I , with y = y w .
Let ε > 0 be given, and define the sets
Sε j = {s ∈ I − ε ≤ g3 j ( s, y ( s )) ≤ 0,} ,
S 'ε j = {s ∈ I g3 j ( s, y ( s )) ≤ −ε } .
Let m ' be such that
g3 j ( s, y m ( s )) − g3 j ( s, y ( s )) ≤ ε , s ∈ I , for m ≥ m ' ,
By the definition of λ3m , we have
λ3mj ( s ) = 0 , s ∈ S 'ε j , for m ≥ m ' .
It follows that
η mj :=
∫
T
0
≤ 2ε λ3mj
λ3mj ( s) g3 j ( s), y m ( s))ds =
≤ 2ε λ3mj
L1 ( Sε j )
L1 ( I )
∫
Sε j
λ3mj ( s ) g3 j ( s ), y m ( s ))ds
≤ 2cε , for m ≥ m ' .
Therefore, by the involved weak star and uniform convergences
0 = lim η mj =
m →∞
∫
T
0
λ3 j (ds) g3 j ( s, y ( s)) ,
j = 1,..., m3 ,
or equivalently (since λ3 ≥ 0 and g3 ( s, y ( s )) ≤ 0 , in the limit)
∫
T
0
λ3 ( ds ) g3 ( s, y ( s )) = 0 .
On the other hand, since λ3m ≥ 0 and λ3m → λ3 weak star, we have
2
2
m3
1 + ∑ λlm + ∫ λ3m ( s )ds = 1 + ∑ λlm + ∫ {∑ [1⋅ λ3mj ( s )]}ds
l =0
I
l =0
T
0
j =1
1488
I. Chryssoverghi, J. Coletsos and B. Kokkinis
2
m3
2
j =1
l =0
→ 1 + ∑ λl + ∫ {∑ [1 ⋅ λ j 3 ( ds )]} = 1 + ∑ λl + λ3 * = a ≠ 0 .
T
0
l =0
We clearly have also λ0 = 1 , λ2 ≥ 0 , and since λ3m ≥ 0 and λ3m → λ3 in M ( I ) m3 weak
star, we have also λ3 ≥ 0 . Dividing all λl by a , w is thus weakly extremal classical.
(ii) Let ( wm ) be a subsequence (same notation) of the sequence generated in Step 3
that converges in R to some r in R . The admissibility of r is proved as in (i).
Suppose as in (i) that λlm → λl , l = 1, 2 , and λ3m → λ3 in M ( I ) m3 weak star. By Step
2, we have
DG m ( wm , w '− wm ) + (γ / 2) w '− wm
2
L2
= DG0 ( w , w '− w ) + λ DG1 ( w , w '− wm ) + λ2m DG2 ( wm , w 'n − wm )
m
nm
m
1
m
+ ∫ λ3m ( s ) DG3 ( wm , w '− wm )( s )ds + (γ / 2) w '− wm
I
2
L2
≥ dm ,
for every w ' ∈ W , which can be written
2
∑λ ∫
m
l
l =0
T
0
[ zlm (t ) f u (t , y m (t ), wm (t )) + glu (t , y m (t ), wm (t ))][ w '(t ) − wm (t )]dt
T
+ ∫ λ3m (ds ) g3 y ( s, y m ( s )) Z ( s ) −1
0
( ∫ Z (t) f (t, y (t), w (t))[w '(t) − w (t)]dt )
s
m
m
m
u
0
2
T
+ (γ / 2) ∫ w '(t ) − wm (t ) dt ≥ d m ,
0
for every w ' ∈ W . Choosing now any Caratheodory function φ ∈ B( I × U ;U ) and
setting w '(t ) := φ (t , w(t )) in this inequality, by the continuity of the relaxed state and
adjoint operators (Theorems 2.2-3) and using Proposition 2.1 in [1] (also for each
fixed s - the integrals
∫
s
0
are equicontinuous and converge for each s , hence
uniformly), and the convergences λ3m → λ3 in M ( I ) m3 weak star and wm → r in R ,
we can pass to the limit in this inequality and obtain
2
∑λ ∫
l
l =0
T
0
[ zl (t ) f u (t , y (t ), r (t )) + glu (t , y (t ), r (t ))][φ (t , r (t )) − r (t )]dt
T
+ ∫ λ3 ( ds ) g3 y ( s, y ( s )) Z ( s ) −1
0
T
(∫ Z (t) f (t, y(t), r(t))[φ (t, r(t)) − r(t)]dt )
s
0
u
+(γ / 2) ∫ φ (t , r (t )) − r (t ) dt ≥ 0 , for every such φ ,
0
2
which implies, by a θ -argument similar to (i), the same inequality, but without the
last integral term, and with multipliers as in the optimality conditions. The
transversality conditions are proved similarly to (i).
(iii) In any of the above cases (i) or (ii), suppose that the limit control is admissible
and that the classical, or relaxed, problem has no admissible, abnormal extremal,
controls. Suppose that the multipliers are not all bounded. Then, dividing the
corresponding inequality resulting from Step 2 by the greatest multiplier norm and
passing to the limit for a subsequence, we readily see that we obtain an extremality
inequality where the first multiplier is zero, and that the limit control is abnormal
extremal, a contradiction. Therefore, the sequences of multipliers are bounded, and by
(i) or (ii), this limit control is extremal as above.
1489
Optimal Control Problems
The proofs in the absence of state constraints are similar.
If the additional assumptions of Theorem 2.4 are also satisfied (classical case),
then Algorithm 1 computes optimal classical controls in case (i).
Next, we define the penalized functionals on R
m1
m2
2
1
G m (r ) = G0 (r ) + {M 1m ∑ G1 j (r ) + M 2m ∑ [max(0, G2 j (r ))]2
2
j =1
j =1
m3
+ M 3m ∑ ∫ [max(0, G3 j (r )( s ))]2 ds} .
j =1
I
The directional derivative of G m is given by
DG m (r , r '− r ) = DG0 (r , r '− r )
+M
+M
m
1
m1
∑G
1j
j =1
(r ) DG1 j (r , r '− r ) + M
m3
m
3
∑ ∫ max(0, G
j =1
3j
I
m
2
m2
∑ max(0, G
j =1
2j
(r )) DG2 j (r , r '− r )
(r )( s )) DG3nj (r , r '− r )( s )ds .
The relaxed penalized conditional descent method is described by the following
Algorithm, where U is not necessarily convex.
Algorithm 2
Step 1. Set k := 0 , m := 1 , and choose an initial control r01 ∈ R .
Step 2. Find rk m ∈ R such that
d k := DG m (rkm , rk m − rkm ) = min DG m (rkm , r '− rkm ).
r '∈R
m
Step 3. If d k ≤ β , set r := r , r := rk m , d m := d k , m := m + 1 , and go to Step 2.
m
m
m
k
Step 4. Find the lowest integer value s (positive or not), say s , such that
α ( s ) = c sζ k ∈ (0,1] and α ( s ) satisfies the inequality
G m (rkm + α ( s )(rk m − rkm )) − G m (rkm ) ≤ α ( s )bd k ,
and then set α k := α ( s ) .
Step 5. Choose any rkm+1 ∈ R such that
G l (rkl+1 ) ≤ G l (rkl + α k (rkl − rkl )),
set k := k + 1 , and go to Step 2.
With r m as defined in Step 3, define the sequences of multipliers
λ1m := M 1mG1 (r m ), λ2m := M 2mmax(0, G2 (r m )),
λ3m ( s) := M 3mmax(0, G3 (r m )( s )) = M 3mmax(0, g3 ( s, y m ( s))) .
Theorem 3.2 We suppose that the derivatives in u are excluded in the last
assumptions of Section 2.
(i) In the presence of state constraints, if a subsequence (r l )l∈L of the sequence
generated by Algorithm 2 in Step 3 converges to some r in R and the sequences
(λlm ) , l = 1, 2,3 , (λ3m ) in L1 ( I )m3 , are bounded, then r is admissible and strongly
1490
I. Chryssoverghi, J. Coletsos and B. Kokkinis
extremal relaxed for the relaxed problem. In the absence of state constraints, if a
subsequence ( rk ) k∈K (no index m ) converges to some r in R , then r is strongly
extremal relaxed for the relaxed problem.
(ii) In case (i) with state constraints, suppose that the relaxed problem has no
admissible, abnormal extremal, controls. If r is admissible, then the sequences (λmm )
are bounded and r is also strongly extremal relaxed for the relaxed problem.
Proof. (State constraints present) Suppose by contradiction, as in the proof of
Theorem 3.1, that m remains constant after a finite number of iterations in k , and so
we drop here the index m . Since R is compact, let (rk )k∈K , (rk ) k∈K be subsequences
of the sequences generated by Algorithm 2 that converge to some r% ∈ R , r% ∈ R ,
respectively. By Theorem 2.3, we have
d := lim d k = lim DG (rk , rk − rk ) = DG (r%, r% − r% ) ≤ 0.
k →∞ , k∈K
k →∞ , k∈K
Suppose that d < 0 . The function Φ(α ) := G (r + α (r '− r )) is continuous on [0,1] , and
since the directional derivative DG (r , r '− r ) is linear in r '− r , Φ is differentiable on
(0,1) and has derivative Φ '(α ) = DG (r + α (r '− r ), r '− r ). By the mean value theorem,
for each α ∈ (0,1] there exists α '(α ) ∈ (0, α ) such that
G (rk + α (rk − rk )) − G (rk ) = α DG (rk + α '(α )(rk − rk ), rk − rk ) .
Therefore, by Theorem 2.3
G (rk + α (rk − rk )) − G (rk ) = α (d + ε kα ), for α ∈ [0,1] ,
where ε kα → 0 as k → ∞ , k ∈ K , and α → 0+ . We then show, similarly to Theorem
3.1, that d = 0 , hence d k → 0 , which leads also to a contradiction. Therefore,
m → ∞ in Algorithm 2.
(i) Let (r m ) be a subsequence (same notation) of the sequence generated in Step 3 of
Algorithm 2, that converges to some r ∈ R as m → ∞ . The admissibility of r is
shown similarly to Theorem 3.1. Suppose that the sequences (λmm ) are bounded and,
up to subsequences, that λmm → λm . Now, by Steps 2 and 3 we have, for any r ' ∈ R
2
DG m (r m , r '− r m ) = ∑ λlm DGl (r m , r '− r m ) + ∫ λ3m ( s ) DG3 (r m , r '− r m )( s )ds ≥ d m .
I
l =0
Since d m ≤ β m → 0 by Step 3, using the above convergences and Theorem 2.3, we
can pass to the limit and obtain
2
∑ λ DG (r , r '− r ) + ∫ λ (s) DG (r , r '− r )(s)ds ≥ 0,
l =0
l
l
I
3
3
for every r ' ∈ R ,
with multipliers λl as in the optimality conditions. Finally, we find in the limit the
transversality and other conditions similarly to Theorem 3.1. Therefore, r is also
weakly extremal relaxed for the relaxed problem.
The proof in the absence of state constraints is similar.
(ii) The proof is similar to that of Theorem 3.1 (iii).
If the additional assumptions of Theorem 2.4 are also satisfied (relaxed case),
then Algorithm 2 computes optimal relaxed controls.
1491
Optimal Control Problems
One can see (using Filippov’s selection theorem, see [13]) that a classical
(measurable) control rk m in Step 2 can be found for every k by appropriately
minimizing H (t , y m (t ), z m (t ), u ) on U , for each t ∈ I (in practice, this is trivially
done for discrete, e.g. piecewise constant, controls).
Algorithm 2 can be implemented as follows. Suppose for simplicity that there
are no state constraints. We first choose the initial discrete control in Step 1 to be of
Gamkrelidze type, i.e. equal for each t to a convex combination of d + 1 Dirac
measures at d + 1 points of U , where d is the dimension of the system. Suppose, by
induction, that the control rkm computed in Algorithm 2 is of Gamkrelidze type. Since
the control rk m in Step 2 is chosen to be classical, i.e. piecewise Dirac, the control
r%km := rkm + α (rk m − rkm ) in Step 5 is piecewise equal to a convex combination of d + 2
Dirac measures. Using now a known property of convex hulls of finite vector sets,
we can construct a Gamkrelidze control rkm+1 equivalent to r%km , i.e. such that
f (t , y% km (t ), rkm+1 (t )) = f (t , y% km (t ), r%km (t )) ∈ IR d , t ∈ I ,
where y% km corresponds to r%km , by selecting only d + 1 appropriate points in U among
the d + 2 ones defining r%km , for each t . The control rkm+1 yields then the same state,
values of functionals and G m as r%km . Therefore, the constructed control rkm is of
Gamkrelidze type for every k .
In practice, by choosing in Algorithms 1 and 2 moderately growing sequences
m
( M l ) and a sequence ( β m ) relatively fast converging to zero, the resulting
sequences of multipliers (λlm ) are often kept bounded. One can choose a fixed
ζ k := ζ ∈ (0,1] in Step 4; a usually faster and adaptive procedure is to set ζ 0 := 1 , and
then ζ k := α k −1 , for k ≥ 1 .
Gamkrelidze Formulation Approach
When directly applied to nonconvex optimal control problems whose solutions are
non-classical relaxed controls, the classical method yields often very poor
convergence (due to highly oscillating extremal controls). For this reason, we describe
here an alternative approach, assuming that U is convex, following [7], that uses the
Gamkrelidze formulation. For simplicity, we consider only the case without state
constraints. Consider the following relaxed problem, with state equation
y '(t ) = f (t , y (t ), r (t )) , t ∈ I , y (0) = y 0 ,
control constraint r ∈ R , and cost functional
G (r ) := g ( y (T )) .
For each t fixed, the vector f (t , y (t ), r (t )) in IR d belongs to the convex hull of the
set
f (t , y (t ), U ) ⊂ IR d . Hence, we can write
d +1
f (t , y (t ), r (t )) = ∑ v j (t ) f (t , y (t ), w j (t )) , with 0 ≤ v j (t ) ≤ 1 ,
j =1
d +1
∑ v (t ) = 1 ,
j =1
j
and by Filippov’s selection theorem, we can suppose that these functions v j , w j are
measurable. Therefore, the control r yields the same state y as the Gamkrelidze
1492
I. Chryssoverghi, J. Coletsos and B. Kokkinis
d +1
control rG := ∑ v j (t )δ w j (t ) . Conversely, every such a control rG is clearly a relaxed
j =1
control r that yields the same state. Therefore, the above relaxed control problem is
equivalent to the following extended classical problem, with state equation
d +1
y '(t ) = ∑ v j (t ) f (t , y (t ), w j (t )) in I ,
y (0) = y 0 ,
j =1
classical controls v = ( v j ) , w = ( w j ) , convex control constraints
d +1
∑ v (t ) = 1 , 0 ≤ v (t ) ≤ 1 , w (t ) ∈ U ,
j
j
j =1
j
j = 1,..., d + 1 ,
and cost functional G ( v, w ) := g ( y (T )) . We can therefore apply the classical method
(Algorithm 1) to this problem. The main disadvantage of this approach is that the
dimension of the control space is rapidly increased; it can be successfully applied for
relatively small dimensions d , d ' . In the general case, i.e. if U is not convex, one can
use Algorithm 2 to solve such highly nonconvex problems.
Finally, Gamkrelidze relaxed controls (in practice discrete ones) computed as
above, or by Algorithm 2, can then be approximated, and simulated, by classical
controls using a standard procedure, see [2], [4].
4 Numerical examples
Let I := [0,1] .
Example 1. Define the reference state and control
y1 (t ) = e − t , y2 (t ) = e −2t , y3 (t ) = e −3t ,
w(t ) = min(1, −1 + 2.5t ) ,
and consider the following optimal control problem, with state equations
y1 ' = − y1 + y3 − e −3t + sin y1 − sin y1 + w1 − w ,
y2 ' = y1 − 2 y2 − e − t + w2 − w,
y3 ' = y2 − 3 y3 − e −2t + w3 − w,
t∈I ,
y1 (0) = y2 (0) = y3 (0) = 1,
control constraint set U = [−1,1] , and cost functional
3
G0 ( w) := 0.5∫ {∑ [( yi − yi ) 2 + ( wi − w) 2 ]}dt.
1
0
i =1
The optimal control and state are clearly w*:= ( w, w, w) and y*:= ( y1 , y2 , y3 ) .
Algorithm 1, without penalties, was applied to this example, using the midpoint
scheme (step size 0.002) for solving the differential equations, with piecewise
constant classical controls, and γ = 0.5 . After 15 iterations in k , we obtained
G0 ( wK ) = 1.366 ⋅10−12 , d k = −4.979 ⋅10−15 , ε k = 4.316 ⋅10−7 , ζ k = 4.789 ⋅10−7 ,
Optimal Control Problems
1493
where d k was defined in Step 2 of Algorithm 1, ε k is the max-error for the state at
the nodes, and ζ k the max-error for the control at the midpoints of the intervals.
Figure 1 shows the first component of the last computed control.
Example 2. With the same state equations, cost, and parameters as in Example 1, but
with constraint set U = [−0.75,1] , additional pointwise state constraints
G3 ( w)(t ) = 0.8 − 0.4t − y1 (t ) ≤ 0 , t ∈ I ,
and applying here the penalized Algorithm 1, we obtained after 85 iterations in k
G0 ( wk ) = 3.765340220 ⋅10−3 , d k = −4.502 ⋅10−6 ,
the maximum state constraint violation
θ k := max[max(0, 0.8 − 0.4ti − y1i ,k )] = 1.444 ⋅10−5 ,
i
and the first control and state components shown in Figures 3 and 4.
Example 3. Consider the nonconvex problem, with state equations
y1 ' = − y1 + w, y2 ' = 0.5( y1 − y ) 2 − w2 , t ∈ I ,
y1 (0) = 1, y2 (0) = 0,
where y (t ) = e − t , control constraint set U = [−1,1] , and cost G ( w) = y2 (1) . The
unique optimal relaxed control is clearly r * (t ) = (δ −1 + δ 1 ) / 2 ( δ −1 , δ1 Dirac
measures), with optimal state y* = y and optimal cost G ( r*) = −1 . Note that the
optimal relaxed cost can be approximated as closely as desired with a classical
control, but cannot be attained for such a control. Since here the set f (t , y ,U ) is a
continuous arc, hence a connected set, the Gamkrelidze formulation involves only
three controls v, u, w
1
y1 ' = − y1 + vu + (1 − v) w, y2 ' = ( y1 − y ) 2 − vu 2 − (1 − v) w2 , t ∈ I ,
2
y1 (0) = 1, y2 (0) = 0,
with v ∈ [0,1] and u, w ∈ [ −1,1] . Applying to this problem Algorithm 1 without
penalties, we obtained after 15 iterations the control vk ≈ 0.5 with max-error ≈ 3 ⋅10−6
at the midpoints, the controls uk = −1 , wk = 1 exactly, the optimal state with maxerror ≈ 3.8 ⋅10−6 at the nodes, the approximate cost G (vk , uk , wk ) = −0.999999999997 ,
and d k = −1.134 ⋅10−6 .
Example 4. Consider the following problem, with state equations
y1 ' = − y2 + w1 , y2 ' = − y1 + w2 , t ∈ [0,0.5) ,
y1 ' = − y2 + w1 − t + 0.5 , y2 ' = − y1 + w2 − t + 0.5 , t ∈ [0.5,1] ,
y1 (0) = y2 (0) = 1 ,
nonconvex control constraint set
U = {(u1 , 0) ∈ IR 2 0 ≤ u1 ≤ 1} ∪ {(0, u2 ) ∈ IR 2 0 ≤ u2 ≤ 1} ,
and cost functional
1
G0 ( w) = 0.5∫ [( y1 − y ) 2 + ( y2 − y ) 2 ]dt ,
0
−t
where y = e . Clearly, the unique optimal relaxed control is
r * (t ) = δ (0,0) , t ∈ [0,0.5) (classical)
1494
I. Chryssoverghi, J. Coletsos and B. Kokkinis
r * (t ) = 2(1 − t )δ (0,0) + (t − 0.5)δ (1,0) + (t − 0.5)δ (0,1) , t ∈ [0.5,1] (non-classical),
where δ denotes Dirac measures, which yields the optimal state ( y1*, y2 *) = ( y , y ) ,
and cost G0 ( r*) = 0 . Algorithm 2, without penalties, was applied here, using also the
midpoint scheme (step size 0.002), here with piecewise constant relaxed controls.
After 120 iterations in k , we obtained
G0 (rk ) = 2.552 ⋅10−6 , d k = −3.393 ⋅10−5 .
Figures 4 and 5 show the last relaxed control probability functions
⎧1, t ∈ [0, 0.5)
p0 (t ) = rk (t )({(0, 0)}) ≈ ⎨
⎩2(1 − t ), t ∈ [0.5,1)
⎧0, t ∈ [0, 0.5)
p1 (t ) = rk (t )({(1, 0)}) ≈ ⎨
⎩t − 0.5, t ∈ [0.5,1)
We also obtained p2 (t ) = rk (t )({(0,1)}) = 1 − p0 (t ) − p1 (t ) .
Example 5. Consider the following problem, with state equation
y ' = − y + w , t ∈ I , y (0) = 1 ,
control constraint set U = [−1,1] , state constraints
G1 ( w) = y (1) − 0.5 = 0 , G3 ( w)( s ) = 0.3 − y ( s ) ≤ 0 , s ∈ I ,
and nonconvex cost functional
1
G0 ( w) = ∫ (0.5 y 2 − w2 )dt .
0
Writing the solution of the state equation in closed form via the fundamental solution,
first forward with initial condition y (0) = 1 and then backward with terminal
condition y (1) = 0.5 , we can see that the unique optimal relaxed control and state are
⎧δ −1 , t ∈ [0, ρ ) (classical)
⎪
r * (t ) = ⎨0.35δ −1 + 0.65δ 1 , t ∈ [ ρ , σ ) (non-classical)
⎪δ , t ∈ [σ ,1] (classical)
⎩ 1
⎧−1 + 2e − t , t ∈ [0, ρ )
⎪
y * (t ) = ⎨0.3, t ∈ [ ρ , σ )
⎪1 − 0.5e1−t , t ∈ [σ ,1]
⎩
where ρ = − ln 0.65 ≈ 0.43 is such that −1 + 2e− ρ = 0.3 , and σ = 1 − ln1.4 ≈ 0.66 such
that 1 − 0.5e1−σ = 0.3 . The optimal cost is G0 (r*) ≈ −0.868398913624 . Applying here
the penalized Algorithm 2, we obtained after 200 iterations in k the results
G0 (rkm ) = −0.868200567558 , G1 (rkm ) = −9.732 ⋅10−6 ,
max[max(0, 0.3 − y1mi,k )] = 2.962 ⋅10−4 , d k = −1.236 ⋅10−3 .
i
Figure 6 shows the last relaxed control probability function
⎧0, t ∈ [0, ρ )
⎪
m
p1 (t ) = rk (t )({(1)}) ≈ ⎨0.65, t ∈ [ ρ , σ )
⎪1, t ∈ [σ ,1]
⎩
and we also obtained p0 (t ) = rkm (t )({−1}) = 1 − p1 (t ) . Figure 7 shows the last state.
1495
Optimal Control Problems
W1
1
0.5
1
t
-1
Figure 1. Example 1: Last control (1st component)
W1
1
0.5
1
- 0.75
Figure 2. Example 2: Last control (1st component)
t
1496
I. Chryssoverghi, J. Coletsos and B. Kokkinis
Y1
1
0.5
0.4
0.5
1
t
Figure 3. Example 2: Last state (1st component)
P0
1
0.5
1
t
Figure 4. Example 4: Last relaxed control probability p0
P1
1
0.5
0.5
1
t
Figure 5. Example 4: Last relaxed control probability p1
1497
Optimal Control Problems
P1
1
0.65
с
у
1
t
Figure 6. Example 5: Last relaxed control probability p1
Y
1
0.5
0.3
с
у
1
t
Figure 7. Example 5: Last state
References
[1] I. Chryssoverghi, Nonconvex optimal control of nonlinear monotone parabolic
systems, Systems and Control Letters, 8 (1986), 55-62.
[2] I. Chryssoverghi, A. Bacopoulos, Discrete approximation of relaxed optimal
control problems, Journal of Optimization Theory and Applications, 65, 3 (1990),
395-407.
[3] I. Chryssoverghi, A. Bacopoulos, J. Coletsos and B. Kokkinis, Discrete
approximation of nonconvex hyperbolic optimal control problems with state
constraints, Control and Cybernetics, 27, 1 (1998), 29-50.
[4] I. Chryssoverghi, J. Coletsos and B. Kokkinis, Discrete relaxed method for
semilinear parabolic optimal control problems, Control and Cybernetics, 28, 2 (1999),
157-176.
[5] I. Chryssoverghi, J., Coletsos, B., Kokkinis, Approximate relaxed descent method
for optimal control problems, Control and Cybernetics, 30, 4 (2001), 385-404.
1498
I. Chryssoverghi, J. Coletsos and B. Kokkinis
[6] I. Chryssoverghi, Discrete gradient projection method with general Runge-Kutta
schemes and control parameterizations for optimal control problems, Control and
Cybernetics, 34, (2005), 425-451.
[7] I. Chryssoverghi, Discretization methods for semilinear parabolic optimal control
problems, International Journal of Numerical Analysis and Modeling, 3 (2006), 437458.
[8] A.L. Dontchev, W.W. Hager and V.M. Veliov, Second-order Runge-Kutta
approximations in control constrained optimal control, SIAM Journal on Numerical
Analysis, 38, 1 (2000), 202-226.
[9] T. Roubíček, A Convergent computational method for constrained optimal relaxed
control problems, Journal of Optimization Theory and Applications, 69 (1991), 589603.
[10] T. Roubíček, Relaxation in Optimization Theory and Variational Calculus,
Walter de Gruyter, Berlin (1997).
[11] V.M. Veliov, On the time-discretization of control systems, SIAM Journal on
Control and Optimization, 35, 5 (1997), 1470-1486.
[12] A. Schwartz and E. Polak, Consistent approximations for optimal control
problems based on Runge-Kutta integration, SIAM Journal on Control and
Optimization, 34, 4 (1996), 1235-1269.
[13] J. Warga, Optimal Control of Differential and Functional Equations, Academic
Press, New York (1972).
[14] J. Warga, Steepest Descent with Relaxed Controls, SIAM Journal on Control and
Optimization, 15 (1977) 674-682.
Received: May 18, 2006