Yugoslav Journal of Operations Research
16 (2006), Number 2, 153-160
FIRST AND SECOND ORDER NECESSARY OPTIMALITY
CONDITIONS FOR DISCRETE OPTIMAL CONTROL
PROBLEMS
Boban MARINKOVIĆ
Faculty of Mining and Geology, University of Belgrade
Belgrade, Serbia
[email protected]
Received: October 2004 / Accepted: June 2005
Abstract: Discrete optimal control problems with varying endpoints are considered. First
and second order necessary optimality conditions are obtained without normality
assumptions.
Keywords: Discrete optimal control, mathematical programming, abnormal extremal.
1. INTRODUCTION
Consider discrete optimal control problem with varying endpoints.
N −1
minimize ∑ fi ( xi , ui ) ;
(1)
xi +1 = ϕi ( xi , ui ) , i = 0, N − 1 ,
(2)
K ( x0 , xN ) = 0 ,
(3)
i =1
where fi ( x, u ) : R n × R r → R is twice continuously differentiable function, ϕi ( x, u ) :
R n × R r → R n and K ( x0 , xN ) : R n × R n → R k are twice continuously differentiable
mappings. Here xi ∈ R n is state variable, ui ∈ R r is a control parameter, N is given
number of steps. Vector ε = ( x0 , x1 ,..., xn ) is called a trajectory, ω = (u0 , u1 ,..., u N −1 ) is
called a control, x0 is a starting point and xN is an end point of corresponding trajectory.
154
B. Marinković / First and Second Order Necessary Optimality Conditions
Let x0 be a starting point and let be a control. Then the pair ( x0 , ω ) defines the
corresponding directory ε = ( x0 , x1 ,..., xn ) . If the condition (3) is satisfied then we say
that the pair ( x0 , ω ) is feasible.
The discrete optimization problem is to minimize the function
N −1
J ( x0 , ω ) = ∑ fi ( xi , ui )
i =0
on the set of feasible pairs.
The aim of this paper is to obtain first and second order necessary optimality
conditions for the problem (1)-(3) without normality assumptions.
2. FIRST ORDER NECESSARY OPTIMALITY CONDITIONS
Let ( xˆ0 , ωˆ ) be a feasible pair and let εˆ = ( x0 , x1 ,..., xN ) be a corresponding
trajectory. Suppose that the pair ( xˆ0 , ωˆ ) is optimal.
Put
∂ϕ
∂ϕ
= Ck ,
= Dk
∂x
∂u
and
∂ 2ϕ k
∂ 2ϕ k
∂ 2ϕ k
= Ck2 ,
= Dk2 ,
= M k2 .
2
2
∂x∂u
∂x
∂u
Let (h, v), h = (h0 , h1 ,..., hN ) ∈ R n ( N +1) , v = (v0 , v1 ,..., vN −1 ) ∈ R rN , be the vector for
'
'
which there exists a vector ( h , v ) ∈ R
satisfied:
k −1
k −2
s =0
j = 0 j < s ≤ k −1
hk = ∏ Cs h0 + ∑
∏
n ( N +1)
× R rN such the following conditions are
Cs D j v j + Dk −1vk −1 , k = 1, N ,
(4)
∂K
( xˆ0 , xˆ N )(h0 , hN ) = 0 ,
∂ ( x0 , xN )
(5)
−Ck2 [hk ]2 − Dk2 [vk ]2 − 2M k2 [hk , vk ] = hk' +1 − Ck hk' − Dk vk' , k = 0,..., N − 1.
(6)
Here [-,-] stands for the arguments of a bilinear form (or, more generally,
bilinear mapping).
Denote by K set of all vectors (h,v) for which the preceding conditions are
satisfied.
Define the functions
H Ai ( x, u , p, q, λ0 , h, v) = 〈 p, ϕi ( x, u )〉 + 〈 q,
−λ0 fi ( x, u ), i = 0, N − 1 ,
∂ϕi
∂ϕ
( x, u ) h + i ( x , u ) v〉 −
∂x
∂u
B. Marinković / First and Second Order Necessary Optimality Conditions
l A ( x0 , xN , λ 1 , λ 2 , h0 , hN ) = 〈 λ1 , K ( xˆ0 , xˆ N )〉 + 〈 λ2 ,
155
∂K
( xˆ0 , xˆ N )(h0 , hN )〉 ,
∂ ( x0 , xN )
where
λ0 ∈ R, λ1 , λ2 ∈ R k , p, q ∈ R n .
Theorem 1: Let ( xˆ0 , ωˆ ) be the optimal solution for the problem (1)-(3). Then there exists
Lagrange multiplier
λ A = (λ0 , λ1 , λ2 , p, q), λ0 ∈ R, λ1 , λ2 ∈ R k , p ∈ R n ( N +1) , q ∈ R nN ,
λ0 + (q, λ2 ) ≠ 0 such that for every (h, v) ∈ K the following conditions are satisfied:
(i)
p0 =
∂l A
( xˆ0 , xˆ N , λ1 , λ2 , h0 , hN ) ,
∂x0
(7)
pi =
∂H Ai
( xˆi , uˆi , pi +1 , qi +1 , λ0 , hi , vi ), i = 0, N − 1 ,
∂x
(8)
pN = −
∂l A
( xˆ0 , xˆ N , λ1 , λ2 , h0 , hN ),
∂xN
∂H Ai
( xˆi , uˆi , pi +1 , qi +1 , λ0 , hi , vi ), i = 0, N − 1 .
∂u
(9)
(10)
(ii) There exists vector (h′, v′) ∈ R n ( N +1) × R rN such that
pk = hk′ − Ck −1hk′ −1 − Dk −1vk′ −1 , k = 1, N ,
λ1 =
∂K 0
( xˆ0 , xˆ N )h0′ ,
∂x0
(11)
(12)
(iii)
−C0* q1 +
∂K
( xˆ0 , xˆ N )* λ2 = 0 ,
∂x0
(13)
qk − Ck* qk +1 = 0, k = 1,...N − 1 ,
(14)
∂K
( xˆ0 , xˆ N )* λ2 = 0 ,
∂xN
(15)
− Dk*−1qk = 0, k = 1,..., N .
(16)
qN −
Proof: We shall formulate the problem (1)-(3) as a mathematical programming problem,
and we shall apply results from [2].
156
B. Marinković / First and Second Order Necessary Optimality Conditions
Define the functions
f (ε , ω ) : R n ( N +1) × R rN → R, Fi (ε , ω ) : R n ( N +1) × R rN → R n , i = 1, N ,
and
F (ε , ω ) : R n ( N +1) × R rN → R nN ,
by
N −1
f (ε , ω ) = ∑ fi ( xi , ui ) ,
i =0
Fi +1 (ε , ω ) = xi +1 − ϕi ( xi , ui ), i = 0, N − 1 ,
F (ε , ω ) = ( F1 (ε , ω ),..., FN (ε , ω )) .
Consider the following mathematical programming problem:
Minimize f (ε , ω ) ;
(17)
F (ε , ω ) = 0 ,
(18)
K ( x0 , xN ) = 0 .
(19)
The point (εˆ, wˆ ) is the local minimum for preceding problem.
Consider the operator
F (ε , ω ) : R n ( N +1) × R rN → R nN × R k
given by
F (ε , ω ) = ( F (ε , ω ), K ( x0 , xN )) .
Define the Lagrange – Avakov function
LA (ε , ω , λA , h, v) : R n ( N +1) × R rN × R × R nN + k × Κ → R
by
∂F
LA (ε , ω , λA , h, v) = λ0 f (ε , ω ) + p , F (ε , ω ) + q ,
(ε , ω )(h, v) ,
∂ (ε , ω )
where λA = (λ0 , p , q ), λ0 ∈ R, p , q ∈ R nN + k and K is the set of all (h, v) ∈ R n ( N +1) × R rN
such that the following conditions are satisfied:
1)
∂F
(ε, ωˆ )(h, v ) = 0 ,
∂ (ε , ω )
2)
∂2 F
∂F
(ε, ωˆ )[(h, v), (h, v)] ∈ im
(ε, ωˆ ) .
∂ (ε , ω )
∂ (ε , ω ) 2
Note that the vector (h,v) is the parameter in the Lagrange – Avakov function.
B. Marinković / First and Second Order Necessary Optimality Conditions
Put A =
157
∂F
(ε, ω ) .
∂ (ε , ω )
From [2], theorem 10.1, we have that ∃λA , λ0 ≥ 0, λ0 + q ≠ 0 such that for every
(h, v) ∈ K ,
∂LA
(ε, ω , λA , h, v) = 0, p ∈ im A, q ∈ (im A) ⊥ .
∂ (ε , ω )
(20)
From the fact that
N −1
N −1
i =0
i =0
LA (ε , ω , λA , h, v) = λ0 ∑ fi ( xi , ui ) + ∑ pi +1 , xi +1 − ϕ ( xi , ui ) + λ1 , K ( x0 , x N ) +
N −1
+ ∑ qi +1 , hi +1 − Ci hi − Di vi + λ2 ,
i =0
∂K
( x0 , x N )(h0 , hN ) ,
∂ ( x0 , xN )
where p = ( p1 ,..., pN , λ1 ) and q = (q1 ,..., qN , λ2 ) , we have
∂L A ∂l A
∂H A0
=
( x0 , x N , λ1 , λ2 , h0 , hN ) −
( x0 , u0 , p, q, λ0 , h, v) = 0 ,
∂x0 ∂x0
∂x
(21)
∂L A
∂H Ai
= pi −
( xi , ui , pi +1 , qi +1 , λ0 , hi , vi ) = 0 i = 1, N − 1 ,
∂xi
∂x
(22)
∂L A
∂l
= pN + A ( x0 , x N , λ1 , λ2 , h0 , hN ) = 0 ,
∂xN
∂xN
(23)
∂Lˆ A ∂H Ai
( xˆi , uˆi , pi +1 , qi +1 , λ0 , hi , vi ) = 0 i = 0, N − 1 .
=
∂ui
∂u
(24)
Put
p0 =
∂H A0
( xˆ0 , uˆ0 , p, q, λ0 , h, v) .
∂x
(25)
From (25) and (21) we have
p0 =
∂l A
( xˆ0 , xˆ N , λ1 , λ2 , h0 , hN ) .
∂x0
Obviously that from (22), (23) and (24) we obtain that (8), (9) and (10) hold.
Put p = ( p0 , p1 ,..., pN ), q = (q1 , q2 ,..., qN ) and λ A = (λ0 , λ1 , λ2 , p, q ) . We proved
that for considered λ A hold (7), (8), (9) and (10).
B. Marinković / First and Second Order Necessary Optimality Conditions
158
From
(h′, v ′) ∈ R
the
n ( N +1)
×R
rN
fact
that
p ∈ im A
we
have
that
there
exists
vector
such that (11) and (12) hold.
Since q ∈ (im A) ⊥ we have that A∗ ⋅ q = 0 or, in other words, we obtain that
(13)-(16) hold.
From A(h, v) = 0 we obtain the system of the equations
hk − Ck −1hk −1 − Dk −1vk −1 = 0, k = 1, N
(26)
with the boundary conditions
∂K
( xˆ0 , xˆ N )(h0 , hN ) = 0 .
∂ ( x0 , xN )
Solving the equations (26) we obtain that
k =1
k −2
s =0
j = 0 j < s ≤ k −1
hk = ∏ Cs h0 + ∑
∏
Cs D j v j + Dk −1vk −1 , k = 1, N ,
holds. It’s easy to see that from
∂2 F
(εˆ, ωˆ )[(h, v), (h, v)] ∈ im A
∂ (ε , ω )2
we obtain the equation (6). We conclude that K = K .
3. SECOND ORDER NECESSARY OPTIMALITY CONDITIONS
Suppose that the function fi ( x, u ) and mappings ϕ ( x, u ) and K ( x0 , xN ) are the
three times continuously differentiable.
Put
q
∂ 2 lˆA ∂ 2 l A
λ2
∂ 2 Hˆ Ai ∂ 2 H Ai
ˆ
ˆ
x
x
h
h
=
λ
=
(
,
,
,
,
,
),
( xˆi , uˆi , pi +1 , i +1 , λ0 , hi , vi ) .
0
1
0
N
N
3
3
∂x02
∂x02
∂x 2
∂x 2
For a given Lagrange multiplier λ A = (λ0 , λ1 , λ2 , p, q ) define the bilinear form
Ω[(h, v), (h, v)] =
N −1
−∑
i =0
Where
∂ 2 lˆA
∂ 2 lˆA
∂ 2 lˆA
[
,
]
[
,
]
2
[h0 , h0 ] −
h
h
+
h
h
+
0
0
N
N
∂x0 ∂xN
∂x02
∂xN2
N −1 2 ˆ i
N −1 2 ˆ i
∂ 2 Hˆ Ai
∂ HA
∂ HA
[hi , hi ] − ∑
[vi , vi ] − 2 ∑
[hi , vi ] ,
2
2
∂x
i = 0 ∂u
i = 0 ∂x∂u
∂ 2 Hˆ Ai ∂ 2 Hˆ Ai ∂ 2 Hˆ Ai
∂ 2 Hˆ Ai
are introduced analogously as above.
,
,
and
∂x∂u
∂xN2
∂u 2 ∂x0 ∂xN
B. Marinković / First and Second Order Necessary Optimality Conditions
159
Theorem 2. Let ( xˆ0 , ωˆ ) be the optimal solution for the problem (1)-(3). Then there exists
Lagrange multiplier λ A = (λ0 , λ1 , λ2 , p, q ) such that the assertions of the theorem 1 hold,
and for every (h, v) ∈ K holds:
Ωλ [(h, v), (h, v)] ≥ 0 .
(27)
Proof: Analogously as in the proof of theorem 1 we consider the mathematical
programming problem (17)-(19). It’s easy to see that for Lagrange – Avakov function
hold:
∂ 2 LA
∂ 2 LA
≡ 0, ∀(i, j ) : i ≠ j , (i, j ) ≠ (0, N );
≡ 0, ∀i ≠ j
∂xi ∂x j
∂xi ∂u j
From [2], theorem 10.2, and from preceding facts we obtain that the assertion of theorem
2 hold.
4. CONCLUDING REMARKS
First we shall compare the number of variables and number of equations from
theorem 1.
We have that the number of variables εˆ, ωˆ , (h′, v ′) and λ A from the theorem 1 is
equal to:
2(n( N + 1) + rN ) + 1 + 2(nN + k ) .
From (2), (3), and (8)-(17) we obtain:
N n + k + n( N + 1) + rN + nN + k + n( N + 1) + rN
equations.
Since λ A ≠ 0 then, without loss of generality, we may take that λ A = 1 . It
follows that the number of variables is equal to the number of the equations and we have
complete system of the equations associated with εˆ, ωˆ , (h′, v′) and λ A .
The pair ( xˆ0 , ωˆ 0 ) is said to be extreme for the problem (1)-(3) if feasible and if
we have satisfied conditions from theorem 1.as in [3] we shall define the normal extreme
for problem (1)-(3).
Definition 1: The extremal is to be normal if
im A = R nN + k
∂F
(εˆ, ωˆ ) .
∂ (ε , ω )
First we suppose that the extreme ( xˆ0 , ωˆ 0 ) is normal. Then we have that
and abnormal otherwise. Note that A =
(q, λ2 ) = 0 and we obtain classical first order optimality conditions which are known for
160
B. Marinković / First and Second Order Necessary Optimality Conditions
a long time (see [4, 7, 8]). Also theorem 2 becomes a known second order optimality
conditions (see [9]).
Suppose that the extreme ( xˆ0 , ωˆ 0 ) is abnormal. We shall define two regular
constraint mapping for the problem (1)-(3).
Denote by K ( h ,v ) the set
⎧
⎫
∂2 F
(εˆ, ωˆ )[(h, v), (h, v)] ∈ im A⎬ .
K ( h ,v ) = ⎨(h, v) : A(h, v) = 0,
2
∂ (ε , ω )
⎩
⎭
Definition 2: The constraint mapping F (ε , ω ) is sad to be 2- regular at the point
(εˆ, ωˆ ) with respect to a direction (h, v) ∈ K if
co dim K ( h,v ) = nN + k .
Suppose the extreme ( xˆ0 , ωˆ 0 ) is abnormal and that for every (h, v) ∈ K the
mapping F (ε , ω ) is not 2-regular at the point ( xˆ , ωˆ ) with respect to a direction (h, v) .
Then we have that assertions of the theorem 1 and theorem 2 are satisfied for every
minimizing function f . It follows that in that case we have trivial theorem.
The most interesting case is when the mapping F (ε , ω ) is 2-regular at the point
(εˆ, ωˆ ) with respect to a direction (h, v) ∈ K . Then we obtain nontrivial first and second
order optimality conditions for abnormal extremes.
For details of the preceding facts we refer to [2].
REFERENCES
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
Avakov, E.R., "Extremum conditions in smooth problems with equality type of
constraints", Comput. Math. and Math. Phys., 25 (5) (1985) 690-693 (in Russian).
Arutyunov, A.V., Optimality conditions: Abnormal and Degenerate Problems, Moscow,
1997 (in Russian).
Arutyunov, A.V., "Second order conditions in optimal control problems", Doklady Math., 371
(1) (2000) 10-13 (in Russian).
Boltyanskii, V.G., Optimal Control of Discrete Systems, Moscow, 1973 (in Russian).
Vasilev, F.P., Numerical Methods of Optimal Control Problems, Moscow, 1988 (in Russian).
Vasilev, F.P., Optimization Methods, Moscow, 2002 (in Russian).
Ioffe, A.D., and Tihomirov, V.M., Theory of Extremal Problems, Moscow 1974 (in Russian).
Propoii, A.I., Elements of the Theory of Optimal Discrete Processes, Moscow, 1973 (in
Russian).
Hilscher, R., and Zeidan, V., "Discrete optimal control: second order optimality conditions",
J. Differ. Equations Appl., Vol.8 (10) (2002) 875-896.