1496
JOURNAL OF COMPUTERS, VOL. 8, NO. 6, JUNE 2013
Improved Feasible SQP Algorithm for Nonlinear
Programs with Equality Constrained SubProblems
Zhijun Luo1, Guohua Chen3 and Simei Luo4
Department of Mathematics & Applied Mathematics, Hunan University of Humanities, Science and Technology, Loudi,
China
Email: [email protected]
Zhibin Zhu2
School of Mathematics and Computing Science, Guilin University of Electronic Technology, Guilin, China
Email: [email protected]
Abstract—This paper proposed an improved feasible
sequential quadratic programming (FSQP) method for
nonlinear programs. As compared with the existing SQP
methods which required solving the QP sub-problem with
inequality constraints in single iteration, in order to obtain
the feasible direction, the method of this paper is only
necessary to solve an equality constrained quadratic
programming sub-problems. Combined the generalized
projection technique, a height-order correction direction is
yielded by explicit formulas, which can avoids Maratos
effect. Furthermore, under some mild assumptions, the
algorithm is globally convergent and its rate of convergence
is one-step superlinearly. Numerical results reported show
that the algorithm in this paper is effective.
Index Terms—Nonlinear programs, FSQP method, Equality
constrained quadratic programming, Global convergence,
Superlinear convergence rate
I. INTRODUCTION
Consider the following nonlinear programs
min f ( x)
s.t. g j ( x) ≤ 0, j ∈ I = {1, 2,
Where
f ( x), g j ( x) : R n → R ( j ∈ I ) are
, m},
(1)
continuously
differentiable functions. Denote the feasible set for (1) by
X = {x ∈ R n | g j ( x) ≤ 0, j ∈ I } .
The Lagrangian function associated with (1) is defined
as follows:
m
L ( x, λ ) = f ( x ) + ∑ λ j g j ( x )
j =1
A point x ∈ X is said to be a KKT point of (1), if it is
satisfies the equalities
This work was supported in part by the National Natural Science
Foundation (11061011) of China, and the Educational Reform Research
Fund of Hunan University of Humanities, Science and Technology
(NO.RKJGY1030), corresponding author, E-mail: [email protected] .
© 2013 ACADEMY PUBLISHER
doi:10.4304/jcp.8.6.1496-1503
m
∇f ( x) + ∑ λ j ∇g j ( x) = 0,
j =1
λ j g j ( x) = 0, j ∈ I ,
where λ = (λ1 , , λm )T is nonnegative, and λ is said to
be the corresponding KKT multiplier vector.
Method of Sequential Quadratic Programming (SQP)
is an important method for solving nonlinearly
constrained optimization [1, 2, 18]. It generates
iteratively the main search direction d 0 by solving the
following quadratic programming (QP) sub-problem:
1
min ∇f ( x)T d + d T Hd
2
s.t. g j ( x) + ∇g j ( x)T d ≤ 0, j ∈ I ,
(2)
where H ∈ R n× n is a symmetric positive definite matrix.
However, such type SQP algorithms have two serious
shortcomings:
1) SQP algorithms require that the relate QP subproblems (2) must be consistency;
2) There exists Matatos effect.
Many efforts have been made to overcome the
shortcomings through modifying the quadratic subproblem (2) and the direction d [4, 5, 7, 8]. Some
algorithms solve the problem (1) by using the idea of
filter method or trust-region [13, 16, 17].
For the problem (2), it is also a hot topic to solve the
QP problem like (2) in the field of optimization. By using
the idea of active constraints set, some algorithms solve
step by step a series of corresponding QP problems with
only equality constraints to obtain the optimum solution
to the QP sub-problem (2). P. Spellucci [6] proposed a
new method, the d 0 is obtained by solving QP subproblem with only equality constraints:
JOURNAL OF COMPUTERS, VOL. 8, NO. 6, JUNE 2013
1497
1
min ∇f ( x)T d + d T Hd
2
s.t. g j ( x) + ∇g j ( x)T d = 0, j ∈ A ⊆ I ,
complementarity. Furthermore, its global and superlinear
convergence rate is obtained under some suitable
conditions.
This paper is organized as follows: In Section II, we
state the algorithm; the well-defined of our approach is
also discussed, the accountability of which allows us to
present global convergence guarantees under common
conditions in Section III, while in Section IV we deal
with superlinear convergence. Finally, in Section V,
numerical experiments are implemented.
(3)
where the so-called working set A ⊆ I is suitably
determined. If d 0 = 0 and λ ≥ 0 ( λ is said to be the
corresponding KKT multiplier vector.), the algorithm
stops. The most advantage of these algorithms is merely
necessary to solve QP sub-problems with only equality
constraints. However, if d 0 = 0 , but λ < 0 , the algorithm
will not implement successfully. In [10], proposed an
SQP method for general constrained optimization. Firstly,
make use of the technique which handle the general
constrained optimization as an inequality parametric
programming, then, consider a new quadratic
programming with only equality constraints as follow:
1
min ∇f ( x)T d + d T Hd
2
s.t. g j ( x) + ∇g j ( x)T d = − min{0, π ( x)}, j ∈ J ( x).
Where π ( x) is a suitable vector, J ( x) is a suitable
approximate active set. But the QP problems may no
solution under some conditions. Recently, Zhu [14]
Consider the following QP sub-problem:
1
min ∇f ( x)T d + d T Hd
2
s.t. p j ( x) + ∇g j ( x)T d = 0, j ∈ L.
The active constraints set of (1) is denoted as follows:
I ( x) = { j ∈ I | g j ( x) = 0, j ∈ I }.
approximate active, which guarantees to hold that if
d 0 = 0 , then x is a KKT point of (1), i.e., if d 0 = 0 , then
it holds that λ ≥ 0 . Depended strictly on the strict
complementarity, which is rather strong and difficult for
testing, the superlinear convergence properties of the
SQP algorithm are obtained. For avoiding the superlinear
convergence depend strictly on the strict complementarity,
Another some SQP algorithms (see [15]) have been
proposed, however it is regretful that these algorithms are
infeasible SQP type and nonmonotone. In [16], a feasible
SQP algorithm is proposed. Using generalized projection
technique, the superlinear convergence properties are still
obtained under weaker conditions without the strict
complementarity.
We will develop an improved feasible SQP method for
solving optimization problems based on the one in [14].
The traditional FSQP algorithms, in order to prevent
iterates from leaving the feasible set, and avoid Maratos
effect, it needs to solve two or three QP sub-problems
like (2). In our algorithm, per single iteration, it is only
necessary to solve an equality constrained quadratic
programming, which is very similar to (4). Obviously, it
is simpler to solve the equality constrained QP problem
than to solve the QP problem with inequality constraints.
In order to void the Maratos effect, combined the
generalized projection technique, a height-order
correction direction is computed by an explicit formula,
and it plays an important role in avoiding the strict
(5)
Now, the following algorithm is proposed for solving
the problem (1).
Algorithm A:
Step 0 Initialization:
Given a starting point x 0 ∈ X , and an initial
symmetric positive definite matrix H 0 ∈ R n×n . Choose
1
parameters ε 0 ∈ (0,1), α ∈ (0, ), τ ∈ (2,3) . Set k = 0 ;
2
Step 1. Computation of an approximate active set J k .
Step 1.1. For the current point x k ∈ X , set i = 0,
ε i ( x k ) = ε 0 ∈ (0,1) .
(4)
where p j ( x ) is a suitable vector, L is a suitable
© 2013 ACADEMY PUBLISHER
II. DESCRIPTION OF ALGORITHM
Step
1.2.
If
det( Ai ( x k )T Ai ( x k )) ≥ ε i ( x k )
,
let
J k = J ( x ), Ak = A( x ), i ( x ) = i , and go to Step 2.
Otherwise go to Step 1.3, where
k
k
k
J i ( x k ) = { j ∈ I | −ε i ( x k ) ≤ g j ( x k ) ≤ 0},
Ai ( x k ) = (∇gi ( x k ), j ∈ J i ( x k )).
(6)
1
Step 1.3. Let i = i + 1, ε i ( x k ) = ε i −1 ( x k ) , and go to
2
Step1. 2.
Step 2. Computation of the vector d 0k .
Step 2.1
Bk = ( AkT Ak ) −1 AkT , v k = (v kj , j ∈ J k ) = − Bk ∇f ( x k ),
k
k
⎪⎧ −v j , v j < 0
p kj = ⎨
k
k
⎪⎩ g j ( x ), v j ≥ 0
p k = ( p kj , j ∈ J k ).
(7)
Step 2.2 Solve the following equality constrained QP
Sub problem at x k :
1
min ∇f ( x k )T d + d T H k d
2
k
k T
s.t. p j + ∇g j ( x ) d = 0, j ∈ J k .
Let
d 0k
be
the
KKT
point
of
(8)
(8),
and
b = (b , j ∈ J k ) be the corresponding multiplier vector.
k
k
j
If d 0k = 0 , STOP. Otherwise, CONTINUE;
1498
JOURNAL OF COMPUTERS, VOL. 8, NO. 6, JUNE 2013
Step 3
Computation of the feasible direction with descent d k :
~
J k ,i +1 ≡ J k ,i
Lk for i large enough. From (6) and (13),
with i → ∞ , we obtain
~
d k = d 0k − δ k Ak ( AkT Ak ) −1 ek
Where ek = (1,
δk =
L k = I ( x k ), det( AIT( xk ) AI ( xk ) ) = 0.
(9)
This is a contradiction to H 2.2, which shows that the
statement is true.
2) Suppose K is an infinite index set such that
{x k }k∈K → x* . We suppose that the conclusion is false,
,1)T ∈ R| J k | , and
|| d 0k || (d 0k )T H k d 0k
, μ k = −( AkT Ak ) −1 AkT ∇f ( x k )
2 | μ kT ek ||| d 0k || +1
i.e., there exists K ' ⊆ K (| K ' |= ∞) , such that
ε k ,i → 0, k ∈ K ' , k → ∞.
k
Step 4. Computation of the high-order revised direction
k
d :
d k = − Ak ( Ak T Ak )−1 (|| d0k ||τ ek + g Jk ( xk + d k )),
(10)
~
Let Lk = J k ,ik −1 . From the definition of ε k ,ik , it holds,
for k ∈ K ' , k large enough, that
~
det( AT~ AT~ ) = 0, − 2ε k ,ik ≤ g j ( x k ) ≤ 0, j ∈ Lk . (14)
where
Lk
g J k ( x k + d k ) = g J k ( x k + d k ) − g J k ( x k ) − ∇ g J ( x k )T d k .
Lk
Since there are only finitely many choices for sets
k
Step 5. Line search:
Compute tk , the first number t in the sequence
1 1 1
{1, , , ,...} satisfying
2 4 8
~
Lk ⊆ I
, it is sure that there exists K '' ⊆ K ' (| K '' |= ∞) ,
~
~
such that Lk ≡ L, (k ∈ K '' ) , for k
~
f ( x k + td k + t 2 d k ) ≤ f ( x k ) + α t∇f ( x k )T d k ,
g j ( x + td + t d ) ≤ 0, j ∈ I .
k
k
2
(11)
Denote A = {∇g j ( x* ) | j ∈ L} , then, let k ∈ K '' , k → ∞ ,
from (14), it holds that
~
Step 6. Update:
Obtain H k +1 by updating the positive definite matrix H
using
some
quasi-Newton
formulas.
Set
x k +1 = x k + tk d k + t 2 d k , and k = k + 1 . Go back to step 1.
~
~
det( AT A) = 0, g j ( x* ) = 0, j ∈ L ⊆ I ( x* ).
(12)
k
large enough.
~
This is a contradiction to H 2.2, too, which shows that
the statement is true.
k
Throughout this paper, following basic assumptions
are assumed.
H2.1 The feasible set X ≠ Φ , and functions f ( x),
g j ( x), j ∈ I are twice continuously differentiable.
H2.2 ∀ x ∈ X , the vectors {∇g j ( x), j ∈ I ( x)} are
linearly independent.
d 0k = 0 , then x is a KKT point of (1). If d 0k ≠ 0 , then
k
x k computed in step 4 is a feasible direction with descent
of (1) at x k .
Proof.
By the KKT conditions of QP sub-problem (8), we
have
∇f ( x k ) + H k d 0k + Ak b k = 0,
p kj + ∇g j ( x k )T d 0k = 0, j ∈ J k ,
Lemma 2.1 Suppose that H2.1and H2.2 hold, then
1) For any iteration, there is no infinite cycle in step 1.
2) If a sequence {x k } of points has an accumulation
_
point, then there exists a constant ε > 0 such that
_
ε k ,ik > ε for k large enough.
Proof.
1) Suppose that the desired conclusion is false, that is
to say, there exists some k, such that there is an infinite
cycle in Step 1, then we obtain, ∀i = 1, 2, , that Ak ,i is
not of full rank, i.e., it holds that
det( AkT,i Ak ,i ) = 0, i = 1, 2,
Lemma 2.2 For the QP sub-problem (8) at x k , if
,
(13)
If d 0k = 0 , we obtain
∇f ( x k ) + Ak b k = 0, p kj = 0, j ∈ J k ,
Thereby, from (7) and x k ∈ X , ∀k implies that
g j ( x k ) = 0, v kj ≥ 0, j ∈ J k .
In addition, we have b k = − Bk ∇f ( x k ) = v k , in a word,
we obtain
∇f ( x k ) + Ak b k = 0, g j ( x k ) = 0, b kj ≥ 0, j ∈ J k ,
Let b kj = 0, j ∈ I \ J k , which shows that x k is a KKT
point of (1).
If d 0k ≠ 0 , we have
g J k ( x k )T d k = AkT d k = − p k − δ k ek ,
And by (6), we can know that J k ,i +1 ⊆ J k ,i . Since there
are only finitely many choices for J k ,i ⊆ I , it is sure that
© 2013 ACADEMY PUBLISHER
∇f ( x ) d 0 = − ( d 0 ) H k d 0 + b p ,
k
So,
T
k
k
T
k
kT
k
JOURNAL OF COMPUTERS, VOL. 8, NO. 6, JUNE 2013
1499
∇f ( x k )T d k = ∇f ( x k )T d 0k − δ k ∇f ( x k )T Ak ( AkT Ak ) −1 ek
prove that d 0* = 0 . Suppose by contradiction that d 0* ≠ 0 .
1
1
≤ − (d 0k )T H k d 0k + b kT p k ≤ − (d 0k )T H k d 0k < 0.
2
2
Thereby, we know that d k is a feasible descent
direction of (1) at x k .
Then, from Lemma 2.2, it is obvious that d * is welldefined, and it holds that
III. GLOBAL CONVERGENCE OF ALGORITHM
∇f ( x* )T d * < 0, ∇g j ( x* )T d * < 0, j ∈ I ( x* ) ⊆ J
Thus, from (16), it is easy to see that the step-size tk
obtained in step 5 are bounded away from zero on
K , i .e .
In this section, firstly, it is shown that Algorithm A
given in section 2 is well-defined, that is to say, for every
k, that the line search at Step 5 is always successful
Lemma 3.1 The line search in step 5 yields a stepsize
1
tk = ( )i for some finite i = i (k ) .
2
Proof.
It is a well-known result according to Lemma 2.2. For
(11),
s f ( x k + td k + t 2 d k ) − f ( x k ) − α t∇f ( x k )T d k
= ∇f ( x k )T (td k + t 2 d k ) + o(t ) − α t∇f ( x k )T d k
= (1 − α )t∇f ( x k )T d k + o(t ).
For (12), if
tk ≥ t* = inf{tk , k ∈ K } > 0, k ∈ K .
In addition, from (11) and Lemma 2.2, it is obvious
that { f ( x k )} is monotonous decreasing. So, according to
assumption
H 2.1, the fact that {x k }K → x* implies that
f ( x k ) → f ( x* ), k → ∞.
(18)
So, from (11), (16), (17), it holds that
0 = lim( f ( x k ) − f ( x* )) ≤ lim(α tk ∇f ( x k )T d k )
x∈K
x∈K
1
≤ α t*∇f ( x* )T d * < 0,
2
which is a contradiction thus lim d 0k = 0 . Thus, x* is a
KKT point of (1).
j ∉ I (x ), g j ( x ) < 0;
k
j ∈ I ( xk ), g j ( xk ) = 0, ∇g j ( xk )T d k < 0,
IV. THE RATE OF CONVERGENCE
so we have
g j ( x k + td k + t 2 d k ) = ∇f ( x k )T (td k + t 2 d k ) + o(t )
= α t∇g j ( x k )T d k + O(t ).
In the sequel, the global convergence of Algorithm A
is shown. For this reason, we make the following
additional assumption.
H3.1 {x k } is bounded, which is the sequence
generated by the algorithm, and there exist constants
b ≥ a > 0 , such that a || y ||2 ≤ yT H k y ≤ b || y ||2 , for all k
and all y ∈ R n .
Since there are only finitely many choices for sets
J k ⊆ I , and the sequence {d 0k , d1k , d k , v k , b k } is bounded,
we can assume without loss of generality that there exists
a subsequence K, such that
x → x , H k → H* , d → d , d → d , d → d ,
*
(17)
x →∞
k
k
(16)
k
0
*
0
k
b k → b* , v k → v * , J k ≡ J ≠ Φ , k ∈ K ,
*
k
*
(15)
where J is a constant set.
Now we discuss the convergent rate of the algorithm,
and prove that the sequence {x k } generated by the
algorithm is one-step super-linearly convergent under
some mild conditions without the strict complementarily.
For this purpose, we add some regularity hypothesis.
H 4.1 The sequence {x k } generated by Algorithm A is
bounded, and possess an accumulation point x* , such
that the KKT pair ( x* , u* ) satisfies the strong secondorder sufficiency conditions, i.e.,
d T ∇ 2xx L( x* , u * )d > 0,
∀d ∈ Ω {d ∈ R n : d ≠ 0, ∇g I + ( x* )T d = 0},
L( x, u ) = f ( x ) + ∑ u j g j ( x), I + = { j ∈ I : u *j > 0}.
j∈ I
Lemma 4.1 Suppose that assumptions H 2.1-H 3.1 hold,
then,
1) There exists a constant ζ > 0 , such that
|| ( AkT Ak ) −1 ||≤ ζ ;
2)
lim d 0k = 0; lim d k = 0; lim d k = 0;
k →∞
k →∞
k →∞
Theorem 3.2 The algorithm either stops at the KKT
point x k of the problem (1) in finite number of steps, or
generates an infinite sequence {x k } any accumulation
3)
point x* of which is a KKT point of the problem (1).
Proof.
The first statement is easy to show, since the only
stopping point is in step 3. Thus, assume that the
algorithm generates an infinite sequence {x k } , and (15)
holds. According to Lemma 2.2, it is only necessary to
Proof.
1) By contradiction, suppose that sequence
{|| ( AkT Ak ) −1 ||} is unbounded, then there exists an infinite
subset K, such that
|| ( AkT Ak ) −1 ||→ ∞, (k ∈ K ).
© 2013 ACADEMY PUBLISHER
|| d k ||∼|| d 0k ||, || d k ||= O(|| d k ||2 ),
||d k − d 0k ||=O (|| d 0k ||3 ), || d k ||= O(|| d k ||2 ).
.
1500
JOURNAL OF COMPUTERS, VOL. 8, NO. 6, JUNE 2013
In view of the boundedness of {x k } and J k being a
subset of the finite set I = {1, 2, , m} as well as
Lemma 2.1, we know that there exists an infinite index
set K ' ⊆ K such that
x k → x, J k ≡ J ' , ∀k ∈ K ' , det( AkT Ak ) ≥ ε , ε k ≥ ε .
As a result,
lim(
AkT Ak ) = ∇g J ' ( x)T ∇g J ' ( x),
'
k ∈K
−1
continuously differentiable, for k
Hence, we obtain || ( A Ak ) ||→|| ∇g J ' ( x) ∇g J ' ( x) ||,
T
this contradict || ( AkT Ak ) −1 ||→ ∞, (k ∈ K ). So the first
conclusion 1) follows.
2) We firstly show that lim d 0k = 0 .
k →∞
We suppose by contradiction that lim d 0k ≠ 0,
enough, if v < 0 , we have
p j ( x k ) + ∇g j0 ( x* )T d 0k = −v kj0 + ∇g j0 ( x* )T d 0k
0
1
≥ − v kj0 > 0.
2
Otherwise,
p j ( x k ) + ∇g j0 ( x* )T d 0k
k →∞
then
1
= g j0 ( x k ) + ∇g j0 ( x* )T d 0k ≤ − β < 0, (v kj0 ≥ 0)
2
which is contradictory with (8) and the fact j0 ∈ J k . So,
J k ≡ I* (for k large enough).
Prove that b k → uI* = (u *j , j ∈ I* ), v k → (u *j , j ∈ I* ) .
For the v k → (u *j , j ∈ I* ) statement, we have the
there exist an infinite index set K and a constant σ > 0
such that || d 0k ||> σ holds for all k ∈ K . Taking notice
following results from the definition of v k ,
v k → − − B*∇f ( x* ) = −( A*T A* ) −1 A*T ∇f ( x* )
of the boundedness of {x k } , by taking a subsequence if
necessary, we may suppose that
x k → x, J k ≡ J ' , ∀k ∈ K .
Using Taylor expansion, we analyze the first search
inequality of Step 5, combining the proof of Theorem 3.2,
the fact that x k → x* , (k → ∞) implies that it is true.
i.e.
The proof of lim d = 0; lim d = 0 are elementary
k
k
k →∞
k →∞
from the result of 1) as well as formulas (9) and (10).
3) The proof of 3) is elementary from the formulas (9),
(10) and assumption H2.1.
Lemma
4.2.
Let H2.1 to H4.1 holds,
lim || x k +1 − x k ||= 0 . Thereby, the entire sequence {x k }
k →∞
converges to x* i.e. x k → x* , k → ∞ .
Proof.
From the Lemma 4.1, it is easy to see that
lim || x k +1 − x k ||= lim(|| tk d k + tk2 d k ||)
k →∞
k →∞
≤ lim(|| d k || + || d k ||) = 0
In addition, since x* is a KKT point of (1), it is
evident that
∇f ( x* ) + A*uI* = 0, uI* = − B*∇f ( x* )
.
Moreover, together with Theorem 1.1.5 in [4], it shows
that x k → x* , k → ∞
Lemma 4.3 It holds, for k large enough, that
1) J k ≡ I ( x* ) I* , b k → uI* = (u*j , j ∈ I* ), v k → (u *j , j ∈ I* )
2) I + ⊆ Lk = { j ∈ J k : g j ( x k ) + ∇g j ( x k )T d 0k = 0} ⊆ J k .
Proof.
1) Prove J k ≡ I* .
On one hand, from Lemma 2.1, we know, for k large
enough, that I* ⊆ J k . On the other hand, if it doesn’t
hold that J k ⊆ I* , then there exist constants j0 and
β > 0 , such that
g j0 ( x* ) ≤ − β < 0, j0 ∈ J k .
So, according to d 0k → 0 and the functions g j ( x),
−1
uI* = −( A A* ) A ∇f ( x ).
T
*
T
*
*
Otherwise, from (8), the fact that d 0k → 0 implies that
∇f ( x k ) + H k d 0k + Ak b k = 0, b k → − B*∇f ( x* ) = u I* .
The claim holds.
2) For lim( x k , d 0k ) = ( x* , 0) , we have Lk ⊆ I ( x* ) .
k →∞
Furthermore, it has lim u Ik+ = uI*+ > 0 , so the proof is
x →∞
finished.
In order to obtain super-linear convergence, a crucial
requirement is that a unit step size is used in a
neighborhood of the solution. This can be achieved if the
following assumption is satisfied.
H4.2 Let || (∇ 2xx L( x k , u Jkk ) − H k )d k ||= o(|| d k ||) , where
L( x, u Jkk ) = f ( x) +
k →∞
© 2013 ACADEMY PUBLISHER
large
k
j0
0
det(∇g J ' ( x)T ∇g J ' ( x)) ≥ ε > 0.
T
k
( j ∈ I ) are
∑u
j∈ J k
k
Jk
g j ( x) .
Lemma 4.4 Suppose that Assumption H 2.1 to H 4.2
are all satisfied. Then, the step size in Algorithm A
always one, i.e. tk ≡ 1 , if k is large enough.
Proof.
It is only necessary to prove that
f ( x k + d k + d k ) ≤ f ( x k ) + α∇f ( x k )T d k ,
g j ( x k + d k + d k ) ≤ 0, j ∈ I .
For
(12)
k
if
j ∈ I \ I*
we
have
(19)
(20)
g j ( x* ) < 0
,
( x k , d k , d ) → ( x* , 0, 0)(k → ∞), then, it is easy to
obtain g j ( x k + d k + d k ) ≤ 0 holds.
If j ∈ I* we have
JOURNAL OF COMPUTERS, VOL. 8, NO. 6, JUNE 2013
g j (xk + dk + dk ) = g j (xk + dk ) +∇g j (xk + dk )T d k + O(|| d k ||2 )
=g j (xk + dk ) +∇g j (xk )T dk + O(|| dk |||| d k ||) + O(|| d k ||2 )
1501
− ∑ u kj ∇g j ( x k ) (d k + d )
(21)
j∈Lk
=
=g j (x + d ) +∇g j (x ) d + O(|| d |||| d ||).
k
k
k T
k
k
k
∑ u ∇g
j∈Lk
In addition, from (9) and (10),
∇g j ( x k )T d k = ∇g j ( x k )T d 0k − δ k ,
k
(xk )
(26)
⎞ k
k 2
⎟⎟ d + o(|| d || ),
⎠
k
∇f ( xk )T (d k + d )
so, for τ ∈ (2,3) we have
= −(d k )T Hk d k − ∑ ukj ∇g j ( xk )
g j ( xk + d k + d k )
(27)
j∈Lk
=− || d0k ||τ + g j ( xk ) + ∇g j ( x k )T d0k − δ k + O(|| d k |||| d k ||)
≤ − || d || +O(|| d |||| d ||) ≤ 0.
Hence, the second inequalities of (20) hold for t = 1
and k is sufficiently large.
The next objective is to show the first inequality of (19)
holds. From Taylor expansion and taking into account
Lemma 4.1 and Lemma 4.3, we have
s
T
j
From (25) and (26), we have
+ g j ( x k ) + ∇g j ( x k )T d k ,
τ
k
j
⎛
T
1
+ (d k )T ⎜⎜ ∑ u kj ∇ 2 g j ( x k )
2
⎝ j∈Lk
∇g j ( x k )T d = − || d 0k ||τ − g j ( x k + d k )
k
0
k
T
k
⎛
T ⎞
1
+ (d k )T ⎜⎜ ∑ ukj ∇2 g j ( xk ) ⎟⎟ d k + o(|| d k ||2 )
2
⎝ j∈Lk
⎠
k
(22)
k
k
2
k
j
T
k
1
+ (d k )T ∇ 2 L( x k , u Jkk ) − H k d k + o(|| d k ||2 ).
2
Then, together assumption H 3.1 and H 4.2 as well as
u kj g j ( x k ) ≤ 0, shows that
(
∇f ( x k ) = − H k d 0k − ∑ u kj ∇g j ( x k )
= − H k d k − ∑ u kj ∇g j ( x k ) + o(|| d k ||2 ),
2
j ∈ Lk
On the other hand, from the KKT condition of (8) and
the active set Lk defined by Lemma 4.3 one has
j∈Lk
1
⎛
⎞
+ (d k )T ⎜ ∇ f ( x ) + ∑ u ∇ g ( x ) − H ⎟ d k + o(|| d k ||2 )
2
⎝
⎠
1 k T
=(α − )(d ) H k d k + (1 − α ) ∑ u kj g j ( x k )
2
j∈Lk
j
f ( x k + d k + d k ) − f ( x k ) + α ∇ f ( x k )T d k
1
=∇f ( x k )T (d k + d k ) + (d k )T ∇ 2 f ( x k )T d k
2
− α∇f ( x k )T d k + o(|| d k ||2 ).
Substituting (27) and (24) into (22), it holds that
1
s (α − )(d k )T H k d k + (1 − α ) ∑ u kj g j ( x k )
2
j∈Lk
(23)
)
1
1
s ≤ (α − )a || d k ||2 +o(|| d k ||2 ) ≤ 0. (α ∈ (0, )).
2
2
Hence, the inequality of (19) holds.
j∈Lk
Furthermore, in a way similar to the proof of Theorem
5.2 in [5] and in [19, Theorem 2.3], we may obtain the
following theorem:
So, from (23) and Lemma 4.3, we have
∇f ( x k )T d k
= −(d k )T H k d k − ∑ u kj ∇g j ( x k ) d k + o(|| d k ||2 )
T
(24)
j∈Lk
= − (d k )T H k d k − ∑ u kj ∇g j ( x k ) d 0k + o(|| d k ||2 )
T
Theorem 4.5 Under all above-mentioned assumptions,
the algorithm is superlinearly convergent, i.e., the
sequence {x k } generated by the algorithm satisfies that
j∈Lk
|| x k +1 − x* ||= o(|| x k − x* ||).
k
∇f ( xk )T (d k + d )
= −(d k )T Hk d k − ∑ ukj ∇g j ( xk ) (d k + d ) + o(|| d k ||2 ),
T
k
(25)
j∈Lk
Again, from (21) and Taylor expansion, it is clear that
Proof.
From Lemma 4.1 and Lemma 4.4, we can know that
the sequence {x k } yielded by Algorithm A has the form
of
x k +1 = x k + d k + d
k
k
= x k + d 0k + (d k + d − d 0k )
o(|| d || ) = g j (x + d + d )
2
k
k
k
k
1
=g j (x ) +∇g j (x ) (d + d ) + (d k )T ∇2 g j (xk )T d k + o(|| d k ||2 )
2
where j ∈ Lk , then, we obtain
k
k T
k
k
x k + d 0k + d .
k
k
k
where d = (d k + d − d 0k ) (for k large enough) and
k
|| d ||= O(|| d 0k ||3 ) . Consequently, we can obtain the
result together with Ref. [5] and [19].
V. NUMERICAL RESULTS
© 2013 ACADEMY PUBLISHER
1502
JOURNAL OF COMPUTERS, VOL. 8, NO. 6, JUNE 2013
In this section, we carry out numerical experiments
based on the Algorithm A. The code of the proposed
algorithm is written by using MATLAB 7.0 and utilized
the optimization toolbox. The results show that the
algorithm is effective. During the numerical experiments,
it is chosen at random some parameters as follows:
ε 0 = 0.5, α = 0.25, τ = 2.25, H 0 = I ,
where I is the n × n unit matrix. H k is updated by the
BFGS formula [2].
H k +1 = BFGS ( H k , s k , y k ),
Where,
^
CPU: the total time taken by the process (unit:
millisecond) ;
FV: the final value of the objective function.
TABLE III.
THE APPROXIMATE VALUE OF THE DIRECTION
d 0k
FOR TABLE I
NO.
n, m
|| d 0k ||
HS12
HS43
HS66
HS100
HS113
2, 1
4, 3
3, 8
7, 4
10, 8
7.329773437334 E-09
5.473511535838 E-09
8.327832675386 E-09
8.595133692328 E-09
6.056765632745 E-09
s k = x k +1 − x k , y k = θ y k + (1 − θ ) H k s k ,
^
m
y k = ∇f ( x k +1 ) − ∇f ( x k ) + ∑ u kj (∇g j ( x k +1 ) − ∇g j ( x k )),
j =1
kT
⎧1,
if y s k ≥ 0.2( s k )T H k s k ,
⎪⎪
θ = ⎨ 0.8( s k )T H k s k
, otherwise.
⎪ k T
kT
k
k
⎪⎩ ( s ) H k s − y s
In the implementation, the stopping criterion of Step 2
is changed to “If || d 0k ||≤ 10−8 , STOP.”
TABLE I.
THE DETAIL INFORMATION OF NUMERICAL EXPERIMENTS
NO.
HS12
HS43
HS66
HS100
HS113
n, m
2, 1
4, 3
3, 8
7, 4
10, 8
NT
10
17
14
18
45
CPU
0
10
0
62
50
TABLE II.
THE APPROXIMATE OPTIMAL SOLUTION
NO.
HS12
HS43
HS66
HS100
HS113
x* FOR TABLE I
*
the approximate optimal solution x
(1.999999999995731, 3.000000000011285) T
(0.000000000000000, 1.000000000000000,
2.000000000000000, −1.000000000000000) T
(0.184126482757009, 1.202167866986839,
3.327322301935746) T
(2.330499372903103, 1.951372372923884,
-0.477541392886392, 4.365726233574537,
-0.624486970384889, 1.038131018506466,
1.594226711671913) T
(2.171996371254668, 2.363682973701174,
8.773925738481299, 5.095984487967813,
0.990654764957730, 1.430573978920189,
1.321644208159091, 9.828725807883636,
8.280091670090108, 8.375926663907775) T
This algorithm has been tested on some problems from
Ref.[20], a feasible initial point is either provided or
obtained easily for each problem. The results are
summarized in Table 1 to Tabe 4. The columns of this
table have the following meanings:
No.: the number of the test problem in [20];
n: the number of variables;
m: the number of inequality constraints;
NT: the number of iterations;
© 2013 ACADEMY PUBLISHER
TABLE IV.
THE FINAL VALUE OF THE OBJECTIVE FUNCTION FOR TABLE I
NO.
HS12
HS43
HS66
HS100
HS113
FV
-29.999999999999705
-44.000000000000000
0.518163274181542
6.806300573744022e+002
24.306209068179822
VI. CONCLUSION
This paper proposed an improved feasible sequential
quadratic programming (FSQP) method for nonlinear
programs. As compared with the existing SQP methods
which required solving the QP sub-problem with
inequality constraints in single iteration, in order to obtain
the feasible direction, the method of this paper is only
necessary to solve an equality constrained quadratic
programming sub-problems. Combined the generalized
projection technique, a height-order correction direction
is yielded by explicit formulas, which can avoids Maratos
effect. Furthermore, under some mild assumptions, the
algorithm is globally convergent and its rate of
convergence is one-step super-linearly. Numerical results
reported show that the algorithm in this paper is effective.
ACKNOWLEDGMENT
The author would like to thank the editors, whose
constructive comments led to a considerable revision of
the original paper.
REFERENCES
[1] S. P. Han. “Superlinearly Convergent Variable Metric
Algorithm for General Nonlinear Programming Problems”,
Mathematical Programming, Berlin, vol.11 (1), pp. 263–
282, December 1976.
[2] M. J. D. Powell. “A Fast Algorithm for Nonlinearly
Constrained Optimization Calculations”, In: Waston, G.A.
(ed). Numerical Analysis. Springer, Berlin, pp. 144–157,
1978.
[3] P. T. Boggs, J. W. Tolle. “A Strategy for Global
Convergence in a Sequential Quadratic Programming
Algorithm”, SIAM J. Num. Anal., Philadelphia, vol. 26 (1),
pp. 600–623, June 1989.
[4] E. R. Panier, A. L. Tits. “On Combining Feasibility,
Descent and Superlinear Convergence in Inequality
JOURNAL OF COMPUTERS, VOL. 8, NO. 6, JUNE 2013
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
Constrained Optimization”, Mathematical Programming,
Berlin, vol. 59, pp. 261–276, 1993.
J.F. Binnans, G. Launay. “Sequential quadratic
programming with penalization the displacement”, SIAM J.
Optimization, Philadelphia, vol. 5 (4), pp. 796–812, 1995.
P. Spellucci. “An SQP Method for General Nonlinear
Programs Using Only Equality Constrained Subproblems”,
Mathematical Programming, Berlin, vol. 82 (3), pp. 413–
448, August 1998.
C. T. Lawarence, and A.L.Tits. “A Computationally
Efficient Feasible Sequential Quadratic Programming
Algorithm”, SIAM J.Optim., Philadelphia, vol. 11, pp.
1092–1118, 2001.
L. Qi, Y. F. Yang. “Globally and Superlinearly Convergent
QP-free
Algorithm
for
Nonlinear
Constrained
Optimization”, Journal of Optimization Theory and
Applications, Berlin, vol. 113 (2), pp. 297–323, May 2002.
(7)
J.B.Jian, C.M.Tang. “An SQP Feasible Descent Algorithm
for Nonlinear Inequality Constrained Optimization without
Strict Complementarity”, An International Journal
Computers and Mathematics with application, vol. 49, pp.
223–238, 2005.
Z.J. Luo, Z.B. Zhu, L.R Wang, and A.J Ren. “An SQP
algorithm with equality constrained sub-problems for
general constrained optimization”, International Journal of
Pure and Applied Mathematics, vol. 39 (1): pp. 125-138,
2007.
Richard H. Byrd, Frank E. Curtis, and Jorge Nocedal.
“Infeasibility Detection and SQP Methods for Nonlinear
Optimization”, SIAM J. Optim. Vol. 20 (5), pp. 2281-2299,
2010.
A. F. Izmailov and M. V. Solodov. “A Truncated SQP
Method Based on Inexact Interior-Point Solutions of Sub-
© 2013 ACADEMY PUBLISHER
1503
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
problems”, SIAM J. Optim. Vol. 20 (5), pp. 2584-2613,
2010.
C.G. Shen, W.J. Xue and X.D. Chen. “Global convergence
of a robust filter SQP algorithm”, European Journal of
Operational Research, vol. 206 (1), pp. 34–45, 2010.
Z. B. Zhu, W.D. Zhang, and Z.J. Geng. “A feasible SQP
method for nonlinear programming”, Applied Mathematics
and Computation, vol. 215, pp. 3956–3969, 2010.
Z. J. Luo, G. H. Chen, and L. R. Wang, "A Modified
Sequence Quadratic Programming Method for Nonlinear
Programming, " cso, pp.458-461, 2011 Fourth
International Joint Conference on Computational Sciences
and Optimization, 2011.
C. Gu, D.T. Zhu. “A non-monotone line search
multidimensional filter-SQP method for general nonlinear
programming”, Numerical Algorithms, vol. 56 (4),
pp.537–559, 2011.
C.L. Hao, X.W. Liu. “A trust-region filter-SQP method for
mathematical programs with linear complementarity
constraints”, Journal of Industrial and Management
Optimization, vol. 7 (4), pp. 1041–1055, 2011.
C.L. Hao, X.W. Liu. “Global convergence of an SQP
algorithm for nonlinear optimization with over determined
constraints”, Numerical algebra control and optimization,
vol. 2 (1), pp. 19–29, 2012.
J.B. Jian. “Two extension models of SQP and SSLE
algorithms for optimization and their superlinear and
quadratical convergence”, Applied Mathematics--A
Journal of Chinese Universities, Set. A, vol. 16 (4),
pp.435-444, 2001.
W. Hock, K. Schittkowski, Test examples for nonlinear
programming codes, Lecture Notes in Economics and
Mathematical Systems, vol. 187, Springer, Berlin, 1981.