CurncmZwn in Operations Researah and Systems Analysis~ University of
North Carolina at Chapel HiZl~ 27514. This work was supported in part by the
Office of Naval Researah under grant nwnber NOOO-14-67-A-0321-0003
(NR047-096) •
New
CoNDITIONS FOR Ex..I\CTNESS OF A
SIMPLE PENALlY FUNCTION
by
Stephen Howe
Department of Statistics
University of North Carolina at ChapeZ Hi'Ll
Institute of Statistics Mimeo Series No. 778
October, 1971
.
1
,
j
j
NB-I CoNDITIONS FOR ExACTNESS OF A SIMPLE PENALTY FuNCTION.:l~
by
Stephen Howe
ABsTRACT
We consider penalty function methods for finding the maximum of a function
f
over the set
New conditions, extending earlier work done by Pietrzykowski, are presented
under which the penalty function
is locally exact.
The relationships among the new conditions, Pietrzykowski's
conditions, and Kuhn-Tucker constraint qualifications are explored.
i~EH CONDITIONS FOR ExACTI\~ESS OF A SU1PLE PENALTY FUNCTION
Stephen HO\'le*
1. IfITRODUCTION.
In 1969 Pietrzykowski [1], [2] presented conditions under
which a local maximum
S
X
o
of the function
f
on the set
o =
is also an unconstrained isolated (strong) local maximum, for
~
sufficiently
small, of the penalty function
p(x,~)
where
=
~f(x)
-
I
U(gi(X»
r
-
i=l
Ihj(X)
I
j=l
U: R + R is defined by
U(n)
and where
=
{:
f, gl' ••• , gm' hI' ••• , h
valued functions on
n
R •
p
if
a. >
0
if
a.
~
0
are continuously differentiable real-
Specifically, Pietrzykowski's condition is that
be an isolated local maximum of
f
on
So
X
o
and that the set of "active
gradients"
be linearly independent, where
I = {i: gi(x ) =
O
a}.
It is worth noting that
the linear independence of the gradients, as specified, constitutes a constraint qualification which validates the Kuhn-Tucker necessary optimality
*.
CupricuZum in Operations Research and Systems Analysis, University of
North Carolina at Chapel Hitt, 27514. This work was supported in part by the
Office of Naval Research under grant number NOOO-14-67-A-0321-0003
(NR047-096J.
2
criterion at
This suggests that any constraint qualification which vali-
dates the Kuhn-Tucker criterion might also guarantee the above result at an
isolated local maximum of
f
on
SO'
In this note a new condition for the above result is presented.
The new
condition, unlike that of Pietrzykowski, depends upon the objective function
f, and furthermore it neither implies nor is implied by Pietrzykowski's condition.
It is also shown that the weak constraint qualification [3] does not
guarantee the result at an isolated local maximum of
f
on
SO'
and hence
that the Kuhn-Tucker constraint qualifications are not sufficient for this
result in general.
2,
RESULT.
f, gl' ••• , gm' hI' ••• , h p ' SO' x o' I
Let
and
p(x,~)
be defined as above.
THEOREM. Suppose the following condition is satisfied.
G:
For every nonzero
and IJh/xO)
T
y
n
T
such that Vgi(xO) y S 0 each
each j = l, ••• ,p it follows that
yER
=o
iE:!
T
Vf(xo) y < O.
Then there is a
~O >
0 such that if 0 <
strained isolated local maximum at
Remark: It can be shown that
~ S ~O
then
p(x,~)
has an unaon-
x •
O
G is a sufficient, but not necessary,
condition that
be an isolated local maximum of
f
on
Proof: Since gl, ••• ,gm are continuous, there is a 0
gi(x)
radius
<
0
for every
about
XEB(XO'O)
and
i4I.
Here
B(xO'o)
> 0
such that
is the open ball of
Now suppose, to get a contradiction, that there is a
3
{~}
sequence
p(x'~n)
n.
n
~
0
as
n
~
00
does not have an unconstrained isolated local maximum at
Then for each
but for which
Y ~ 0
n
as
pact set
n
YO€R
~
of positive scalars such that
n
n
there is a point
p(x,~) ~
n
n
n
n
X
o
for each
0 < Ilxn-xOII < min{o'~n}
such that
Denote Y .. x -x
p(xO,V ).
n
n
x
and where
0
each
n,
noting that
n ~ 00. Consider the sequence {y /1 Iy I I} of vectors in the comn
n
n
{z€R : I Izi I=l}. This sequence must have a subsequential limit
I IYol I ..
such that
Yn/IIYnll ~ YO
as
n ~
1.
Assume without loss of generality that
00.
Now suppose that for some
r€I,
T
Vgr(x ) YO > O.
O
Then for each
n
p(xn'~n)-p(xO'~n) ~ ~n[f(xn)-f(xO)]-U(gr(xn»·
Note that
Clearly for
n
sufficiently large
whence for such
g (x ) > 0
r n
so that
n
p(x,~)
n
vgi(xO)T yo ~ 0
T
Vhj(X ) YO .. 0
O
it follows that
..
g (x ),
r n
n
implying that for
but the latter converges to
large
U(g (x »
r n
n
< p(xO'~)
n
for each
for
i€I.
which is a contradiction.
Thus it follows that
Similarly, it can be shown that
j " l, ••• ,p,
and hence from the assumption that
But then
G holds
4
+
implying that for
n
p(xn'~n) < p(xO'~n)·
3.
Vf(x )
O
sufficiently large
T
YO < 0
as
f(x ) < f(x )
O
n
n +
m
and hence
This final contradiction establishes the theorem.
o
b<PMPLES.
Example 1. Let f,gl,82: R + R be defined by f(x). x, gl(x) = x,
g2 (x)
= -x.
Both constraints are active at
point and hence the constrained maximum.
that
ygi'(O)
gl'(O) .. 1
S
0
and
new condition
for
g2'(0)
i
= 1,2,
= -1
x " 0,
o
which is the only feasible
Since there is no nonzero
y~R
such
G holds trivially, but the gradients
are not linearly independent.
Therefore, the
G does not imply Pietrzykowski's condition.
Example 2. Let f,g: R + R be defined by f(x) = x 3 , g(x) = x. The
constraint is active at the constrained maximum
and that for
y
a
-1
yg'(O)
= -1
S
0 but
Pietrzykowski's condition does not imply
Example 3. Let f,gl,g2: R2
+
f(x,y)
o = O.
X
yf'(O)" O.
Note that
g'(O)
= 1,
Therefore,
G.
R be defined by
..
y
+ x4
gl(x,y)
g2(x,y)
(see Figure 1 below).
It can be shown that the weak constraint qualification
[3] is satisfied at the isolated local maximum
ever, that for
~
sufficiently small the point
(0,0).
It is not true, how-
(0,0)
is a local maximum of
5
p«x,y),~).
for any
~
>
o.
~
To see this, define
> 0,
noting that
h
Hence the condition that
h: R
+
R by
h(x). p«x,o),~)
has a local minimum at
X
o
x
=0
= ~x4_x6
for any
be an isolated local constrained maxi-
mum at which the weak constraint qualification holds does not guarantee that
the penalty function will be exact.
-2
-1
1
2
-1
g = 0
2
-2
/1
-3
~
f = 0
-4
Figure 1
REFER8'~CES
[1]
[2]
[3]
T. Pietrzykouski..
"The potential method for conditional maxima in the
locally compact metric spaces," Nwne1". Math. 14, 1970, 325-329.
, "An exact potential method for constrained maxima,"
SIAM Journal of Nwneriaal AnalYBiB~ 6, No.2, June 1969.
F.J. Gould and Jon \1. Tolle~
"A necessary and sufficient constraint
qualification for constrained optimization,"
SIAM J. Appl. Math.~ 20, No.2, March 1971.