On Minimizing the Sum of a
Convex Function and a Concave
Function
Polyakova, L.N.
IIASA Collaborative Paper
June 1984
Polyakova, L.N. (1984) On Minimizing the Sum of a Convex Function and a Concave Function. IIASA
Collaborative Paper. IIASA, Laxenburg, Austria, CP-84-028 Copyright © June 1984 by the author(s).
http://pure.iiasa.ac.at/2549/ All rights reserved. Permission to make digital or hard copies of all or part of this
work for personal or classroom use is granted without fee provided that copies are not made or distributed for
profit or commercial advantage. All copies must bear this notice and the full citation on the first page. For other
purposes, to republish, to post on servers or to redistribute to lists, permission must be sought by contacting
[email protected]
NOT FOR QUOTATION
WITHOUT PERMISSION
OF THE AUTHOR
ON M I N I M I Z I N G THE SUM OF A CONVEX FUNCTION
AND A CONCAVE FUNCTION
L,N,
POLYAKOVA
June 1984
CP-84-28
CoZZaborative Papers r e p o r t work which h a s n o t been
performed s o l e l y a t t h e I n t e r n a t i o n a l I n s t i t u t e f o r
A p p l i e d Systems A n a l y s i s and which
l i m i t e d review,
Views or opinions
do n o t n e c e s s a r i l y r e p r e s e n t t h o s e
its- N a t i o n a l Member O r g a n i z a t i o n s ,
z a t i o n s s u p p o r t i n g t h e work,
has received only
expressed herein
of the I n s t i t u t e ,
o r o t h e r organi-
INTERNATIONAL INSTITUTE FOR APPLIED SYSTEMS ANALYSIS
A-2361 L a x e n b u r g , A u s t r i a
PREFACE
I n t h i s p a p e r , t h e a u t h o r p r e s e n t s an a l g o r i t h m f o r minimizing t h e sum of a convex f u n c t i o n and a concave f u n c t i o n .
The f u n c t i o n s i n v o l v e d a r e n o t n e c e s s a r i l y smooth and t h e r e s u l t i n g f u n c t i o n i s q u a s i d i f f e r e n t i a b l e . The main p r o p e r t y of
s u c h f u n c t i o n s i s t h e non-uniqueness of d i r e c t i o n s of s t e e p e s t
d e s c e n t (and a s c e n t ) , and t h e r e f o r e s p e c i a l p r e c a u t i o n s must be
taken t o guarantee t h a t t h e algorithm converges t o a s t a t i o n a r y
point.
T h i s .paper i s a c o n t r i b u t i o n t o r e s e a r c h on n o n d i f f e r e n t i a b l e
o p t i m i z a t i o n c u r r e n t l y underway w i t h i n t h e System and D e c i s i o n
S c i e n c e s Program.
ANDRZEJ WIERZBICXI
Chairman
System and D e c i s i o n S c i e n c e s
ON MINIMIZING THE SUM OF A CONVEX FUNCTION
AND A CONCAVE FUNCTION
L.N. POLYAKOVA
Department o f A p p l i e d M a t h e m a t i c s , L e n i n g r a d S t a t e U n i v e r s i t y ,
U n i v e r s i t e t s k a y a nab. 7 / 9 , L e n i n g r a d 199164, USSR
Received 27 December 1 9 8 3
Revised 2 4 Rarch 1 9 8 4
We consider here the problem of minimizing a particular
subclass of quasidifferentiable functions: those which
may be represented as the sum of a convexfunctionanda
concave function. It is shown that in an n-dimensional
space this problem is equivalent to the problem of
minimizing a concave function on a convex set. A
successive approximations cethod is suggested; this
makes use-of'some of the principles of E-steepest-descenttype approaches.
Key w o r d s :
Quasidifferentiable Functions, Convex Functions,
Concave Functions, E-Steepest-Descent Methods.
1
.
Introduction
The problem of minimizing nonconvex nondifferentiable func-
tions poses a considerable challenge to specialists in mathematical programming.
Most of the difficulties arise from the
fact that there may be several directions of steepest descent.
To solve this problem requires both a new technique and a new
approach.
In this paper we discuss a special subclass of non-
differentiable functions:
the form
those which can be represented in
where f
1
is a finite function which is convex on En and f2 is a
.
finite function which is concave on En
Then f is continuous
and quasidifferentiable on En , with a quasidifferential at
x E En which may be taken to be the pair of sets
where
In other words, af(x) is the subdifferential of the convex
function f
1
at x E En (as defined in convex analysis) and af (x)
is the superdifferential of the concave function f2 at x E En
.
Consider the problem of calculating
inf f (x)
xEEn
.
Quasidifferential calculus shows that for x
*
E E tobe a minimum
n
point of f on En it is necessary that
We shall now show that the problem of minimizing f on the space
En can be reduced to that of minimizing a concave function on
a convex set.
-
L e t R d e n o t e t h e e p i g r a p h o f t h e convex f u n c t i o n f l
R
= e p i f = { z = [ x , ~ ]E En x E l h ( z )
and d e f i n e t h e f o l l o w i n g f u n c t i o n on En
x El
-
f l (XI
,
i.e.,
11 4 01
t
:
S e t R i s c l o s e d and convex and f u n c t i o n $ i s q u a s i d i f f e r e n t i a b l e a t any p o i n t z E E n
x
El
.
Take a s i t s q u a s i d i f f e r e n t i a l
a t z = [ x , p ] t h e p a i r o f s e t s D $ ( z ) = [{O}
where 0 E En+l
,
af2(x)
x
ill]
,
.
L e t u s now c o n s i d e r t h e problem of f i n d i n g
I t i s well-known
(see, e . g . ,
[ 31) t h a t i f a concave f u n c t i o n
a c h i e v e s i t s i n f i m a l v a l u e on a convex s e t , t h i s v a l u e i s
a c h i e v e d on t h e boundary o f t h e s e t .
Theorem 1 .
For a p o i n t x * t o b e a s o l u t i o n o f p r o b l e m ( I ) , i t
i s b o t h n e c e s s a r y and s u f f i c i e n t t h a t p o i n t
t i o n t o problem ( 3 ) ,
where P* = f (x*)
[ x * , p * ] be a s o l u -
.
Proof
Necessity.
11 + f
2
L e t x* be a s o l u t i o n o f problem ( 1 )
(x) ? f l ( x )
.
Then
+ f 2 ( x ) > f l ( x * ) + f 2 1 x * ) V p 2 f l ( x ) , VxGEn=
(4)
But ( 4 ) i m p l i e s t h a t
where p* = f l ( x * )
.
Thus t h e r e e x i s t s a z * = [ x * , p * ] E R
such t h a t
$(z) 1
)I
(z*)
Vz E R
.
This proves t h a t t h e c o n d i t i o n is necessary.
Sufficiency.
That t h e c o n d i t i o n i s a l s o s u f f i c i e n t c a n b e proved
i n a n a n a l o g o u s way by a r g u i n g Backwards from i n e q u a l i t y ( 5 ) .
2.
A numerical algorithm
Set
E
2 0
.
A point xo E E
n
p o i n t of t h e f u n c t i o n f on En i f
i s c a l l e d an E - i n f - s t a t i o n a r y
where
i.e.
, 'aE f ( x o )
f l a t xO
.
i s t h e E - s u b d i f f e r e n t i a l o f t h e convex f u n c t i o n
F i x g E En and s e t
Theorem 2.
For a p o i n t xo t o be an E - i n f - s t a t i o n a r y p o i n t o f
t h e f u n c t i o n f on E~
, it
i s b o t h n e c e s s a r y and s u f f i c i e n t t h a t
a,£ (x,)
min
Il g ll = 1
Necessity.
a4
2 0 .
Let xo be an E-inf-stationary point of f on En
.
Then from (6) it follows that
Eience
min
max
I I ~ I I =z&+a,f(~~)
I -
(z,~)
and thus for every g E En, lgll=l
Vw
sf(xol
,
we have
min
w a f (xo)
However, this means that
proving that the condition is necessary.
That it is also suf-
ficient can be demonstrated in an analogous way, arguing backwards from the inequality ( 9 )
-
-6-
Note that since the mapping
is Hausdorff-continuous if
> 0 (see, e.g., [I]), then the
E
following theorem holds.
If
Theorem 3.
E
> 0 then the function
max
(v,g) i s convaaEf (x)
En
.
t i n u o u s i n x on E
n f o r any f i x e d g E
Assume that xo is not an E-in£-stationary point.
Then we
can describe the vector
gE (x0)= arg
min
Il g ll = 1
aEf(xO)
a9
as a direction of E-steepest-descent of function f at point
Xo
It is not difficult to show that the direction
where vOE
E
CEf(xO)
-
WO E
gf(x0) and
max
min
Ilv+wl
WET£ (x") fiaEf(x,)
=
-llv
OE
+ w l =aE(xo)
,
o
is a direction of E-steepest-descent of function f at point x
0
Now let us consider the following method of successive ap-
proximations.
-
Fix
xO E En
> 0 a n d c h o o s e an a r b i t r a r y i n i t i a l a p p r o x i m a t i o n
E
.
Suppose t h a t t h e Lebesque s e t
i s bounded.
-
-af ( x k )
If
Assume t h a t a p o i n t x
C
, then
a_,£ ( x k )
E E~
h a s a l r e a d y been found.
xk i s a n E - i n f - s t a t i o n a r y
point
of f on E~ ; i f n o t , t a k e
X
k+ 1 =
Xk
where g E k = g , ( x k )
Xk
+ akgEk
a k = a r g min f (xk+ag
&k
a20
,
i s an E-steepest-descent d i r e c t i o n of f a t
'
Theorem 4 .
The f o l l o w i n g r e l a t i o n h o l d s :
l i m a, (xk) = 0
kProof.
.
W e s h a l l p r o v e t h e t h e o r e m by c o n t r a d i c t i o n .
Assume
t h a t a s u b s e q u e n c e i x k 1 of s e q u e n c e { x k } a n d a number a>O e x i s t
S
such t h a t
(The r e q u i r e d s u b s e q u e n c e must e x i s t s i n c e D ( x o ) i s compact.)
W i t h o u t l o s s o f g e n e r a l i t y , w e c a n assume t h a t x
(clearly, x
I:
E D
( x O ))
.
Then
-x
ks
*
where
The t e r m o ( a t g E k ) a p p e a r s i n t h e above e q u a t i o n due t o t h e conc a v i t y of f 2
.
s
The f a c t t h a t f u n c t i o n f 2 i s concave i m p l i e s t h a t
o ( a t g E k s) 5 0
' d a > O t
WgEk
E En
t
S
and t h e r e f o r e
f
(xk
s
+ agEk
(xk 1 +
)
s
S
la
max
(vtgEk )dr
O v E a f l (xk +Tg,k )
s
S
+
a
min
6af2(xk
S
(wt g
EkS
S
Since
a Ef 1 (x)
3 af
1
(x) f o r every x
max
eaE
(xkf+TgEk
l
)
S
and t h u s
S
( v t g ~ k2s
E En
,
max
we have
t
+
f ( x k +ag
s
EkS
s f(xk
s
)
+la
O
max
vEaEf
1 (xk +rgEk )
S
+
a
S i n c e t h e mapping a E f l i s Hausdorff-continuous
>
>
0 such t h a t
and hence
Therefore
a t the point x
*
0 such t h a t
where S r ( z ) = {x E E n ] llx-zll 6 r }
K
S
min
~ ( Xa 1f(w,gEks
ks
~
there exists a 6
) d +~
.
Also, t h e r e e x i s t s a number
,
Inequality (10) contradicts the fact that sequence {f(xk)} is
bounded, thus proving the theorem.
References
[I] E.A. Nurminski, "On the continuity of &-subgradient mappings", C y b e r n e t i c s 5 (1977) 148-1 49.
[2] L.N. Polyakova, "Necessary condtitions for an extremum of
a quasidifferentiable function", V e s t n i k L e n i n g r a d s k o g o
U n i v e r s i t e t a 13 ( 1 980) 57-62 (translated in V e s t n i k Leningrad
U n i v . Math. 13 (1981) 241-247).
[3] R. T. Rockafellar , Convex Ana Z y s i s (Princeton University
Press, Princeton, New Jersey, 1970).