JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS: Vol. 121, No. 3, pp. 597–612, June 2004 (Ó 2004)
Solving Continuous Min-Max Problems by an
Iterative Entropic Regularization Method1,2
R. L. SHEU3
AND
J. Y. LIN4
Communicated by P. Tseng
Abstract. We propose a method of outer approximations, with each
approximate problem smoothed using entropic regularization, to solve
continuous min-max problems. By using a well-known uniform error
estimate for entropic regularization, convergence of the overall method
is shown while allowing each smoothed problem to be solved inexactly.
In the case of convex objective function and linear constraints, an
interior-point algorithm is proposed to solve the smoothed problem
inexactly. Numerical examples are presented to illustrate the behavior of
the proposed method.
Key Words. Min-max problems, entropic regularization, interior-point
algorithms, semi-infinite programming.
1. Introduction
Consider the following continuous min-max problem (P):
min FðuÞ;
u2W
where W is a compact subset of Rn ,
FðuÞ ¼ maxfft ðuÞg;
t2T
T is a compact metric space, and ft ðuÞ is continuous on T W. Notice that
T may be a finite set, which is a compact metric space in the discrete
1
This research work was partially supported by the National Science Council of Taiwan under
Project NSC 89-2115-M-006-009.
2
The authors thank Professor Paul Tseng and two anonymous referees for valuable comments
and suggestions.
3
Professor, Department of Mathematics, National Cheng-Kung University, Tainan, Taiwan.
4
PhD Candidate, Department of Mathematics, National Cheng-Kung University, Tainan,
Taiwan.
597
0022-3239/04/0600-0597/0 Ó 2004 Plenum Publishing Corporation
598
JOTA: VOL. 121, NO. 3, JUNE 2004
topology. In general, finite min-max problems have been studied more
extensively than continuous min-max problems.
A classical approach to solving problem (P) involves approximating T
by a finite subset Tm of m points in T. Correspondingly, F(u) is
approximated by
F m ðuÞ ¼ maxfft ðuÞg:
t2Tm
m
Since F is not differentiable, various smooth approximations Fpm ; p > 0;
with the property that
lim Fpm ðuÞ ¼ F m ðuÞ;
for all u 2 W;
p#0
have been proposed; (see e.g. Refs. 1–3). We will focus on entropic
regularization/smoothing,
(
)
X
m
Fp ðuÞ ¼ ð1=pÞ log
expfpft ðuÞg ;
ð1Þ
t 2 Tm
which admits a well-known uniform error estimate (Refs. 4–5)
0 Fpm ðuÞ F m ðuÞ logðmÞ=p:
ð2Þ
A related approach, proposed by Wu and Fang (Ref. 6), smooths directly
FðuÞ using an integral analog of (1),
Z
Fp ðuÞ ¼ ð1=pÞ log
expðpft ðuÞÞdt :
ð3Þ
T
However, due to the lack of a uniform error estimate such as (2),
convergence has been proven under the assumptions that fft ðuÞg are both
uniformly bounded and superuniformly continuous. These assumptions are
restrictive, since they require effectively the members of fft ðuÞg to behave
alike. Moreover, introducing an integral into the objective function
increases the computational effort.
Another approach, also involving an integration, was due to Polak et al.
(Ref. 7) who proposed a reciprocal barrier function,
Z
pðu; nk Þ ¼
dt=½nk ft ðuÞ;
T¼½0;1
where nk is an estimate of the optimal objective value at the kth iteration. By
minimizing pðu; nk Þ over the domain
Cðnk Þ ¼ fu 2 Rn jFðuÞ < nk g;
their method computes a u kþ1 whose average ft value is minimum. To ensure
that pðu; nk Þ ! 1 as u approaches a boundary point of Cðnk Þ, a Lipschitz
condition on ft in terms of t, uniformly with respect to any compact subset
JOTA: VOL. 121, NO. 3, JUNE 2004
599
in the u-space, is assumed. However, such a uniform Lipschitz condition
could fail to hold p
even
ffiffi on simple examples. For example,
min max u t
u 2 ½0;1 t 2 ½0;1
pffiffi
has an optimal value 0, but ft ðuÞ ¼ u t is not Lipschitz continuous in
t 2 ½0; 1 for any u.
In general, it seems that additional assumptions on ft ; t 2 T, are
unavoidable if one works at each iteration with the full set of functions
fft gt 2 T . In contrast, the method of outer approximations (Ref. 8), which
reduces problem (P) into a sequence of finite min-max problems, needs
typically no additional assumptions for convergence. At each iteration, T is
approximated by a finite subset Tm ¼ ft1 ; t2 ; . . . ; tm g, an approximate
solution
umþ1 2 arg min F m ðuÞ
u2W
is computed, and
tmþ1 2 arg max ft ðumþ1 Þ
t2T
is added to Tm to form Tmþ1 . This method is analogous to the exchange
method in semi-infinite programming (SIP) when applied to the following
problem:
minfnjft ðuÞ n 0; t2T; u2Wg:
ð4Þ
Under certain circumstances, a subset of ft1 ; . . . ; tm g can be dropped so that
each subproblem is kept to a limited size. We refer the readers to Ref. 9 and
the book edited by Reemtsen and Rückmann (Ref. 10) for detailed
discussions of SIP.
In this paper, we study the method of outer approximations, with each
approximate problem smoothed using the entropic regularization (1). We
call this algorithm ‘iterative entropic regularization’ (IER). By using the
uniform error estimate (2), we prove convergence of the IER algorithm,
while allowing minu 2 W Fpm ðuÞ to be solved inexactly. When each ft is smooth
convex and W is a linear constraint set, we propose to use an interior-point
algorithm to compute an inexact solution. Numerical examples are
presented to illustrate the behavior of the IER algorithm.
2. Iterative Entropic Regularization (IER)
Considering any approximation Fpm ðuÞ having a uniform error estimate,
ðm; pÞ Fpm ðuÞ F m ðuÞ ðm; pÞ;
8u 2 W;
ð5Þ
600
JOTA: VOL. 121, NO. 3, JUNE 2004
where ðm; pÞ 0 is a decreasing function of p having the following
properties:
(i) for any fixed m, limp ! 1 ðm; pÞ ¼ 0;
(ii) there exists some pðmÞ such that
limm!1 ðm; pðmÞÞ ¼ 0:
lim m ! 1 pðmÞ ¼ 1 and
In the case of the entropic regularization (1), we may choose
ðm; pÞ ¼ logðmÞ=p; pðmÞ ¼ ðlog mÞ2 :
Algorithm 2.1. IER Algorithm.
Step 1. Select t1 2 T and let T1 ¼ ft1 g; m ¼ k ¼ 1. Choose d 2 ð0; 1Þ;
p > 0:
Step 2. Find um
p 2 W satisfying
Fpm ðupm Þ min Fpm ðuÞ þ dk ;
u2W
ð6Þ
increase the iteration count k by 1.
k
m m
Step 3. If (i) Fðum
p Þ Fp ðup Þ and (ii) d þ ðm; pÞ is below a desired
tolerance, stop. If (i) is violated, then choose any
tmþ1 2 arg maxt2T ft ðum
p Þ; set Tmþ1 ¼ Tm [ ftmþ1 g, increase m
by 1, select p pðmÞ, and go to Step 2. If (ii) is violated,
increase p by a constant factor, and go to Step 2.
Theorem 2.1. The IER algorithm either stops in Step 3 with a
ðdk þ ðm; pÞÞ optimal solution or else it generates an infinite sequence fum
pg
in Wðm ! 1; p pðmÞÞ, any cluster point of which is a global minimum of
(P).
Proof. Suppose that (i) in Step 3 is violated finitely often. Then, m is
and p ! 1, k ! 1, so that ðm;
pÞ ! 0 [by
fixed eventually at some m
property (i) of ðm; pÞ] and dk ! 0. Then, (5) and (6) imply
m
m
Fðum
p Þ Fp ðup Þ
Fpm ðu Þ þ dk
pÞ
F m ðu Þ þ dk þ ðm;
pÞ;
Fðu Þ þ dk þ ðm;
pÞÞ optimal
where u is any optimal solution of (P). Thus, upm is a ðdk þ ðm;
solution.
Suppose that (i) in Step 3 is violated infinitely often. If m ! 1, then
p pðmÞ ensures that ðm; pÞ ðm; pðmÞÞ ! 0, as well as dk ! 0. Also, (5)
and (6) imply
JOTA: VOL. 121, NO. 3, JUNE 2004
601
m m
F m ðum
p Þ Fp ðup Þ þ ðm; pÞ
Fpm ðuÞ þ dk þ ðm; pÞ
F m ðuÞ þ dk þ 2ðm; pÞ
FðuÞ þ dk þ 2ðm; pÞ;
8u 2 W:
ð7Þ
(um
p ; tmþ1 )
t) be any cluster point of
Let (u;
as m ! 1 with p pðmÞ. Then,
there exists an infinite M f1; 2; . . .g such that, as m ! 1; m 2 M, we have
t) in the compact space W T. Then,
ðum
p ; tmþ1 Þ converging to (u;
m
ftmþ1 ðum
p Þ ¼ Fðup Þ
F m ðum
pÞ
ftiðmÞþ1 ðum
p Þ;
ð8Þ
where
iðmÞ ¼ maxfiji 2 M \ f1; 2; . . . ; mgg:
Thus,
tiðmÞþ1 ! t;
as m ! 1; m 2 M:
implying
By the continuity of ft ðuÞ, both sides of (8) tend to ftðuÞ,
m m
m
F ðup Þ ! Fðup Þ:
since F is continuous. Using this in (7) yields in the limit
Also, Fðum
p Þ ! FðuÞ,
FðuÞ; for all u 2 W;
FðuÞ
so u is a global minimum of (P).
(
Suppose that the IER algorithm generates an infinite sequence fum
p g.
For each fixed u 2 W; fF m ðuÞg1
forms
an
increasing
sequence
bounded
m¼1
above by FðuÞ. Define
FðuÞ ¼ lim F m ðuÞ;
m!1
8u 2 W:
Theorem 2.2. Suppose that the IER algorithm generates an infinite
sequence fum
p g. Then, minimizing FðuÞ over W is equivalent to minimizing
FðuÞ
over W. Namely,
min FðuÞ ¼ min FðuÞ:
u2W
u2W
Proof. Replacing the last inequality in (7) by F m ðuÞ FðuÞ, We obtain
as in the proof of Theorem 2.1 that
k
F m ðum
p Þ FðuÞ þ d þ 2ðm; pÞ;
8u 2 W;
602
JOTA: VOL. 121, NO. 3, JUNE 2004
and that, in the limit as m ! 1 with p pðmÞ,
FðuÞ;
FðuÞ
where u is any cluster point of um
p . Since
FðuÞ FðuÞ;
we have
¼ FðuÞ:
FðuÞ
So,
FðuÞ;
FðuÞ
and this shows that u also minimizes F over W.
h
Recall that F is continuous on W. By the Dini theorem (Ref. 11) [the
theorem states that, if an increasing sequence of real-valued continuous
functions fF m g1
m¼1 converges pointwise to a continuous function g on a
compact metric space, then it converges uniformly to g], we have the
uniform convergence property below. This property will be used in
Section 3.
Theorem 2.3. Suppose that the IER algorithm generates an infinite
m 1
sequence fum
p g. Then, fF gm¼1 converges uniformly to F on W.
3. Continuous Convex Min-Max Problems with Linear Constraints
In this section, we consider the special case of (P) in which ft ; t 2 T, are
smooth convex functions and
W ¼ fu 0jAu ¼ bg;
where A is a q n matrix and b 2 Rq . The convexity of ft implies that Fpm
given by (1) is also convex. We propose to use an interior-point algorithm to
implement Step 2 of the IER algorithm. In particular, we use a pathfollowing algorithm (Ref.12) to compute um
p 2 W satisfying (6). Such an
algorithm has been shown to be efficient for solving convex programs with
linear constraints. We call this algorithm ‘IER-interior point’ (IER-IP)
algorithm.
Algorithm 3.1. IER-IP Algorithm.
Step 1. This is the same as in the IER algorithm.
Step 2. Let l ¼ dk =2n. Use any algorithm to finitely generate
m m
um
p > 0; yp ; sp > 0 satisfying
JOTA: VOL. 121, NO. 3, JUNE 2004
603
Au ¼ b; s ¼ rFpm ðuÞ At y; kUs=l ek1 < 1;
ð9Þ
where e ¼ ð1; 1; :::; 1Þt and U ¼ diagðu1 ; u2 ; :::; un Þ. Increase k
by 1.
Step 3. This is the same as in the IER algorithm.
Step 2 can be realized by various interior-point algorithms
(Refs. 3,12–13) or by any feasible-descent method applied to the linearly
constrained convex problem
n
X
min
Fpm ðuÞ l
log ui :
u 0; Au ¼ b
i¼1
In the following theorem, we show that l ¼ dk =2n ensures that um
p generated
by the IER-IP algorithm satisfies (6).
Theorem 3.1. The um
p generated in Step 2 of the IER-IP algorithm
satisfies (6).
Proof. By attaching Lagrange multipliers y 2 Rq ; s 2 Rn to the constraints Au ¼ b; u 0; the dual problem of minu2W Fpm ðuÞ can be written as
sup fFpm ðuÞ rFpm ðuÞt u þ yt bg;
y;s;u
s.t.
s ¼ rFpm ðuÞ At y 0:
m
m
Thus, the pair ym
p ; sp > 0 generated in Step 2 is dual feasible, while up is
primal feasible.
By the weak duality theorem, the objective value of any dual feasible
solution provides a lower bound on that of any primal feasible solution.
Consequently,
m m t m
m t
m
m m
Fpm ðum
p Þ rFp ðup Þ up þ ðyp Þ b min Fp ðuÞ Fp ðup Þ;
u2W
which gives the following error estimate:
m
Fpm ðum
p Þ min Fp ðuÞ
u2W
m m
m m t m
m t
Fpm ðum
p Þ ½Fp ðup Þ rFp ðup Þ up þ ðyp Þ b
m t
m
m t
¼ ðspm Þt um
p þ ðyp Þ Aup ðyp Þ b
¼ ðspm Þt um
p:
on the other hand, since
kUpm spm =l ek1 < 1;
we have also
ð10Þ
604
JOTA: VOL. 121, NO. 3, JUNE 2004
0 ðUpm spm Þi 2l;
8i ¼ 1; 2; . . . ; n:
ð11Þ
Combining (10) and (11) yields
k
m
m t m
Fpm ðum
p Þ min Fp ðuÞ ðsp Þ up 2nl ¼ d :
(
u2W
Theorems 2.1 and 3.1 show that any cluster point of the primal sequence
fum
p g solves (P). The following theorem shows that, in addition, any cluster
m
point of fðym
p ; sp Þg solves the dual problem of (P). In other words, the
primal and dual cluster points satisfy the Karush-Kuhn-Tucker (KKT)
conditions,
Au ¼ b; u 0
0 2 f@FðuÞ At y sg;
ð12aÞ
s 0;
ðuÞt s ¼ 0:
ð12bÞ
ð12cÞ
Theorem 3.2. Suppose that the IER-IP algorithm generates an infinite
m m
m m
sequence of um
p ; yp ; sp . Then, any cluster point of fðyp ; sp Þg, if it exists,
solves the dual of (P).
m
Proof. Let ðy ; s Þ be any cluster point of fðym
p ; sp Þg. By passing to a
m m
subsequence if necessary, we may assume that fðyp ; sp Þg ! ðy ; s Þ: Since
fum
p g W, it has a cluster point u 2 W. By further passing to a subsequence if
necessary, we may assume that fum
p g ! u . We show below that ðu ; y ; s Þ
satisfies (12).
From the Proof of Theorem 3.1, we observe that
k
0 ðspm Þt um
p 2nl ¼ d :
Since dk ! 0; this yields in the limit
ðs Þt u ¼ 0:
m
Also, um
p > 0 and Aup ¼ b yield in the limit that
u 0 and Au ¼ b:
Thus, it remains only to verify the condition
0 2 @Fðu Þ At y s ;
which by Theorem 2.2 is equivalent to
At y þ s 2 @Fðu Þ:
Recall that F is convex. By the convexity of Fpm ðuÞ; we have
m m t
m
Fpm ðuÞ Fpm ðum
p Þ rFp ðup Þ ðu up Þ; 8u 2 W:
ð13Þ
JOTA: VOL. 121, NO. 3, JUNE 2004
605
Based on the following triangular inequality:
jFpm ðum
p Þ Fðu Þj
m m
m m
m
m
jFpm ðum
p Þ F ðup Þj þ jF ðup Þ Fðup Þj þ jFðup Þ Fðu Þj;
ð14Þ
we claim that
Fpm ðum
p Þ ! Fðu Þ:
This is true because the first term of (14) is uniformly bounded by ðm; pÞ
[see (5)], the second term tends to 0 by the uniform convergence of F m ðuÞ to
FðuÞ (Theorem 2.3), while the last term tends to 0 due to the continuity of
FðuÞ. By a similar argument as in (14),
Fpm ðuÞ ! FðuÞ:
Finally,
t m
m
t rFpm ðum
p Þ ¼ A yp þ sp ! A y þ s :
Now, pass to the limit in (13) to obtain
FðuÞ Fðu Þ ðAt y þ s Þt ðu u Þ;
8u 2 W:
This shows that
At y þ s 2 @Fðu Þ
and completes the proof.
h
4. Numerical Examples
In this section, we present three numerical examples to illustrate the
practical behavior of the IER/IER-IP algorithm. The first two are selected
from Polak et al. (Ref. 7), while the last one is a convex linearly constrained
problem.
We implemented the IER Algorithm using MATLAB 6.5 on an Athlon
1.4GHz PC. Key steps of the algorithm include:
k
m m
m
(i) finding um
p that satisfies Fp ðup Þ minu2W Fp ðuÞ þ d ;
(ii) computing tmþ1 ;
(iii) checking the stopping criterion.
For (i), we utilize the MATLAB subroutine fminsearch which allows the
users to set the tolerance dk .
606
JOTA: VOL. 121, NO. 3, JUNE 2004
For (ii), it is a global optimization problem. We subdivide T and select
tmþ1 to be the point on the grid which attains the largest functional value
ft ðum
p Þ. The partition is made finer and finer as the iteration count k increases.
For (iii), p is updated by pnew ¼ pð1 þ eÞ and the algorithm is stopped
when dk g1 and 1=p g2 ; with g1 ; g2 being two preset positive constants.
Although the IER Algorithm does not drop any point from Tm ; various
constraint dropping schemes such as those in Refs. 10, 14–15 can be
employed to reduce the computational effort. The idea is to keep only active
or -active constraints in each subproblem, while ignoring all others. A
second version of the IER algorithm, which drops all ‘nonbinding’
objectives5 from Tm is implemented for comparison. Such constraint
dropping reduces the number of summation terms in (1). A similar issue
arises in SIP and it was suggested (Refs. 10, 16) that the sup-norm should be
used in place of the L1 –norm on large-scale problems.
A greater concern for the IER algorithm, due to the use of exponential
functions in (1), is numerical overflow resulting from large positive values in
pftj ðuÞ; especially when ftj ðuÞ is already large. We resolved this issue by
working with small values of p, thus sacrificing to some extent the precision
of the solutions found. Alternatively, one could replace the exponential
function by a polynomial approximation whenever its argument exceeds a
threshold.
Example 4.1. This example, adopted from Ref. 7, has the following
form:
min maxfhðuÞ; max fhðuÞ þ 100½u1 þ u2 eu3 t þ e2t 2sinð4tÞgg:
u
t 2 ½0;1
ð15Þ
The numerical behavior of the IER algorithm is shown in Figure 1 and
Table 1 below. The initial parameters were set at
t1 ¼ 0; p ¼ 104 ; ¼ 1:3; g1 ¼ 106 ; g2 ¼ 0:005; d ¼ 1=2:
At the first iteration of the algorithm, fminsearch started from
u ¼ ð1; 1; 1Þt ; which is the same initial point used in Ref. 7. As can be
seen from Figure 1, the objective value jumped from 1265.09 to 7229.76
after one iteration and then quickly declined. This is because the IER
algorithm minimizes Fpm ðuÞ; which is a poor approximation of FðuÞ
initially. As m and p increases, this approximation appears to improve
quickly.
5
m
m
0
By nonbinding objectives, we mean ft 0 ðum
p Þ such that ft 0 ðup Þ 6¼ maxt 2 T ft ðup Þ, where t 2 Tm
m
and up is the iterate point to generate tmþ1 .
607
JOTA: VOL. 121, NO. 3, JUNE 2004
Fig. 1. Numerical behavior of the IER algorithm on solving Example 4.1 with X=iterations
and Y=FðuÞ.
Table 1. IER algorithm for Example 4.1.
Dropping
points
Time
u1
u2
u3
Optimal
Value
No
Yes
3.564
2.829
)0.213230
)0.213230
)1.361000
)1.361000
1.854000
1.854000
5.334900
5.334900
In general, the IER algorithm converges rapidly to a neighborhood of
the optimal solution, after which the convergence slows. About half of the
iterations (12 out of 25 iterations) were spent on improving the precision
level by one decimal place. This behavior is typical of cutting-plane
methods, including IER.
Listed in Table 1 are numerical results with and without the dropping rules.
We observe that dropping nonbinding objectives improves the run times.
Example 4.2. This example, also selected from Ref. 7, minimizes the
maximum of two functions f1 ðuÞ and f2 ðuÞ, with
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
2
2
f1 ðuÞ ¼ u1 u1 þ u2 cos
u21 þ u22
þ 0:005 u21 þ u22 ;
608
JOTA: VOL. 121, NO. 3, JUNE 2004
Table 2. IER algorithm for Example 4.2.
e
u1
u2
Optimal Value
Time[sec]
1.3
)4.2352e)020
6.6466e)008
4.4398e)015
0.219
f2 ðuÞ ¼
u2 qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
u21 þ u22 sin
u21 þ u22
þ 0:005 u21 þ u22 :
It was considered to be difficult by Polak et al. (Ref. 7) due to the spiral
contours of the max function. However, using the same starting point
u ¼ ð1:41831; 4:79462Þt as in Ref. 7 to initialize fminsearch, the IER
algorithm moved quickly to a near-optimal solution up to an accuracy of
1015 in less than one second (Table 2). Initial parameters used in this
example were
p ¼ 105 ; g1 ¼ 107 ; g2 ¼ 105 :
Example 4.3. This example has a convex objective function and linear
constraints. We took Au ¼ b; called afiro, from www.netlib.org and added
the constraint 0 u 5 to make W compact. Next, slack variables were
introduced to convert the inequalities u 5 into equalities. Furthermore,
the matrix A was modified to make ðe; 4eÞt an initial feasible solution. The
final form of the A-matrix has a size of 781026 .
To implement Step 2 in the IER-IP algorithm, we use a primal pathfollowing method (e.g., Refs. 3, 13). Starting from an interior feasible
solution um
p ; we compute
m2 t 1
m2
m m
ym
p ¼ ðAUp A Þ AUp ðrFp ðup Þ leÞ;
ð16Þ
t m
spm ¼ rFpm ðum
p Þ A yp ;
ð17Þ
m m
wm
p ¼ ð1=lÞUp sp e:
ð18Þ
If kwm
p k1 1; a line search
m
m m
up :¼ um
p rUp wp
ð19Þ
is invoked, where the step size r is computed by golden section, applied to
!
n
X
m
logui
:
min Fp ðuÞ l
r0
i¼1
m m
u¼um
p rUp wp
In this example, T=[0, a] and the objective function is
6
It can be found at our web site http://math.ncku.edu.tw/jylin/a.mat
609
JOTA: VOL. 121, NO. 3, JUNE 2004
ft ðuÞ ¼
102
X
tðui sin tÞ2 :
i¼1
For any t > 0; ft ðuÞ is convex in u. However, for any u 0; it is a
nonperiodical function of t with different oscillations (Figure 2).
In general, the computational time was insensitive to the choice of a
(Table 3). Moreover, the pattern of convergence is quite similar. Let us take
a=20 and use Figure 3 as an example for explanation. The IER-IP
algorithm started from the point u ¼ ðe; 4eÞt with an objective value about
Fig. 2. Graph of ft ðuÞ with u fixed and t 2 ½0; 50.
Table 3. IER-IP algorithm for Example 4.3 without dropping rule.
T=[0, a]
a=10
a=20
a=30
a=40
a=50
Time for Step 2
Time for Step 3
Total time (sec)
Number of points added
Final objective value
2nl þ logðmÞ=p
Relative error
14.641
16.814
31.487
2
10755
69.682
0.00630
21.608
17.032
38.704
7
23919
192.02
0.00794
19.9
16.857
36.818
5
41227
159.16
0.00381
21.377
17.14
38.58
5
49889
176.97
0.00315
18.082
20.234
38.395
6
67220
701.9
0.01044
610
JOTA: VOL. 121, NO. 3, JUNE 2004
Fig. 3. Behavior of the IER-IP algorithm for the case a=20.
2.565104 . After one interior-point step (16)–(19), a point near the central
path satisfying kwm
p k1 < 1 was found and the objective value was quickly
improved to 2.44104 . Then, the parameters p, l were updated with a new
point t2 adding to T1 , which yielded another drop in the objective value to
about 2.395104 .
Although the newly selected member ftmþ1 shifted the central path, the
time to recompute a solution near the central path is less than before. After
adding more t-points (the fifth row in Table 3), the process quickly
converges.
On this example, numerical overflow is an issue because the objective
function is the sum of 102 terms each of which is in the order of hundreds or
thousands. Thus, we started IER-IP from p ¼ 105 and finished at
g1 ¼ 102 and g2 ¼ 100, namely, p 2 ½105 ; 102 : Other initial parameters
included t1 ¼ 0:1; l ¼ 10; ¼ 3; d ¼ 1=2. According to Theorem 2.1, if the
IER algorithm stops when there is no additional t-point to include, we have
a ðdk þ ðm; pÞÞ optimal solution. Since
dk ¼ 2nl and ðm; pÞ ¼ logðmÞ=p;
the quantity 2nl þ logðmÞ=p provides a bound on the error between the
solution obtained by IER-IP and the true optimal solution; see the final two
611
JOTA: VOL. 121, NO. 3, JUNE 2004
Table 4.
IER-IP algorithm for Example 4.3 with dropping rule.
T=[0, a]
a=10
a=20
a=30
a=40
a=50
Time for Step 2
Time for Step 3
Total time (sec)
Number of points retained
Final objective value
2nl þ 1=p
Relative error
14.467
17.204
31.703
1
10755
99.648
0.00908
17.923
17.282
35.268
1
23919
99.648
0.00408
19.454
17
36.486
1
41227
99.648
0.00237
18.31
17.173
35.515
1
49889
99.648
0.00196
13.018
20.063
33.144
1
67220
392.62
0.00584
rows in Tables 3 and 4, which were obtained using the same initial
parameters. This error is in the hundreds for most cases, indicating that the
solutions obtained by IER-IP are not precise in absolute sense. However,
when 2nl þ logðmÞ=p is divided by the final objective value, the relative
error bound is about 103 , which we feel is acceptable.
Finally, Table 4 suggests that the IER-IP algorithm equipped with the
dropping rule converges faster generally than the original IER-IP, except
when IER-IP adds very few t-points as in the case a=10.
References
1. BERTSEKAS, D. P., Constrained Optimization and Lagrange Multiplier Methods,
Academic Press, New York, NY, 1982.
2. POLYAK, R. A., Smooth Optimization Methods for Minmax Problems, SIAM
Journal on Control and Optimization, Vol. 26, pp. 1274–1286, 1988.
3. SHEU, R. L., and WU, S. Y., Combined Entropic Regularization and PathFollowing Method for Solving Finite Convex Min-Max Problems Subject to
Infinitely Many Linear Constraints, Journal of Optimization Theory and
Applications, Vol. 101, pp. 167–190,1999.
4. CHANG, P. L., A Minimax Approach to Nonlinear Programming, Doctoral
Dissertation, Department of Mathematics, University of Washington, Seattle,
Washington, 1980.
5. BEN-TAL, A., and TEBOULLE, M., A Smoothing Technique for Nondifferentiable
Optimization Problems, Lecture Notes in Mathematics, Springer Verlag, Berlin,
Germany, Vol. 1405, pp. 1–11, 1989.
6. WU, S. Y., and FANG, S. C., Solving Min-Max Problems and Linear Semi- Infinite
Programs, Computers and Mathematics with Applications, Vol. 32, pp. 87–93,
1996.
7. POLAK, E., HIGGINS, J. E., and MAYNE, D. Q., A Barrier Function Method for
Minimax Problems, Mathematical Programming, Vol. 54, pp. 155–176, 1992.
8. POLAK, E., Optimization: Algorithms and Consistent Approximations, Springer
Verlag, New York, NY, 1994.
612
JOTA: VOL. 121, NO. 3, JUNE 2004
9. HETTICH, R., and KORTANEK, K. O., Semi-Infinite Programming: Theory,
Methods, and Applications, SIAM Review, Vol. 35, pp. 380–429, 1993.
10. REEMTSEN, R., and GÖRNER, S., Numerical Methods for Semi-Infinite Programming: A Survey, Semi-Infinite Programming, Edited by R. Reemtsen and
J.J. Rückmann, Kluwer Academic Publishers, Dordrecht, Netherlands, pp. 195–
275, 1998.
11. DIEUDONNE, J., Foundations of Modern Analysis 10–1, Academic Press, New
York, NY, 1969.
12. DEN HERTOG, D., Interior-Point Approach to Linear, Quadratic, and Convex
Programming, Kluwer Academic Publishers, Dordrecht, Netherlands, 1994.
13. SHEU, R. L., A Generalized Interior-Point Barrier Function Approach for Smooth
Convex Programming with Linear Constraints, Journal of Information and
Optimization Sciences, Vol. 20, pp. 187–202, 1999.
14. GONZAGA, C. C., and POLAK, E., On Constraint Dropping Schemes and
Optimality Functions for a Class of Outer Approximations Algorithms, SIAM
Journal on Control and Optimization, Vol. 17, pp. 477–493, 1979.
15. POWELL, M. J. D., A Tolerant Algorithm for Linearly Constrained Optimization
Calculations, Mathematical Programming, Vol. 45, pp. 547–566, 1989.
16. ABBE, L., A Logarithmic Barrier Approach and Its Regularization Applied to
Convex Semi-Infinite Programming Problems, PhD Dissertation, Universität
Trier, Trier, Germany, 2001.