[convex-1]
class 26
CONVEXITY
A taxonomy of mathematical programming problems
Consider the optimization problem
max/min f(x)
s.t. x,K fS d Rn
where
f(x) is the objective function that is real and continuous,
K is that part of the feasible region that is closed & bounded,
S is the feasible region, and
Rn is the n-dimensional Euclidean space.
Under these conditions, the problem has an optimal solution x*,K.
In the case of K being continuous, a taxonomy for solution algorithms can be mapped
out.
Figure 1 - Convex vs nonconvex programming
Review of Convexity
Convex feasible-region
A set S d Rn is convex if for every x1, x2 , S and every c, 0#c#1, the point cx1+(1-c)x2
, S.
Sets are either convex or nonconvex.
Example 1:
Let S be x1+x2#1; x1=(0,0), x2=(0.5,0.5) , S; and c=0.5;
then 0.5(0,0)+(1&0.5)(0.5,0.5)=(0.25,0.25) , S.
Figure 2 - R2 convexity example
Example 2:
Let S be 0#x#10; x1=5, x2=10,S; and c=0.4;
then 0.4(5)+(1-0.4)10=8 , S.
Figure 3 - R1 convexity example
1
In general
Figure 4 - Illustrating a convex and nonconvex set
LP always deals with a convex feasible region. Intersection of two convex sets is also
convex
Convex objective function
An objective function f defined on a convex set S is a convex function if for every x1,
x2 , S and every c, 0#c#1,
f[cx1+(1&c)x2]#cf(x1)+(1&c)f(x2).
It is strictly convex (concave) function if # ($) can be replaced by < (>).
Figure 5 - Illustration of a convex objective function in R1
Sum of two convex (concave) functions is also convex (concave).
Figure 6 - Illustrating addition of 2 convex objective functions
Application of convexity tests
Theorem
If objective function f(x) is convex (concave) over a convex feasible region S d
Rn, then any local minimum (maximum) is also a global minimum (maximum).
Figure 7 - Maximum illustration
Theorem
If objective function f(x) is convex (concave) over a convex feasible region S d
Rn, then the maxima (minima) of f(x) occur at an extreme point.
Figure 8 - Maxima illustration
Optimization on a nonconvex feasible region
Figure 9 - Illustration in R2
2
[convx-program]
Convex vs. nonconvex programming
min (max) f(x)
s.t. x∈K
Evaluate convexity of set K
K convex
K nonconvex
Evaluate convexity of f(x)
f(x) strictly
convex
(concave)
Case I
f(x)
convex
(concave)
f(x) not
convex
(or concave)
Case II
Single unique
global
minimum
(maximum)
Any local
minimum
(maximum) is
a global
minimum
(maximum)
Convex program
Case III
Find all local
minima (maxima)
to determine
global minimum
(maximum)
Nonconvex program
R2 convexity example
x2
1
x2
= (0.5,0.5)
(0.25,0.25)
x1 = (0,0)
S
1
x1
R1 convexity example
0
5
10
8
S
x
Illustrating a convex & nonconvex set
nonconvex
x1
S convex
S
x2
Illustration of
a convex objective function in R1
f(x1)
cf(x1)+(1-c)f(x2)
f(x2)
f[cx1+(1-c)x2]
x1
cx1+(1-c)x2
S
x2
Illustrating addition of 2 convex objective functions
f(x)
f1(x) + f2(x)
f1(x)
f2(x)
x
Maximum illustration
f(x)
Global
max
x
[
S
]
Maxima illustration
f(x)
Local maxima
x
[
S
]
Example Illustration in R2
x2
Global
min
Local min
x1