Convex Optimization EE364a:
Review Session 2
Convex Functions and CVX
Stanford University
Winter Quarter, 1/22/2013
1
Outline
Announcements
Convex functions
Operations that preserve convexity
CVX
2
Outline
Announcements
Convex functions
Operations that preserve convexity
CVX
Announcements
3
Announcements
◮
download and install CVX
◮
start looking through the user guide
Announcements
4
Outline
Announcements
Convex functions
Operations that preserve convexity
CVX
Convex functions
5
Tools for establishing convexity
◮
definition of convexity
◮
first-order condition
◮
restrict function to a line
◮
simple examples (negative log, norms, quadratic-over-linear,
log-sum-exp,. . . )
◮
convexity-preserving operations
Convex functions
6
Example: sigmoid / logistic function
f (x) =
1
,
1 + e −x
dom f = R
◮
is it convex? concave?
◮
is it quasiconvex? quasiconcave?
◮
is it log-convex? log-concave?
Convex functions
7
Convex or concave?
f (x) =
1
1 + e −x
1
1
1+e −x
0.8
0.6
0.4
0.2
0
−0.2
−10
−8
−6
−4
−2
0
x
2
4
6
8
10
Solution
◮
◮
by looking at the graph, it is neither convex nor concave
> 0 if x < 0
e −x (1−e −x )
′′
formally, f (x) = − (1+e −x )3
≤ 0 if x ≥ 0
Convex functions
8
Restricting domain
convexity of a function may depend on the domain
for example, is
g (x) =
1
,
1 + e −x
dom g = R+
concave? what about
(
g (x),
g̃ (x) =
−∞,
if x ≥ 0
if x < 0,
the extended-value extension?
Solution
◮
g concave, since g ′′ (x) ≤ 0 if x ≥ 0
◮
g̃ also concave
◮
note that f , g , and g̃ are different functions
Convex functions
9
Quasiconvex or quasiconcave?
f (x) =
1
1 + e −x
Solution
◮
sublevel sets Sα = {x ∈ dom f | f (x) ≤ α} are convex ⇒
quasiconvex
◮
◮
◮
◮
◮
for α ≤ 0, Sα = ∅
for α ≥ 1, Sα = R
for 0 < α < 1, Sα = (−∞, f −1 (α)]
similarly, superlevel sets are convex ⇒ quasiconcave
for x ∈ R, f (x) monotonic ⇔ quasiconvex and
quasiconcave
Convex functions
10
Log-convex or log-concave?
log(f (x)) = − log(1 + e −x )
2
0
1
1+e −x
−2
−4
log
−6
−8
−10
−12
−10
−8
−6
−4
−2
0
x
2
4
6
8
10
Solution
◮
not log-convex
◮
is log-concave (log f (x) is negative of log-sum-exp,
evaluated at z1 = 0, z2 = −x)
Convex functions
11
Outline
Announcements
Convex functions
Operations that preserve convexity
CVX
Operations that preserve convexity
12
Operations that preserve convexity
Operation on functions
Pointwise maximum and supremum
Composition with an affine function
Nonnegative weighted sum
Perspective transformation
Minimization (over convex sets)
Composition rules
Related operation for sets
Intersection
Affine transformation
Affine transformation
Perspective transformation
Projection
◮
convex functions and convex sets are related via the
epigraph
◮
composition rules are extremely important
Operations that preserve convexity
13
Example:
is the following a convex function (in x, y , z ∈ R)?
1
(x − z)2
+ max 1 − y + √ , e z , 0
f (x, y , z) =
y +1
x
(with domain x > 0, y + 1 > 0)
Solution
◮
◮
◮
◮
◮
◮
(x−z)2
y +1
2
is composition of quadratic-over-linear function st
with affine function that maps (x, y , z) to (x − z, y + 1), so
is convex
√1 is a negative-power function, so convex in x
x
1 − y is affine, so 1 − y + √1x is convex in x and y
e z is exponential, so convex in z
max term is convex, since its arguments are convex
sum of left and right terms is convex
Operations that preserve convexity
14
Example:
is the maximum eigenvalue of a symmetric matrix a convex
function on dom f = Sm ?
f (X ) = sup{y T Xy |
m
X
i =1
yi2 ≤ 1}
Solution
◮
◮
y T Xy is a linear function of X
f is the supremum of a collection of uncountably-many
linear functions, so convex
Operations that preserve convexity
15
Example:
what about g with dom g = Sm ?
T
g (X ) = sup{y Xy |
m
X
i =1
|yi |1/2 ≤ 1}
1
0.8
0.6
0.4
0.2
0
−0.2
−0.4
−0.6
−0.8
−1
−1
−0.5
0
0.5
1
Solution
◮
convex; still a supremum of a collection of linear functions
Operations that preserve convexity
16
Composition rules
composition of g : Rn → Rk and h : Rk → R:
f (x) = h(g (x)) = h(g1 (x), g2 (x), . . . , gk (x))
e.g., f is convex if gi concave, h convex, h̃ nonincreasing in each
argument
Operations that preserve convexity
17
Example: 3.22(b)
Show that the following function is convex:
p
f (x, u, v ) = − uv − x T x
on dom f = {(x, u, v ) | uv > x T x, u, v > 0}. Use the fact
√
that x T x/u is convex in (x, u) for u > 0, and that − z1 z2 is
convex on R2++ .
Operations that preserve convexity
18
Example: 3.22(b) solution
Solution
p
f (x, u, v ) = − uv − x T x
◮
◮
◮
take f (x, u, v ) = −
p
u(v − x T x/u)
g1 (u, v , x) = u and g2 (u, v , x) = v − x T x/u are concave
the function
h(z1 , z2 ) =
√
− z1 z2
0
if z 0
otherwise
is convex and decreasing in each argument
◮
f (u, v , x) = h(g (u, v , x)) is convex
Operations that preserve convexity
19
Outline
Announcements
Convex functions
Operations that preserve convexity
CVX
CVX
20
CVX
CVX
◮
written by Michael Grant and Stephen Boyd
◮
MATLAB-based modeling system for convex optimization
◮
solves problems that obey the disciplined convex
programming ruleset
◮
converts a problem to standard form (LP/SOCP/SDP) and
solves it using a core solver (either SDPT3 or SeDuMi)
21
Example: linear program
minimize 1T x
subject to Ax = b
x 0,
where x ∈ Rn , b ∈ Rm .
◮
◮
CVX
CVX source code:
cvx_begin
variable x(n)
minimize(ones(1,n)*x)
subject to
A*x == b
x >= 0
cvx_end
objective function can also be written as sum(x)
22