Convex Sets
Daniel P. Palomar
Hong Kong University of Science and Technology (HKUST)
ELEC5470 - Convex Optimization
Fall 2016-17, HKUST, Hong Kong
Outline of Lecture
• Definition convex set
• Hyperplanes, halfspaces, polyhedra
• Balls and ellipsoids
• Convex hull
• Cones: norm cones, PSD cone
• Operations that preserve convexity
• Generalized inequalities
(Acknowledgement to Stephen Boyd for material for this lecture.)
Daniel P. Palomar
1
Definition of Convex Set
• A set C ∈ Rn is said to be convex if the line segment between any
two points is in the set: for any x, y ∈ C and 0 ≤ θ ≤ 1,
θx + (1 − θ) y ∈ C.
Daniel P. Palomar
2
Examples: Hyperplanes and Halfspaces
• Hyperplane:
T
T
C= x|a x=b
where a ∈ Rn, b ∈ R.
• Halfspace:
C= x|a x≤b
Daniel P. Palomar
3
Example: Polyhedra
• Polyhedron:
C = {x | Ax ≤ b, Cx = d}
where A ∈ Rm×n, C ∈ Rp×n, b ∈ Rm, d ∈ Rp.
Daniel P. Palomar
4
Examples: Euclidean Balls and Ellipsoids
• Euclidean ball with center xc and radius r:
B (xc, r) = {x | kx − xck2 ≤ r} = {xc + ru | kuk2 ≤ 1} .
• Ellipsoid:
n
o
T
E (xc, P ) = x | (x − xc) P −1 (x − xc) ≤ 1 = {xc + Au | kuk2 ≤ 1}
with P ∈ Rn×n 0 (positive definite).
Daniel P. Palomar
5
Convex Combination and Convex Hull
• Convex combination of x1, . . . , xk : any point of the form
x = θ 1 x 1 + θ 2 x 2 + · · · + θ k xk
with θ1 + · · · + θk = 1, θi ≥ 0.
• Convex hull of a set: set of all convex combinations of points in
the set.
Daniel P. Palomar
6
Convex Cones
• A set C ∈ Rn is said to be a convex cone if the ray from each
point in the set is in the set: for any x1, x2 ∈ C and θ1, θ2 ≥ 0,
θ1x1 + θ2x2 ∈ C.
Daniel P. Palomar
7
Norm Balls and Norm Cones
• Norm ball with center xc and radius r: {x | kx − xck ≤ r} where
k·k is a norm.
n+1
• Norm cone: (x, t) ∈ R
| kxk ≤ t .
• Euclidean norm cone or second-order cone (aka ice-cream cone):
Daniel P. Palomar
8
Positive Semidefinite Cone
• Positive semidefinite (PSD) cone:
Sn+
• Example:
Daniel P. Palomar
x y
y z
= X∈R
n×n
T
|X=X 0 .
∈ S2+
9
Operations that Preserve Convexity
How do we establish the convexity of a given set?
1. Applying the definition:
x, y ∈ C, 0 ≤ θ ≤ 1 =⇒ θx + (1 − θ) y ∈ C
which can be cumbersome.
2. Showing that C is obtained from simple convex sets (hyperplanes,
halfspaces, norm balls, etc.) by operations that preserve convexity:
•
•
•
•
intersection
affine functions
perspective function
linear-fractional functions
Daniel P. Palomar
10
Intersection
• Intersection: if S1, S2, . . . , Sk are convex, then S1 ∩ S2 ∩ · · · ∩ Sk
is convex.
• Example: a polyhedron is the intersection of halfspaces and hyperplanes.
• Example:
S = {x ∈ Rn | |px (t)| ≤ 1 for |t| ≤ π/3}
where px (t) = x1 cos t + x2 cos 2t + · · · + xn cos nt.
Daniel P. Palomar
11
Affine Function
• Affine composition: the image (and inverse image) of a convex set
under an affine function f (x) = Ax + b is convex:
S ⊆ Rn convex =⇒ f (S) = {f (x) | x ∈ S} convex.
• Examples: scaling, translation, projection.
• Example:
(x, t) ∈ R
n+1
n
| kxk ≤ t is convex, so is
T
x ∈ R | kAx + bk ≤ c x + d .
• Example: solution set of LMI: {x ∈ Rn | x1A1 + · · · + xnAn B}.
Daniel P. Palomar
12
Perspective and Linear-Fractional Functions
Perspective function: P : Rn+1 −→ Rn:
P (x, t) = x/t,
dom P = {(x, t) | t > 0} .
• Images and inverse images of convex sets under perspective functions
are convex.
Linear-fractional function: f : Rn −→ Rm:
Ax + b
,
f (x) = T
c x+d
T
dom P = x | c x + d > 0 .
• Images and inverse images of convex sets under linear-fractional
functions are convex.
Daniel P. Palomar
13
Generalized Inequalities
• A convex cone K ⊆ Rn is a proper cone if it is closed, solid, and
pointed.
• Examples:
– nonnegative orthant:
K = Rn+ = {x ∈ Rn | xi ≥ 0, i = 1, . . . , n}
– positive semidefinite cone:
n
n×n
T
K = S+ = X ∈ R
|X=X 0
– nonnegative polynomials on [0, 1]:
n
2
n−1
K = x ∈ R | x1 + x2 t + x 3 t + · · · xn t
≥ 0 for t ∈ [0, 1] .
Daniel P. Palomar
14
• A generalized inequality is defined by a proper cone K:
y K x ⇐⇒ y − x K 0 or y − x ∈ K.
• Examples:
– componentwise inequality (K = Rn+):
y Rn+ x ⇐⇒ yi ≥ xi, i = 1, . . . , n
– matrix inequality (K = Sn+):
Y Sn+ X ⇐⇒ Y − X is positive semidefinite.
Daniel P. Palomar
15
References
Chapter 2 of
• Stephen Boyd and Lieven Vandenberghe, Convex Optimization.
Cambridge, U.K.: Cambridge University Press, 2004.
http://www.stanford.edu/˜boyd/cvxbook/
Daniel P. Palomar
16