Lecture: Convex Sets
http://bicmr.pku.edu.cn/~wenzw/opt-2015-fall.html
Acknowledgement: this slides is based on Prof. Lieven Vandenberghe’s lecture notes
1/24
Introduction
affine and convex sets
some important examples
operations that preserve convexity
generalized inequalities
separating and supporting hyperplanes
dual cones and generalized inequalities
2/24
Affine set
e through x1, x2: all points
line through x1 , x2 : all points
x x==θx
+(1(1−−
θx11 +
θ)xθ)x
R) ∈ R)
2 2 (θ ∈ (θ
x1
θ = 1.2
θ=1
θ = 0.6
x2
θ=0
θ = −0.2
contains
linethrough
through any
points
in theinset
ffineaffine
set: set:
contains
thethe
line
anytwo
twodistinct
distinct
points
the s
example: solution set of linear equations {x| Ax = b}
ample:
solution set of linear equations {x | Ax = b}
(conversely, every affine set can be expressed as solution set of
system of linear equations)
nversely, every affine set can be expressed as solution set of syste
3/24
1
Convex set
x = θx1 + (1 − θ)x2
≤1 1segment between x1 and x2 : all points
θ θ≤line
x = θx1 + (1 − θ)x2
ontains
line segment
between
any two
pointspoints
in thein
set:
contains
line
segment
between
et:
contains
line
segment
between
anyany
twotwo
points in the
with 0 ≤ θ ≤ 1
convex set: contains line segment between any two points in the set
xx2x
∈, C,
θxθx
θ)x−2 θ)x
∈ CC∈
1+
x ∈∈C,0C,≤0θ≤
0≤≤
θ≤
1 =⇒=⇒
θx(1 −
+ (1
θ 1≤
1 =⇒
1 ,1x2 2
1 +1(1 − θ)x2 ∈ 2
x1 , x2 ∈ C,
0≤θ≤1
=⇒
θx1 + (1 − θ)x2 ∈ C
examples two
(one convex,
two nonconvex
sets)
e(one
convex,
nonconvex
sets)
es
(one
convex,
two
nonconvex
sets)
convex,
two
nonconvex
sets)
4/24
nvex combination of x1,. . . , xk : any point x of the form
Convex combination
x = θ1x1 +and
θ2x2convex
+ · · · +hull
θk xk
x = θ1 x1 + θ2 x2 + · · · + θk xk
combination
· · convex
· + θk =
1, θi ≥ 0of x1 , ..., xk : any point x of the form
h θ1 + · · · + θk = 1, θi ≥ 0
x = θ1 x1 + θ2 x2 + ... + θk xk
with θ1 + ... + θk = 1, θi ≥ 0
ull
conv S: set of all convex combinations of points in S
nvex hull conv S: set of all convex combinations of points in S
convex hull convS: set of all convex combinations of points in S
5/24
c Convex
(nonnegative)
cone combination of x1 and x2: any point of the fo
x = θ1xof1 x+
θ2xx22 : any point of the form
conic (nonnegative) combination
1 and
x = θ1 x1 + θ2 x2
θ1 ≥ 0, θ2 ≥ 0
with θ1 ≥ 0, θ2 ≥ 0
x1
x2
0
convex cone: set that contains all conic combinations of points in the set
vex cone: set that contains all conic combinations of points in th
6/24
Hyperplanes and halfspaces
x0
x
aT x = b
halfspace: set of the form
hyperplane: set
of the
form
yperplanes
and
halfspaces
T 6= 0)
{x|aT{x
x ≤| a
b}(a
halfspace:
set of the form
x ≤ b} (a 6= 0)
{x|aT x = b}(a
6= 0)
he form {x | aT x = b} (a 6= 0)
a
a
x0
x0
x
T
a x=b
aT x ≥ b
aT x ≤ b
is the
vector
form {x
| aT x•≤a b}
(a 6=normal
0)
a is the normal vector
a
• hyperplanes
are affine and convex; halfspaces are convex
hyperplanes are affine and convex; halfspaces are convex
aT x ≥ b
x
0
Convex sets
T
7/24
clidean)
ball with
xc and radius r:
Euclidean
ballscenter
and ellipsoids
(Euclidean) ball with center xc and radius r:
B(xc, r) = {x | kx − xck2 ≤ r} = {xc + ru | kuk2 ≤ 1}
B(xc , r) = {x| kx − xc k2 ≤ r} = {xc + ru| kuk2 ≤ 1}
ellipsoid:
the form
soid:
set of set
theofform
{x|(x − xc )T P−1 (x − xc ) ≤ 1}
{x | (x − xc)T P −1(x − xc) ≤ 1}
with P ∈ Sn++ (i.e., P symmetric positive definite)
representation:
{xc + Au| kuk
with A square and nonsingular
P other
∈ Sn++
(i.e., P symmetric
positive
2 ≤ 1} definite)
xc
8/24
= 0Norm balls and norm cones
norm: a function k · k that satisfies
kxk ≥ 0; kxk = 0 if and only if x = 0
ktxk = |t| kxk for t ∈ R
norm;kxk+· kyksymb
is particular norm
≤ kxk + kyk
notation:
k ·x
k ciskgeneral
: {x
| kx −
≤ r} (unspecified) norm; k · ksymb is particular norm
norm ball with center xc and
radius r: {x| kx − xc k ≤ r}
1
norm cone: {(x, t)| kxk ≤ t}
Euclidean norm cone is called
second-order cone
t
0.5
0
1
1
0
x2
0
−1 −1
x1
norm balls and cones are
convex
9/24
Polyhedra
solution set of finitely many linear inequalities and equalities
solution set of finitely many linear inequalities and equalities
Ax b,
Cx = d
Ax b,
Cx = d
m×n
p×n
m×n,, C
(A(A∈ ∈RR
Rp×n
componentwiseinequality)
inequality)
C∈
∈R
, ,isiscomponentwise
a1
a2
P
a5
a3
a4
polyhedron
is intersection
finitenumber
numberofofhalfspaces
halfspaces
and
polyhedron
is intersection
of of
finite
and
hyperplanes
hyperplanes
10/24
Positive
Positive semidefinite
cone semidefinite cone
notation:
notation:
n
• S
is set of symmetric n × n matrices
n
S is set of symmetric n × n matrices
n
n
• S
+ = {X ∈ S | X 0}: positive semidefinite n × n matrices
Sn+ = {X ∈ Sn |X 0}: positive semidefinite n × n matrices
n
⇐⇒ zT Xz
zT ≥
Xz0 ≥
forz all z
XX
∈ S∈n+S+ ⇐⇒
for0all
n nis a convex cone
SS
++ is a convex cone
n n = {X ∈ Sn |Xn 0}: positive definite n × n matrices
++ = {X ∈ S | X ≻ 0}: positive definite n × n matrices
•SS
++
1
0.5
z
x xy y
2
example:
∈+S2+
example: y z ∈ S
y z
0
1
1
0
y
0.5
−1 0
x
11/24
Operations that preserve convexity
practical methods for establishing convexity of a set C
1
apply definition
x1 , x2 ∈ C,
2
0≤θ≤1
=⇒
θx1 + (1 − θ)x2 ∈ C
show that C is obtained from simple convex sets (hyperplanes,
halfspaces, norm balls, ... ) by operations that preserve convexity
intersection
affine functions
perspective function
linear-fractional functions
12/24
intersection
of (any
of) convex
sets issets
convex
the
intersection
of number
(any number
of) convex
is convex
Intersection
the intersection of (any number of ) convex sets is convex
mple:
example:
m
example:S = {x
∈ {x
Rm∈| R
|p(t)|
≤ 1 for
≤ |t|
π/3}
S=
| |p(t)|
≤ 1|t|for
≤ π/3}
m
S = {x ∈ R | |p(t)| ≤ 1 for |t| ≤ π/3}
rewhere
p(t)
=p(t)
x1 cos
+
2t
+
·2t
· ...
·+
++·xx·mm
cos
mt
=
cosxt 2t+cos
+x2xcos
·+
xmt
cos mt
where
p(t)
=x
xt11cos
2t +
cos
2 cos
m
= 2:
mfor
=for2:
mm =
2:
0
0
x2
p(t)
p(t)
0
−1
−1
0 π/3
π/32π/3
t
t
2π/3 π
π
2
1
1
0
x2
1
1
2
0
−1
−1
−2
−2
−2
−2
−1
S
−10
x1
S
x011
12
2
13/24
Affine function
suppose f : Rn → Rm is affine (f (x) = Ax + b with A ∈ Rm×n , b ∈ Rm )
the image of a convex set under f is convex
S ⊆ Rn convex
=⇒
f (S) = {f (x)|x ∈ S} convex
the inverse image f −1 (C) of a convex set under f is convex
C ⊆ Rm convex
=⇒
f −1 (C) = {x ∈ Rn |f (x) ∈ C} convex
examples
scaling, translation, projection
solution set of linear matrix inequality {x|x1 A1 + ... + xm Am B}
(with Ai , B ∈ Sp )
hyperbolic cone {x|xT Px ≤ (cT x)2 , cT x ≥ 0} (with P ∈ Sn+ )
14/24
Perspective and linear-fractional function
perspective function P : Rn+1 → Rn :
P(x, t) = x/t,
dom P = {(x, t)|t > 0}
images and inverse images of convex sets under perspective are
convex
linear-fractional function f : Rn → Rm :
f (x) =
Ax + b
,
cT x + d
dom f = {x|cT x + d > 0}
images and inverse images of convex sets under linear-fractional
functions are convex
15/24
1 1
x x
x1x+
x21+ 1
x1 +
2+
example of a linear-fractional
f (x)
=
f (x)
= function
f (x) =
1
−1 −1
−1 −1
C C
0
0
x1 x1
1
1
0
x2
0
1
1
x2
x2
x2
1
0
1
x
x1 + x2 + 1
0
−1 −1
−1 −1
f (C)
f (C)
0
0
x1 x1
1
1
16/24
Separating
hyperplane
Separating
hyperplane
theorem theorem
if CDand
are disjoint
convex
a 6=
b such
C and
areDdisjoint
convex
sets, sets,
then then
therethere
existsexists
a 6= 0,
b 0,
such
that that
aT x ≤ b for x ∈ C, T aT x ≥ b for x ∈ D
aT x ≤ b for x ∈ C,
a x ≥ b for x ∈ D
aT x ≥ b
aT x ≤ b
D
C
a
the hyperplane
b} separates
C and
hyperplane
{x | aT{x|a
x =T xb}=separates
C and
D D
strict separation
requires
additional
assumptions
C is closed,
ct separation
requires
additional
assumptions
(e.g., C(e.g.,
is closed,
D is aD
is
a
singleton)
gleton)
17/24
Supporting hyperplane theorem
Supporting hyperplane theorem
supporting hyperplane to set C at boundary point x0:
supporting hyperplane to set C at boundary point x0 :
{x{x|a
| aTTxx==aTaxT0x}0}
T
T
where
0 and
x ≤aTaxx0 for
for all
all xx∈∈CC
where
a 6=a06=and
aT ax ≤
0
a
C
x0
supporting
hyperplane
theorem:
if Cisisconvex,
convex,then
thenthere
there exists
exists aa
supporting
hyperplane
theorem:
if C
supporting hyperplane at every boundary point of C
supporting hyperplane at every boundary point of C
18/24
Generalized inequalities
a convex cone K ⊆ Rn is a proper cone if
K is closed (contains its boundary)
K is solid (has nonempty interior)
K is pointed (contains no line)
examples
nonnegative orthant K = Rn+ = {x ∈ Rn |xi ≥ 0, i = 1, ..., n}
positive semidefinite cone K = Sn+
nonnegative polynomials on [0, 1]:
K = {x ∈ Rn |x1 + x2 t + x3 t2 + ... + xn tn−1 ≥ 0 for t ∈ [0, 1]}
19/24
generalized inequality defined by a proper cone K :
x K y
⇐⇒
y − x ∈ K,
x ≺K y
⇐⇒
y − x ∈ int K
examples
componentwise inequality (K = Rn+ )
x Rn+ y
⇐⇒
xi ≤ yi ,
i = 1, ..., n
matrix inequality (K = Sn+ )
X Sn+ Y
⇐⇒
Y − X positive semidefinite
these two types are so common that we drop the subscript in K
properties: many properties of K are similar to ≤ on R, e.g.,
x K y,
u K v
=⇒
x + u K y + v
20/24
Dual cones and generalized inequalities
dual cone of a cone K :
K ∗ = {y|yT x ≥ 0 for all x ∈ K}
examples
K = Rn+ : K ∗ = Rn+
K = Sn+ : K ∗ = Sn+
K = {(x, t)| kxk2 ≤ t} : K ∗ = {(x, t)| kxk2 ≤ t}
K = {(x, t)| kxk1 ≤ t} : K ∗ = {(x, t)| kxk∞ ≤ t}
first three examples are self-dual cones
dual cones of proper cones are proper, hence define generalized
inequalities:
y K ∗ 0
⇐⇒
yT x ≥ 0 for all x K 0
21/24
nd minimal elements
Minimum and minimal elements
dering : we can have x 6K y and y 6K x
not in general a linear ordering : we can have x K y and y K x
K is
nt ofS
with
respect to K if
x ∈ S is the minimum element of S with respect to K if
=⇒ x K y
y ∈ S =⇒ x K y
∈ S respect
is a minimal
of S with respect to K if
f S xwith
to Kelement
if
K x
=⇒
y = x y ∈ S,
y K x
=⇒
y=x
example (K = R2+ )
S1
S1
S2
x2
x1 is the minimum element of S1
x2 is a minimal element of S2
x1
22/24
Minimum and minimal elements via dual inequalities
Minimum and minimal elements via du
minimum element w.r.t. K
Minimum and minimal elements via dual inequalities
minimum element w.r.t. K
minimum element w.r.t. K
x is minimum
element
of S iff for all
x is xminimum
element
is minimum
elementofofSSiffifffor
forall
all
∗
λ
≻
0,
x
is
the
unique
minimizerS
K
λ λK ∗≻0,
isxthe
unique
is the
uniqueminimizer
minimizer
K ∗x0,
T
T
T zλover
of λ z over S
z over
of λof
S S
x
x
minimal element w.r.t. K
minimal element w.r.t. K
minimal
element
K S for some λ ≻K ∗ 0, then x is minimal
• if x
minimizesw.r.t.
λT z over
T
λ1 λ z over S for some λ ≻K ∗ 0, then
• if x minimizes
if x minimizes λT z over S for some
λ K ∗ 0, then x is minimalx1
λ1
S
if x is a minimal element of a convex
x1λ2
S
x2
set S, then there exists a nonzero
λ2
λ K ∗ 0 such that x minimizes λT z
x2
• ifSx is a minimal element of a convex set S, then there exists a nonzero
over
λ
∗
0 such that x minimizes λT z over S
23/24
optimal production frontier
different production methods use different amounts of resources
n
fferent amounts
x ∈ Rn of resources x ∈ R
x for all production
possible production
methodsvectors x for all possible production
set P : resource
methods
correspond to resource vectors x
efficient (Pareto optimal) methods correspond to resource
vectors x that are minimal w.r.t. Rn+
fuel
P
example (n = 2)
x1 , x2 , x3 are efficient; x4 , x5 are not
x1
x2 x5 x4
λ
x3
labor
24/24