Convex sets
I
affine and convex sets
I
some important examples
I
operations that preserve convexity
I
separating and supporting hyperplanes
I
generalized inequalities
I
dual cones and generalized inequalities
IOE 611: Nonlinear Programming, Fall 2016
2. Convex sets
Affine set
Page 2–1
Affine set
line through
x1 , xx2, :x all
points
line through
: all points
1
2
x = ✓x +
✓)x
22
+ (1 −
θ)x
θx1(1
1
x=
PSfrag replacements
(✓∈2R)R)
(θ
x1
θ = 1.2
θ=1
θ = 0.6
x2
θ=0
θ = −0.2
affine set: contains the line through any two distinct points in the set
affine example:
set: contains
the line through any two distinct points in the
solution set of linear equations {x | Ax = b}
set
(conversely, every affine set can be expressed as solution set of system of
linear equations)
example: solution set of linear equations {x | Ax = b}
(conversely,
every affine set can be expressed as solution set of2–2
Convex sets
system of linear equations)
IOE 611: Nonlinear Programming, Fall 2016
2. Convex sets
Page 2–2
Convex set
Convex set
line segment between x1 and x2 : all points
line segment between x1 and x2: all points
x = ✓x1 + (1
with 0  ✓  1
✓)x2
x = θx1 + (1 − θ)x2
with 0 ≤ θ ≤ 1
convex set: contains line segment between any two points in the
set
convex set: contains line segment between any two points in the set
x1 ,xx12, x22 C
∈ ,C, 0 0≤✓θ
≤ 11
=) θx
✓x1 1++(1(1
2C
=⇒
− θ)x✓)x
2 ∈2 C
examples
(oneconvex,
convex,
nonconvex
examples (one
twotwo
nonconvex
sets) sets)
Convex sets
2–3
IOE 611: Nonlinear Programming, Fall 2016
2. Convex sets
Page 2–3
Convex combination and convex hull
Convex
combination
convex
combination
of x1and
,. . . ,convex
xk : any hull
point x of the form
convex combination of x1 ,. . . , xk : any point x of the form
x = θ1x1 + θ2x2 + · · · + θk xk
x = ✓ 1 x1 + ✓ 2 x 2 + · · · + ✓ k x k
with θ1 + · · · + θk = 1, θi ≥ 0
with ✓1 + · · · + ✓k = 1, ✓i 0
convex
hull conv
S: set
combinations
of points
in in
SS
convex
hull conv
S: of
setall
of convex
all convex
combinations
of points
Convex
IOE 611: sets
Nonlinear Programming, Fall 2016
2. Convex sets
Page 2–4
2–4
Convex cone
Convex cone
cone:
set C such that
for any xof2x1Cand
and
✓x of2the
C form
conica (nonnegative)
combination
x2:✓any 0,
point
conic (nonnegative) combination of x1 and x2 : any point of the
x = θ1x1 + θ2x2
form
x = ✓ 1 x1 + ✓ 2 x 2
with θ1 ≥ 0, θ2 ≥ 0
with ✓1
0, ✓2
0
PSfrag replacements
x1
x2
0
convexcone:
cone: asetcone
that that
contains
all conic combinations
points in the
convex
is convex,
i.e., set thatof contains
all set
conic combinations
Convex sets
{✓1 x1 + ✓2 x2 + · · · + ✓k xk | ✓i
0}
2–5
of points in the set
IOE 611: Nonlinear Programming, Fall 2016
2. Convex sets
Page 2–5
Examples of affine/convex sets and convex cones
Before we get to interesting examples, some trivial ones:
I
An empty set ;, a singleton {x0 } ⇢ Rn , the whole space Rn
are affine and convex sets in Rn
I
Any line is affine. If it passes through 0, it is also a convex
cone
I
A line segment is convex, but not affine (unless a singleton)
I
A ray (set {x0 + ✓v | ✓ 0}) is convex but not affine; if
x0 = 0, it is a convex cone
I
A subspace (a subset of Rn closed under sums and scalar
multiplications) is affine, convex, and is a convex cone
IOE 611: Nonlinear Programming, Fall 2016
2. Convex sets
Page 2–6
Hyperplanes and halfspaces
Hyperplanes and
halfspaces
Hyperplanes
and
halfspaces
T x = b} (a 6= 0)
hyperplane: set of the form {x
|
a
T
hyperplane:
the form
form {x
==
b} b}
(a ̸=
hyperplane:
setsetofofthe
{x| |aaxT x
(a0)̸= 0)
a
PSfrag replacements
PSfrag replacements
a
x0
x
x0
T
xa x = b
T
halfspace: set of the form {x | aT x ≤ b} (a ̸= 0) a x = b
T
halfspace:
set of the form {x | a x  b} (a 6= 0)
halfspace: set of the form {x | aT x ≤ b}a (a ̸= 0)
aT x a
≥b
x0
PSfrag replacements
T
aTxx ≤ ba x ≥ b
PSfrag replacements
0
• a is the normal vector
aT x ≤ b
• hyperplanes are affine and convex; halfspaces are convex
• a is the normal vector
Convex sets
2–6
• hyperplanes are affine and convex; halfspaces are convex
I
a is the normal vector
I sets
Convex
hyperplanes are affine and convex; halfspaces are convex
IOE 611: Nonlinear Programming, Fall 2016
2. Convex sets
2–6
Page 2–7
Euclidean balls and
ellipsoids
Euclidean
balls and ellipsoids
(Euclidean) ball with center xc and radius r :
(Euclidean) ball with center xc and radius r:
B(xc , r ) = {x | kx
xc k2  r } = {xc + ru | kuk2  1}
B(xc, r) = {x | ∥x − xc∥2 ≤ r} = {xc + ru | ∥u∥2 ≤ 1}
ellipsoid:
setofofthethe
form
ellipsoid: set
form
1
{x{x| |(x(x −xxcc))TT P
(x − xxcc) )≤1}1}
P −1(x
n
with
(i.e.,P Psymmetric
symmetric
positive
definite)
positive
definite)
with PP ∈2SSn++
++(i.e.,
PSfrag replacements
xc
other representation:
| ∥u∥2 ≤ 1} with A square and nonsingular
c + Au
other
representation:{x{x
c + Au | kuk2  1} with A square and
nonsingular
Convex sets
2–7
IOE 611: Nonlinear Programming, Fall 2016
2. Convex sets
Page 2–8
Norm balls and norm cones
Norm balls and norm cones
norm: a function k · k that satisfies
I
I
I
norm: a function ∥ · ∥ that satisfies
kxk
0; kxk = 0 if and only if x = 0
• ∥x∥ ≥ 0; ∥x∥ = 0 if and only if x = 0
ktxk = |t| · kxk for t 2 R
• ∥tx∥ = |t| ∥x∥ for t ∈ R
kx + y k  kxk + ky k
• ∥x + y∥ ≤ ∥x∥ + ∥y∥
notation: k · k is a general (unspecified) norm; k · ksymb is a
notation:
∥ · ∥ is general (unspecified) norm; ∥ · ∥symb is particular norm
particular
norm
PSfrag replacements
normnorm
ball ball
withwith
center
radius
: {x
kxxc∥ ≤
xcr}
k  r}
and radius
r:r {x
| ∥x| −
centerxcxcand
1
0.5
t
norm cone:
cone: {(x, t) |n+1
{(x, t)norm
| kxk
 t} ⇢ R ∥x∥ ≤ t}
Euclidean
norm
coneisiscalled
called secondEuclidean
norm
cone
order cone
second-order cone
0
1
1
0
0
x2
−1 −1
x1
conesare
areconvex
convex
norm norm
ballsballs
andand
cones
Convex sets
2–8
Polyhedra
IOE 611: Nonlinear Programming, Fall 2016
2. Convex sets
Page 2–9
solution
set of finitely many linear inequalities and equalities
Polyhedra
solution set of finitely many linear inequalities and equalities
Ax ≼ b,
Ax
Cx = d
b,
Cx = d
m⇥n
∈, RCp×n
≼ ,is componentwise
(A ∈ Rm×n
(A 2 ,RC
2 R, p⇥n
is componentwiseinequality)
inequality)
a1
a2
PSfrag replacements
P
a5
a3
a4
polyhedron
is intersection
of finite
number
halfspacesand
and hyperplanes
polyhedron
is intersection
of finite
number
of of
halfspaces
hyperplanes
IOE 611: Nonlinear Programming, Fall 2016
Convex sets
2. Convex sets
Page 2–10
2–9
Positive semidefinite cone
notation:
I
I
Positive
semidefinite cone
Sn is set of symmetric n ⇥
n matrices
notation:
Sn+ = {X 2
Sn | X ⌫ 0}: positive semidefinite n ⇥ n matrices
• Sn is set of symmetric n × n matrices
n
T
• SX
X ≽ 0}: zpositive
n ×zn matrices
2{X
Sn+∈ Sn |()
Xz semidefinite
0 for all
+ =
X ∈ Sn+ ⇐⇒ z T Xz ≥ 0 for all z
n
S+ is a convex
cone
n
S
is
+ n a convex cone
I Sn
X ∈ Sn0}:
positive definite n ⇥ n matrices
++ = {X •2SnS =| {X
| X ≻ 0}: positive definite n × n matrices
++
PSfrag replacements
1
x
y
y
2
example:
z
"
! 2
Sx+ y ∈ S2
+
y z
0.5
z
example:

0
1
1
0
y
0.5
x
−1 0
Convex sets
IOE 611: Nonlinear Programming, Fall 2016
2–10
2. Convex sets
Page 2–11
Operations that preserve convexity
practical methods for establishing convexity of a set C
1. apply definition
x 1 , x2 2 C ,
0✓1
=)
✓x1 + (1
✓)x2 2 C
2. show that C is obtained from simple convex sets
(hyperplanes, halfspaces, norm balls, . . . ) by operations that
preserve convexity
I
I
I
I
I
Cartesian product (S1 ⇥ S2 = {(x1 , x2 ) | x1 2 S1 , x2 2 S2 }; if
S1 and S2 are convex, so is S1 ⇥ S2 )
intersection
affine functions
perspective function
linear-fractional functions
IOE 611: Nonlinear Programming, Fall 2016
2. Convex sets
Page 2–12
Intersection
the intersection of (any number of) convex sets is convex
Intersection
the intersection of (any number of) convex sets is convex
example:
example:
S = {x ∈ Rm | |p(t)| ≤ 1 for |t| ≤ π/3}
S = {x 2 Rm | |px (t)|  1 for |t|  ⇡/3}
where p(t) = x1 cos t + x2 cos 2t + · · · + xm cos mt
where px (t) = x1 cos t + x2 cos 2t + · · · + xm cos mt
forfor
mm
==
2: 2:
PSfrag replacements
2
PSfrag replacements
1
0
x2
p(t)
1
−1
S
0
−1
π/3
0
S = {x 2 Rm |
Convext sets
t
−2
−2
π
2π/3
−1
x01
1
2
1  (cos t, . . . , cos mt)T x  1}; S = \|t|⇡/3 St .
IOE 611: Nonlinear Programming, Fall 2016
2. Convex sets
Page 2–13
Affine function
suppose f : Rn ! Rm is affine (f (x) = Ax + b with A 2 Rm⇥n ,
b 2 Rm )
I
the image of a convex set under f is convex
S ✓ Rn convex
I
I
=)
examples: scaling, translation, projection, sum of sets
the inverse image f
C ✓ Rm convex
I
I
f (S) = {f (x) | x 2 S} ✓ Rm convex
1 (C )
=)
of a convex set under f is convex
f
1
(C ) = {x 2 Rn | f (x) 2 C } convex
example: solution set of linear matrix inequality
{x | x1 A1 + · · · + xm Am B} (with Ai , B 2 Sp )
example: hyperbolic cone {x | x T Px  (c T x)2 , c T x
(with P 2 Sn+ )
IOE 611: Nonlinear Programming, Fall 2016
2. Convex sets
Page 2–14
0}
2–12
Perspective and linear-fractional function
perspective function P : Rn+1 ! Rn :
dom P = {(x, t) | t > 0}
P(x, t) = x/t,
images and inverse images of convex sets under perspective are
convex
linear-fractional function f : Rn ! Rm :
f (x) =
Ax + b
,
cT x + d
dom f = {x | c T x + d > 0}
images and inverse images of convex sets under linear-fractional
functions are convex
IOE 611: Nonlinear Programming, Fall 2016
2. Convex sets
Page 2–15
example of a linear-fractional function
example of a linear-fractional function
1
f (x) =
1x2 + 1 x
x
+
1
f (x) =
x
x1 + x2 + 1
1
1
x2
C
0
−1
−1
0
1
x1
IOE 611: Nonlinear Programming, Fall 2016
Convex sets
x2
PSfrag replacements
PSfrag replacements
2. Convex sets
0
−1
−1
f (C)
0
x1
1
Page 2–16
2–15
Separating hyperplane theorem
Separating hyperplane theorem
if C and D are disjoint convex sets, then there exists a 6= 0, b such
that if C and D are disjoint convex sets, then there exists a ̸= 0, b such that
aT x 
C , aT x ≥aTb for
x x ∈ bD for x 2 D
b forxx 2
∈ C,
aT xb≤for
aT x ≥ b
aT x ≤ b
PSfrag replacements
D
C
a
the hyperplane {x | aT x = b} separates C and D
the hyperplane {x | aT x = b} separates C and D
strict separation requires additional assumptions (e.g., C is closed, D is a
singleton)
strict separation (aT x < b for x 2 C ,
aT x > b for x 2 D)
requires additional assumptions (e.g., C is closed, D is a singleton;
see example 2.20)
Convex sets
IOE 611: Nonlinear Programming, Fall 2016
2–19
2. Convex sets
Page 2–17
Supporting
hyperplane theorem
Supporting hyperplane
theorem
supporting hyperplane
to to
set set
C atC boundary
point point
x 0 : x0 :
supporting
hyperplane
at boundary
T
{x
= aaTTxx00}}
{x| |aaT xx =
T
where
6=00and
andaTaxT x≤ 
0 for
aT ax0 xfor
all xall∈ xC 2 C
where aa ̸=
a
PSfrag replacements
C
x0
supporting hyperplane theorem: if C is convex, then there exists a
supporting
hyperplane
theorem:
C is of
convex,
then there
supporting hyperplane
at every
boundaryif point
C
exists a supporting hyperplane at every boundary point of C
Convex sets
IOE 611: Nonlinear Programming, Fall 2016
2–20
2. Convex sets
Page 2–18
Generalized inequalities
a convex cone K ✓ Rn is a proper cone if
I
K is closed (contains its boundary)
I
K is solid (has nonempty interior)
I
K is pointed (contains no line)
examples
I
I
I
nonnegative orthant
K = Rn+ = {x 2 Rn | xi
0, i = 1, . . . , n}
positive semidefinite cone K = Sn+
nonnegative polynomials of degree n
1 on [0, 1]:
K = {x 2 Rn | x1 + x2 t + x3 t 2 + · · · + xn t n
IOE 611: Nonlinear Programming, Fall 2016
2. Convex sets
1
0 for t 2 [0, 1]}
Page 2–19
Generalized inequalities
generalized inequality defined by a proper cone K :
x
K
y
()
x 2 K,
y
x
K
()
y
y
x 2 int K
examples
I componentwise inequality (K = Rn
+)
x
I
Rn+ y
()
xi  y i ,
i = 1, . . . , n
matrix inequality (K = Sn+ )
X
()
Sn+ Y
Y
X positive semidefinite
these two types are so common that we drop the subscript in K
properties: many properties of K are similar to  on R, e.g.,
x
K
IOE 611: Nonlinear Programming, Fall 2016
y,
u
K
v
2. Convex sets
=)
x +u
K
y +v
Page 2–20
Minimum and minimal elements
is not in general a linear ordering: we can have x 6
y6 K x
K
K
Minimum
minimal
elements
x 2 S is the minimum
elementand
of S
with respect
to
y and
if
K
≼
is not in general a linear ordering : we can have x ̸≼K y and y ̸≼K x
=) x K y (equivalently, S ✓ x + K ).
x ∈ S is the minimum element of S with respect to ≼K if
y 2KS
y∈S
=⇒
x≼
y
K
x 2 S is a minimal element of S with respect
to
y 2 S,
K
if
x
with respect to ≼(x
K if K )\S
y ∈ SKisxa minimal
=) element
y = xof S(equivalently,
y ∈ S,
y ≼K x
2
2
example example
(K = R(K
+ )= R+)PSfrag replacements
x1 is the minimum element of S1
x1 is the minimum element of S1
x2 is a minimal
element of S2 S
x is a minimal element of
2
IOE 611: Nonlinear Programming,
ConvexFall
sets2016
2
=⇒
= {x}).
y=x
S2
x2
S1
x1
2. Convex sets
Page 2–21
2–18
optimal production frontier
I
I
I
di↵erent production methods use di↵erent amounts of
resources x 2 Rn
optimal production frontier
production set P: resource vectors x for all possible
• different production methods use different amounts of resources x ∈ Rn
production
methods
• production set P : resource vectors x for all possible production methods
efficient (Pareto optimal) methods correspond to resource
• efficient
(Pareto
methods
to resource vectors x
vectors
x that
areoptimal)
minimal
w.r.t.correspond
Rn+
n
that are minimal w.r.t. R+
PSfrag replacements fuel
example (n = 2)
x1, (n
x2 , =
x3 2)
are efficient; x4, x5 are not
example
x1 , x2 , x3 are efficient; x4 , x5 are not
P
x1
x2 x5 x4
λ
x3
labor
IOE 611: Nonlinear Programming,
Convex setsFall 2016
2. Convex sets
Page 2–22
2–23
Dual cones and generalized inequalities
dual cone of a cone K :
K ⇤ = {y | y T x
0 for all x 2 K }
examples
I
K = Rn+ : K ⇤ = Rn+
I
K = Sn+ : K ⇤ = Sn+
I
K = {(x, t) | kxk2  t}: K ⇤ = {(y , s) | ky k2  s}
I
K = {(x, t) | kxk1  t}: K ⇤ = {(y , s) | ky k1  s}
first three examples are self-dual cones
dual cones of proper cones are proper, hence define generalized
inequalities:
y ⌫K ⇤ 0
IOE 611: Nonlinear Programming, Fall 2016
()
yT x
0 for all x ⌫K 0
2. Convex sets
Page 2–23
Minimum and
minimal
elements
via dual
inequalities
Minimum
and minimal
elements
via dual
inequalities
minimum element w.r.t. ≼K
minimum
element
Minimum
and w.r.t.
minimalKelements via dual inequalities
x
is
minimum
element
for all
x is minimum element of ofS Si↵iff for
λ ≻K ∗ 0, x is the unique minimizer
S
w.r.t.
≼K
all minimum
xoveris Sthe
unique
K ⇤λT0,zelement
of
T z over
x is minimum
of S iffPSfrag
for allreplacements
minimizer
of element
x
λ ≻K ∗ 0, x is the unique minimizer
of λTminimal
z over S element w.r.t. ≽K
S
PSfragKreplacements
minimal• ifelement
w.r.t.
x minimizes λT z over
S for some λ ≻Kx∗ 0, then x is minimal
T
I if x minimizes
z
over
Sλ1for some
K ⇤ 0, then x is
minimal element
≽K
PSfragw.r.t.
replacements
minimal
T
• if x minimizes λ z over S forxsome
λ ≻K ∗ 0, then x is minimal
1
S
λ1
PSfrag replacements
x1
x2
λ2
S
2
• if x is a minimal element ofxa2 convex setλS,
then there exists a nonzero
T
λ ≽K ∗ 0 such that x minimizes λ z over S
I
• if Convex
x is asetsminimal element of a convex set S, then there exists a nonzero 2–22
ifλ x≽ is∗ 0a such
minimal
element of a convex set S, then there exists
that x minimizes λT z over S
K
a nonzero
Convex sets
IOE 611: Nonlinear Programming, Fall 2016
⌫K ⇤ 0 such that x minimizes
2. Convex sets
Tz
over S
Page 2–24
2–22
A theorem of alternative for linear systems
Theorem Given A 2 Rm⇥n and b 2 Rm , exactly one of
(i) Ax < b and (ii) AT
= 0, b T
 0,
0,
6= 0
has a solution.
Proof
I
Easy to see (i) and (ii) can’t both have solutions.
I
If (i) does not have a solution, C = {b Ax | x 2 Rn } and
D = {y 2 Rm
++ } are convex disjoint sets and can be separated.
IOE 611: Nonlinear Programming, Fall 2016
2. Convex sets
Page 2–25