CS/ECE/ISyE 524
Introduction to Optimization
Spring 2016–17
12. Cones and semidefinite constraints
ˆ Geometry of cones
ˆ Second order cone programs
ˆ Example: robust linear program
ˆ Semidefinite constraints
Laurent Lessard (www.laurentlessard.com)
What is a cone?
ˆ A set of points C ∈ Rn is called a cone if it satisfies:
I
αx ∈ C whenever x ∈ C and α > 0.
I
x + y ∈ C whenever x ∈ C and y ∈ C .
ˆ Similar to a subspace, but α > 0 instead of α ∈ R.
(this is a critical difference!)
ˆ Simple examples: |x| ≤ y and y ≥ 0
-1.5 -1.0 -0.5
1.5
1.5
1.0
1.0
0.5
0.5
0.5
1.0
1.5
-1.5 -1.0 -0.5
0.5
1.0
1.5
12-2
What is a cone?
ˆ A slice of a cone is its intersection with a subspace.
ˆ We are interested in convex cones (all slices are convex).
ˆ Can be polyhedral, ellipsoidal, or something else...
12-3
What is a cone?
Polyhedral cone recipe:
1. Begin with your favorite polyhedron Ax ≤ b where x ∈ Rn
2. {Ax ≤ bt, t ≥ 0} is a polyhedral cone in (x, t) ∈ Rn+1
3. The slice t = 1 is the original polyhedron.
2.5
2.0
1.5
1.0
0.5
0.5
1.0
1.5
2.0
2.5
12-4
What is a cone?
Ellipsoidal cone recipe:
1. Ellipsoid x T Px + q T x + r ≤ 0 where P 0 and x ∈ Rn
2. Complete the square ⇐⇒ kAx + bk ≤ c
3. {kAx + btk ≤ ct} is an ellipsoidal cone in (x, t) ∈ Rn+1
4. The slice t = 1 is the original ellipsoid.
2.5
2.0
1.5
1.0
0.5
0.5
1.0
1.5
2.0
2.5
12-5
Second-order cone
A second-order cone is the set of points x ∈ Rn satisfying
kAx + bk ≤ c T x + d
Special cases:
ˆ If A = 0, we have a linear constraint (hyperplane)
ˆ If c = 0, can square both sides, (ellipsoid)
In general, you cannot just square both sides!
12-6
Second-order cone
A second-order cone is the set of points x ∈ Rn satisfying
kAx − bk ≤ c T x + d
Counterexample:
0
If A = 1 0 and c =
and b = d = 0, we have:
1
|x| ≤ y
Square both sides, we get a nonconvex quadratic constraint!
x2 − y2 ≤ 0
12-7
Special case: rotated second-order cone
A rotated second-order cone is the set x ∈ Rn , y , z ∈ R:
x T x ≤ yz,
y ≥ 0,
z ≥0
With n = 1, this looks like:
12-8
Special case: rotated second-order cone
A rotated second-order cone is the set x ∈ Rn , y , z ∈ R:
x T x ≤ yz,
y ≥ 0,
z ≥0
Can put into standard form:
4x T x ≤ 4yz
4x T x + y 2 + z 2 ≤ 4yz + y 2 + z 2
4x T x + (y − z)2 ≤ (y + z)2
p
4x T x + (y − z)2 ≤ y + z
2x y −z ≤y +z
12-9
SOCPs
A second-order cone program (SOCP) has the form:
minimize c T x
x
subject to: kAi x + bi k ≤ ciT x + di
for i = 1, . . . , m
ˆ Every LP is an SOCP (just make each Ai = 0)
ˆ Every convex QP and QCQP is an SOCP
I
convert quadratic cost to epigraph form (add a variable)
I
convert quadratic constraints to SOCP (complete square)
12-10
Implementation details
A second-order cone program (SOCP) has the form:
minimize c T x
x
subject to: kAi x + bi k ≤ ciT x + di
for i = 1, . . . , m
ˆ In JuMP, you can specify SOCP using:
@constraint(m, norm(A*x+b) <= dot(c,x)+d)
works with ECOS, SCS, Mosek, Gurobi, Ipopt.
ˆ Can also specify rotated cones directly in Mosek, Ipopt.
12-11
Example: robust LP
Consider a linear program with each linear constraint
separately written out:
maximize c T x
x
subject to: aiT x ≤ bi
for i = 1, . . . , m
Suppose there is uncertainty in some of the ai vectors. Say
for example that ai = āi + ρu where āi is a nominal value and
u is the uncertainty.
ˆ box constraint: kuk∞ ≤ 1
ˆ ball constraints: kuk2 ≤ 1
12-12
Example: robust LP
Substituting ai = āi + ρu into aiT x ≤ bi , obtain:
āiT x + ρu T x ≤ bi
for all uncertain u
box constraint:
If this must hold for all u with kuk∞ ≤ 1, then it holds for the
worst-case u. Therefore:
T
u x=
n
X
i=1
u i xi ≤
n
X
|ui ||xi | ≤
n
X
i=1
|xi | = kxk1
i=1
Then we have
āiT x + ρkxk1 ≤ bi
12-13
Robust LP with box constraint
With a box constraint ai = āi + ρu with kuk∞ ≤ 1
maximize c T x
x
subject to: aiT x ≤ bi
for i = 1, . . . , m
Is equivalent to the optimization problem
maximize c T x
x
subject to: āiT x + ρkxk1 ≤ bi
for i = 1, . . . , m
12-14
Robust LP with box constraint
With a box constraint ai = āi + ρu with kuk∞ ≤ 1
maximize c T x
x
subject to: aiT x ≤ bi
for i = 1, . . . , m
... which is equivalent to the linear program:
maximize c T x
x,t
subject to:
āiT x
+ρ
n
X
tj ≤ bi
for i = 1, . . . , m
j=1
− tj ≤ xj ≤ tj
for j = 1, . . . , n
12-15
Example: robust LP
Substituting ai = āi + ρu into aiT x ≤ bi , obtain:
āiT x + ρu T x ≤ bi
for all uncertain u
ball constraint:
If this must hold for all u with kuk2 ≤ 1, then it holds for the
worst-case u. Using Cauchy-Schwarz inequality:
u T x ≤ kuk2 kxk2 ≤ kxk2
Then we have
āiT x + ρkxk2 ≤ bi
(a second-order cone constraint!)
12-16
Robust LP with ball constraint
With a ball constraint ai = āi + ρu with kuk2 ≤ 1
maximize c T x
x
subject to: aiT x ≤ bi
for i = 1, . . . , m
Is equivalent to the optimization problem
maximize c T x
x
subject to: āiT x + ρkxk2 ≤ bi
for i = 1, . . . , m
which is an SOCP
12-17
Ball constraint example
1.5
1.5
1.0
1.0
0.5
0.5
-1.5 -1.0 -0.5
-0.5
0.5 1.0 1.5 2.0
-1.5 -1.0 -0.5
-0.5
-1.0
-1.0
-1.5
-1.5
aiT x ≤ bi
0.5 1.0 1.5 2.0
aiT x + 0.2kxk2 ≤ bi
ˆ New region is smaller, no longer a polyhedron
ˆ More robust to uncertain constraints
12-18
Matrix variables
Sometimes, the decision variable is a matrix X .
ˆ Can always just think of X ∈ Rm×n as x ∈ Rmn .
ˆ Linear functions:
mn
X
ck x k = c T x
k=1
m
n
XX
Cij Xij = trace(C T X )
i=1 j=1
ˆ Linear program:
maximize
X
subject to:
trace(C T X )
trace(ATi X ) ≤ bi
for i = 1, . . . , k
12-19
Matrix variables
If a decision variable is a symmetric matrix X = X T ∈ Rn×n ,
we can represent it as a vector x ∈ Rn(n+1)/2 .
 
x1
x2 


 
x1 x2 x3
x3 
 
x2 x4 x5 
⇐⇒
x4 
 
x3 x5 x6
x5 
x6
The constraint X 0 is called a semidefinite constraint.
What does it look like geometrically?
12-20
The PSD cone
The set of matrices X 0 are a convex cone in Rn(n+1)/2
x y
Example: The set
0 of points in R3 satisfy:
y z
xz ≥ y 2 ,
x ≥ 0,
z ≥0
This is a rotated second-order cone! Equivalent to:
2y x −z ≤x +z
12-21
More complicated example
"
The set of (x, y , z) satisfying
X ∈ R3×3 ,
X 0,
1
x
y
x
1
z
X11 = 1,
y
z
1
#
0 is the solution of:
X22 = 1,
X33 = 1
12-22
Spectrahedra
ˆ Two common set representations:
I
variables x1 , . . . , xk , constants Qi = QiT , and constraint:
Q0 + x1 Q1 + . . . xk Qk 0
I
(linear matrix inequality)
variable X 0 and the constraints:
trace(AT
i X ) ≤ bi
(linear constraint form)
ˆ These sets are called spectrahedra.
ˆ Very rich set, lots of possible shapes.
12-23
Semidefinite program (SDP)
Standard form #1: (looks like the standard form for an LP)
maximize
X
subject to:
trace C T X
trace(ATi X ) ≤ bi
X 0
for i = 1, . . . , m
Standard form #2:
maximize c T x
x
subject to: Q0 +
m
X
xi Q i 0
i=1
12-24
Relationship with other programs
Every LP is an SDP:
a11 a12 x1
b
≤ 1
a21 a22 x2
b2
is the same as:
a11 0
a12 0
b1 0
x1
+ x2
0 a21
0 a22
0 b2
(polyhedra are special cases of spectrahedra)
12-25
Relationship with other programs
Every SOCP is an SDP:
kAx + bk ≤ c T x + d
is the same as:
T
(c x + d)I Ax + b
0
(Ax + b)T c T x + d
This isn’t obvious — proof requires use of Schur complement.
(second-order cones are special cases of spectrahedra)
12-26