Decision Optimization
IBM CPLEX
Global Non-Convex MIQP
Christian Bliek & Pierre Bonami
Decision Optimization
Disclaimer
IBM’s statements regarding its plans, directions, and intent are subject to
change or withdrawal without notice at IBM’s sole discretion. Information
regarding potential future products is intended to outline our general product
direction and it should not be relied on in making a purchasing decision. The
information mentioned regarding potential future products is not a
commitment, promise, or legal obligation to deliver any material, code or
functionality. Information about potential future products may not be
incorporated into any contract. The development, release, and timing of any
future features or functionality described for our products remains at our sole
discretion.
Performance is based on measurements and projections using standard IBM®
benchmarks in a controlled environment. The actual throughput or
performance that any user will experience will vary depending upon many
factors, including considerations such as the amount of multiprogramming in
the user's job stream, the I/O configuration, the storage configuration, and the
workload processed. Therefore, no assurance can be given that an individual
user will achieve results similar to those stated here.
© 2013 IBM Corporation
Global Non-Convex MIQP
Quadratic Program (QP)
Standard form
1
Min x' Qx + c' x
2
Ax = b
x ≥ 0
Convex or Positive Semi-Definite
x' Qx ≥ 0
Indefinite
3
any Q
© 2013 IBM Corporation
Global Non-Convex MIQP
Non-Convex QP
Local optimum
Available since IBM CPLEX 12.3
Interior Point Algorithm
Solution target Parameter FIRSTORDER
4
© 2013 IBM Corporation
Global Non-Convex MIQP
Local Non-Convex QP Benchmark
Performance Cplex versus Ipopt with Wsmp
1,6
1,4
relative time
1,2
1
time
0,8
iterations
0,6
0,4
0,2
0
[0,1)
[1,10)
[10,100)
[100,1k)
[1k,10k)
problem time
5
© 2013 IBM Corporation
Global Non-Convex MIQP
Non-Convex MIQP
Global optimum
NEW in CPLEX 12.6
Branch and Bound
6
© 2013 IBM Corporation
Global Non-Convex MIQP
Example
Min xy
− 2 ≤ x ≤ 1
−1 ≤ y ≤ 1
y
Local Optimum
Global Optimum
7
x
© 2013 IBM Corporation
Global Non-Convex MIQP
Global Non-Convex QP
Even if Q has only 1 negative eigenvalue,
Non-Convex QP is NP-hard
Checking if a feasible solution is not a local minimum
is NP-complete
Checking if a Non-Convex QP is unbounded is NPcomplete
8
© 2013 IBM Corporation
Global Non-Convex MIQP
Overview
We consider 2 formulations
1. Original
2. Factorized Eigenvalue
9
© 2013 IBM Corporation
Global Non-Convex MIQP
Factorized Eigenvalue Formulation
1
Min x' Qx + c' x
2
Ax = b
x≥0
10
© 2013 IBM Corporation
Global Non-Convex MIQP
Factorized Eigenvalue Formulation
1
Min x' Qx + c' x
2
Ax = b
x≥0
1
Min y' By + c' x
2
Ax = b
L' x = y
x≥0
Q = LBL '
11
© 2013 IBM Corporation
Global Non-Convex MIQP
Factorized Eigenvalue Formulation
1
Min x' Qx + c' x
2
Ax = b
x≥0
12
1
Min y' By + c' x
2
Ax = b
L' x = y
x≥0
© 2013 IBM Corporation
Global Non-Convex MIQP
Factorized Eigenvalue Formulation
1
Min x' Qx + c' x
2
Ax = b
x≥0
1
Min y' By + c' x
2
Ax = b
L' x = y
x≥0
1
Min z ' Λz + c' x
2
Ax = b
L' x = y
Φ' y = z
x≥0
B = ΦΛΦ '
13
© 2013 IBM Corporation
Global Non-Convex MIQP
Factorized Eigenvalue Formulation
1
Min x' Qx + c' x
2
Ax = b
x≥0
14
1
Min y' By + c' x
2
Ax = b
L' x = y
x≥0
1
Min z ' Λz + c' x
2
Ax = b
L' x = y
Φ' y = z
x≥0
© 2013 IBM Corporation
Global Non-Convex MIQP
Factorized Eigenvalue Formulation
1
Min x' Qx + c' x
2
Ax = b
x≥0
1
Min y' By + c' x
2
Ax = b
L' x = y
x≥0
1
Min z ' Λz + c' x
2
Ax = b
L' x = y
Φ' y = z
x≥0
Advantage
– Sparse
– Efficient
– Proper identification of negative eigenvalues
15
© 2013 IBM Corporation
Global Non-Convex MIQP
Example
1. Original Formulation
Min xy
− 2 ≤ x ≤ 1
−1 ≤ y ≤ 1
2. Factorized Eigenvalue Formulation
0

Q =
 1
1 = 1
0 
2
1 2 2
Min u − v
2
(
16
x
x
−
−
+ y
− y
2 ≤
1 ≤
1
 1
1  1
− 1   0
0  1
− 1   1
1 
− 1 
)
=
2u
=
2v
x ≤ 1
y ≤ 1
© 2013 IBM Corporation
Global Non-Convex MIQP
Overview
We consider 2 formulations
1. Original
2. Factorized Eigenvalue
17
© 2013 IBM Corporation
Global Non-Convex MIQP
Overview
We consider 2 formulations
1. Original
2. Factorized Eigenvalue
Automatically select most promising one
18
© 2013 IBM Corporation
Global Non-Convex MIQP
Overview
We consider 2 formulations
1. Original
2. Factorized Eigenvalue
Automatically select most promising one
Do Term by Term McCormick Relaxation
19
© 2013 IBM Corporation
Global Non-Convex MIQP
Relaxation of Non-Convex MIQP
Min
1

q
x
x
+
q
x
x
 ∑ ij i j ∑ ij i j  + c ' x
2 P
N

Ax = b
x ≥ 0
1

Min  ∑ q ij x i x j + ∑ q ij z ij  + c ' x
2 P
N

Ax = b
z ij ≥ q ij x i x j
x ≥ 0
20
© 2013 IBM Corporation
Global Non-Convex MIQP
Relaxation of Non-Convex MIQP
Relaxation of individual Non-Convex quadratic
terms using McCormick envelopes
21
© 2013 IBM Corporation
Global Non-Convex MIQP
Overview
We consider 2 formulations
1. Original
2. Factorized Eigenvalue
Automatically select most promising one
Do Term by Term McCormick Relaxation
22
© 2013 IBM Corporation
Global Non-Convex MIQP
Overview
We consider 2 formulations
1. Original
2. Factorized Eigenvalue
Automatically select most promising one
Do Term by Term McCormick Relaxation
Branch and Bound
23
© 2013 IBM Corporation
Global Non-Convex MIQP
Branching for Non-Convex MIQP
Branch on continuous variables and update
envelopes
24
© 2013 IBM Corporation
Global Non-Convex MIQP
Other Ingredients
QP simplex for convex QP relaxation
Pseudocost branching
Local interior point solver for incumbents
Bound strengthening
Detection of unboundedness
Linearize quadratic terms involving binaries
25
© 2013 IBM Corporation
Global Non-Convex MIQP
Global Non-Convex QP Benchmark
internal non-convex miqp testset
globallib GAMS
minlp.org
boxqp
From miqp testset generated 50% mixed miqp set
Comparison with SCIP and Couenne on 1 thread
26
© 2013 IBM Corporation
Global Non-Convex MIQP
Global Non-Convex QP Benchmark
CPLEX versus SCIP on individual testsets
at most one timeout
no timeouts
0,35
1,2
0,3
1
binary
0,2
50% binary
0,15
continuous and integer
0,1
0,8
binary
0,6
50% binary
continuous and integer
0,4
0,2
0,05
0
0
[0,10k]
[1,10k]
[10,10k] [100,10k] 1k,10k]
problem time
27
relative time
relative time
0,25
[0,1)
[1,10)
[10,100)
[100,1k)
[1k,10k)
problem time
© 2013 IBM Corporation
Global Non-Convex MIQP
Global Non-Convex QP Benchmark
CPLEX versus SCIP and Couenne on combined testset
at most one timeout
no timeouts
0,35
1,2
0,3
1
0,2
scip
couenne
0,15
0,1
0,8
scip
0,6
couenne
0,4
0,2
0,05
0
0
[0,10k]
[1,10k]
[10,10k]
problem time
28
relative time
relative time
0,25
[100,10k]
1k,10k]
[0,1)
[1,10)
[10,100)
[100,1k)
[1k,10k)
problem time
© 2013 IBM Corporation
Global Non-Convex MIQP
Global Non-Convex QP Benchmark
CPLEX versus SCIP and Couenne on combined testset
no timeouts
at most one timeout
1,6
0,7
1,4
0,6
r e l a ti v e n o d e s
0,4
scip
couenne
0,3
r e l a ti v e n o d e s
1,2
0,5
1
scip
0,8
couenne
0,6
0,2
0,4
0,1
0,2
0
0
[0,10k]
[1,10k]
[10,10k]
problem time
29
[100,10k]
1k,10k]
[0,1)
[1,10)
[10,100)
[100,1k)
[1k,10k)
problem time
© 2013 IBM Corporation
Global Non-Convex MIQP
Global Non-Convex QP Benchmark
CPLEX 1 versus 4 threads on combined testset
no timeouts
at most one timeout
1,2
0,9
0,8
1
0,6
0,5
4thread
0,4
0,3
relative time
relative time
0,7
0,8
0,6
4thread
0,4
0,2
0,2
0,1
0
0
[0,10k]
[1,10k]
[10,10k]
problem time
30
[100,10k]
1k,10k]
[0,1)
[1,10)
[10,100)
[100,1k)
[1k,10k)
problem time
© 2013 IBM Corporation
Global Non-Convex MIQP
How to use it
Available in CPLEX 12.6
By default Non-Convex MIQP are not accepted
Set Solution Target Parameter to OPTIMALGLOBAL
31
© 2013 IBM Corporation
Global Non-Convex MIQP
Global Non-Convex QP Benchmark
CPLEX versus SCIP and Couenne on combined testset
at most one timeout
no timeouts
0,35
1,2
0,3
1
0,2
scip
couenne
0,15
0,1
0,8
scip
0,6
couenne
0,4
0,2
0,05
0
0
[0,10k]
[1,10k]
[10,10k]
problem time
32
relative time
relative time
0,25
[100,10k]
1k,10k]
[0,1)
[1,10)
[10,100)
[100,1k)
[1k,10k)
problem time
© 2013 IBM Corporation