Solving Linear and Integer
Programs: Time is Money
Robert E. Bixby
1
War of 1812
December 24, 1814: Treaty of Ghent – “ending war”
January 8, 1815: 5300 British troops attack New Orleans
2
War of 1812
December 24, 1814: Treaty of Ghent – “ending the war”
January 8, 1815: 5300 British troops attack New Orleans
Ghent
Boston
New Orleans
Total travel time Ghent-New Orleans:
Approximately 3 weeks
1865: Transatlantic telegraph cable completed –
Communication time reduced from weeks to seconds
3
Linear Programming
T George Dantzig, 1947
T
Introduces LP and recognized it as more than a conceptual
tool: Computing answer important.
T
Invented “primal” simplex algorithm.
T
First LP solved: Laderman, 9 cons., 77 vars., 120 MANDAYS.
T What is the single most important event in LP since
Dantzig?
4
Linear Programming
T George Dantzig, 1947
T
Introduces LP and recognized it as more than a conceptual
tool: Computing answer important.
T
Invented “primal” simplex algorithm.
T
First LP solved: Laderman, 9 cons., 77 vars., 120 MANDAYS.
T What is the single most important event in LP since
Dantzig? We now (since ~1990) have 3 algorithms to
solve LPs
5
T
Primal Simplex Algorithm (Dantzig 1947)
T
Dual Simplex Algorithm (Lemke, 1954)
T
Barrier Algorithm (Karmarkar, 1984)
LP
PDS Models
“Patient Distribution System”: Carolan, Hill, Kennington, Niemi, Wichmann,
An empirical evaluation of the KORBX algorithms for military airlift
applications, Operations Research 38 (1990), pp. 240-248
MODEL
ROWS
pds02
2953
pds06
9881
pds10 16558
pds20 33874
pds30 49944
pds40 66844
pds50 83060
pds60 99431
pds70 114944
6
CPLEX1.0
1988
0.4
26.4
208.9
5268.8
15891.9
58920.3
122195.9
205798.3
335292.1
CPLEX5.0
1997
0.1
2.4
13.0
232.6
1154.9
2816.8
8510.9
7442.6
21120.4
Primal
Simplex
Dual
Simplex
CPLEX8.0
2002
0.1
0.9
2.6
20.9
39.1
79.3
114.6
160.5
197.8
Dual
Simplex
SPEEDUP
1.0Æ8.0
4.0
29.3
80.3
247.3
406.4
743.0
1066.3
1282.2
1695.1
LP
Not just faster -- Growth with size:
Quadratic then & Linear now !
400000.00
250.00
350000.00
CPLEX 1.0 seconds
250000.00
150.00
200000.00
100.00
150000.00
100000.00
50.00
50000.00
.00
.00
.00
10.00
20.00
30.00
40.00
50.00
60.00
# Time Periods: PDS02 -- PDS70
7
70.00
80.00
CPLEX 8.0 seconds
200.00
300000.00
LP
BIG TEST: The testing methodology
T Not possible for one test to cover 10+ years:
Combined several tests.
T The biggest single test:
T
T
Assembled 680 real LPs (up to 7 million consts.)
Test runs: Using a time limit (4 days per LP) two
chosen methods would be compared as follows:
P Run method 1: Generate 680 solve times
P Run method 2: Generate 680 solve times
P Compute 680 ratios and form GEOMETRIC MEAN
(not arithmetic mean!)
The same methodology was applied throughout.
8
LP
Progress: 1988 – Present
T Algorithms
T
Best simplex
T
Best simplex + barrier
960x
2360x
T Machines
9
T
Simplex algorithms
800x
T
Barrier algorithms
13000x
LP
Algorithm comparison: Extracted
from the previous results …
10
T Dual simplex vs. primal:
Dual 2x faster
T Best simplex vs. barrier:
About even
T Best of three vs. primal:
Best 7.5x faster
Mixed-Integer
Programming
11
MIP
A mixed-integer program (MIP) is
an optimization problem of the form
cT x
Ax = b
l≤ x≤u
some or all x j integer
Maximize
Subject to
12
Branch-and-Cut
Solve LP relaxation:
v=3.5 (fractional)
Root
3
z≤
2
4
1
x≤
≥
v≥
z≥
x
3
0
v≤
Upper Bound
G
A
P
Lower Bound
1
y≤
y≥
0
Integer
z≥
z≤
0
Infeas
1
Integer
Remarks:
(1) GAP = 0 ⇒ Proof of optimality
(2) In practice: Often good enough to have good Solution
13
MIP
Example 1: LP still can be HARD
SGM: Schedule Generation Model
157323 rows, 182812 columns, 6348437 nzs
T LP relaxation at root node:
T
Barrier: Solve time estimate 3-6 days.
T
Primal steepest edge: 18 hours
T Branch-and-cut
T
368 nodes enumerated, infeasibility reduced by 3x.
T
Time: 2 weeks.
T Does not appear to be difficult.
Likely solvable
to optimality – with only a 1000x improvement
in LP speed!
14
MIP
Example 2: MIP really is HARD
A customer model: 44 cons, 51 vars, 167 nzs, maximization
51 general integer variables (and no bounds)
Branch-and-cut: Initial integer solution -2186.0
Initial upper bound
-1379.4
…after 1.4 days, 32,000,000 B&C nodes, 5.5 Gig tree
Integer solution and bound: UNCHANGED
What’s wrong? Bad modeling. Free GIs
chase each other off to infinity.
15
MIP
Example 3: A typical situation today –
Supply-chain scheduling
T
T
T
16
Model description:
T
Weekly model, daily buckets: Objective to minimize end-ofday inventory.
T
Production (single facility), inventory, shipping (trucks),
wholesalers (demand known)
Initial modeling phase
T
Simplified prototype + complicating constraints (production
run grouping req’t, min truck constraints)
T
RESULT: Couldn’t get good feasible solutions.
Decomposition approach
T
Talk to manual schedulers: They first decide on “producibles”
schedule. Simulate using Constraint Programming.
T
Fixed model: Fix variables and run MIP
MIP
Supply-chain scheduling (continued):
Solving the fixed model
CPLEX 5.0:
Integer optimal solution (0.0001/0): Objective =
1.5091900536e+05
Current MIP best bound =
1.5090391809e+05 (gap = 15.0873)
Solution time = 3465.73 sec. Iterations = 7885711 Nodes = 489870 (2268)
CPLEX 8.0:
Implied bound cuts applied:
Flow cuts applied: 200
55
Integer optimal solution (0.0001/1e-06): Objective =
1.5091904146e+05
Current MIP best bound =
1.5090843265e+05 (gap = 10.6088, 0.01%)
Solution time =
1.53 sec. Iterations = 3187 Nodes = 58 (2)
17
Original model: Now solves to optimality
in 2 hours (20% improvement in solution
quality)
MIP
Computational History:
1950 –1998
T
1954 Dantzig, Fulkerson, S.
Johnson: 42 city TSP
T
T
1957 Gomory
T
T
Cutting plane algorithm: A
complete solution
1960 Land, Doig, 1965 Dakin
T
18
Solved to optimality using
cutting planes and LP
Branch-and-bound (B&B)
T
1971 MPSX/370, Benichou
et al.
T
1972 UMPIRE, Forrest,
Hirst, Tomlin (Beale)
T
1972 – 1998 Good B&B
remained the state-of-theart in commercial codes, in
spite of
T
T
T
T
T
1973 Padberg
1974 Balas (disjunctive
programming)
1983 Crowder, Johnson,
Padberg: PIPX, pure 0/1 MIP
1987 Van Roy and Wolsey:
MPSARX, mixed 0/1 MIP
Grötschel, Padberg, Rinaldi
…TSP (120, 666, 2392 city
models solved)
MIP
1998… A new generation of MIP codes
T Linear programming
T
Stable, robust dual simplex
T Variable/node selection
T
T
ideas
T
Influenced by traveling
salesman problem
T Primal heuristics
T
T Presolve – numerous small
8 different tried at root
Retried based upon success
T Cutting planes
T
T Node presolve
T
19
Fast, incremental bound
strengthening (very similar
to Constraint Programming)
Probing in constraints:
∑ xj ≤ (∑ uj) y, y = 0/1
Î xj ≤ ujy (for all j)
T
Gomory, knapsack covers,
flow covers, mix-integer
rounding, cliques, GUB
covers, implied bounds,
path cuts, disjunctive cuts
Various extensions
P Aggregation
MIP
The BEST of the cuts:
A view of Gomory Mixed Cuts
(Nemhauser & Wolsey)
20
MIP
y
x + y ≥ 3.5, x,y ≥ 0, y integral
y ≥ 3.5
4.0
(0, 3.5)
3.0
2.0
y ≤ 3.5
1.0
21
1.0
2.0
3.0
4.0
x
MIP
y
x + y ≥ 3.5, x,y ≥ 0, y integral
y ≥ 3.5
4.0
General formula: x + y ≥ b ⇒ x/f + y ≥ b
3.0
CUT:
2x + y ≥ 4
2.0
y ≤ 3.5
1.0
22
1.0
2.0
3.0
4.0
x
MIP
Computational Results I: 964 models
(30 hour time limit)
Finding Feasible Solutions
Solving to Optimality
100%
100%
5%
8%
80%
80%
60%
11%
98% (19 MIPs)
60%
40%
40%
74%
74%
56%
20%
20%
0%
0%
CPLEX 5.0
CPLEX 8.0
CPLEX 8.0
Setting: "MIP emphasis feasibility"
Integer Solution with > 10% Gap
Integer Solution with < 10% Gap
23
Solved to provable optimality
MIP
Computational Results II: 651 models
(all solvable to optimality)
T Ran for 30 hours using defaults
T Relative speedups:
24
T
All models
(651):
12x
T
CPLEX 5.0 > 1 second
(447):
41x
T
CPLEX 5.0 > 10 seconds
(362):
87x
T
CPLEX 5.0 > 100 seconds
(281): 171x
MIP
Computational Results III: 78 Models
Which single feature helps most?
T Cuts
25
33.3x
T Presolve
7.7x
T Old variable selection
2.7x
T CPLEX 5.0 presolve
2.6x
T Node presolve
1.3x
T Heuristics
1.1x
T Probing on dives
1.1x
Conclusions
T Through a combination of advances in algorithms and
computing machines, combined with developments in
data availability and modern modeling languages,
what is possible today could only have been dreamed
of even 10 years ago.
T The result is that whole new application domains have
been enabled
26
T
Larger, more accurate models and multiple scenarios
T
Tactical and day-of-operations are possible, not just
planning
T
Disparate components of the extended enterprise can now
be “optimized” in concert.
Acknowledgement – The CPLEX
development team: Mary Fenelon,
Zonghao Gu, Javier Lefuente, Ed
Rothberg, Roland Wunderling (and
Irv Lustig, in a previous life).
27