INFORMS Journal on Computing
Vol. 23, No. 3, Summer 2011, pp. 392–403
issn 1091-9856 — eissn 1526-5528 — 11 — 2303 — 0392
doi 10.1287/ijoc.1100.0405
© 2011 INFORMS
INFORMS holds copyright to this article and distributed this copy as a courtesy to the author(s).
Additional information, including rights and permission policies, is available at http://journals.informs.org/.
Predicting the Solution Time of Branch-and-Bound
Algorithms for Mixed-Integer Programs
Osman Y. Özaltın
Department of Industrial Engineering, University of Pittsburgh, Pittsburgh, Pennsylvania 15261,
[email protected]
Brady Hunsaker
Google Inc., Pittsburgh, Pennsylvania 15213, [email protected]
Andrew J. Schaefer
Department of Industrial Engineering, University of Pittsburgh, Pittsburgh, Pennsylvania 15261,
[email protected]
T
he most widely used progress measure for branch-and-bound (B&B) algorithms when solving mixed-integer
programs (MIPs) is the MIP gap. We introduce a new progress measure that is often much smoother than the
MIP gap. We propose a double exponential smoothing technique to predict the solution time of B&B algorithms
and evaluate the prediction method using three MIP solvers. Our computational experiments show that accurate
predictions of the solution time are possible, even in the early stages of B&B algorithms.
Key words: solution time prediction; mixed-integer programming; branch-and-bound algorithm
History: Accepted by Karen Aardal, Area Editor for Design and Analysis of Algorithms; received June 2009;
revised March 2010; accepted June 2010. Published online in Articles in Advance August 31, 2010.
1.
Introduction
nodes typically indicates that the algorithm will terminate soon, this is not always the case. Furthermore,
while the number of unprocessed nodes is increasing,
it may be difficult to assess the algorithm’s progress.
Therefore, its suitability as a progress measure may
be limited until termination is imminent.
This paper introduces a new progress measure for
B&B algorithms, the sum of subtree gaps (SSG), that
extends the MIP gap. The SSG partitions the active
B&B tree into subtrees and calculates a weighted sum
of the subtree optimality gaps. We show that the
SSG is a nonincreasing progress measure. Moreover,
it decreases every time the MIP gap decreases, and it
may decrease or stay constant when the MIP gap is
constant. In this sense, the decreasing pattern of the
SSG is more steady than that of the MIP gap.
In §4, we develop a method to predict the solution
time of B&B algorithms for mixed-integer programs.
This method predicts the total solution time based on
past values of a progress measure by using a forecasting technique called double exponential smoothing
(Gardner and Dannenbring 1980). In §5, we present
computational experiments on MIPLIB (Achterberg
et al. 2006) and COR@L (COR@L 2010) instances
using three widely used open-source MIP solvers.
Finally, in §6 we conclude and discuss future research
areas.
Mixed-integer programming (MIP) solvers rely on
branch-and-bound (B&B) algorithms, which are typically very time consuming for practical instances. On
the one hand, a user might terminate the search prematurely if the current solution is close enough to
the optimal solution. However, this might result in an
unfavorable outcome if the algorithm were close to
a better solution at termination. On the other hand,
B&B algorithms may spend an enormous amount of
time verifying the optimality of a solution. Therefore,
predicting the progress of B&B algorithms is of great
practical importance.
MIP solvers provide a variety of numerical measures regarding the progress of B&B algorithms. The
most widely used one is the MIP gap, which is the
percent difference between the current upper and
lower bounds on the optimal objective function value.
The MIP gap is a nonincreasing measure that diminishes to zero at the optimal solution for feasible MIPs.
Empirically, its value often stays near constant for a
while and then drops suddenly when a new incumbent is found. This staggered pattern may prevent the
user from getting a clear understanding of the algorithm’s progress.
Another widely used progress measure for B&B
algorithms is the number of unprocessed nodes.
Although a decrease in the number of unprocessed
392
Özaltın, Hunsaker, and Schaefer: Predicting the Solution Time of B&B Algorithms for MIPs
393
INFORMS Journal on Computing 23(3), pp. 392–403, © 2011 INFORMS
INFORMS holds copyright to this article and distributed this copy as a courtesy to the author(s).
Additional information, including rights and permission policies, is available at http://journals.informs.org/.
2.
Literature Review
There are online and off-line algorithms for estimating
the size of a general backtrack tree (i.e., the number
of nodes). Off-line algorithms can be used after terminating the search. Knuth (1975) proposed a method
using random probing. His algorithm estimates the
number of nodes in the tree as the average of 1 +
Q
d1 + d1 d2 + · · · + ki=1 di over several random probes,
where di is the branching rate observed at depth i of
a probe. Knuth proved that his method is an unbiased estimator of the tree size, but he also noted some
limitations. One is that it does not directly apply to
B&B trees because the depth of a random probe is
not known a priori in B&B algorithms. Another is
the large variance of the estimator caused by random probing. Purdom (1978) observed that the variance of the estimator becomes a more serious problem
for unbalanced trees and modified Knuth’s algorithm.
Chen (1992) proposed a heuristic algorithm based on
stratified sampling to address the same issue.
Online algorithms are more useful for our purposes. Kilby et al. (2006) proposed two online methods for estimating the size of a general backtracking
tree. The first method is based on a weighted sum
over the branches that are visited by chronological
backtracking. The second one focuses on the explored
branches assuming that the unexplored branches will
be similar to the explored ones. Both of these methods were shown to be unbiased estimators of the
tree size. Cornuéjols et al. (2006) predicted the size
of a B&B tree based on its ƒ-sequence—the ratios
that describe the change of width from one level to
the next. Their method first estimates the maximum
depth, the widest and the last full level of the tree
according to a partial tree, which is obtained by running the B&B algorithm for a specified amount of
time. These estimations are then used to extrapolate
the B&B tree’s ƒ-sequence by a linear model. The
restriction of this approach is that the linear extrapolation model is developed based on empirical observations of B&B trees developed by CPLEX (ILOG 2010)
for a given parameter setting.
Our approach differs from previous efforts in that
we do not predict the solution time based on the
search tree size. Instead, we first develop a measure
that is capable of showing the continuous progress of
B&B algorithms, and we then predict the total solution time based on past trends of that progress measure using statistical forecasting methods.
3.
Sum of Subtree Gaps
A B&B tree has a single root node and the MIP gap is
calculated for that single-rooted tree. The main rationale behind the SSG is to partition the active B&B tree
into subtrees. Once partitioned, the optimality gap of
each individual subtree is measured instead of the
overall MIP gap. The SSG value is calculated by summing the subtree gaps and scaling the sum to preserve
the nonincreasing property. Rather than maintaining
a fixed set of subtrees, we repartition the active B&B
tree every time a new incumbent solution is found.
Another strategy might be setting a limit for the duration of time when the incumbent solution stays constant and repartitioning the tree every time this limit
is exceeded as well.
Let t be the time elapsed since the B&B algorithm
began. We define Uk to be the set of indices of the
unprocessed (active) nodes in the tree right after finding the kth (k > 0) incumbent solution at time tk ≤ t
and pruning the tree. We partition the active B&B tree
into —Uk — subtrees such that each subtree i ∈ Uk initially consists of a single active node from which a
search tree emanates as the B&B algorithm progresses
after time tk . Let gi 4t5 be the optimality gap of subtree
i ∈ Uk at time t ≥ tk . Then gi 4t5 is equal to the percent
difference between the linear programming relaxation
of the best active node in subtree i and the incumbent
solution found anywhere in the overall B&B tree.
Remark 1. The MIP gap = maxi∈Uk 8gi 4t59 and gi 4t5
is nonincreasing ∀ i ∈ Uk .
For t ≥ tk , define
X
fk 4t5 =
gi 4t50
(1)
i∈Uk
Proposition 1. fk 4t5 is nonincreasing over t for tk ≤
t < tk+1 .
Proposition 1 holds since gi 4t5 is nonincreasing over
t for tk ≤ t < tk+1 , ∀i ∈ Uk , and the number of subtrees
is constant for a particular incumbent solution.
Proposition 2. fk 4t5 is not necessarily nonincreasing
over k.
Proposition 2 follows since the size of Uk+1 may
be smaller or larger than that of Uk , depending on
the number of active nodes in the tree when these
sets are defined. While calculating the progress measure SSG4t5, we scale fk 4t5 between 0 and 1 so that it
becomes nonincreasing over k as well. We define scaling parameter sk by the following recursive equation:


if k = 11

1
sk =
fk−1 4tk 5


sk−1
fk 4tk 5
(2)
if k > 10
Then we define the sum of subtree gaps at time t by

1
if t < t1 1
SSG4t5 =
(3)

sk fk 4t5 if tk ≤ t < tk+1 0
Özaltın, Hunsaker, and Schaefer: Predicting the Solution Time of B&B Algorithms for MIPs
394
INFORMS Journal on Computing 23(3), pp. 392–403, © 2011 INFORMS
We set SSG4t5 = 1 for t < t1 because subtree optimality gaps cannot be calculated without an incumbent
solution.
(a) pk1
Progress measures: pk1, COIN/CBC
1.0
0.9
0.8
0.7
SSG4t 00 5 ≤ SSG4tk 5
= sk fk 4tk 5 = sk−1
= sk−1 fk−1 4tk 51
Measure
Proof. SSG4t5 is nonincreasing between tk and tk+1
for every incumbent solution k by Proposition 1.
We show that SSG4t5 is also nonincreasing between
two consecutive incumbent solutions when k > 1. Let
tk−1 ≤ t 0 < tk ≤ t 00 < tk+1 . Then,
0.6
0.5
SSG
0.4
MIP gap
0.3
0.2
0.1
(4)
0
fk−1 4tk 5
f 4t 5
fk 4tk 5 k k
0
100
200
300
400
500
600
700
800
900
Time (s)
(5)
(b) aflow30a
Progress measures: aflow30a, COIN/CBC
1.0
where inequality (4) follows from Proposition 1, and
Equation (5) holds by definition of sk given in (2). As
a result of Proposition 1, Equation (5), and the definition of SSG4t5, SSG4t 00 5 ≤ sk−1 fk−1 4tk 5 ≤ sk−1 fk−1 4t 0 5 =
SSG4t 0 50 ƒ
0.9
0.8
Measure
0.7
Proposition 4. For a feasible MIP instance, the SSG
decreases every time the MIP gap decreases. It may decrease
or stay constant when the MIP gap is constant.
Proposition 4 shows that the SSG is at least as
good as the MIP gap when it comes to decreasing. This result directly follows from Remark 1 and
Proposition 1.
The difference between the SSG and the MIP
gap comes from the fact that if a subtree optimality gap, which does not define the largest optimality
gap, decreases, then the SSG will decrease while the
MIP gap will not. Figure 1 plots the SSG and the MIP
gap to illustrate this fact. The SSG steadily diminishes to zero, but the MIP gap stays constant for long
stretches of time. The relatively smooth pattern of
the SSG gives a better understanding of the progress
for the instances in Figure 1. However, neither measure always follows a pattern of steady decrease (see
Figure 2).
Note that although the SSG often decreases more
steadily than the MIP gap, it does not always provide
better information. Consider the hypothetical B&B
tree depicted in Figure 3.
There are three subtrees rooted at nodes A, B, and C.
Suppose that subtree A has the largest optimality gap.
Then, the MIP gap is equal to the optimality gap of
subtree A and the SSG is equal to a weighted average of all subtree optimality gaps. The MIP gap is a
better measure of progress towards the optimal solution at this snapshot in time, because the optimal solution is attained at the solid node located in subtree A.
Note that the reason that the SSG may decrease when
the MIP gap stays constant is exactly the case when
the optimality gap of subtree B and/or C decreases
0.6
0.5
0.4
0.3
SSG
0.2
0.1
0
MIP gap
0
2,000
4,000
6,000
8,000
10,000
12,000
Time (s)
Figure 1
The MIP Gap and the SSG by Time for pk1 (a) and aflow30a
(b) Instances (Achterberg et al. 2006)
Note. The values are calculated every second.
while the optimality gap of subtree A stays constant.
However, that decrease in the SSG does not reflect the
progress towards the optimal solution. Yet it may still
Progress measures: p2756, SYMPHONY
1.0
MIP gap
0.9
0.8
0.7
Measure
INFORMS holds copyright to this article and distributed this copy as a courtesy to the author(s).
Additional information, including rights and permission policies, is available at http://journals.informs.org/.
Proposition 3. SSG4t5 is nonincreasing over t.
0.6
0.5
0.4
0.3
SSG
0.2
0.1
0
80
100
120
140
160
180
200
220
240
Time (s)
Figure 2
Neither the MIP Gap nor the SSG Is Smooth for the p2756
Instance (Achterberg et al. 2006)
Note. The values are calculated every second.
Özaltın, Hunsaker, and Schaefer: Predicting the Solution Time of B&B Algorithms for MIPs
INFORMS Journal on Computing 23(3), pp. 392–403, © 2011 INFORMS
C
The worst gap
A
and let ã4t5 = 4”4t5 − ”4t 0 55/4t − t 0 5. Then we define
the updating equations of the forecasting model by
S4t5 = ”4t5 + 41 − 5S4t 0 5 0 ≤  ≤ 11
b4t5 = ƒã4t5 + 41 − ƒ5b4t 0 5 0 ≤ ƒ ≤ 10
INFORMS holds copyright to this article and distributed this copy as a courtesy to the author(s).
Additional information, including rights and permission policies, is available at http://journals.informs.org/.
B
Optimal solution
Figure 3
A Hypothetical B&B Tree for Which the MIP Gap Is a Better
Measure of Progress Than the SSG
be useful for predicting the remaining solution time,
because all nodes in subtrees B and C have to be processed before terminating the search.
4.
Predicting Solution Time
In this section, we develop a solution time prediction method for B&B algorithms. We formulate a forecasting model with two drifting parameters, a slope
and an intercept, which are updated over time as
new progress measurements become available from
the B&B tree. The underlying idea is that changes
in the progress measure can be modeled by a linear
trend equation based on the complete history of the
algorithm’s performance.
A standard way of modeling parameter drift is to
use the Kalman filter, which minimizes the estimated
covariance of unknown parameters using a predictor–
corrector mechanism (Welch and Bishop 1995). We
refrain from this approach because it has long running times and is hard to implement. Instead, we
employ a double exponential smoothing technique
(Gardner and Dannenbring 1980), which calculates
the weighted average of time series data by assigning
exponentially decreasing weights to the past observations. Widely used in economic forecasting models (Box 1991, Snyder 2002), this technique uses only
two parameters, and it is much simpler to implement
and faster than the Kalman filter. Moreover, LaViola
(2003) reported that double exponential smoothing
and Kalman filter-based predictions have roughly the
same accuracy.
Let ” be a nonincreasing progress measure between
0 and 1, which reduces to 0 at the optimal solution.
Whereas we use the SSG and the MIP gap for ”,
our prediction method can also be applied with other
nonincreasing progress measures. We calculate the
total solution time predictions based on the past values of ”. Let  and ƒ be constant coefficients. Denote
by t 0 < t the last time ” was calculated before time t,
395
(6)
In (6), S4t5 is the predicted value of the progress
measure, and b4t5 is the estimated trend in the
progress measure at time t. There are various ways to
initialize S4 · 5 and b4 · 5; see Gardner and Dannenbring
(1980) and NIST/Sematech (2006). Each time a new
incumbent is found, we reinitialize S4 · 5 and b4 · 5 and
do not make any prediction until we observe a specified number of progress measurements for the new
incumbent. We set this threshold to four based on
empirical experience. Let ln denote the time when the
nth progress measure value is calculated. After having
four progress measurements at time l4 , we initialize
S4l4 5 and b4l4 5 by
S4l4 5 = ”4l4 5 + 41 − 5”4l3 51
b4l4 5 = 4”4l4 5 − ”4l3 55/4l4 − l3 50
We calculate predictions with discrete time intervals. Denote by F 4t5 the total solution time prediction
at time t. In addition, let at and nt be the number of
unprocessed and processed nodes in the B&B tree at
time t, respectively. The first solution time prediction
at t = l4 is given by F 4t5 = at 4t/nt 5 + t. This expression
divides the current time t by the number of processed
nodes nt to compute how much time was spent for
each node on average thus far. The result is multiplied
by the number of unprocessed nodes at and added to
the current time. Let ‹ be a constant coefficient. Then
for t > l4 , F 4t5 is defined by

S4t5

−
+ t if ã4t5 < ‹b4t 0 51



min4ã4t51
b4t55





at
0
0
if ã4t5 ≥ ‹b4t 0 5 (7)
F 4t5 = 4F 4t 5 − t 5 a 0 + t
t




and F 4t 0 5 ≤ t1





F 4t 0 5
otherwise.
In (7), ‹ is used to assess the significance of the
decrease in the progress measure. If the progress measure decreases significantly from time t 0 to t, we calculate the total solution time prediction by making a
linear interpolation. Note that b4t5 ≤ 0 for all t. If the
decrease is not significant and the previous prediction
is less than the current time, we compute the prediction by weighting the last calculated remaining solution time prediction with the rate of increase in the
number of unprocessed nodes. Otherwise, we repeat
the previous total solution time prediction.
Özaltın, Hunsaker, and Schaefer: Predicting the Solution Time of B&B Algorithms for MIPs
396
Computational Tests
80,000
MIP gap
Elapsed time
70,000
Prediction of total time
5.1. Implementation
We modified open-source MIP solvers CBC 1.01
(Forrest 2007), SYMPHONY 5.1 (Ralphs 2007), and
GLPK 4.15 (Makhorin 2007) to have them report sufficient information about the B&B tree. We use these
MIP solvers not to compare their performance but to
demonstrate that our proposed prediction method is
generally applicable. Unfortunately, we are unable to
obtain sufficient information about the B&B tree from
commercial solvers, so our tests are restricted to opensource MIP solvers. Note that although we demonstrate the use of the SSG and the prediction method
on B&B algorithms for MIPs, they actually apply to
any implicit enumeration algorithm that develops a
search tree providing upper and lower bounds on the
optimal objective value.
We use a Python script to summarize the solver
output about tree development. We call this tool the
“branch-and-bound analysis kit” (BAK) (Özaltın et al.
2007). BAK stores the information about each node
of the tree into memory. It also stores an additional
variable for each unprocessed node that represents
its subtree index. This additional variable is updated
every time the active B&B tree is repartitioned. Therefore, the computational effort to calculate the SSG
during the B&B algorithm is linear in the number of
unprocessed nodes.
Test instances for the experiments conducted in this
paper are from MIPLIB (Achterberg et al. 2006) and
COR@L (COR@L 2010) libraries. We run the tests
using default solver settings on a computer with an
AMD Opteron 240 processor and 3.6 GB memory.
We report only those instances that are solved optimally within an eight-hour time limit. Furthermore,
we do not report an instance if it takes less than two
minutes to solve or if there is no prediction for it,
which occurs when optimality is proven before making a prediction.
We implement the prediction method in BAK and
run it for all of the test instances by setting , ‹, and
ƒ to 0.5, as this is one of the best settings we found
in our preliminary experiments (see Appendix III of
the Online Supplement, available at http://joc.pubs
.informs.org/ecompanion.html).
We simultaneously calculate two separate predictions of the total solution time: one based on the
SSG and the other based on the MIP gap. Both
of these predictions are plotted over time. Example
prediction plots for the aflow30a instance are given
in Figure 4. Prediction plots for all reported test
instances are available in Appendix II of the Online
Supplement.
In Figure 4, the horizontal axis depicts the points
in time when a prediction is calculated, and the vertical axis shows the total solution time predictions. The
(a) The MIP gap-based total solution time predictions
Prediction plot: aflow30a.dat, COIN/CBC
60,000
50,000
40,000
30,000
20,000
10,000
0
0
2,000
4,000
6,000
8,000
10,000
12,000
Time (s)
(b) The SSG-based total solution time predictions
Prediction plot: aflow30a.dat, COIN/CBC
80,000
(SSG)
Elapsed time
70,000
Prediction of total time
5.
INFORMS Journal on Computing 23(3), pp. 392–403, © 2011 INFORMS
60,000
50,000
40,000
30,000
20,000
10,000
0
0
2,000
4,000
6,000
8,000
10,000
12,000
Time (s)
Figure 4
The (a) MIP Gap- and (b) SSG-Based Total Solution Time Predictions for the aflow30a Instance (Achterberg et al. 2006)
Note. Values calculated every 10 seconds.
dashed line on the diagonal represents the solution
time elapsed since the B&B algorithm began.
5.2. Accuracy of Predictions
In this section, we estimate the accuracy of remaining solution time predictions. Note that the prediction method returns total solution time predictions.
However, remaining solution time predictions can also
be calculated easily by subtracting the time of a prediction from itself. Given a scalar k > 1 and the true
remaining solution time t ∗ , a prediction t̂ is k-accurate
if 1/k ≤ t̂/t ∗ ≤ k. Note that one shortcoming of this
definition of accuracy is that near termination, predictions that are close in an absolute sense may be considered inaccurate.
Tables 1–3 report the ratio of 2-accurate SSG-based
and MIP gap-based predictions using each of the three
MIP solvers. Similar tables for k = 1051 31 and 4 are
provided in Appendix I of the Online Supplement. We
also report the geometric mean of the absolute differ-
Özaltın, Hunsaker, and Schaefer: Predicting the Solution Time of B&B Algorithms for MIPs
397
INFORMS Journal on Computing 23(3), pp. 392–403, © 2011 INFORMS
Table 1
Ratio of 2-Accurate Predictions and the Geometric Mean of the Absolute Differences Between the Predictions and the True Remaining
Solution Time for CBC
2-Accuracy
No. of predictions
Solution
Overall
Instance
Overall
Postdec.
Time (s)
MIP gap
set1ch
qiu
aflow30a
mod011
harp2
pk1
mas76
bell5
21298
11219
11099
528
326
72
74
28
21272
11219
11099
525
326
72
74
28
231271003
121285004
111154042
51364075
31957093
846088
766028
324051
0010
0010
0010
0010
0013
0014
0012
0018
227
97
62
67
56
5
224
90
0
0
56
0
503043
295016
171067
161035
143071
112011
0013
0030
0016
0012
0011
0040 Ø
11483
11401
686
631
491
501
480
159
449
365
355
327
203
145
153
142
121
114
102
88
87
75
77
63
40
54
11478
11391
686
631
477
499
478
123
449
349
355
327
203
144
153
142
121
114
102
82
87
75
72
63
16
8
151423083
141233085
71097019
71048077
51691058
51374044
51130022
41996044
41645094
31877068
31591089
31291001
21255011
11696007
11647099
11476061
11371002
11237023
11098078
11016090
944010
862047
821086
792047
697098
616094
0010
0010
0011
0011
0011
0011
0011
0026 Ø
0011
0011
0010
0011 Ø
0014
0041
0012
0011
0012
0012
0012
0013
0011 Ø
0013
0012
0016
0063 Ø
0015
147
147
217
167
152
181
143
135
121
71
13
122
147
215
167
152
149
143
133
121
71
0
528031
451044
445008
413065
384066
347066
308062
294069
261087
261022
249039
0012
0005
0011
0013
0016
0013
0012 Ø
0013
0015
0013
0000 Ø
Geometric mean of deviations
Postdecrease
SSG
MIP gap
Overall
SSG
MIP gap
0058 Ø
0062 Ø
0092 Ø
0042 Ø
0040 Ø
1000 Ø
0093 Ø
0093 Ø
98096
99093
98016
99089
97024
92059
99041
96022
Postdecrease
SSG
MIP gap
SSG
41090 Ø
36078 Ø
18044 Ø
51061 Ø
65036 Ø
22014 Ø
25041 Ø
20051 Ø
98094
99093
98016
99089
97024
92059
99041
96022
41057 Ø
36078 Ø
18044 Ø
51094 Ø
65036 Ø
22014 Ø
25041 Ø
20051 Ø
66043 Ø
71055
64064 Ø
87059 Ø
33072 Ø
27044 Ø
101024
72000
66017 Ø
55056 Ø
96015
33072 Ø
MIPLIB with 10 s interval
0058 Ø
0062 Ø
0092 Ø
0043 Ø
0040 Ø
1000 Ø
0093 Ø
0093 Ø
0010
0010
0010
0010
0013
0014
0012
0018
MIPLIB with 1 s interval
air05
air04
l152lav
stein45
misc07
nw04
0048 Ø
0067 Ø
0063 Ø
0013 Ø
0068 Ø
0040 Ø
0013
0032
0049 Ø
0072 Ø
0011
0068 Ø
101007
70069 Ø
83008
99047
96015
41065
COR@L with 10 s interval
neos17
bienst2
sp98ir
neos-430149
dano3_5
neos-504674
neos-512201
neos-799716
binkar10_1
neos-829552
neos-1440447
neos-906865
neos3
neos-504815
neos-803219
neos-803220
neos-860300
neos-538867
neos-686190
neos-611838
neos-1582420
bienst1
neos-570431
neos-806323
neos-1171448
neos-612162
0071 Ø
0048 Ø
0044 Ø
0033 Ø
0021 Ø
0088 Ø
0098 Ø
0026 Ø
0098 Ø
0042 Ø
0052 Ø
0001
0028 Ø
0080 Ø
0089 Ø
0075 Ø
0034 Ø
0089 Ø
0086 Ø
0081 Ø
0000
0067 Ø
0056 Ø
0090 Ø
0063 Ø
0089 Ø
0010
0010
0011
0011
0012
0011
0011
0032 Ø
0011
0011
0010
0011 Ø
0014
0041
0012
0011
0012
0012
0012
0013
0011 Ø
0013
0013
0016
1000 Ø
1000 Ø
0071 Ø
0048 Ø
0044 Ø
0033 Ø
0021 Ø
0089 Ø
0098 Ø
0032 Ø
0098 Ø
0044 Ø
0052 Ø
0001
0028 Ø
0080 Ø
0089 Ø
0075 Ø
0034 Ø
0089 Ø
0086 Ø
0087 Ø
0000
0067 Ø
0060 Ø
0090 Ø
1000 Ø
1000 Ø
99091
30029 Ø
97078
46086 Ø
90016
67056 Ø
99079
60058 Ø
98044
73017 Ø
85093
26031 Ø
92097
24089 Ø
85008 Ø
85008 Ø
94048
17045 Ø
99041
65059 Ø
99097
44084 Ø
99066 Ø 51124031
87071 Ø 115031
69062
34063 Ø
98079
25013 Ø
98066
36039 Ø
90011 Ø 116007
93057
30078 Ø
99044
33007 Ø
99025
27014 Ø
98065 Ø 11340071
85078
48044 Ø
99071
64070 Ø
94098
22073 Ø
223051
142041 Ø
96031
28092 Ø
99091
97077
90016
99079
98051
85089
92095
84008 Ø
94048
99062
99097
99066 Ø
87071 Ø
69048
98079
98066
90011 Ø
93057
99044
99039
98065 Ø
85078
99071
94098
749068
96034
30017 Ø
46066 Ø
67056 Ø
60058 Ø
72062 Ø
26010 Ø
24085 Ø
84008 Ø
17045 Ø
64048 Ø
44084 Ø
51124031
115031
34065 Ø
25013 Ø
36039 Ø
116007
30078 Ø
33007 Ø
25042 Ø
11340071
48044 Ø
62079 Ø
22073 Ø
242095 Ø
84084 Ø
0043 Ø
0010 Ø
0086 Ø
0044 Ø
0093 Ø
0077 Ø
0012 Ø
0035 Ø
0032 Ø
0056 Ø
89062
113017
78069
97088
77061
95079
99051 Ø
98077
97040
90099
648084 Ø
92065
113017
78058
97088
77061
95052
99051 Ø
98081
97040
90099
36043 Ø
105005Ø
23095 Ø
46011 Ø
19029 Ø
23039 Ø
99051 Ø
69055 Ø
64099 Ø
56012 Ø
COR@L with 1 s interval
bc1
neos-555771
neos-839859
neos-612143
neos-807705
neos-807639
neos-912015
neos-825075
neos-603073
aligninq
neos-826250
0039 Ø
0010 Ø
0085 Ø
0044 Ø
0093 Ø
0064 Ø
0012 Ø
0036 Ø
0032 Ø
0056 Ø
0000 Ø
0015
0005
0011
0013
0016
0016
0012 Ø
0013
0015
0013
41015 Ø
105005Ø
24038 Ø
46011 Ø
19029 Ø
29027 Ø
99051 Ø
67022 Ø
64099 Ø
56012 Ø
648084 Ø
Özaltın, Hunsaker, and Schaefer: Predicting the Solution Time of B&B Algorithms for MIPs
398
Table 1
INFORMS Journal on Computing 23(3), pp. 392–403, © 2011 INFORMS
(Continued)
2-Accuracy
No. of predictions
Instance
Overall
Postdec.
Solution
Overall
Geometric mean of deviations
Postdecrease
Time (s)
MIP gap
SSG
MIP gap
221037
180046
177017
172066
120010
116012
108098
71046
65060
0017 Ø
0022
0014
0016
0015 Ø
0024
0048
1000 Ø
0020 Ø
0000
0064 Ø
0022 Ø
0041 Ø
0015 Ø
0081 Ø
0062 Ø
0080
0020 Ø
SSG
Overall
MIP gap
Postdecrease
SSG
MIP gap
SSG
96050 Ø
99029
51585020 Ø
94032
77094
51019
256002
114025
59057 Ø
51585020 Ø
30015 Ø
26058 Ø
13092 Ø
231086 Ø
COR@L with 1 s interval
neos-612125
neos-941698
neos2
prod1
neos-494568
neos-801834
neos-598183
neos-810326
neos-892255
Table 2
72
50
74
90
26
37
50
20
30
0
0
74
90
2
37
50
16
16
0014
0016
0000 Ø
0024
0048
1000 Ø
0038 Ø
0022 Ø
0041 Ø
0000 Ø
0081 Ø
0062 Ø
1000 Ø
0038 Ø
95081 Ø
99032
96050 Ø
99029
153018 Ø
94032
77094
38092
160081
941049
58025 Ø
114025
59057 Ø
153018 Ø
30015 Ø
26058 Ø
26027 Ø
152053 Ø
Ratio of 2-Accurate Predictions and the Geometric Mean of the Absolute Differences Between the Predictions and the True Remaining
Solution Time for GLPK
2-Accuracy
No. of predictions
Solution
Overall
Instance
Overall
Postdec.
Time (s)
MIP gap
air05
pp08aCUTS
misc07
pp08a
vpm2
roll3000
10teams
l152lav
fiber
gesa3o
rout
stein45
nw04
mitre
11158
525
295
275
215
210
88
113
110
93
77
57
23
22
11155
525
295
275
215
198
88
113
110
93
77
57
23
19
111887094
51315020
31011075
21896088
21250031
21229031
21041030
11274052
11253025
985026
881034
616019
403056
283001
0011
0010
0011
0021
0008
0011 Ø
0020 Ø
0042 Ø
0013
0012
0013 Ø
0012
0026
0014 Ø
53
113
53
113
714069
159032
1000 Ø
0008 Ø
11573
11132
684
616
637
460
185
482
475
191
334
329
291
229
226
193
11573
11132
684
616
637
460
185
482
475
191
334
329
291
229
226
193
151773016
111413064
71247048
61730000
61421028
51261066
51123027
51047017
41844030
41421073
31572059
31463028
31064036
21611083
21466092
21080089
0010 Ø
0010 Ø
0027
0015
0010
0012
0050 Ø
0011 Ø
0010
0024 Ø
0028 Ø
0003
0010
0025
0031
0009 Ø
Geometric mean of deviations
Postdecrease
SSG
MIP gap
Overall
Postdecrease
SSG
MIP gap
SSG
MIP gap
SSG
0041 Ø
0081 Ø
0064 Ø
0061 Ø
0010 Ø
0012 Ø
0011
0034
0032 Ø
0039 Ø
0000
0093 Ø
0035 Ø
0000
83002 Ø
99080
99093
84065
113021 Ø
97018 Ø
82085 Ø
71091 Ø
86052 Ø
99003
85034 Ø
96001
76088 Ø
84035 Ø
91033
28006 Ø
46056 Ø
81022 Ø
401089
97018 Ø
141092
135047
121051
90039 Ø
914007
29005 Ø
330055
11396033
82098 Ø
99080
99093
84065
113021 Ø
97021 Ø
82085 Ø
71091 Ø
86052 Ø
99003
85034 Ø
96001
76088 Ø
86009 Ø
91031
28006 Ø
46056 Ø
81022 Ø
401089
97021 Ø
141092
135047
121051
90039 Ø
914007
29005 Ø
330055
11538021
1000 Ø
0006
93081 Ø
120085 Ø
93081 Ø
11050008
93081 Ø
120085 Ø
93081 Ø
11050008
0010 Ø
0005
0061 Ø
0078 Ø
0059 Ø
0072 Ø
0050 Ø
0005
0019 Ø
0024 Ø
0000
0053 Ø
0065 Ø
0067 Ø
0075 Ø
0009 Ø
99098
94059 Ø
80091
79061
99060
95085
48028 Ø
99083 Ø
99089
94091 Ø
80010 Ø
101006
87098
88090
78063
99049
92048 Ø
11034037
53066 Ø
30071 Ø
56083 Ø
36067 Ø
48028 Ø
11169011
75009 Ø
94091 Ø
11235070
64018 Ø
41089 Ø
35085 Ø
38099 Ø
98086 Ø
99098
94059 Ø
80091
79061
99060
95085
48028 Ø
99083 Ø
99089
94091 Ø
80010 Ø
101006
87098
88090
78063
99049
92048 Ø
11034037
53066 Ø
30071 Ø
56083 Ø
36067 Ø
48028 Ø
11169011
75009 Ø
94091 Ø
11235070
64018 Ø
41089 Ø
35085 Ø
38099 Ø
98086 Ø
MIPLIB with 10 s interval
0041 Ø
0081 Ø
0064 Ø
0061 Ø
0010 Ø
0011 Ø
0011
0034
0032 Ø
0039 Ø
0000
0093 Ø
0035 Ø
0000
0011
0010
0011
0021
0008
0012 Ø
0020 Ø
0042 Ø
0013
0012
0013 Ø
0012
0026
0016 Ø
MIPLIB with 1 s interval
enigma
mod008
1000 Ø
0006
1000 Ø
0008 Ø
COR@L with 10 s interval
neos11
neos-1582420
sp98ir
bc1
neos-570431
prod1
neos-827175
neos-892255
neos-1058477
neos-810286
neos-810326
neos-480878
bienst2
neos-860300
neos-839859
neos10
0010 Ø
0005
0061 Ø
0078 Ø
0059 Ø
0072 Ø
0050 Ø
0005
0019 Ø
0024 Ø
0000
0053 Ø
0065 Ø
0067 Ø
0075 Ø
0009 Ø
0010 Ø
0010 Ø
0027
0015
0010
0012
0050 Ø
0011 Ø
0010
0024 Ø
0028 Ø
0003
0010
0025
0031
0009 Ø
Özaltın, Hunsaker, and Schaefer: Predicting the Solution Time of B&B Algorithms for MIPs
399
INFORMS Journal on Computing 23(3), pp. 392–403, © 2011 INFORMS
Table 2
(Continued)
2-Accuracy
No. of predictions
Instance
Overall
Postdec.
neos-906865
neos-912015
neos-796608
neos-807705
neos4
neos-717614
Test3
aligninq
neos-807639
neos-686190
dano3_5
neos-801834
neos-806323
neos-584851
neos-565815
neos-825075
bienst1
qap10
neos-803220
neos-501453
neos8
945
550
11002
787
65
608
428
183
507
314
171
162
232
207
16
228
181
13
133
109
13
945
550
11002
787
65
608
428
183
507
314
171
162
232
207
16
228
181
13
133
109
13
Solution
Overall
Time (s)
MIP gap
11509012
11170072
11117022
11110036
857052
892081
873078
816006
728070
536042
475002
362067
353025
294042
258045
250017
232077
138069
153067
137038
122091
0009
0029 Ø
0010 Ø
0000
0015 Ø
0010
0011
0089 Ø
0010
0019
0009
0014
0044 Ø
0009
0000 Ø
0010
0011
0031 Ø
0011
0011 Ø
0038 Ø
Geometric mean of deviations
Postdecrease
SSG
MIP gap
Overall
Postdecrease
SSG
MIP gap
SSG
0039 Ø
0029 Ø
0010 Ø
0023 Ø
0015 Ø
0037 Ø
0022 Ø
0015
0047 Ø
0029 Ø
0030 Ø
0046 Ø
0000
0045 Ø
0000 Ø
0062 Ø
0045 Ø
0031 Ø
0089 Ø
0011 Ø
0038 Ø
99099
79046 Ø
99098
124082 Ø
100047 Ø
98088 Ø
99074
10059 Ø
99097
81081 Ø
96096
87096
28003 Ø
99008
756071 Ø
99023
99073
54077 Ø
91007
99096
67080 Ø
54050 Ø
79046 Ø
99097 Ø
318032
100089
129090
80056 Ø
11212076
65017 Ø
208018
57007 Ø
52097 Ø
11502037
49052 Ø
756071 Ø
52047 Ø
75009 Ø
54077 Ø
32066 Ø
95088 Ø
67080 Ø
MIP gap
SSG
COR@L with 1 s interval
Table 3
0039 Ø
0029 Ø
0010 Ø
0023 Ø
0015 Ø
0037 Ø
0022 Ø
0015
0047 Ø
0029 Ø
0030 Ø
0046 Ø
0000
0045 Ø
0000 Ø
0062 Ø
0045 Ø
0031 Ø
0089 Ø
0011 Ø
0038 Ø
0009
0029 Ø
0010 Ø
0000
0015 Ø
0010
0011
0089 Ø
0010
0019
0009
0014
0044 Ø
0009
0000 Ø
0010
0011
0031 Ø
0011
0011 Ø
0038 Ø
99099
79046 Ø
99098
124082 Ø
100047 Ø
98088 Ø
99074
10059 Ø
99097
81081 Ø
96096
87096
28003 Ø
99008
756071 Ø
99023
99073
54077 Ø
91007
99096
67080 Ø
54050 Ø
79046 Ø
99097 Ø
318032
100089
129090
80056 Ø
11212076
65017 Ø
208018
57007 Ø
52097 Ø
11502037
49052 Ø
756071 Ø
52047 Ø
75009 Ø
54077 Ø
32066 Ø
95088 Ø
67080 Ø
Ratio of 2-Accurate Predictions and the Geometric Mean of the Absolute Differences Between the Predictions and the True Remaining
Solution Time for SYMPHONY
2-Accuracy
No. of predictions
Solution
Overall
Instance
Overall
Postdec.
Time (s)
MIP gap
mas76
pk1
stein45
aflow30a
gesa2
nw04
pp08aCUTS
mod011
pp08a
gesa2o
l152lav
vpm2
bell3a
21298
223
588
265
273
193
228
182
131
98
92
74
27
21298
223
588
265
273
193
228
182
131
98
92
74
27
231165033
121362017
81221008
51041016
31122047
21644045
21418058
21152088
11605027
11199032
11122033
11044013
312014
0010
0056
0027
0027
0011
0013
0011
0011
0014
0013
0015
0016
0015
75
92
59
91
51
75
92
59
91
51
306031
258047
236006
233022
125069
0020
0018
0014
0013
0016 Ø
605
673
903
552
265
449
141904000
111317000
91949000
0012
0006
0041 Ø
Geometric mean of deviations
Postdecrease
SSG
MIP gap
SSG
Overall
MIP gap
Postdecrease
SSG
MIP gap
SSG
MIPLIB with 10 s interval
0088 Ø
0093 Ø
0074 Ø
0097 Ø
0073 Ø
0029 Ø
0091 Ø
0075 Ø
1000 Ø
0061 Ø
0023 Ø
0053 Ø
0089 Ø
0010
0056
0027
0027
0011
0013
0011
0011
0014
0013
0015
0016
0015
0088 Ø
0093 Ø
0074 Ø
0097 Ø
0073 Ø
0029 Ø
0091 Ø
0075 Ø
1000 Ø
0061 Ø
0023 Ø
0053 Ø
0089 Ø
99090
92068
73017
63048
93067
88016 Ø
99059
91039
97027
94070
88024 Ø
90038
96042
19046 Ø
29090 Ø
42041 Ø
15035 Ø
34093 Ø
98029
20091 Ø
34027 Ø
12094 Ø
59088 Ø
232011
56048 Ø
21045 Ø
99090
92068
73017
63048
93067
88016 Ø
99059
91039
97027
94070
88024 Ø
90038
96042
19046 Ø
29090 Ø
42041 Ø
15035 Ø
34093 Ø
98029
20091 Ø
34027 Ø
12094 Ø
59088 Ø
232011
56048 Ø
21045 Ø
95050
98022
79005 Ø
97018
92069 Ø
26080 Ø
30057 Ø
83090
60005 Ø
92069 Ø
95050
98022
79005 Ø
97018
92069 Ø
26080 Ø
30057 Ø
83090
60005 Ø
92069 Ø
75019 Ø
88028 Ø
64061 Ø
80089
11239049
50024 Ø
73043 Ø
340042 Ø
57067
MIPLIB with 1 s interval
blend2
fixnet6
p2756
stein27
dsbmip
0081 Ø
0072 Ø
0031 Ø
0063 Ø
0016 Ø
0020
0018
0014
0013
0016 Ø
0081 Ø
0072 Ø
0031 Ø
0063 Ø
0016 Ø
COR@L with 10 s interval
neos13
neos-810326
neos-829552
0024 Ø
0057 Ø
0033
0014
0016
0082 Ø
0027 Ø
0064 Ø
0038
82013
243096
70082
Özaltın, Hunsaker, and Schaefer: Predicting the Solution Time of B&B Algorithms for MIPs
400
Table 3
INFORMS Journal on Computing 23(3), pp. 392–403, © 2011 INFORMS
(Continued)
2-Accuracy
No. of predictions
Instance
Overall
Postdec.
Solution
Overall
Time (s)
MIP gap
Geometric mean of deviations
Postdecrease
SSG
MIP gap
SSG
Overall
Postdecrease
MIP gap
SSG
MIP gap
SSG
15082 Ø
98003
61174093
99073
87073
287084
82054
92067 Ø
99083 Ø
45033
80003
90025
71017 Ø
108077 Ø
40086 Ø
64026 Ø
81094
95066 Ø
82054
61020 Ø
68000 Ø
98020
94078 Ø
97049
98015 Ø
96050
94054 Ø
83003
39062 Ø
83008
49082
99026
76051 Ø
33041
95051 Ø
132007 Ø
63041 Ø
75079 Ø
131032 Ø
45007 Ø
11993000
99083 Ø
26036 Ø
68092 Ø
52077 Ø
90080
171032
87099
31600096
57024 Ø
488032
45019 Ø
62019
99077
47009 Ø
196019
44033 Ø
98015 Ø
92055 Ø
11208057
75078 Ø
247015
13098 Ø
39056 Ø
95061 Ø
440023
15082 Ø
98037
61174093
99099
88016
913086
82029
92067 Ø
33041
95079 Ø
132007 Ø
59067 Ø
75082 Ø
86077 Ø
44057 Ø
11993000
45033
80003
90025
69081 Ø
109005 Ø
12071 Ø
64026 Ø
81040
95066 Ø
82054
52049 Ø
68000 Ø
98020
94078 Ø
97049
98085 Ø
96049
82059
81048
39062 Ø
83008
49082
99026
76051 Ø
26036 Ø
68092 Ø
52077 Ø
83054
173044
15050
31600096
56008 Ø
488032
45019 Ø
56047
99077
47009 Ø
196019
44033 Ø
98085 Ø
95025 Ø
40026 Ø
73038 Ø
247015
13098 Ø
39056 Ø
95061 Ø
440023
78095
65051
117093 Ø
90009
97097
599092
48098 Ø
98034
90058
70085 Ø
38001 Ø
160095
44075 Ø
70099 Ø
128067 Ø
482001
90031 Ø
87055 Ø
81054
65051
196084
90009
97097
623007
48098 Ø
98034
90058
68095 Ø
38001 Ø
164052 Ø
44075 Ø
70052 Ø
124098 Ø
482001
90031 Ø
87055 Ø
COR@L with 10 s interval
neos20
neos-799711
dano3_5
neos11
neos-820879
neos10
neos-906865
neos-603073
neos-547911
neos-598183
neos-860300
neos-538867
neos-839859
neos-480878
neos-522351
neos-504674
neos-775946
neos-686190
bienst2
neos-570431
prod1
neos-1440447
neos7
neos-807639
neos-912015
neos-892255
neos-538916
neos-555771
neos-807705
neos-803220
neos-803219
neos-825075
neos-555694
40
755
26
538
396
24
365
264
203
73
121
254
211
143
130
108
133
97
110
114
107
111
97
104
91
100
90
70
44
56
54
65
52
40
740
26
472
384
8
359
264
0
73
121
254
198
140
47
108
128
97
110
85
107
111
97
104
76
86
24
61
44
56
54
65
52
91922000
81745000
81539000
61124000
51125000
31887000
31757000
31749000
31195000
31082000
21748000
21724000
21500000
21447000
11850000
11619000
11466000
11455000
11383000
11293000
11183000
11142000
11141000
11131000
11110000
11095000
11027000
813000
787000
780000
712000
699000
681000
1000 Ø
0012 Ø
0000
0012
0017
0025
0024
0009 Ø
0014 Ø
1000 Ø
0047 Ø
0017
0022
0013
0037
0013 Ø
0028
0005
0015
0058
0035
0012
0011
0011
0013 Ø
0012
0012
0014
0080 Ø
0014
0050
0011
0015 Ø
neos-801834
bienst1
neos-1489999
neos-717614
neos-584851
neos-565815
neos-504815
aligninq
neos-880324
332
375
379
275
258
24
102
98
20
280
375
112
275
252
23
102
98
20
558000
453000
424000
373000
278000
267000
254000
118000
42000
0013
0038
0009
0013
0011
0000
1000 Ø
0011
0015 Ø
1000 Ø
0012 Ø
0077 Ø
0030 Ø
0037 Ø
0058 Ø
0060 Ø
0003
0014 Ø
0096
0026
0054 Ø
0043 Ø
0041 Ø
0047 Ø
0008
0034 Ø
0006 Ø
0057 Ø
0061 Ø
0053 Ø
0066 Ø
0038 Ø
0064 Ø
0013 Ø
0035 Ø
0027 Ø
0040 Ø
0018
0084 Ø
0065 Ø
0040 Ø
0000
1000 Ø
0012 Ø
0000
0013
0018
0000
0025
0009 Ø
1000 Ø
0012 Ø
0077 Ø
0034 Ø
0039 Ø
1000 Ø
0061 Ø
0003
1000 Ø
0047 Ø
0017
0023
0014
1000 Ø
0013 Ø
0029
0005
0015
0078 Ø
0035
0012
0011
0011
0016 Ø
0014
0046
0016
0080 Ø
0014
0050
0011
0015 Ø
0096
0026
0054 Ø
0046 Ø
0041 Ø
0091
0008
0035 Ø
0006 Ø
0057 Ø
0069
0053 Ø
0066 Ø
0038 Ø
0064 Ø
0016 Ø
0037 Ø
1000 Ø
0046 Ø
0018
0084 Ø
0065 Ø
0040 Ø
0000
COR@L with 1 s interval
0020 Ø
0073 Ø
0017 Ø
0077 Ø
0038 Ø
0033 Ø
0030
0053 Ø
0015 Ø
ences between the predictions and the true remaining
solution time for each instance in Tables 1–3.
We initially use a 10-second time interval to
compute the progress measure. If fewer than
20 predictions are obtained with 10-second time
intervals, the experiment is repeated using 1-second
time intervals. To investigate whether the predictions
become more accurate over time during the solution
process, in Tables 1–3 we report the accuracy of
predictions that are calculated after the first decrease
0015
0038
0029 Ø
0013
0012
0000
1000 Ø
0011
0015 Ø
0023 Ø
0073 Ø
0010
0077 Ø
0039 Ø
0035 Ø
0030
0053 Ø
0015 Ø
in the number of unprocessed nodes (“postdecrease”
predictions) separately. We select this time point
because it can be detected while the B&B algorithm is
running.
During our computational tests, we observed that
predictions get more accurate as the B&B algorithm
comes closer to finding the optimal solution. In particular, postoptimal predictions that are calculated after
finding the optimal solution (but before it is proven
to be optimal) are very accurate. One possible expla-
Özaltın, Hunsaker, and Schaefer: Predicting the Solution Time of B&B Algorithms for MIPs
401
INFORMS Journal on Computing 23(3), pp. 392–403, © 2011 INFORMS
Table 4
Average Ratio of 1.5-, 2-, 3-, and 4-Accurate Predictions
Over All Instances
CBC
k
GLPK
Table 5
SYMPHONY
CBC
MIP gap
SSG
MIP gap
SSG
MIP gap
SSG
k
0016
0017
0021
0024
0044
0055
0067
0073
0018
0019
0022
0025
0027
0036
0048
0054
0020
0023
0028
0033
0040
0049
0060
0067
1.5
2
3
4
1.5
2
3
4
Average Ratio of 1.5-, 2-, 3-, and 4-Accurate Predictions
Over All Predictions
nation for this intuitive result is that all remaining
unfathomed nodes must be considered regardless of
the node selection rule after the optimal solution has
been found. Furthermore, the SSG-based predictions
may be more accurate as the search tree is not subsequently repartitioned after the optimal solution has
been found. Unfortunately, it is impossible to know
when the optimal solution has been found before
termination.
We make two comparisons based on Tables 1–3.
First, we investigate which of the MIP gap-based and
the SSG-based predictions are more often accurate
over all instances. We mark the more often accurate
prediction type with a check mark.
As seen in Tables 1–3 the SSG-based predictions are
more often accurate than the MIP gap-based predictions for the majority of instances for all MIP solvers.
This is a result of the generally more steady decreasing pattern of the SSG. However, there are also many
instances for which the MIP gap-based predictions are
more accurate than the SSG-based predictions. This is
because the SSG considers the optimality gap of all
subtrees, whereas actually only one subtree that contains the optimal solution is relevant.
Next, for both the MIP gap-based and the SSG-based
predictions, we compare the ratio of accurate postdecrease predictions with the ratio of accurate overall
predictions. For most instances, the first decrease in
SSG
MIP gap
SSG
MIP gap
SSG
0012
0012
0014
0015
0043
0058
0075
0082
0014
0014
0017
0021
0025
0035
0048
0054
0017
0019
0025
0029
0043
0052
0065
0071
the number of unprocessed nodes occurs early (as seen
when the number of postdecrease predictions is close
to the total number of predictions), and so the ratio of
accurate postdecrease predictions is close to that of all
predictions. However, if the time of the first decrease
in the number of unprocessed nodes is close to half of
the total solution time (as seen when the number of
postdecrease predictions is about half of the total number of predictions), we observe that the postdecrease
predictions are generally more accurate.
Tables 4 and 5 report the average ratio of 105-, 2-,
3-, and 4-accurate MIP gap-based and SSG-based predictions over all instances and over all predictions,
respectively. As seen in these tables, for each k and for
each of the three MIP solvers, the SSG-based predictions are more accurate on average than those based
on the MIP gap.
The accuracy measure should be interpreted cautiously as the predictions calculated based on a
progress measure that stays constant during the solution process might still be accurate. For instance, consider the aligninq instance from the COR@L library
that is reported in Table 2 and depicted in Figures 5
and 6. In Figure 5, the MIP gap is constant and the
SSG is decreasing very slowly. Therefore, from Equation (7), the MIP gap-based predictions are calculated
based on the rate of increase in the number of unprocessed nodes. As seen in Figure 6, these naïve time
Progress measures: aligninq, GLPK
1.0
0.9
Measure
SYMPHONY
MIP gap
Progress measures: aligninq, GLPK
1.0
0.9
MIP gap
0.8
0.8
0.7
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0
500
550
600
650
700
750
Time (s)
Figure 5
GLPK
The MIP Gap and SSG for the aligninq Instance
800
850
0
500
SSG
550
600
650
700
Time (s)
750
800
850
Özaltın, Hunsaker, and Schaefer: Predicting the Solution Time of B&B Algorithms for MIPs
402
INFORMS Journal on Computing 23(3), pp. 392–403, © 2011 INFORMS
Prediction plot: aligninq, GLPK
Prediction plot: aligninq, GLPK
2,000
14,000
Elapsed time
Prediction of total time
12,000
1,500
10,000
(SSG)
8,000
1,000
(MIP gap)
6,000
4,000
500
2,000
0
500
(MIP gap)
550
600
650
700
750
800
850
0
500
550
600
Time (s)
Figure 6
700
750
800
850
Time (s)
The MIP Gap-Based and SSG-Based Predictions for the aligninq Instance
predictions have turned out to be accurate. On the
other hand, the SSG-based predictions, which are calculated based on a near-flat slope, overestimate the
solution time.
6.
650
Conclusions
We introduced a new progress measure, the SSG,
that is often more steady than the MIP gap, which
enables the user to get a better understanding of the
progress of B&B algorithms. The additional computational work to calculate the SSG is linear in the number
of unprocessed nodes.
We proposed a method for predicting the total solution time of B&B algorithms. We performed computational experiments on the instances from MIPLIB and
COR@L libraries using the SSG and the MIP gap as a
progress measures. We showed that making accurate
predictions of the total solution time is possible for
many MIP instances even in the early stages of B&B
algorithms if the progress measure follows a steady
decrease pattern.
During our computational tests, we observed that
the SSG-based predictions are more accurate than the
MIP gap-based predictions for most instances; however, once the MIP gap-based predictions become
accurate, they sometimes exhibit smaller variation
than the SSG-based predictions. Therefore, some
refinements on the prediction method to reduce the
variance will improve the quality of predictions.
In our prediction method, we employed a double
exponential smoothing technique to update the model
parameters because of its ease of implementation and
fast running times. The performance of other smoothing techniques remains an open question.
We did not consider the time of a prediction when
evaluating its accuracy or its deviation from the actual
solution time. Intuitively, early predictions are more
error prone because there is less information about the
instance. Future research may address that issue by
developing more-advanced performance measures.
Acknowledgments
The authors thank to two anonymous referees for constructive comments that improved a previous version of this
paper. The authors are partially supported by National Science Foundation Grants CMMI-0826141 and DMI-0620780.
References
Achterberg, T., T. Koch, A. Martin. 2006. MIPLIB 2003. Oper. Res.
Lett. 34(4) 361–372.
Box, G. E. P. 1991. George’s column: Understanding exponential
smoothing: A simple way to forecast sales and inventory. Quality Engrg. 4(1) 143–151.
Chen, P. C. 1992. Heuristic sampling: A method for predicting the
performance of tree search programs. SIAM J. Comput. 21(2)
295–315.
COR@L. 2010. Coral: Mixed-integer programming instances.
Lehigh University, Bethlehem, PA. http://coral.ie.lehigh.edu/
data-sets/mixed-integer-instances.
Cornuéjols, G., M. Karamanov, Y. Li. 2006. Early estimates of the
size of branch-and-bound trees. INFORMS J. Comput. 18(1)
86–96.
Forrest, J. J. 2007. COIN-OR branch and cut 1.01. Retreived March
13, 2010, http://projects.coin-or.org/Cbc.
Gardner, E. S., Jr., D. G. Dannenbring. 1980. Forecasting with exponential smoothing: Some guidelines for model selection. Decision Sci. 11(2) 370–383.
ILOG. 2010. IBM ILOG CPLEX optimizer. IBM ILOG, Armonk,
NY. Retrieved March 13, 2010, http://www.ilog.com/
products/cplex.
Kilby, P., J. Slaney, S. Thiébaux, T. Walsh. 2006. Estimating search
tree size. Proc. 21st Natl. Conf. Artificial Intelligence, Boston.
AAAI Press, Menlo Park, CA, 1014–1019.
Knuth, D. E. 1975. Estimating the efficiency of backtrack programs.
Math. Comput. 29(129) 121–136.
LaViola, J. J. 2003. Double exponential smoothing: An alternative to
Kalman filter-based predictive tracking. Proc. Workshop Virtual
Environments (EGVE ’03), Zurich, Vol. 39. ACM, New York,
199–206.
Makhorin, A. 2007. GLPK (GNU Linear Programming Kit), 4.15.
Retrieved March 13, 2010, http://www.gnu.org/software/glpk.
Özaltın, Hunsaker, and Schaefer: Predicting the Solution Time of B&B Algorithms for MIPs
INFORMS Journal on Computing 23(3), pp. 392–403, © 2011 INFORMS
NIST/Sematech. 2006. Double exponential smoothing. eHandbook of Statistical Methods. Retrierved March 13, 2010,
http://www.itl.nist.gov/div898/handbook/pmc/section4/
pmc433.htm.
Özaltın, O. Y., B. Hunsaker, T. K. Ralphs. 2007. BAK: Branch-andbound analysis kit. Retrieved March 13, 2010, http://www
.rosemaryroad.org/projects/BAK.
Purdom, P. W. 1978. Tree size by partial backtracking. SIAM J.
Comput. 7(4) 481–491.
403
Ralphs, T. K. 2007. SYMPHONY 5.1. Lehigh University, Bethlehem, PA. Retrieved March 13, 2010, http://www.coin-or
.org/SYMPHONY.
Snyder, R. D., A. B. Koehler, J. K. Ord. 2002. Forecasting for inventory control with exponential smoothing. Internat. J. Forecasting
18(1) 5–18.
Welch, G., G. Bishop. 1995. An introduction to the Kalman filter.
Technical Report TR 95-041, Department of Computer Science,
University of North Carolina at Chapel Hill, Chapel Hill.