Quality of LP-based
Approximations for Highly
Combinatorial Problems
Lucian Leahu and Carla Gomes
Computer Science Department
Cornell University
Motivation

Increasing interest in combining Constraint Satisfaction Problem
(CSP) formulations and Linear Programming (LP) based techniques
for solving hard computational problems.

Successful results for solving problems that are a mixture of linear
constraints – where LP excels – and combinatorial constraints –
where CSP excels.
In a purely combinatorial setting,
surprisingly difficult to effectively
integrate LP- and CSP-based techniques
Goal
Study and characterize the quality of
LP based heuristics
for highly combinatorial problems.
Research Questions

Is the quality of LP-based Approximations related to
the structure of the problem? (Typical case, rather
than worst case)

How is the quality of LP-based Approximations
influenced by different formulations of the problem?

Does the LP relaxation provide a global perspective
of the search space? Is the LP relaxation good as a
heuristic to guide complete solvers?
Outline


A highly combinatorial search problem --quasigroup completion problem (QCP)
LP-based formulations for QCP





Assignment based formulation
Packing formulation
Quality of LP based approximations
LP as a global search heuristic
Conclusions
Latin Squares or Quasigroups

Given an N X N matrix, and given N colors, a
quasigroup of order N is a a colored matrix, such
that:



all cells are colored.
each color occurs exactly once in each row.
each color occurs exactly once in each column.
Quasigroup or Latin Square
(Order 4)
Latin Squares/Quasigroups
Completion Problem

Given a partial assignment of colors (10 colors in this
case), can the partial Latin Square be completed so we
obtain a full square?
Latin Squares/Quasigroups
Completion Problem

Given a partial assignment of colors (10 colors in this
case), can the partial Latin Square be completed so we
obtain a full square?
Example:
Structure of this problem characterizes several real-world applications: e.g.,
Timetabling, sports scheduling, rostering, routing, etc.
Quasigroup with Holes (QWH)
Given
a full quasigroup, “punch” holes into it
32% holes
QWH is NP-Hard.
Advantage: we know the optimal value.
LP-based formulations for QCP
Assignment Formulation
n3
cell i, j has color k; i, j,k 1, 2, ...,n.
Variables -
x
ijk
x  {0,1}
ijk
x
1
s.t. PLS  k
ij
i, j,k
i, j,k
Max
number of colored cells
s.t.
at most one color per cell:
a color appears at most once per row
a color appears at most once per column
Assignment Formulation
Max value of LP Relaxation
No of backtracks
New Phase Transition Phenomenon:
Integrality of LP
Sudden phase Transition in
solution integrality of LP relaxation
and it coincides with the hardest area
holes/n^1.55
Note: standard phase
transition curves are w.r.t
existence of solution)
Packing formulation
Families of patterns

(partial patterns are not shown)
Max
number of colored cells in the
selected patterns
s.t.
one pattern per family
a cell is covered at most by
one pattern
Packing formulation
Previous Results

0.5 approximation based on Assignment formulation –
Kumar et al. – 1999

(1-1/e ≈ 0.63) approximation based on Packing
formulation – Gomes, Regis, Shmoys – 2003

Use of LP to select variables and values and to prune
search trees – Refalo et al. – 1999, 2000

No typical case results on the quality of LP based
approximation
Quality of LP-based Approximations
Approximation Schemes

LP Formulations:



Approximation scheme:


Assignment formulation;
Packing formulation;
solve the LP relaxation and interpret the resulting solution as a
probability distribution;
Order for Variable Setting
Uniformly at Random
 Greedy Random
 Greedy Deterministic

Increasing greediness
Uniformly at Random
Uniformly at Random
% of colored holes
% of colored holes
Uniformly at Random
holes/n^1.55
holes/n^1.55
% of colored holes
Uniformly Random - Comparison
holes/n^1.55

Drop in quality of approximation as
we enter the critically constrained
area

The quality stabilizes in the under
constrained area

Random LP Packing does better,
since the corresponding LP relaxation
is stronger

Random LP Packing is a 1 – 1/e≈0.63
approximation, while LP assignment
½ approximation.
Greedy Random
Greedy Random
% of colored holes
% of colored holes
Greedy Random
holes/n^1.55
holes/n^1.55
% of colored holes
Greedy Random - Comparison
holes/n^1.55

Drop in quality of approximation as
we enter the critically constrained
area

The quality increases in the under
constrained area --- info provided by
LP is used in a more greedy way
(more valuable); forward checking
also improves quality.

Random LP Packing does slightly
worse, since it optimizes an entire
matching
Greedy Deterministic
Greedy Deterministic
% of colored holes
Greedy Deterministic - Comparison
holes/n^1.55

Drop in quality of approximation as
we enter the critically constrained
area

The quality increases in the under
constrained area --- info provided by
LP is used in a more greedy way and
deterministically (more valuable);
forward checking also improves
quality.

Random LP Packing does slightly
worse, since it is less greedy (sets an
entire matching), doesn’t use as
much lookahead
% of colored holes
Comparison with Pure Random Strategy
holes/n^1.55
LP as a Global Search Heuristic

Can LP guide complete solvers?

Use an LP relaxation to set a certain percent of variables
(the highest values)

Run a complete solver on the resulting instances and
check if it is still completable (we start with a PLS that is
completable)
LP as a Global Search Heuristic - Results
5%
% of satisfiable instances
% of satisfiable instances
1 hole
holes/n^1.55
holes/n^1.55
Conclusions

Quality of approximation is directly correlated with phase
transition phenomenon – closely related to constrainedness
regions of the problem (sharp decrease in the critical region) 
New phase transition in the integrality of the LP relaxation solution

Typical case analysis – although theoretical bounds for LP packing
are stronger, the empirical results for enhanced versions of
approximations (with forward-checking) seem to indicate that LP
approximations based on the assignment formulation are better (but
difficult to analyze theoretically)

LP can provide useful high level guidance + should be combined
with random restart strategies to recover from potential mistakes
made at the top of the tree
Quality of LP-based
Approximations for Highly
Combinatorial Problems
Lucian Leahu and Carla Gomes
Computer Science Department
Cornell University