Backdoors in the Context of Learning
(short paper)
Bistra Dilkina, Carla P. Gomes, Ashish Sabharwal
Cornell University
SAT-09 Conference
Swansea, U.K., June 30, 2009
SAT 2009
Ashish Sabharwal
1
SAT: Gap between theory & practice
• Boolean Satisfiability or SAT :
– Given a Boolean formula F in conjunctive normal form
e.g. F = (a or b) and (¬a or ¬c or d) and (b or c)
determine whether F is satisfiable
– NP-complete [note: “worst-case” notion]
– widely used in practice, e.g. in hardware & software verification, design
automation, AI planning, …
• Large industrial benchmarks (10K+ vars) are solved within seconds by
state-of-the-art complete/systematic SAT solvers
• Even 100K or 1M not completely out of question
• Good scaling behavior seems to defy “NP-completeness”!
Real-world problems have tractable sub-structure
“Backdoors” help explain how solvers can
get “smart” and solve very large instances
SAT 2009
Ashish Sabharwal
not quite Horn-SAT
or 2-SAT…
2
Backdoors to Tractability
A notion to capture “hidden structure”
(~500 vars)
Informally:
A backdoor to a given problem is a subset of its variables
such that, once assigned values, the remaining instance
simplifies to a tractable class.
Formally:
define a notion of a poly-time “sub-solver”
handles tractable substructure of problem instance
e.g. unit prop., pure literal elimination, CP filtering, LP solver, …
• Weak backdoors for finding feasible solutions
• Strong backdoors for finding feasible solutions or proving
unsatisfiability
SAT 2009
Ashish Sabharwal
3
Are backdoors small in practice?
Domain
graph coloring
planning
game theory
car configuration
car configuration
verification
verification
verification
Instance
gcp
map_50_97
pne
C210_FS_RZ
C210_FW_UT
ssa0432-003
bf2670-001
bf1355-638
Vars
1500
38364
5000
1755
2024
435
1393
2177
Clause
%Vars in B
187556
0.43
438840
0.25
98930.79
0.64
5764.333
0.70
9720
0.74
1027
3.91
3434
2.80
6768
10.66
Enough to branch on backdoor variables to “solve” the formula
 heuristics need to be good on only a few vars
The notion of backdoors has provided powerful insights, leading to
techniques like randomization, restarts, and algorithm portfolios for SAT
SAT 2009
Ashish Sabharwal
4
This Talk: Motivation
• “Traditional” backdoors are defined for a basic tree-search
procedure, such as pure DPLL
– Oblivious to the now-standard (and essential) feature of
learning during search, i.e, clause learning for DPLL
• Note: state-of-the-art SAT solvers rely heavily on clause
learning, especially for industrial and crafted instances
– provably leads to shorter proofs for many unsatisfiable formulas
– significant speed-up on satisfiable formulas as well
Does clause learning allow for smaller backdoors
when capturing hidden structure in SAT instances?
SAT 2009
Ashish Sabharwal
5
This Talk: Contribution
Affirmative answer:
1. First, must extend the notion of backdoors to clause
learning SAT solvers: take ‘order-sensitivity’ into account
2. Theoretically, learning-sensitive backdoors for SAT solvers
with clause learning (“CDCL solvers”) can be
exponentially smaller than traditional strong backdoors
3. Initial empirical results suggesting that in practice,
–
–
SAT 2009
More learning-sensitive backdoors than traditional (of a given size)
SAT solvers often find much smaller learning-sensitive backdoors
than traditional ones
Ashish Sabharwal
6
DPLL Search with Clause Learning
Input: CNF formula F
At every search node:
“sub-solver”
for SAT
– branch by setting a variable to True or False;
current partial variable assignment: 
– consider simplified sub-formula F|
SAT 2009
– apply a poly-time inference procedure to F|
(e.g. unit prop., pure literal test, failed literal test / “probing”)
 Contradiction  learn a conflict clause
 Solution
 declare satisfiable and exit
 Not solved
 continue branching
Ashish Sabharwal
7
Backdoors and Search with Learning
Backdoor
{
x
=1
y
Search Tree to Solution
Backdoor?
Traditional Backdoor
=0
{
=0
x
y
=1
y
=1
=0
Early contradiction
due to previously
learned clause
Sub-solver
infers solution
with help from
learned clauses
z
=1
w
=1
Contradiction:
Conflict clause
learned
Sub-solver
infers solution
SAT 2009
=0
Search order matters!
Ashish Sabharwal
8
“Traditional” Backdoors
Definition [Williams, Gomes, Selman ’03]:
A subset B of variables is a strong backdoor
(for F w.r.t. a sub-solver S)
if for every truth assignment  to variables in B,
S “solves” F|.
either finds a satisfying assignment for F
or proves that F is unsatisfiable
Issue: oblivious to “previously” learned clauses; sub-solver
must infer contradiction on F| for every  from scratch.
SAT 2009
Ashish Sabharwal
9
New: Learning-Sensitive Backdoors
Definition:
A subset B of variables is a learning-sensitive backdoor
(for F w.r.t. a sub-solver S)
if there exists a search order s.t. a clause learning solver
– branching only on the variables in B
– in this search order
– with S as the sub-solver at each leaf
“solves” F.
either finds a satisfying assignment for F
or proves that F is unsatisfiable
SAT 2009
Ashish Sabharwal
10
Theoretical Results
Learning-Sensitive Backdoors Can
Provably be Much Smaller
Setup:
• Sub-solver: unit propagation
• Clause learning scheme: 1-UIP
used Rsat for experiments
• Comparison w.r.t. traditional strong backdoors
Theorem 1: There are unsatisfiable SAT instances for which
learning-sensitive backdoors are exponentially smaller than
the smallest traditional strong backdoors.
Theorem 2: There are satisfiable SAT instances for which
learning-sensitive backdoors are smaller than the smallest
traditional strong backdoors.
SAT 2009
Ashish Sabharwal
12
Proof Idea: Simple Example
{x} is a learning-sensitive backdoor (of size 1) :
p1
a
q
x=0
contradiction
p2
a
x=1
b
r
q
(x appears only
in a “long” clause)
SAT 2009
Learn 1-UIP clause:
(q)
b
contradiction
With clause learning, branching on x
in the right order suffices to prove unsatisfiability
Ashish Sabharwal
13
Proof Idea: Simple Example
In contrast, without clause learning, must branch on
at least 2 variables in every proof of unsatisfiability!
 every “traditional” strong backdoor is of size ≥ 2
Why?
•every variable, in at least one polarity, only in “long” clauses
e.g., p1, q, r, a do not appear in any 2-clauses
•therefore, no unit prop. or empty clause generation by fixing
this variable to this value
•therefore, this variable by itself cannot be a strong backdoor
SAT 2009
Ashish Sabharwal
14
Proof Idea: Exponential Separation
Construct an unsatisfiable formula F on n vars. such that
1. certain long clauses must be used in every refutation
(i.e., removing a long clause makes F satisfiable)
2. many variables in at least one polarity appear only in such
long clauses with (n) variables
 Controlled unit propagation / empty clause generation
 Must branch on essentially all variables of the long clauses to
derive a contradiction
 Such variables must be part of every traditional backdoor set
3. With learning: conflict clauses from previous branches on
O(log n) “key variables” enable unit prop. in long clauses
SAT 2009
Ashish Sabharwal
15
Order-Sensitivity of Backdoors
Corollary (follows from the proof of Theorem 1) :
There are unsatisfiable SAT instances for which learningsensitive backdoors w.r.t. one value ordering are
exponentially smaller than the smallest learning-sensitive
backdoors w.r.t. another value ordering.
SAT 2009
Ashish Sabharwal
16
Experimental evaluation
Learning-Sensitive Backdoors in Practice
Preliminary evaluation of smallest backdoor size
Reporting “best found” backdoors over 5000 runs of
Rsat (with clause learning) or Satz-rand (no learning) :
•
•
SAT 2009
up to 10x smaller than traditional on satisfiable instances
often 2x or less smaller than traditional on unsatisfiable instances
Ashish Sabharwal
18
How hard is it to find small backdoor
sets with learning?
Recently reported in a paper at CPAIOR-09
(backdoors in the context of optimization problems)
• Considering only the size of the smallest backdoor does
not provide much insight into this question
• One way to assess this difficulty:
– How many backdoors are there of a given cardinality?
• Experimental setup:
– For each possible backdoor size k, sample uniformly at random
subsets of cardinality k from the (discrete) variables of the problem
– For each subset, evaluate whether it is a backdoor or not
SAT 2009
Ashish Sabharwal
19
Backdoor Size Distribution
E.g., for a Mixed Integer Programming (MIP)
optimization instance:
SAT 2009
Ashish Sabharwal
20
Added Power of Learning
E.g., for a Mixed Integer Programming (MIP)
optimization instance:
SAT 2009
Ashish Sabharwal
21
Summary
• Defined backdoors in the context of learning during search
(in particular, clause learning for SAT solvers)
• Proved that learning-sensitive backdoors can be smaller
than traditional strong backdoors
– Exponentially smaller on unsatisfiable instances
– Somewhat smaller on satisfiable instances (open?)
• Branching order affects backdoor size as well
Future work: stronger separation for satisfiable instances;
detailed empirical study
SAT 2009
Ashish Sabharwal
22