Exact Algorithms via Monotone Local
Search
Saket Saurabh
The Institute of Mathematical Sciences, India
and University of Bergen, Norway.
(Joint work with Fedor V. Fomin, Daniel Lokshtanov and Serge
Gaspers )
Hong Kong Theory Day 2017,
Hong Kong, January 6, 2017
1
Outline
Introduction to Parameterized and Exact Algorithms
2
Outline
Introduction to Parameterized and Exact Algorithms
Basic Questions
2
Outline
Introduction to Parameterized and Exact Algorithms
Basic Questions
Local Search and Monotone Local Search
2
Outline
Introduction to Parameterized and Exact Algorithms
Basic Questions
Local Search and Monotone Local Search
Our Algorithm
2
Outline
Introduction to Parameterized and Exact Algorithms
Basic Questions
Local Search and Monotone Local Search
Our Algorithm
Proof of Correctness and Running Time Analysis
2
Outline
Introduction to Parameterized and Exact Algorithms
Basic Questions
Local Search and Monotone Local Search
Our Algorithm
Proof of Correctness and Running Time Analysis
Applications
2
Outline
Introduction to Parameterized and Exact Algorithms
Basic Questions
Local Search and Monotone Local Search
Our Algorithm
Proof of Correctness and Running Time Analysis
Applications
Counting and Derandomization
2
Introduction to Parameterized and Exact
Algorithms
3
Vertex Cover
Vertex Cover
Input: A graph G “ pV, Eq and a positive integer k.
Parameter: k
Question: Does there exist a subset V 1 Ď V of size at most k
such that for every edge pu, vq P E either u P V 1 or v P V 1 ?
4
Example of Vertex Cover
5
Example of Vertex Cover
6
Search for 8 sized vertex cover
7
Search for 8 sized vertex cover
7
Search for 8 sized vertex cover
7
Search for 8 sized vertex cover
7
Search for 8 sized vertex cover
7
Search for 8 sized vertex cover
7
Algorithm
If there are n nodes and we are searching for a 8 sized vertex
cover the algorithm essentially takes:
ˆ ˙
n
8
time.
8
Algorithm
If there are n nodes and we are searching for a 100 sized vertex
cover the algorithm essentially takes:
ˆ
n
100
˙
time.
8
Algorithm
If there are n nodes and we are searching for a k sized vertex
cover then the algorithm essentially takes:
ˆ ˙
n
k
time steps.
8
Algorithm
If there are n nodes and we are searching for a k sized vertex
cover then the algorithm essentially takes:
ˆ ˙
n
k
time steps.
If the network/graph has 100’s of nodes then even finding a
100-sized vertex cover can take hundreds’s of centuries by
searching.
8
Algorithm
If there are n nodes and we are searching for a k sized vertex
cover then the algorithm essentially takes:
ˆ ˙
n
k
time steps.
If the network/graph has 100’s of nodes then even finding a
100-sized vertex cover can take hundreds’s of centuries by
searching.
Is searching really necessary?
8
Algorithm
If there are n nodes and we are searching for a k sized vertex
cover then the algorithm essentially takes:
ˆ ˙
n
k
time steps.
If the network/graph has 100’s of nodes then even finding a
100-sized vertex cover can take hundreds’s of centuries by
searching.
Is searching really necessary?
We don’t know.
8
Vertex Cover Continues
Suppose we give up our hope of making a polynomial time
algorithm for Vertex Cover problem and ask if we can
make “some thing like poly time algorithm” but
nevertheless still a good algorithm!
9
Vertex Cover Continues
Suppose we give up our hope of making a polynomial time
algorithm for Vertex Cover problem and ask if we can
make “some thing like poly time algorithm” but
nevertheless still a good algorithm!
For example an algorithm that given a graph on 105 nodes
can find a vertex-cover of size 17, if exists, in time 217 105 .
9
Vertex Cover Continues
Suppose we give up our hope of making a polynomial time
algorithm for Vertex Cover problem and ask if we can
make “some thing like poly time algorithm” but
nevertheless still a good algorithm!
For example an algorithm that given a graph on 105 nodes
can find a vertex-cover of size 17, if exists, in time 217 105 .
In other words an algorithm that given a graph on n nodes
can find a vertex-cover of size k, if exists, in time 2k n.
9
Vertex Cover Continues
Suppose we give up our hope of making a polynomial time
algorithm for Vertex Cover problem and ask if we can
make “some thing like poly time algorithm” but
nevertheless still a good algorithm!
For example an algorithm that given a graph on 105 nodes
can find a vertex-cover of size 17, if exists, in time 217 105 .
In other words an algorithm that given a graph on n nodes
can find a vertex-cover of size k, if exists, in time 2k n.
ˆ ˙
n
versus 2k n
k
9
Vertex Cover Continues
ˆ ˙
n
versus 2k n
k
For example an algorithm that given a graph on 105 nodes
can find a vertex-cover of size 17, if exists, in time 217 105 .
Let us compare these running times.
10
Vertex Cover Continues
ˆ ˙
n
versus 2k n
k
For example an algorithm that given a graph on 105 nodes
can find a vertex-cover of size 17, if exists, in time 217 105 .
Let us compare these running times.
ˆ 5˙
10
versus 217105
17
10
Some Comparision
k“2
k“3
k“5
k “ 10
k “ 20
Table:
n “ 50
625
15, 625
30, 625
1012
1.8 ˆ 1026
The ratio
nk`1
2k n
n “ 100
2, 500
1, 25, 000
62, 50, 000
0.8 ˆ 1014
0.5 ˆ 1031
n “ 150
5, 625
4, 21, 875
3, 16, 40, 625
3.7 ˆ 1016
2.1 ˆ 1035
for various values of n and k.
11
What did we do – A message.
We used the size of a vertex-cover to measure the running
time of the algorithm!
12
What did we do – A message.
We used the size of a vertex-cover to measure the running
time of the algorithm!
This is not what we do in classical complexity!
12
In classical complexity, a decision problem is specified by two
items of information:
The input to the problem – a number n.
The question to be answered.
13
In classical complexity, a decision problem is specified by two
items of information:
The input to the problem – a number n.
The question to be answered.
In multivariate algorithmics, a decision problem is specified by
three items of information:
The input to the problem – a number n.
The aspect of the input that constitutes the parameter – a
number k
The question to be answered.
13
Multivariate Algorithmics and a weak
magnet!
In multivariate algorithmics, a decision problem is specified by
three items of information:
The input to the problem – a number n.
The aspect of the input that constitutes the parameter – a
number k
The question to be answered.
We call an algorithm A, a fixed parameter tractable (or FPT)
algorithm, for a multivaraite problem Π, if it runs in time
f pkq ¨ nc.
14
Vertex Cover
Vertex Cover problem has an algorithm with running
time 2k n, say A, to check whether a k-sized vertex cover
exists in a graph on n nodes.
15
Vertex Cover
Vertex Cover problem has an algorithm with running
time 2k n, say A, to check whether a k-sized vertex cover
exists in a graph on n nodes. That is, Vertex Cover is
FPT.
15
Branching Algorithm for Vertex
Cover
1
Try all subsets of size at most k – Opnk mq.
16
Branching Algorithm for Vertex
Cover
1
For every edge px, yq recursively check whether G ´ x or
G ´ y has a vertex cover of size at most k ´ 1.
16
Branching Algorithm for Vertex
Cover
1
For every edge px, yq recursively check whether G ´ x or
G ´ y has a vertex cover of size at most k ´ 1.
Basic Idea: Given any edge pu, vq either u or v is in the solution.
�∅, G�
�{u},G − u�
�{v},G − v�
no. of nodes ≤ 2k
time spent at each node
= O(m)
total time taken =
O(2k · m)
�{u, x}, G� − x� �{u, y}, G� − y�
16
Fixed Parameter Tractable (FPT)
Algorithms
For decision problems with input size n, and a parameter k,
(which typically is the solution size), the goal here is to design
an algorithm with running time f pkq ¨ nOp1q , where f is a
function of k alone.
Problems that have such an algorithm are said to be fixed
parameter tractable (FPT).
17
Vertex Cover
Vertex Cover
Input: A graph G “ pV, Eq and a positive integer k.
Parameter I: k
Parameter II: n “ |V |
Question: Does there exist a subset V 1 Ď V of size at most k
such that for every edge pu, vq P E either u P V 1 or v P V 1 ?
18
Vertex Cover
Vertex Cover
Input: A graph G “ pV, Eq and a positive integer k.
Parameter I: k
Parameter II: n “ |V |
Question: Does there exist a subset V 1 Ď V of size at most k
such that for every edge pu, vq P E either u P V 1 or v P V 1 ?
What about an algorithm for Vertex Cover with respect to
parameter n?
18
Vertex Cover
Vertex Cover
Input: A graph G “ pV, Eq and a positive integer k.
Parameter I: k
Parameter II: n “ |V |
Question: Does there exist a subset V 1 Ď V of size at most k
such that for every edge pu, vq P E either u P V 1 or v P V 1 ?
What about an algorithm for Vertex Cover with respect to
parameter n?
Try all possible subsets gives us an algorithm with running
time 2n nOp1q .
18
Branching Algorithm for Vertex
Cover
1
For every edge px, yq recursively check whether G ´ x or
G ´ N rxs has a vertex cover of size at most k ´ |N pxq|.
19
Branching Algorithm for Vertex
Cover
1
For every edge px, yq recursively check whether G ´ x or
G ´ N rxs has a vertex cover of size at most k ´ |N pxq|.
19
Branching Algorithm for Vertex
Cover
1
For every edge px, yq recursively check whether G ´ x or
G ´ N rxs has a vertex cover of size at most k ´ |N pxq|.
For any edge pu, vq either u is in or all its neighbors are in the solution.
h;, Gi
h{u},G ui
T (n)  T (n
h{N(v)},G N[v]i
1) + T (n
no. of nodes  1.618
2)
n
time spent at each node
= O(m)
total time taken = 1.618n
19
Vertex Cover admits an algorithm with running time
1.1996n .
20
Vertex Cover admits an algorithm with running time
1.1996n .
For this talk we use n to denote the size of the vertex set or
the number of variables in a cnf-sat formula of an input to
the SAT problem.
20
Vertex Cover admits an algorithm with running time
1.1996n .
For this talk we use n to denote the size of the vertex set or
the number of variables in a cnf-sat formula of an input to
the SAT problem.
This parameterization will be called exact exponential time
algorithms or exact algorithms.
20
Weighted 3-SAT
3-Sat
Input: A cnf-formula ϕ with n variables and m clauses.
Question: Is ϕ satisfiable?
21
Weighted 3-SAT
3-Sat
Input: A cnf-formula ϕ with n variables and m clauses.
Question: Is ϕ satisfiable?
#1’s in a satisfying assignment is called its weight.
21
Weighted 3-SAT
3-Sat
Input: A cnf-formula ϕ with n variables and m clauses.
Question: Is ϕ satisfiable?
#1’s in a satisfying assignment is called its weight.
Weighted 3-Sat
Input: A cnf-formula ϕ with n variables and m clauses.
Parameter: k
Question: Does there exists weight k-satisfying assignment?
21
Weighted 3-SAT
3-Sat
Input: A cnf-formula ϕ with n variables and m clauses.
Question: Is ϕ satisfiable?
#1’s in a satisfying assignment is called its weight.
Weighted 3-Sat
Input: A cnf-formula ϕ with n variables and m clauses.
Parameter I: k
Parameter II: n
Question: Does there exists weight k-satisfying assignment?
21
Weighted 3-SAT
3-Sat
Input: A cnf-formula ϕ with n variables and m clauses.
Question: Is ϕ satisfiable?
#1’s in a satisfying assignment is called its weight.
Weighted 3-Sat
Input: A cnf-formula ϕ with n variables and m clauses.
Parameter I: k
Parameter II: n
Question: Does there exists weight k-satisfying assignment?
3-Sat and Weighted 3-Sat are polynomially equivalent.
21
Weighted 3-SAT
Weighted 3-Sat
Input: A cnf-formula ϕ with n variables and m clauses.
Parameter I: k
Parameter II: n
Question: Does there exists weight k-satisfying assignment?
22
Weighted 3-SAT
Weighted 3-Sat
Input: A cnf-formula ϕ with n variables and m clauses.
Parameter I: k
Parameter II: n
Question: Does there exists weight k-satisfying assignment?
1
Check if all 0̄ is a satisfying assignment.
22
Weighted 3-SAT
Weighted 3-Sat
Input: A cnf-formula ϕ with n variables and m clauses.
Parameter I: k
Parameter II: n
Question: Does there exists weight k-satisfying assignment?
1
Check if all 0̄ is a satisfying assignment.
2
Else, we know that there exists a clause C that has only
positive literals – positive clause.
22
Weighted 3-SAT
Weighted 3-Sat
Input: A cnf-formula ϕ with n variables and m clauses.
Parameter I: k
Parameter II: n
Question: Does there exists weight k-satisfying assignment?
1
Check if all 0̄ is a satisfying assignment.
2
Else, we know that there exists a clause C that has only
positive literals – positive clause.
3
Branch on a positive clause C “ px, y, zq.
22
Weighted 3-SAT
h⌧ = 0̄, , C, ki
x=1
h⌧x , , C, k
z=1
1i
y=1
h⌧y , , C, k
h⌧z , , C, k
T (k)  3T (k
1)
1i
3k nO(1)
1i
⌧x = ⌧ with x changed to 1
23
3-Hitting Set
3-Hitting Set
Input: A universe U with n elements and a family F of sets,
of size at most 3, over U of size m.
Parameter I: k
Parameter II: n
Question: Does there a set S Ď U of size at most k such that
for every F P F, F X S ‰ H?
24
3-Hitting Set
3-Hitting Set
Input: A universe U with n elements and a family F of sets,
of size at most 3, over U of size m.
Parameter I: k
Parameter II: n
Question: Does there a set S Ď U of size at most k such that
for every F P F, F X S ‰ H?
S is called 3-hitting set/hitting set.
24
3-Hitting Set
3-Hitting Set
Input: A universe U with n elements and a family F of sets,
of size at most 3, over U of size m.
Parameter I: k
Parameter II: n
Question: Does there a set S Ď U of size at most k such that
for every F P F, F X S ‰ H?
S is called 3-hitting set/hitting set.
Start with S “ H.
24
3-Hitting Set
3-Hitting Set
Input: A universe U with n elements and a family F of sets,
of size at most 3, over U of size m.
Parameter I: k
Parameter II: n
Question: Does there a set S Ď U of size at most k such that
for every F P F, F X S ‰ H?
S is called 3-hitting set/hitting set.
Start with S “ H.
If S is not a hitting set then Branch on a set
F “ tx, y, zu P F that does not intersect S!
24
3-Hitting Set
3-Hitting Set
Input: A universe U with n elements and a family F of sets,
of size at most 3, over U of size m.
Parameter I: k
Parameter II: n
Question: Does there a set S Ď U of size at most k such that
for every F P F, F X S ‰ H?
S is called 3-hitting set/hitting set.
Start with S “ H.
If S is not a hitting set then Branch on a set
F “ tx, y, zu P F that does not intersect S!
The algorithm runs in time 3k pn ` mqOp1q .
24
Books in the area
25
Basic Questions
26
The main actors of our next part will be:
27
The main actors of our next part will be:
Weighted c-Sat – 3 sized clause replaced by c sized
clause – ck pn ` mqOp1q .
27
The main actors of our next part will be:
Weighted c-Sat – 3 sized clause replaced by c sized
clause – ck pn ` mqOp1q .
c-Hitting Set – 3 sized set replaced by c sized set –
ck pn ` mqOp1q .
Will forget polynomial factor from the running time of the
parameterized algorithm during the talk.
27
Subset Optimization Problems
An implicit set system as a function Φ that takes as input a
string I P t0, 1u˚ and outputs a set system pUI , FI q, where
UI is a universe and FI is a collection of subsets of UI .
28
Subset Optimization Problems
An implicit set system as a function Φ that takes as input a
string I P t0, 1u˚ and outputs a set system pUI , FI q, where
UI is a universe and FI is a collection of subsets of UI .
Input = I
UI
FI = {subsets of UI }
28
Subset Optimization Problems
Φ-Subset
Input: An instance I
Question: A set S P FI if one exists.
29
Subset Optimization Problems
Φ-Subset
Input: An instance I
Question: A set S P FI if one exists.
Input = I
UI
S
FI = {subsets of UI }
29
Examples of Subset Optimization
Problems
Sat or c-Sat : ϕ is a formula.
Uϕ “ V (set of variables)
Fϕ “ tX | X Ď V and setting X to 1 and
V zX to 0 yields a satisfying assignmentu
30
Examples of Subset Optimization
Problems
Sat or c-Sat : ϕ is a formula.
Uϕ “ V (set of variables)
Fϕ “ tX | X Ď V and setting X to 1 and
V zX to 0 yields a satisfying assignmentu
c-Hitting Set : I “ pU, F, kq
UI “ U
FI
“ tX | X Ď V, |X| ď k and X is a hitting set u
30
Examples of Subset Optimization
Problems
Weighted c-Sat : I “ pϕ, kq, where ϕ is a formula.
UI “ V (set of variables of ϕ)
FI
“ tX | X Ď V, |X| ď k and setting X to 1 and
V zX to 0 yields a satisfying assignmentu
31
Examples of Subset Optimization
Problems
Weighted c-Sat : I “ pϕ, kq, where ϕ is a formula.
UI “ V (set of variables of ϕ)
FI
“ tX | X Ď V, |X| ď k and setting X to 1 and
V zX to 0 yields a satisfying assignmentu
Feedback Vertex Set : I “ pG, kq (does G has a set of
size at most k such that GzS is acyclic)
UI “ V pGq
FI
“ tX | X Ď V pGq, |X| ď k and GzX is acyclicu
31
Two Basic Questions?
Q1
Q2
FPT
SAT
EXACT ALGO
SUBSET
OPTIMIZATION
PROBLEMS
32
What is known for the first question?
Proposition
If a subset optimization problem is solvable in time ck , then
there exists an exact algorithm running in time
33
What is known for the first question?
Proposition
If a subset optimization problem is solvable in time ck , then
there exists an exact algorithm running in time
" ˆ ˙*
n
max min ck ,
kďn
k
33
What is known for the first question?
Proposition
If a subset optimization problem is solvable in time ck , then
there exists an exact algorithm running in time
" ˆ ˙*
n
max min ck ,
kďn
k
˘
`n
n
´
εn,
`
εn
, then
For
any
fixed
ε
ą
0,
if
k
R
2
2
`n˘
n
k ď p2 ´ αq for some fixed α ą 0.
33
What is known for the first question?
Proposition
If a subset optimization problem is solvable in time ck , then
there exists an exact algorithm running in time
" ˆ ˙*
n
max min ck ,
kďn
k
˘
`n
n
´
εn,
`
εn
, then
For
any
fixed
ε
ą
0,
if
k
R
2
2
`n˘
n
k ď p2 ´ αq for some fixed α ą 0.
So if c ď p4 ´ εqk then we get p2 ´ ε1 qn algorithm!
33
What is known for the first question?
Proposition
If a subset optimization problem is solvable in time ck , then
there exists an exact algorithm running in time
" ˆ ˙*
n
max min ck ,
kďn
k
˘
`n
n
´
εn,
`
εn
, then
For
any
fixed
ε
ą
0,
if
k
R
2
2
`n˘
n
k ď p2 ´ αq for some fixed α ą 0.
So if c ď p4 ´ εqk then we get p2 ´ ε1 qn algorithm!
Example: Using the 3k algorithm for Weighted 3-SAT,
we get 1.95n algorithm for 3-Sat!
33
Proposition
If a subset optimization problem is solvable in time ck , c ă 4,
then there exists an exact algorithm running in time
" ˆ ˙*
n
ď p2 ´ ε1 qn
max min ck ,
kďn
k
34
Proposition
If a subset optimization problem is solvable in time ck , c ă 4,
then there exists an exact algorithm running in time
" ˆ ˙*
n
ď p2 ´ ε1 qn
max min ck ,
kďn
k
A rather unsatisfactory answer!
34
Proposition
If a subset optimization problem is solvable in time ck , c ă 4,
then there exists an exact algorithm running in time
" ˆ ˙*
n
ď p2 ´ ε1 qn
max min ck ,
kďn
k
A rather unsatisfactory answer!
Proposition
If a subset optimization problem is solvable in time ck , for some
fixed c, then there exists an exact algorithm running in time
p2 ´ f pcqqn .
34
Moving to the second question.
Sat
Random Sampling and doing local search.
35
Moving to the second question.
Sat
Random Sampling and doing local search. Sampling an
assignment σ and doing local search around it.
35
Moving to the second question.
Sat
Random Sampling and doing local search. Sampling an
assignment σ and doing local search around it.
Random decision and inference
35
Moving to the second question.
Sat
Random Sampling and doing local search. Sampling an
assignment σ and doing local search around it.
Random decision and inference
Other Subset Optimization Problems
Branching!
35
Moving to the second question.
Sat
Random Sampling and doing local search. Sampling an
assignment σ and doing local search around it.
Random decision and inference
Other Subset Optimization Problems
Branching!
Pick up some element x from the universe cleverly and then
decide whether it is inside the solution or not!
35
Moving to the second question.
Sat
Random Sampling and doing local search. Sampling an
assignment σ and doing local search around it.
Random decision and inference
Other Subset Optimization Problems
Branching!
Pick up some element x from the universe cleverly and then
decide whether it is inside the solution or not!
Even for Sat branching gave the best algorithm initially.
35
Moving to the second question.
Sat
Random Sampling and doing local search. Sampling an
assignment σ and doing local search around it.
Random decision and inference
Other Subset Optimization Problems
Branching!
Pick up some element x from the universe cleverly and then
decide whether it is inside the solution or not!
Even for Sat branching gave the best algorithm initially.
GOAL: Transfer ideas from Sat to other problems.
35
Towards Goal
Focus
Random Sampling + Local Search for Sat
36
Two Gems in Schöning’s algorithm for
c-Sat: Idea I
Theorem
If Local Search c-Sat can be solved in time α` then c-Sat can be
solved in time
37
Two Gems in Schöning’s algorithm for
c-Sat: Idea I
Theorem
If Local Search c-Sat can be solved in time α` then c-Sat can be
solved in time
ˆ
˙n
2
2´
α`1
37
Two Gems in Schöning’s algorithm for
c-Sat: Idea I
Theorem
If Local Search c-Sat can be solved in time α` then c-Sat can be
solved in time
ˆ
˙n
2
2´
α`1
What is the local search?
What does the Schöning’s algorithm do?
37
Local Search
38
Local Search c-Sat
Local Search c-Sat (LS c-SAT)
Input: A c-cnf-formula ϕ, an assignment σ and a positive
integer `.
Parameter: `
Question: Does there a satisfying assignment σ 1 such that
dpσ, σ 1 q ď `?
39
Local Search c-Sat
Local Search c-Sat (LS c-SAT)
Input: A c-cnf-formula ϕ, an assignment σ and a positive
integer `.
Parameter: `
Question: Does there a satisfying assignment σ 1 such that
dpσ, σ 1 q ď `?
dpσ, σ 1 q counts the number of places σ and σ 1 differs.
Basically given σ (may not be satisfying) can we flip at
most ` variables in σ and obtain a satisfying assignment.
39
Algorithm for Local Search c-Sat
Same as Weighted c-SAT!
40
Algorithm for Local Search c-Sat
Same as Weighted c-SAT!
Algorithm
Pick an unsatisfied clause C, guess the variable in C that needs
to be flipped and flip!
40
Algorithm for Local Search c-Sat
Same as Weighted c-SAT!
Algorithm
Pick an unsatisfied clause C, guess the variable in C that needs
to be flipped and flip!
Gives us c` algorithm for LS c-SAT.
40
Algorithm for Local Search c-Sat
Same as Weighted c-SAT!
Algorithm
Pick an unsatisfied clause C, guess the variable in C that needs
to be flipped and flip!
Gives us c` algorithm for LS c-SAT.
Gives us
ˆ
2´
2
3`1
˙n
“ 1.5n
algorithm for 3-Sat
40
Two Gems in Schöning’s algorithm for
c-Sat: Idea II
Theorem
If Permissive Local Search c-Sat can be solved in time α` then
c-Sat can be solved in time
ˆ
˙n
2
2´
α`1
41
Two Gems in Schöning’s algorithm for
c-Sat: Idea II
Theorem
If Permissive Local Search c-Sat can be solved in time α` then
c-Sat can be solved in time
ˆ
˙n
2
2´
α`1
What is permissive local search?
41
Permissive Local Search c-Sat
Permissive Local Search c-Sat (PLS c-Sat)
Input: A c-cnf-formula ϕ, an assignment σ and a positive
integer `.
Parameter: `
Question: If there exists a satisfying assignment σ 1 such that
dpσ, σ 1 q ď ` then output some satisfying assignment ρ.
42
Permissive Local Search c-Sat
Permissive Local Search c-Sat (PLS c-Sat)
Input: A c-cnf-formula ϕ, an assignment σ and a positive
integer `.
Parameter: `
Question: If there exists a satisfying assignment σ 1 such that
dpσ, σ 1 q ď ` then output some satisfying assignment ρ.
Differences between LS c-Sat and PLS c-Sat
Assume there exists a satisfying assignment σ 1 such that
dpσ, σ 1 q ď `, then
In LS c-Sat we always need to output σ 1 such that
dpσ, σ 1 q ď `. In PLS c-Sat output any satisfying assignment
– even outside the ball.
42
Permissive Local Search c-Sat
Permissive Local Search c-Sat (PLS c-Sat)
Input: A c-cnf-formula ϕ, an assignment σ and a positive
integer `.
Parameter: `
Question: If there exists a satisfying assignment σ 1 such that
dpσ, σ 1 q ď ` then output some satisfying assignment ρ.
Differences between LS c-Sat and PLS c-Sat
Assume there exists a satisfying assignment σ 1 such that
dpσ, σ 1 q ď `, then
In LS c-Sat we always need to output σ 1 such that
dpσ, σ 1 q ď `. In PLS c-Sat output any satisfying assignment
– even outside the ball.
If no such assignment, then LS c-Sat always outputs NO,
but PLS c-Sat could output a satisfying assignment.
42
Algorithm for Permissive Local Search
c-Sat
Schöning’s gave pc ´ 1q` algorithm for PLS c-SAT.
43
Algorithm for Permissive Local Search
c-Sat
Schöning’s gave pc ´ 1q` algorithm for PLS c-SAT.
Gives us
ˆ
2
2´
2`1
˙n
ˆ ˙n
4
“
“ 1.333..n
3
algorithm for 3-Sat
43
Schöning Algorthm
Theorem
If (Permissive) Local Search c-Sat can be solved in time α` then
c-Sat can be solved in time
ˆ
˙n
2
2´
α`1
Question: How does this reduction from local
search to solving works?
44
Schöning Algorthm
Theorem
If (Permissive) Local Search c-Sat can be solved in time α` then
c-Sat can be solved in time
ˆ
˙n
2
2´
α`1
Question: How does this reduction from local
search to solving works?
Based on the universe size n, choose an integer ` (solely
based on n).
44
Schöning Algorthm
Theorem
If (Permissive) Local Search c-Sat can be solved in time α` then
c-Sat can be solved in time
ˆ
˙n
2
2´
α`1
Question: How does this reduction from local
search to solving works?
Based on the universe size n, choose an integer ` (solely
based on n).
Pick a random assignment σ and do permissive local search
with the parameter `.
44
Schöning Algorthm
Theorem
If (Permissive) Local Search c-Sat can be solved in time α` then
c-Sat can be solved in time
ˆ
˙n
2
2´
α`1
Question: How does this reduction from local
search to solving works?
Based on the universe size n, choose an integer ` (solely
based on n).
Pick a random assignment σ and do permissive local search
with the parameter `.
Bigger the value of ` –
44
Schöning Algorthm
Theorem
If (Permissive) Local Search c-Sat can be solved in time α` then
c-Sat can be solved in time
ˆ
˙n
2
2´
α`1
Question: How does this reduction from local
search to solving works?
Based on the universe size n, choose an integer ` (solely
based on n).
Pick a random assignment σ and do permissive local search
with the parameter `.
Bigger the value of ` – better success probability of ending at
closer to a solution –
44
Schöning Algorthm
Theorem
If (Permissive) Local Search c-Sat can be solved in time α` then
c-Sat can be solved in time
ˆ
˙n
2
2´
α`1
Question: How does this reduction from local
search to solving works?
Based on the universe size n, choose an integer ` (solely
based on n).
Pick a random assignment σ and do permissive local search
with the parameter `.
Bigger the value of ` – better success probability of ending at
closer to a solution – more time you take to find the
solution.
44
What is nice about the reduction?
Theorem
If Local Search´c-Sat can¯nbe solved in time α` then c-Sat can be
2
solved in time 2 ´ α`1
.
The reduction has nothing to do with c-Sat
problem!
45
What is nice about the reduction?
Theorem
If Local Search´c-Sat can¯nbe solved in time α` then c-Sat can be
2
solved in time 2 ´ α`1
.
The reduction has nothing to do with c-Sat
problem!
Let Π be a subset optimization problem.
Theorem
If Local Search Π can be solved in time α` then Π can be solved
in time
ˆ
˙n
2
2´
α`1
45
Local Search for Π
Local Search Π (LS Π)
Input: An instance I, a set X Ď UI , and an integer `.
Parameter: `
Question: Does there exists a subset S Ď UI such that S4X
is a solution?
46
Local Search for Π
Local Search Π (LS Π)
Input: An instance I, a set X Ď UI , and an integer `.
Parameter: `
Question: Does there exists a subset S Ď UI such that S4X
is a solution?
Basically remove some element and add some element and
get a solution.
Let us try this for c-Hitting Set
46
3-Hitting Set
3-Hitting Set
Input: A universe U with n elements and a family F of sets,
of size at most 3, over U of size m.
Parameter: k
Question: Does there a set S Ď U of size at most k such that
for every F P F, F X S ‰ H?
47
3-Hitting Set
3-Hitting Set
Input: A universe U with n elements and a family F of sets,
of size at most 3, over U of size m.
Parameter: k
Question: Does there a set S Ď U of size at most k such that
for every F P F, F X S ‰ H?
Local Search 3-Hitting Set
Input: An instance pU, F, kq, a set X Ď U , and an integer `.
Parameter: `
Question: Does there exists a subset S Ď U such that S4X
is a solution of size at most k?
47
3-Hitting Set
Local Search 3-Hitting Set
Input: An instance pU, F, kq, a set X Ď U , and an integer `.
Parameter: `
Question: Does there exists a subset S Ď U such that S4X
is a solution of size at most k?
If there is a set F that is not hit by X, find it and branch
as before.
47
3-Hitting Set
Local Search 3-Hitting Set
Input: An instance pU, F, kq, a set X Ď U , and an integer `.
Parameter: `
Question: Does there exists a subset S Ď U such that S4X
is a solution of size at most k?
If there is a set F that is not hit by X, find it and branch
as before.
If X is hitting set and |X| ą k (for example X could be U )
– what should we do?
47
3-Hitting Set
Local Search 3-Hitting Set
Input: An instance pU, F, kq, a set X Ď U , and an integer `.
Parameter: `
Question: Does there exists a subset S Ď U such that S4X
is a solution of size at most k?
If there is a set F that is not hit by X, find it and branch
as before.
If X is hitting set and |X| ą k (for example X could be U )
– what should we do?
Nothing!
Local Search 2-Hitting Set is W[1]-hard.
47
But adding elements work!
48
But adding elements work!
So let us just do that.
48
Monotone Local Search
49
New Local Search
Pick a random solution S.
Now do a local search around S that only adds elements –
that is only goes upwards – a monotone local search.
50
New Local Search
Pick a random solution S.
Now do a local search around S that only adds elements –
that is only goes upwards – a monotone local search.
Probability of success might not be as good as Schöning
but local search algorithm of this nature could be efficient.
50
Extension Problem
Φ-Extension
Input: An instance I, a set X Ď UI , and an integer k.
Parameter: k
Question: Does there exists a subset S Ď pUI zXq such that
S Y X P FI and |S| ď k?
51
Extension Problem
Φ-Extension
Input: An instance I, a set X Ď UI , and an integer k.
Parameter: k
Question: Does there exists a subset S Ď pUI zXq such that
S Y X P FI and |S| ď k?
If Φ is c-Hitting Set then it has extension algorithm with
running time ck .
51
Extension Problem
Φ-Extension
Input: An instance I, a set X Ď UI , and an integer k.
Parameter: k
Question: Does there exists a subset S Ď pUI zXq such that
S Y X P FI and |S| ď k?
If Φ is c-Hitting Set then it has extension algorithm with
running time ck .
Find an a set that is not hit by X and branch in c ways!
51
Our Algorithm
52
Algorithm
1
Choose an integer t ď k depending on c, n and k.
53
Algorithm
1
Choose an integer t ď k depending on c, n and k.
2
Select a random subset X of UI of size t.
53
Algorithm
1
Choose an integer t ď k depending on c, n and k.
2
Select a random subset X of UI of size t.
3
Do extension from X with ` “ k ´ t.
53
Running Time one Round
1
Choose an integer t ď k depending on c, n and k.
2
Select a random subset X of UI of size t.
3
Do extension from X with ` “ k ´ t.
The running time of the algorithm for one round is:
ck´t “ c`
54
Running Time one Round
1
Choose an integer t ď k depending on c, n and k.
2
Select a random subset X of UI of size t.
3
Do extension from X with ` “ k ´ t.
When does this algorithm succeed:
|S|  k
S
X
55
Running Time one Round
1
Choose an integer t ď k depending on c, n and k.
2
Select a random subset X of UI of size t.
3
Do extension from X with ` “ k ´ t.
When does this algorithm succeed:
|S|  k
S
X
`k ˘
PrrSuccess “ pX Ď Sqs “ `nt ˘
t
55
For constant probability of success ...
`n˘
k´t
`kt ˘ ¨ c
t
56
For constant probability of success ...
#` ˘+
n
min
tďk
`kt ˘
t
¨ ck´t
56
For constant probability of success ...
#` ˘+
n
max min
kďn tďk
`kt ˘
¨ ck´t
t
56
For constant probability of success ...
#` ˘+
n
max min
kďn tďk
`kt ˘
¨ ck´t
t
High school calculus shows this to be:
ˆ
1
2´
c
˙n
56
3-Hitting Set
An algorithm for 3-Hitting Set with running time:
ˆ
1
2´
3
˙n
“ 1.6666...n
57
Looking again at Extension Problem
Φ-Extension
Input: An instance I, a set X Ď UI , and an integer k.
Parameter: k
Question: Does there exists a subset S Ď pUI zXq such that
S Y X P FI and |S| ď k?
58
Looking again at Extension Problem
Φ-Extension
Input: An instance I, a set X Ď UI , and an integer k.
Parameter: k
Question: Does there exists a subset S Ď pUI zXq such that
S Y X P FI and |S| ď k?
Let Φ is c-Hitting Set then it has extension algorithm
with running time ck .
58
Looking again at Extension Problem
Φ-Extension
Input: An instance I, a set X Ď UI , and an integer k.
Parameter: k
Question: Does there exists a subset S Ď pUI zXq such that
S Y X P FI and |S| ď k?
Let Φ is c-Hitting Set then it has extension algorithm
with running time ck .
New Algorithm: Delete all the sets from F that is hit from
X and obtain an instance pU, F 1 , k ´ tq of c-Hitting Set!
58
Looking again at Extension Problem
Φ-Extension
Input: An instance I, a set X Ď UI , and an integer k.
Parameter: k
Question: Does there exists a subset S Ď pUI zXq such that
S Y X P FI and |S| ď k?
Let Φ is c-Hitting Set then it has extension algorithm
with running time ck .
New Algorithm: Delete all the sets from F that is hit from
X and obtain an instance pU, F 1 , k ´ tq of c-Hitting Set!
Extension problem is basically the parameterized problem!
58
Extension to Solving
Extension problem reduces to solving the parameterized
problem.
59
Extension to Solving
Extension problem reduces to solving the parameterized
problem.
3-Hitting Set has an algorithm with running time
p2.0755qk .
59
Extension to Solving
Extension problem reduces to solving the parameterized
problem.
3-Hitting Set has an algorithm with running time
p2.0755qk .
ˆ
1
2´
2.0755
˙n
“ 1.5182n
59
Extension to Solving
Extension problem reduces to solving the parameterized
problem.
3-Hitting Set has an algorithm with running time
p2.0755qk .
ˆ
1
2´
2.0755
˙n
“ 1.5182n
This holds for many many parameterized problems
`
˘nand if
they have ck algorithm we immediately get 2 ´ 1c exact
algorithm.
59
Convincing you that ...
#` ˘+
n
max min
kďn tďk
ˆ
ď
`kt ˘
¨ ck´t
t
1
2´
c
˙n
60
An Algorithm for Subset Optimization
Problem
Question is essentially:
Input: n, k
Question: Find a solution of size at most k if exists?
61
An Algorithm for Subset Optimization
Problem
Question is essentially:
Input: n, k
Question: Find a solution of size at most k if exists?
A deterministic algorithm – try all subsets of size at most k:
ˆ ˙
n
k
61
An Algorithm for Subset Optimization
Problem
Question is essentially:
Input: n, k
Question: Find a solution of size at most k if exists?
A randomized algorithm – pick a random subset of size k, the
success probability is:
1
`n˘
k
61
Analysis
A randomized algorithm that picks a random subset of size k by
picking one vertex at a time and inserting it into the solution.
2
1
1
k k´1 k´2
¨
¨
¨. . .¨
¨
“ `n˘
n n´1 n´2
n ´ pk ´ 2q n ´ pk ´ 1q
k
62
Analysis
A randomized algorithm that picks a random subset of size k by
picking one vertex at a time and inserting it into the solution.
2
1
1
k k´1 k´2
¨
¨
¨. . .¨
¨
“ `n˘
n n´1 n´2
n ´ pk ´ 2q n ´ pk ´ 1q
k
In the beginning of the random process the success
probability of each step is high.
62
Analysis
A randomized algorithm that picks a random subset of size k by
picking one vertex at a time and inserting it into the solution.
2
1
1
k k´1 k´2
¨
¨
¨. . .¨
¨
“ `n˘
n n´1 n´2
n ´ pk ´ 2q n ´ pk ´ 1q
k
In the beginning of the random process the success
probability of each step is high.
It gets progressively worse, and in the very end it is close to
1{n.
62
Analysis
A randomized algorithm that picks a random subset of size k by
picking one vertex at a time and inserting it into the solution.
kk´1
k´t
1
1
...
...
“ `n˘
nn ´ 1 n ´ t
n ´ pk ´ 1q
k
In the beginning of the random process the success
probability of each step is high.
It gets progressively worse, and in the very end it is close to
1{n.
At some point we have picked t vertices and pk ´ tq{pn ´ tq
drops below 1{c.
63
Analysis
A randomized algorithm that picks a random subset of size k by
picking one vertex at a time and inserting it into the solution.
kk´1
k´t
1
1
...
...
“ `n˘
nn ´ 1 n ´ t
n ´ pk ´ 1q
k
In the beginning of the random process the success
probability of each step is high.
It gets progressively worse, and in the very end it is close to
1{n.
At some point we have picked t vertices and pk ´ tq{pn ´ tq
drops below 1{c.
This is the time we run the extension algorithm, spending
time ck´t .
63
If we continued brute force we will get..
ˆ
n´t
k´t
˙
“
n´t n´t´1
n´k`2 n´k`1
¨
¨. . .¨
¨
k´t k´t´1
2
1
Product of k ´ t larger and larger terms, with even the first
and smallest term being greater than c.
64
If we continued brute force we will get..
ˆ
n´t
k´t
˙
“
n´t n´t´1
n´k`2 n´k`1
¨
¨. . .¨
¨
k´t k´t´1
2
1
Product of k ´ t larger and larger terms, with even the first
and smallest term being greater than c.
Thus, any ck algorithm will give some improvement over 2n .
64
Applications
Problem Name
Parameterized
New bound
Previous Bound
Feedback Vertex Set
Feedback Vertex Set
Subset Feedback Vertex Set
Feedback Vertex Set in Tournaments
Group Feedback Vertex Set
Node Unique Label Cover
Vertex (r, ¸)-Partization (r, ¸ Æ 2)
Interval Vertex Deletion
Proper Interval Vertex Deletion
Block Graph Vertex Deletion
Cluster Vertex Deletion
Thread Graph Vertex Deletion
Multicut on Trees
3-Hitting Set
4-Hitting Set
d-Hitting Set (d Ø 3)
Min-Ones 3-SAT
Min-Ones d-SAT (d Ø 4)
Weighted d-SAT (d Ø 3)
Weighted Feedback Vertex Set
Weighted 3-Hitting Set
Weighted d-Hitting Set (d Ø 4)
3k (r)
[13]
3.592k
[30]
k
4
[45]
1.6181k
[32]
4k
[45]
2k
| |
[45]
k
3.3146
[3, 31]
k
8
[8]
6k
[43, 7]
4k
[1]
1.9102k
[5]
k
8
[29]
1.5538k
[28]
2.0755k
[44]
3.0755k
[17]
k
(d ≠ 0.9245)
[17]
k
2.562
[33]
dk
dk
3.6181k
[1]
2.168k
[42]
(d ≠ 0.832)k [17, 42]
1.6667n (r)
1.7217n
1.7500n
1.3820n
1.7500n
(2 ≠ | 1|2 )n
1.6984n
1.8750n
1.8334n
1.7500n
1.4765n
1.8750n
1.3565n
1.5182n
1.6750n
1
(2 ≠ (d≠0.9245)
)n
1.6097n
(2 ≠ d1 )n
(2 ≠ d1 )n
1.7237n
1.5388n
1
(2 ≠ d≠0.832
)n
1.7347n
[21]
1.8638n
[19]
1.4656n
[32]
NPR
NPR
NPR
(2 ≠ Á)n for Á < 10≠20 [4]
(2 ≠ Á)n for Á < 10≠20 [4]
(2 ≠ Á)n for Á < 10≠20 [4]
1.6181n
[17]
NPR
NPR
1.6278n
[44]
1.8704n
[17]
[11, 17]
NPR
NPR
NPR
1.8638n
[18]
1.6755n
[11]
[11]
Table 1: Summary of known and new results for different optimization problems. NPR means that
we are not aware of any previous algorithms faster than brute-force. All bounds suppress factors
polynomial in the input size N . The algorithms in the first row are randomized (r).
65
Extension to Enumeration and
Combinatorial upper bounds
Theorem
Let c ą 1 and
` Φ be
˘ an implicit set system. If Φ is c-uniform,
1 n Op1q
then |FI | ď 2 ´ c n
for every instance I.
66
Applications
Problem Name
c-uniform New bound Previous Bound
Minimal FVSs in Tournaments
Minimal 3-Hitting Sets
Minimal 4-Hitting Sets
Minimal 5-Hitting Sets
Minimal d-Hitting Sets
Minimal d-Sat (d Ø 2)
Minimal FVSs in chordal graphs
Minimal Subset FVSs in chordal graphs
Maximal r-colorable induced subgraphs of perfect graphs
3
3
4
5
d
d
3
3
r+1
1.6667n
1.6667n
1.7500n
1.8000n
(2 ≠ d1 )n
(2 ≠ d1 )n
1.6667n
1.6667n
1
(2 ≠ r+1
)n
1.6740n
1.6755n
1.8863n
1.9538n
(2 ≠ ‘d )n with ‘d < 1/d
NPR
1.6708n
NPR
NPR
[23]
[11]
[11]
[11]
[11]
[24]
Table 2: Summary of known and new results for different combinatorial bounds. NPR means that
we are not aware of any previous results better than 2n . All bounds suppress factors polynomial in
the universe size n.
By a classical theorem of Moon from 1971 [36], the number of maximal transitive subtournaments
in an n-vertex tournament does not exceed 1.7170n . In [23], Gaspers and Mnich improved this
bound to 1.6740n . Our approach yields immediately a better bound of O(1.6667n ) since every
directed 3-cycle needs to be hit. Similarly, in chordal graphs, a set is a feedback vertex set (FVS)
if it hits every 3-cycle. For maximal r-colorable induced subgraphs of perfect graphs it suffices to
67
Conclusion and Derandomization
Theorem
If there exists an algorithm for Φ-Extension with running time
ck nOp1q then there exists a randomized algorithm for Φ-Subset
with running time p2 ´ 1c qn nOp1q .
68
Conclusion and Derandomization
Theorem
If there exists an algorithm for Φ-Extension with running time
ck nOp1q then there exists a randomized algorithm for Φ-Subset
with running time p2 ´ 1c qn nOp1q .
Theorem
If there exists an algorithm for Φ-Extension with running time
ck nOp1q then there exists an algorithm for Φ-Subset with
running time p2 ´ 1c qn`opnq nOp1q .
68
Final Slide
Thank You!
Any Questions?
69