Approximation Algorithms Recap
Advanced Algorithms – COMS31900
An algorithm A is an α-approximation for problem P if,
2013/2014
◦ A runs in polynomial time
◦ A always outputs a solution with value s
within an α factor of Opt
Lecture 18
Approximation Algorithms (part three)
• Here P is an optimisation problem with optimal solution of value Opt
(Fully) Polynomial Time Approximation Schemes
• If P is a maximisation problem,
Benjamin Sach
Opt
α
6 s 6 Opt
• If P is a minimisation problem, Opt 6 s 6 α · Opt
We have seen:
The Subset Sum problem
|S| = m
a 2-approximation for Max Sat
a 3/2-approximation for Bin Packing
a 2-approximation for k -centers
a 3/2-approximation for
scheduling multiple machines
An exact solution
|S| = m
• Let S = {s1 , s2 , s3 . . . sm } be the set of items and Si = {s1 , s2 , . . . , si }
• Let Li be the set of sizes of all S ′ ⊆ Si which are not larger than t
t = 12
4
4
2
(t = 12)
S
7
L4 =
3
2
{0, 2, 4, 6, 7, 8, 9, 11}
7
• Let S be a multi-set of postive integers and t be a positive integer
here S = {4, 2, 4, 7, 2, 3} and t = 12
4
4
2
2
3
′
Decision Problem Is there a subset, S ⊆ S with size t?
the size of S ′ is
Optimisation Problem
P
a∈S ′
a
S4
Find the size of the largest subset of S which is no larger than t
The optimisation version is NP-hard
and the decision version is NP-complete
An exact solution
|S| = m
|S| = m
• Let S = {s1 , s2 , s3 . . . sm } be the set of items and Si = {s1 , s2 , . . . , si }
′
• Let Li be the set of sizes of all S ⊆ Si which are not larger than t
(t = 12)
S
L4 =
{0, 2, 4, 6, 7, 8, 9, 11}
L4 + s 5 = L4 + 2 =
7
2
2
{2, 4, 6, 8, 9, 10, 11}
An exact solution
O(1) time
The algorithm
O(|Li−1 |) time
◦ Let L0 = {0}
◦ For i = 1 . . . m:
O(|Li |) time
◦ Compute (Li−1 + si ) from Li−1
◦ Compute Li = Li−1 ∪ (Li−1 + si )
◦ Output the largest number in Lm
• Each Li is of length |Li | 6 t
• The overall time complexity is therefore O(mt)
S4
s5
L5 = {0, 2, 4, 6, 7, 8, 9, 10, 11}
• The largest subset of S (of size at most t) is the largest number in Lm
• We compute Li from Li−1 :
O(|Lm |) time
Is this polynomial in n?
What even is n?
n is the length of the input (measured in words)
Input
n words
Li = Li−1 ∪ (Li−1 + si )
where (x + si ) ∈ (Li−1 + si ) iff x ∈ Li−1 and x + si 6 t
We don’t have any duplicates in Li - so |Li | 6 t
a w bit word
(conventionally w ∈ Θ(log n))
An exact solution
|S| = m
Pseudo-polynomial time algorithms
O(1) time
The algorithm
O(|Li−1 |) time
◦ Let L0 = {0}
◦ For i = 1 . . . m:
O(|Li |) time
◦ Compute (Li−1 + si ) from Li−1
◦ Compute Li = Li−1 ∪ (Li−1 + si )
◦ Output the largest number in Lm
for some constant c
• The algorithm for Subset Sum given takes O(nt) ∈ O(nc+1 ) time
in this case
• We say that an NP-complete problem is weakly NP-complete if
Is this polynomial in n?
• The overall time complexity is therefore O(mt)
What even is n?
n is the length of the input (measured in words)
s1
if it runs in poly-time when all the numbers are integers 6 nc
O(|Lm |) time
• Each Li is of length |Li | 6 t
Input
• We say that an algorithm is pseudo-polynomial time
s2
s3
there is a pseudo-polynomial time solution for it
• We say that an NP-complete problem is strongly NP-complete if
it is NP-complete when all the numbers are integers 6 nc
t
n words
The input to the Subset Sum problem is a list of the elements of S along with t
encoded in binary in a total of n words
• So Subset Sum is weakly NP-complete
• Bin packing is strongly NP-complete
. . . but t could be as big as 2n
As m 6 n, the time is O(nt)
Polynomial time approximation schemes
Polynomial time approximation schemes
A polynomial time approximation scheme (PTAS) for problem P
A polynomial time approximation scheme (PTAS) for problem P
is a family of algorithms:
is a family of algorithms:
For any constant ǫ > 0 there is an algorithm in the family, Aǫ
For any constant ǫ > 0 there is an algorithm in the family, Aǫ
such that Aǫ is a (1 + ǫ)-approximation for P
such that Aǫ is a (1 + ǫ)-approximation for P
• If we had a PTAS for Subset Sum,
• If we had a PTAS for Subset Sum,
Let ǫ = 0.1 and so A0.1 runs in polynomial time and
Let ǫ = 0.1 and so A0.1 runs in polynomial time and
Opt
Opt
outputs a subset of size at least 1.1 > 0.9 · Opt
outputs a subset of size at least 1.1 > 0.9 · Opt
Let ǫ = 0.01 and so A0.01 also runs in polynomial time and
• A PTAS does not have to have running time polynomial in 1/ǫ
Opt
outputs a subset of size at least 1.01 > 0.99 · Opt
c
Aǫ can have a time complexity of O(n ǫ ) for example
Let ǫ = 0.001 and so A0.001 also runs in polynomial time and
O(n10c ) vs. O(n100c ) vs. O(n1000c ) in our example
Opt
outputs a subset of size at least 1.001 > 0.999 · Opt
S = {4, 2, 4, 7, 2, 3}
t = 11
Polynomial time approximation schemes
S4 = {4, 2, 4, 7}
Let Li be the set of sizes of all S ′ ⊆ Si which are not larger than t
A polynomial time approximation scheme (PTAS) for problem P
is a family of algorithms:
• The exact algorithm for Subset Sum was slow (in general) because
each list of possible subset sizes Li could become very large
For any constant ǫ > 0 there is an algorithm in the family, Aǫ
such that Aǫ is a (1 + ǫ)-approximation for P
Key Idea Constructed a trimmed version of Li (denoted L′i ) so that
◦The length of L′i is polynomial in the input length
• If we had a PTAS for Subset Sum,
◦L′i ⊆ Li and for every y ∈ Li , there is a z ∈ L′i which
Let ǫ = 0.1 and so A0.1 runs in polynomial time and
is almost as big
Opt
outputs a subset of size at least 1.1 > 0.9 · Opt
L5 = {0, 2, 4, 6, 7, 8, 9, 10, 11}
• A PTAS does not have to have running time polynomial in 1/ǫ
Trim(Li , δ): Include Li [j] in L′i iff Li [j] > (1 + δ) · prev
where prev is the previous thing we included in L′i
• A fully PTAS (FPTAS) is also polynomial in 1/ǫ
i.e. the time complexity is O((n/ǫ)c ) for some constant c
c
A PTAS for Subset Sum
c
c
for δ = 1 we include. . .
c
In our example O((10n) ) = O((100n) ) = O((1000n) ) = O(n )
L′5 = {0, 2, 6}
we will pick δ later (and it will be much smaller than 1)
S = {4, 2, 4, 7, 2, 3}
t = 11
Let Li be the set of sizes of all S ′ ⊆ Si which are not larger than t
- L′i is the trimmed version of Li
The algorithm
◦ Let L′0 = {0}, δ = ǫ/(2m)
◦ For i = 0 . . . m:
◦ Compute (L′i−1 + si ) from L′i−1
◦ Compute U = L′i−1 ∪ (L′i−1 + si )
◦ Let L′i = Trim(U, δ)
◦
Li vs. L′i
S4 = {4, 2, 4, 7}
A PTAS for Subset Sum
Lemma For any y ∈ Li there is an z ∈ L′i with
y
(1+δ)i
Proof (by induction)
O(|L′i−1 |) time
O(|L′i−1 |) time
O(|L′i |) time
Output the largest number in L′m
O(|L′m |) time
Base Case: L0 = L′0 = {0}
Inductive step: Assume that the lemma holds for (i − 1)
As y ∈ Li we have that either y ∈ Li−1 or (y − si ) ∈ Li−1
y
if y ∈ Li−1 then there is a x ∈ L′i−1 with
(1+δ)(i−1)
6x6y
By the definition of Trim there is some z ∈ L′i with z 6 x 6 z · (1 + δ)
• This process throws away some subsets,
y
x
So we have that z 6 x 6 y and z > 1+δ
> (1+δ)i
but it outputs a valid solution
y
I.e. that there is an z ∈ L′i with
(1+δ)i
• Two questions remain. . .
How big is |L′i |?
6z6y
How good is the solution given?
The case that (y − si ) ∈ Li−1 is almost identical
Li vs. L′i
Lemma For any y ∈ Li there is an z ∈ L′i with
as required
(we omit it for brevity)
Li vs. L′i
y
(1+δ)i
6z6y
y
(1 +
ǫ m
2m )
!
ǫ m
We need to show that 1 + 2m
6 1 + ǫ (for 0 < ǫ 6 1)
• By setting i = m and δ = ǫ/2m we have that,
For any y ∈ Lm there is a z ∈ L′m with
6z6y
ǫ m
ǫ ǫ 2
ǫ/2
1+
6 e 6 1+ +
6 1+ǫ
2m
2
2
6z6y
This follows from:
• Further, Opt ∈ Lm meaning there is a z ∈ L′m with
Opt
!
6 z 6 Opt
ǫ m
1 + 2m
ex > (1 +
ex =
!
ǫ m
We only need to show that 1 + 2m
6 1 + ǫ ...
Li vs. L′i
!
ǫ m
We need to show that 1 + 2m
6 1 + ǫ (for 0 < ǫ 6 1)
ǫ ǫ 2
ǫ m
ǫ/2
6 e 6 1+ +
6 1+ǫ
1+
2m
2
2
x m
m ) for all
xi
i=0 i!
P∞
x, m > 0
6 1 + x + x2
How big is L′i ?
The time complexity depends on |L′i |. . .
• By the definition of Trim we have that,
any two successive elements, z, z ′ of L′i have
z′
z
• So the output of the algorithm is indeed,
Opt
6 z 6 Opt
1+ǫ
• But how efficient is it?
>1+δ =1+
ǫ
2m
• Further, all elements are no greater than t
• So L′i contains at most O(log(1+δ) t) elements
log(1+δ) t =
x
x+1
(here x = ǫ/2m)
ln(1 + x) >
2m(1 + (ǫ/2m)) ln t
ln t
6
=O
ln(1 + (ǫ/2m))
ǫ
m log t
ǫ
A PTAS for Subset Sum
The algorithm
′
O(|Li−1 |) time
◦ Let L′0 = {0}, δ = ǫ/(2m)
◦ For i = 0 . . . m:
O(|L′i |) time
◦ Compute (L′i−1 + si ) from L′i−1
◦ Compute U = L′i−1 ∪ (L′i−1 + si )
◦ Let L′i = Trim(U, δ)
O(|L′i |) time
◦ Output the largest number in L′m
O(|L′m |) time
•
As |L′i |
∈ O(m log t/ǫ), the algorithm runs in
O(m2 log t/ǫ) ∈ O(n3 /ǫ) time
• The solution outputted is at least
• So this is in fact an FPTAS for Subset Sum
Opt
6 z 6 Opt
1+ǫ