Coping with NP-completeness
11. A PPROXIMATION A LGORITHMS
Q. Suppose I need to solve an NP-hard problem. What should I do?
A. Sacrifice one of three desired features.
‣ load balancing
i.
‣ center selection
ii. Solve problem to optimality.
‣ pricing method: vertex cover
‣ LP rounding: vertex cover
‣ generalized load balancing
‣ knapsack problem
Solve arbitrary instances of the problem.
iii. Solve problem in polynomial time.
ρ-approximation algorithm.
・Guaranteed to run in poly-time.
・Guaranteed to solve arbitrary instance of the problem
・Guaranteed to find solution within ratio ρ of true optimum.
Challenge. Need to prove a solution’s value is close to optimum,
Lecture slides by Kevin Wayne
Copyright © 2005 Pearson-Addison Wesley
without even knowing what optimum value is
http://www.cs.princeton.edu/~wayne/kleinberg-tardos
Last updated on 7/25/17 11:06 AM
2
Load balancing
11. A PPROXIMATION A LGORITHMS
‣ load balancing
‣ center selection
‣ pricing method: vertex cover
‣ LP rounding: vertex cover
Input. m identical machines; n jobs, job j has processing time tj.
・Job j must run contiguously on one machine.
・A machine can process at most one job at a time.
Def. Let J(i) be the subset of jobs assigned to machine i.
The load of machine i is Li = Σj ∈ J(i) tj.
Def. The makespan is the maximum load on any machine L = maxi Li.
‣ generalized load balancing
‣ knapsack problem
Load balancing. Assign each job to a machine to minimize makespan.
machine 1
d Machine 1
a
b
machine 2
0
c
Machine e2
f
g
L1
L2
time
4
Load balancing on 2 machines is NP-hard
Load balancing: list scheduling
Claim. Load balancing is hard even if only 2 machines.
List-scheduling algorithm.
・Consider n jobs in some fixed order.
・Assign job j to machine whose load is smallest so far.
Pf. PARTITION ≤ P LOAD-BALANCE.
NP-complete by Exercise 8.26
List-Scheduling(m, n, t1,t2,…,tn) {
a
b
c
for i = 1 to m {
Li ← 0
d
J(i) ← ∅
e
f
g
d Machine 1
a
for j = 1 to n {
i = argmink Lk
machine 2
c
machine i has smallest load
J(i) ← J(i) ∪ {j}
assign job j to machine i
Li ← Li + tj
update load of machine i
}
return J(1), …, J(m)
f
}
yes
b
jobs assigned to machine i
}
length of job f
machine 1
load on machine i
Machine e2
g
Implementation. O(n log m) using a priority queue.
0
L
time
5
Load balancing: list scheduling analysis
6
Believe it or not
Theorem. [Graham 1966] Greedy algorithm is a 2-approximation.
・First worst-case analysis of an approximation algorithm.
・Need to compare resulting solution with optimal makespan L*.
Lemma 1. The optimal makespan L* ≥ maxj tj.
Pf. Some machine must process the most time-consuming job. ▪
Lemma 2. The optimal makespan L * ≥
1
m∑j
tj.
L* ≥
1
m∑j
tj.
Pf.
・The total processing time is Σj tj .
€ do at least a 1 / m fraction of total work.
・One of m machines must
▪
€
7
8
Load balancing: list scheduling analysis
Load balancing: list scheduling analysis
Theorem. Greedy algorithm is a 2-approximation.
Theorem. Greedy algorithm is a 2-approximation.
Pf. Consider load Li of bottleneck machine i.
Pf. Consider load Li of bottleneck machine i.
・Let j be last job scheduled on machine i.
・When job j assigned to machine i, i had smallest load.
・Let j be last job scheduled on machine i.
・When job j assigned to machine i, i had smallest load.
Its load before assignment is Li – tj ⇒ Li – tj ≤ Lk for all 1 ≤ k ≤ m.
Its load before assignment is Li – tj ⇒ Li – tj ≤ Lk for all 1 ≤ k ≤ m.
・Sum inequalities over all k and divide by m:
Li − tj
blue jobs scheduled before j
Lemma 2
・Now,
j
machine i
1
m
1
m
∑ k Lk
∑ k tk
≤ L*
≤
=
L = Li = (Li − t j ) + t j ≤ 2L *.
%
!
$
€#"#
≤ L*
▪
≤ L*
Lemma 1
€
0
L = Li
Li - tj
time
9
10
Load balancing: list scheduling analysis
Load balancing: list scheduling analysis
Q. Is our analysis tight?
Q. Is our analysis tight?
A. Essentially yes.
A. Essentially yes.
Ex: m machines, m (m – 1) jobs length 1 jobs, one job of length m.
Ex: m machines, m (m – 1) jobs length 1 jobs, one job of length m.
list scheduling makespan = 19 = 2m - 1
optimal makespan = 10 = m
machine 2 idle
machine 3 idle
machine 4 idle
m = 10
m = 10
machine 5 idle
machine 6 idle
machine 7 idle
machine 8 idle
machine 9 idle
machine 10 idle
0
9
19
11
0
9 10
19
12
Load balancing: LPT rule
Load balancing: LPT rule
Longest processing time (LPT). Sort n jobs in descending order of
Observation. If at most m jobs, then list-scheduling is optimal.
processing time, and then run list scheduling algorithm.
Pf. Each job put on its own machine. ▪
Lemma 3. If there are more than m jobs, L* ≥ 2 tm+1 .
Pf.
LPT-List-Scheduling(m, n, t1,t2,…,tn) {
Sort jobs so that t1 ≥ t2 ≥
for i = 1 to m {
Li ← 0
・Consider first m+1 jobs t1, …, tm+1.
・Since the ti’s are in descending order, each takes at least tm+1 time.
・There are m + 1 jobs and m machines, so by pigeonhole principle,
… ≥ tn
load on machine i
J(i) ← ∅
at least one machine gets two jobs. ▪
jobs assigned to machine i
}
for j = 1 to n {
i = argmink Lk
Theorem. LPT rule is a 3/2-approximation algorithm.
Pf. Same basic approach as for list scheduling.
machine i has smallest load
J(i) ← J(i) ∪ {j}
assign job j to machine i
Li ← Li + tj
update load of machine i
L i = (Li − t j ) + t j
%
!
#"#
$
≤ L*
}
return J(1), …, J(m)
}
13
€
≤
3 L *.
2
▪
≤ 12 L*
Lemma 3
( by observation, can assume number of jobs > m )
14
Load Balancing: LPT rule
Q. Is our 3/2 analysis tight?
11. A PPROXIMATION A LGORITHMS
A. No.
‣ load balancing
‣ center selection
Theorem. [Graham 1969] LPT rule is a 4/3-approximation.
‣ pricing method: vertex cover
Pf. More sophisticated analysis of same algorithm.
‣ LP rounding: vertex cover
Q. Is Graham’s 4/3 analysis tight?
‣ generalized load balancing
A. Essentially yes.
‣ knapsack problem
Ex.
・m machines
・n = 2m + 1 jobs
・2 jobs of length m, m + 1, …, 2m – 1 and one more job of length m.
・Then, L / L* = (4m − 1) / (3m)
15
Center selection problem
Center selection problem
Input. Set of n sites s1, …, sn and an integer k > 0.
Input. Set of n sites s1, …, sn and an integer k > 0.
Center selection problem. Select set of k centers C so that maximum
Center selection problem. Select set of k centers C so that maximum
distance r(C) from a site to nearest center is minimized.
distance r(C) from a site to nearest center is minimized.
Notation.
k = 4 centers
・dist(x, y) = distance between sites x and y.
・dist(si, C) = min c ∈ C dist(si, c) = distance from si to closest center.
・r(C) = maxi dist(si, C) = smallest covering radius.
r(C)
Goal. Find set of centers C that minimizes r(C), subject to | C | = k.
Distance function properties.
・dist(x, x) = 0
・dist(x, y) = dist(y, x)
・dist(x, y) ≤ dist(x, z) + dist(z, y)
center
site
[ identity ]
[ symmetry ]
[ triangle inequality ]
17
18
Center selection example
Greedy algorithm: a false start
Ex: each site is a point in the plane, a center can be any point in the plane,
Greedy algorithm. Put the first center at the best possible location for a
dist(x, y) = Euclidean distance.
single center, and then keep adding centers so as to reduce the covering
radius each time by as much as possible.
Remark: search can be infinite!
Remark: arbitrarily bad!
k = 4 centers
k = 2 centers
r(C)
greedy center 1
center
site
center
site
19
20
Center selection: greedy algorithm
Center selection: analysis of greedy algorithm
Repeatedly choose next center to be site farthest from any existing center.
Lemma. Let C* be an optimal set of centers. Then r(C) ≤ 2r(C*).
Pf. [by contradiction] Assume r(C*) < ½ r(C).
・For each site ci ∈ C, consider ball of radius ½ r(C) around it.
・Exactly one ci* in each ball; let ci be the site paired with ci*.
・Consider any site s and its closest center ci* ∈ C*.
・dist(s, C) ≤ dist(s, ci) ≤ dist(s, ci*) + dist(ci*, ci) ≤ 2r(C*).
・Thus, r(C) ≤ 2r(C*). ▪
≤ r(C*) since c * is closest center
GREEDY-CENTER-SELECTION (k, n, s1, s2, … , sn)
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
C ← ∅.
REPEAT k times
i
Δ-inequality
Select a site si with maximum distance dist(si, C).
C ← C ∪ si.
RETURN C.
site farthest
from any center
½ r(C)
½ r(C)
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
ci
½ r(C)
Property. Upon termination, all centers in C are pairwise at least r(C) apart.
C*
Pf. By construction of algorithm.
site
s
ci*
21
22
Center selection
Dominating set reduces to center selection
Lemma. Let C* be an optimal set of centers. Then r(C) ≤ 2r (C*).
Theorem. Unless P = NP, there no ρ-approximation for center selection
problem for any ρ < 2.
Theorem. Greedy algorithm is a 2-approximation for center selection
problem.
Pf. We show how we could use a (2 – ε) approximation algorithm for
CENTER-SELECTION selection to solve DOMINATING-SET in poly-time.
・Let G = (V, E), k be an instance of DOMINATING-SET.
・Construct instance Gʹ of CENTER-SELECTION with sites V and distances
Remark. Greedy algorithm always places centers at sites, but is still within
a factor of 2 of best solution that is allowed to place centers anywhere.
- dist(u, v) = 1 if (u, v) ∈ E
- dist(u, v) = 2 if (u, v) ∉ E
e.g., points in the plane
・Note that Gʹ satisfies the triangle inequality.
・G has dominating set of size k iff there exists k centers C* with r(C*) = 1.
・Thus, if G has a dominating set of size k, a (2 – ε)-approximation
Question. Is there hope of a 3/2-approximation? 4/3?
algorithm for CENTER-SELECTION would find a solution C* with r(C*) = 1
since it cannot use any edge of distance 2. ▪
23
24
Weighted vertex cover
11. A PPROXIMATION A LGORITHMS
Definition. Given a graph G = (V, E), a vertex cover is a set S ⊆ V such that
each edge in E has at least one end in S.
‣ load balancing
Weighted vertex cover. Given a graph G with vertex weights, find a vertex
‣ center selection
cover of minimum weight.
‣ pricing method: vertex cover
‣ LP rounding: vertex cover
2
4
2
4
2
9
2
9
‣ generalized load balancing
‣ knapsack problem
weight = 2 + 2 + 4
weight = 11
26
Pricing method
Pricing method
Pricing method. Each edge must be covered by some vertex. Set prices and find vertex cover simultaneously.
Edge e = (i, j) pays price pe ≥ 0 to use both vertex i and j.
Fairness. Edges incident to vertex i should pay ≤ wi in total.
WEIGHTED-VERTEX-COVER (G, w)
__________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
2
S ← ∅.
4
FOREACH e ∈ E
for each vertex i : ∑ pe ≤ wi
pe ← 0.
e=(i, j)
2
9
WHILE (there exists an edge (i, j) such that neither i nor j is tight)
€
Select such an edge e = (i, j).
Fairness lemma. For any vertex cover S and any fair prices pe : ∑e pe ≤ w(S).
S ← set of all tight nodes.
▪
Pf.
Increase pe as much as possible until i or j tight.
RETURN S.
__________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
each edge e covered by
at least one node in S
sum fairness inequalities
for each node in S
27
28
Pricing method example
Pricing method: analysis
Theorem. Pricing method is a 2-approximation for WEIGHTED-VERTEX-COVER.
Pf.
・Algorithm terminates since at least one new node becomes tight after
each iteration of while loop.
・Let S = set of all tight nodes upon termination of algorithm.
S is a vertex cover: if some edge (i, j) is uncovered, then neither i nor j
is tight. But then while loop would not terminate.
price of edge a-b
・Let S* be optimal vertex cover. We show w(S)
vertex weight
w(S) = ∑ wi = ∑
i∈ S
∑ pe ≤
i∈ S e=(i, j)
all nodes in S are tight
∑
≤ 2 w(S*).
∑ pe = 2 ∑ pe ≤ 2w(S*).
i∈V e=(i, j)
S ⊆ V,
prices ≥ 0
e∈ E
each edge counted twice
fairness lemma
€
29
30
Weighted vertex cover
11. A PPROXIMATION A LGORITHMS
Given a graph G = (V, E) with vertex weights wi ≥ 0, find a min weight subset
of vertices S ⊆ V such that every edge is incident to at least one vertex in S.
‣ load balancing
‣ center selection
‣ pricing method: vertex cover
10
9
6
16
10
7
6
3
9
23
33
7
32
10
‣ LP rounding: vertex cover
‣ generalized load balancing
‣ knapsack problem
total weight = 6 + 9 + 10 + 32 = 57
32
Weighted vertex cover: ILP formulation
Weighted vertex cover: ILP formulation
Given a graph G = (V, E) with vertex weights wi ≥ 0, find a min weight subset
Weighted vertex cover. Integer linear programming formulation.
of vertices S ⊆ V such that every edge is incident to at least one vertex in S.
(ILP )
Integer linear programming formulation.
w i xi
i V
・Model inclusion of each vertex i using a 0/1 variable xi.
KBM
bXiX
x i + xj
xi
" 0 if vertex i is not in vertex cover
xi = #
$ 1 if vertex i is in vertex cover
1
(i, j)
{0, 1}
i
E
V
Vertex covers in 1–1 correspondence with 0/1 assignments:
Observation. If x* is optimal solution to (ILP), then S = { i ∈ V : xi* = 1}
€S = { i ∈ V : xi = 1}.
is a min weight vertex cover.
・Objective function: minimize Σi wi xi.
・For every edge (i, j), must take either vertex i or j (or both):
xi + xj ≥ 1.
33
34
Integer linear programming
Linear programming
Given integers aij, bi, and cj, find integers xj that satisfy:
Given integers aij, bi, and cj, find real numbers xj that satisfy:
KBM
bXiX
cT x
Ax
x
x
KBM
b
0
BMi2;`H
n
KBM
bXiX
cj x j
j=1
bXiX
n
aij xj
bi
1
i
m
j=1
cT x
Ax
x
x
n
KBM
b
0
BMi2;`H
cj x j
j=1
bXiX
n
aij xj
bi
1
i
m
j=1
xj
0
1
j
n
xj
0
1
j
n
xj
BMi2;`H 1
j
n
xj
BMi2;`H 1
j
n
Linear. No x2, xy, arccos(x), x(1 – x), etc.
Observation. Vertex cover formulation proves that INTEGER-PROGRAMMING
is an NP-hard search problem.
Simplex algorithm. [Dantzig 1947] Can solve LP in practice.
Ellipsoid algorithm. [Khachian 1979] Can solve LP in poly-time.
Interior point algorithms. [Karmarkar 1984, Renegar 1988, … ]
Can solve LP in poly-time and in practice.
35
36
LP feasible region
Weighted vertex cover: LP relaxation
LP geometry in 2D.
Linear programming relaxation.
x1 = 0
(LP )
KBM
w i xi
i V
bXiX
xi + xj
1
(i, j)
xi
0
i
E
V
Observation. Optimal value of (LP) is ≤ optimal value of (ILP).
Pf. LP has fewer constraints.
Note. LP is not equivalent to vertex cover.
½
½
x2 = 0
Q. How can solving LP help us find a small vertex cover?
x1 + 2x2 = 6
½
A. Solve LP and round fractional values.
2x1 + x2 = 6
37
38
Weighted vertex cover: LP rounding algorithm
Weighted vertex cover inapproximability
Lemma. If x* is optimal solution to (LP), then S = { i ∈ V : xi* ≥ ½} is a
Theorem. [Dinur–Safra 2004] If P ≠ NP, then no ρ-approximation for
vertex cover whose weight is at most twice the min possible weight.
WEIGHTED-VERTEX-COVER for any ρ < 1.3606 (even if all weights are 1).
Pf. [S is a vertex cover]
・Consider an edge (i, j) ∈ E.
・Since xi* + xj* ≥ 1, either xi* ≥ ½ or
On the Hardness of Approximating Minimum Vertex Cover
xj* ≥ ½ (or both) ⇒ (i, j) covered.
Irit Dinur∗
Pf. [S has desired cost]
・Let S* be optimal vertex cover. Then
∑ wi ≥
i ∈ S*
∑ wi xi* ≥
i∈S
LP is a relaxation
Samuel Safra†
May 26, 2004
1
2
Abstract
∑ wi
We prove the Minimum Vertex Cover problem to be NP-hard to approximate to within
a factor of 1.3606, extending on previous PCP and hardness of approximation technique. To
that end, one needs to develop a new proof framework, and borrow and extend ideas from
several fields.
i∈S
1
xi* ≥ ½
€
Theorem.
The rounding algorithm is a 2-approximation algorithm.
Introduction
The basic purpose of Computational Complexity Theory is to classify computational problems
according to the amount of resources required to solve them. In particular, the most basic task
is to classify computational problems to those that are efficiently solvable and those that are
not. The complexity class P consists of all problems that can be solved in polynomial-time. It
is considered, for this rough classification, as the class of efficiently-solvable problems. While
many computational problems are known to be in P, many others, are neither known to be in
P, nor proven to be outside P. Indeed many such problems are known to be in the class NP,
namely the class of all problems whose solutions can be verified in polynomial-time. When it
comes to proving that a problem is outside a certain complexity class, current techniques are
radically inadequate. The most fundamental open question of Complexity Theory, namely, the
P vs. NP question, may be a particular instance of this shortcoming.
While the P vs NP question is wide open, one may still classify computational problems into
those in P and those that are NP-hard [Coo71, Lev73, Kar72]. A computational problem L
Open research problem. Close the gap.
Pf. Lemma + fact that LP can be solved in poly-time.
39
40
Generalized load balancing
11. A PPROXIMATION A LGORITHMS
‣ load balancing
‣ center selection
Input. Set of m machines M; set of n jobs J.
・Job j ∈ J must run contiguously on an authorized machine in Mj ⊆
・Job j ∈ J has processing time tj.
・Each machine can process at most one job at a time.
M.
Def. Let J(i) be the subset of jobs assigned to machine i.
‣ pricing method: vertex cover
The load of machine i is Li = Σj ∈ J(i) tj.
‣ LP rounding: vertex cover
Def. The makespan is the maximum load on any machine = maxi Li.
‣ generalized load balancing
‣ knapsack problem
Generalized load balancing. Assign each job to an authorized machine to
minimize makespan.
42
Generalized load balancing: integer linear program and relaxation
Generalized load balancing: lower bounds
ILP formulation. xij = time machine i spends processing job j.
Lemma 1. The optimal makespan L* ≥ maxj tj.
Pf. Some machine must process the most time-consuming job. ▪
(IP) min
L
s. t. ∑ x i j
=
tj
for all j ∈ J
≤
L
for all i ∈ M
Lemma 2. Let L be optimal value to the LP. Then, optimal makespan L* ≥ L.
Pf. LP has fewer constraints than IP formulation. ▪
i
∑ xi j
j
xi j
xi j
∈ {0, t j } for all j ∈ J and i ∈ M j
= 0
for all j ∈ J and i ∉ M j
LP relaxation.
€
(LP) min L
s. t. ∑ x i j
= tj
for all j ∈ J
≤ L
for all i ∈ M
≥ 0
= 0
for all j ∈ J and i ∈ M j
for all j ∈ J and i ∉ M j
i
∑ xi j
j
xi j
xi j
43
€
44
Generalized load balancing: structure of LP solution
Generalized load balancing: rounding
Lemma 3. Let x be solution to LP. Let G(x) be the graph with an edge
Rounded solution. Find LP solution x where G(x) is a forest. Root forest G(x)
between machine i and job j if xij > 0. Then G(x) is acyclic.
at some arbitrary machine node r.
・If job j is a leaf node, assign j to its parent machine i.
・If job j is not a leaf node, assign j to any one of its children.
can transform x into another LP solution where
G(x) is acyclic if LP solver doesn’t return such an x
Pf. (deferred)
Lemma 4. Rounded solution only assigns jobs to authorized machines.
Pf. If job j is assigned to machine i, then xij > 0. LP solution can only assign
positive value to authorized machines. ▪
xij > 0
G(x) acyclic
G(x) cyclic
job
job
machine
machine
45
46
Generalized load balancing: analysis
Generalized load balancing: analysis
Lemma 5. If job j is a leaf node and machine i = parent(j), then xij = tj.
Theorem. Rounded solution is a 2-approximation.
Pf.
Pf.
・Let J(i) be the jobs assigned to machine i.
・By LEMMA 6, the load Li on machine i has two components:
・Since i is a leaf, xij = 0 for all j ≠ parent(i).
・LP constraint guarantees Σi xij = tj.
▪
- leaf nodes:
Lemma 6. At most one non-leaf job is assigned to a machine.
LP
Lemma 5
Lemma 2 (LP is a relaxation)
Pf. The only possible non-leaf job assigned to machine i is parent(i). ▪
∑ tj
j ∈ J(i)
j is a leaf
=
∑ xij
j ∈ J(i)
j is a leaf
≤ ∑ xij ≤ L ≤ L *
j ∈ J
optimal value of LP
Lemma 1
€
- parent:
tparent(i) ≤ L *
・Thus,€the overall load Li
job
≤ 2 L*. ▪
machine
47
48
Generalized load balancing: flow formulation
Generalized load balancing: structure of solution
Flow formulation of LP.
Lemma 3. Let (x, L) be solution to LP. Let G(x) be the graph with an edge
from machine i to job j if xij > 0. We can find another solution (xʹ, L) such that
∑ xi j
G(xʹ) is acyclic.
∞
= tj
for all j ∈ J
≤ L
for all i ∈ M
≥ 0
= 0
for all j ∈ J and i ∈ M j
for all j ∈ J and i ∉ M j
i
∑ xi j
Pf. Let C be a cycle in G(x).
j
xi j
xi j
flow conservation maintained
・Augment flow along the cycle C.
・At least one edge from C is removed (and none are added).
・Repeat until G(xʹ) is acyclic. ▪
€
augment flow
along cycle C
3
3
3
3
6
6
2
4
3
4
2
1
Observation. Solution to feasible flow problem with value L are in
4
1-to-1 correspondence with LP solutions of value L.
1
5
3
5
4
G(x)
4
G(x′)
49
50
Conclusions
Running time. The bottleneck operation in our 2-approximation is
solving one LP with m n + 1 variables.
11. A PPROXIMATION A LGORITHMS
Remark. Can solve LP using flow techniques on a graph with m + n + 1 nodes:
‣ load balancing
given L, find feasible flow if it exists. Binary search to find L*.
‣ center selection
‣ pricing method: vertex cover
‣ LP rounding: vertex cover
Extensions: unrelated parallel machines. [Lenstra–Shmoys–Tardos 1990]
・Job j takes tij time if processed on machine i.
・2-approximation algorithm via LP rounding.
・If P ≠ NP, then no no ρ-approximation exists for any ρ < 3/2.
Mathematical Programming 46 (1990) 259-271
North-Holland
‣ generalized load balancing
‣ knapsack problem
259
APPROXIMATION ALGORITHMS FOR SCHEDULING
U N R E L A T E D PARALLEL M A C H I N E S
Jan Karel LENSTRA
Eindhoven University of Technology, Eindhoven, The Netherlands, and
Centre for Mathematics and Computer Science, Amsterdam, The Netherlands
David B. SHMOYS and l~va T A R D O S
Cornell University, Ithaca, NY, USA
Received 1 October 1987
Revised manuscript 26 August 1988
We consider the following scheduling problem. There are m parallel machines and n independent
.jobs. Each job is to be assigned to one of the machines. The processing of .job j on machine i
requires time Pip The objective is to lind a schedule that minimizes the makespan.
Our main result is a polynomial algorithm which constructs a schedule that is guaranteed to
51
Polynomial-time approximation scheme
Knapsack problem
PTAS. (1 + ε)-approximation algorithm for any constant ε > 0.
Knapsack problem.
・Given n objects and a knapsack.
we assume w ≤ W for each i
・Item i has value vi > 0 and weighs wi > 0.
・Knapsack has weight limit W.
・Goal: fill knapsack so as to maximize total value.
・Load balancing. [Hochbaum–Shmoys 1987]
・Euclidean TSP.
[Arora, Mitchell 1996]
i
Consequence. PTAS produces arbitrarily high quality solution,
but trades off accuracy for time.
Ex: { 3, 4 } has value 40.
This section. PTAS for knapsack problem via rounding and scaling.
item
value
weight
1
1
1
2
6
2
3
18
5
4
22
6
5
28
7
original instance (W = 11)
53
54
Knapsack is NP-complete
Knapsack problem: dynamic programming I
KNAPSACK. Given a set X, weights wi ≥ 0, values vi ≥ 0, a weight limit W, and a
Def. OPT(i, w) = max value subset of items 1,..., i with weight limit w.
target value V, is there a subset S ⊆ X such that:
Case 1. OPT does not select item i.
∑ wi ≤ W
・OPT selects best of 1, …, i – 1 using up to weight limit w.
i∈S
∑ vi
≥ V
i∈S
Case 2. OPT selects item i.
・New weight limit = w – wi.
・OPT selects best of 1, …, i – 1 using up to weight limit w – wi.
SUBSET-SUM. Given a set X, values ui ≥ 0, and an integer U, is there a subset S
⊆ X whose elements sum to exactly U ?
€
Pf. Given instance (u1, …, un, U) of SUBSET-SUM, create KNAPSACK instance:
# 0
if i = 0
%
OPT(i, w) = $ OPT(i −1, w)
if wi > w
% max OPT(i −1, w), v + OPT(i −1, w − w )
{
&
i
i } otherwise
Theorem. SUBSET-SUM ≤ P KNAPSACK.
vi = wi = ui
∑ ui
≤ U
i∈S
V = W =U
∑ ui
Theorem. Computes the optimal value in O(n W) time.
≥ U
€
i∈S
55
€
・Not polynomial in input size.
・Polynomial in input size if weights are small integers.
56
Knapsack problem: dynamic programming II
Knapsack problem: dynamic programming II
Def. OPT(i, v) = min weight of a knapsack for which we can obtain a solution
Theorem. Dynamic programming algorithm II computes the optimal value
of value ≥ v using a subset of items 1,..., i.
in O(n2 vmax) time, where vmax is the maximum of any value.
Pf.
・The optimal value V* ≤ n vmax.
・There is one subproblem for each item and for each value v ≤ V*.
・It takes O(1) time per subproblem. ▪
Note. Optimal value is the largest value v such that OPT(n, v) ≤ W.
Case 1. OPT does not select item i.
・OPT selects best of 1, …, i – 1 that achieves value ≥ v.
Remark 1. Not polynomial in input size!
Case 2. OPT selects item i.
Remark 2. Polynomial time if values are small integers.
・Consumes weight wi, need to achieve value ≥ v – vi.
・OPT selects best of 1, …, i – 1 that achieves value ≥ v – vi.
0
OP T (i, v) =
min {OP T (i
1, v), wi + OP T (i
1, v
vi )}
B7 v 0
B7 i = 0 M/ v > 0
Qi?2`rBb2
57
58
Knapsack problem: polynomial-time approximation scheme
Knapsack problem: polynomial-time approximation scheme
Intuition for approximation algorithm.
Round up all values:
・0 <
・vmax
・θ
・Round all values up to lie in smaller range.
・Run dynamic programming algorithm II on rounded/scaled instance.
・Return optimal items in rounded instance.
ε ≤ 1 = precision parameter.
= largest value in original instance.
v̄i =
vi
,
v̂i =
vi
= scaling factor = ε vmax / 2n.
Observation. Optimal solutions to problem with v are equivalent to
optimal solutions to problem with vˆ .
item
value
weight
item
value
weight
1
934221
1
1
1
1
2
5956342
2
2
6
2
€ using v is nearly optimal;
Intuition. v close to v so optimal solution
3
17810013
5
3
18
5
€
programming algorithm II is fast.
vˆ small and integral so dynamic
4
21217800
6
4
22
6
5
27343199
7
5
28
7
original instance (W = 11)
€
€
€
rounded instance (W = 11)
59
60
Knapsack problem: polynomial-time approximation scheme
Knapsack problem: polynomial-time approximation scheme
Theorem. If S is solution found by rounding algorithm and S*
Theorem. For any ε > 0, the rounding algorithm computes a feasible solution
is any other feasible solution, then (1 + )
whose value is within a (1 + ε) factor of the optimum in O(n3 / ε) time.
vi
i S
vi
i S
Pf. Let S* be any feasible solution satisfying weight constraint.
i
i
i
i
i
S
S
S
S
S
vi
vi
vi
vi
vi
=
=
=
=
=
=
=
=
=
=
v̄i
v̄
i S v̄i
i
i S v̄i
i S v̄v̄i
i
i S
v̄
i S v̄i
i
i S v̄i
i S (v
+ )
i S v̄ii
(v + )
i S (vi + )
i
i S (vi + )
i S (v
)
v +
+ n
i S ii
v + n
i S vi + n
i
i S vi + n
1
i S vi + n
2
1
i S
ii S
S vi + 1
vi + 21
i S vi + 12
i S+vi )+ 22vi
(1
i S
(1
i S+ ) i S vi
(1
+ )
v
(1 + ) i S vii
(1 + ) i S vi
(1 + ) ii SSvi
i S
Pf.
・We have already proved the accuracy bound.
・Dynamic program II running time is O(n2 vˆmax ) ,
subset containing
only the item
of largest value
always round up
solve rounded
instance optimally
v̂max =
choosing S* = { max }
never round up
by more than θ
vmax
vmax
vmax
vmax
vKt
vKt
vKt
|S| ≤ n
thus
vKt
vKt
θ = ε vmax / 2n
i S
i S
i S
i S
2
2i
vi +
vi +
1
2
1
2
vi +
vi +
1
2
1
2
S
i S
vmax
=
where
2n
€
vKt
vKt
vKt
vKt
vi
vi
vmax ≤ 2 Σi ∈ S vi
61
62