Principles of Algorithmic Techniques
T-79.4202
Pekka Orponen
Department of Computer Science
Aalto University
Autumn 2015
Lecture 10: NP-completeness
I
Introduction: difficult problems
I
Decision problems
I
Problem complexity class P
I
Problem complexity class NP
I
Examples of NP-type problems
I
Problem transformations (“reductions”)
I
NP-complete problems
T-79.4202 PAT
Autumn 2015
2/32
1. Introduction: difficult problems
Example: The travelling salesman problem (TSP)
I
Given a map of n cities and their pairwise distances, a
travelling salesman needs to find the shortest route that
passes through each city exactly once.
I
Formally, this constitutes a minimal Hamiltonian cycle in a
given weighted complete graph hKn , di.
I
Numerous practical applications: distribution truck route
planning, circuit board manufacturing, flowshop
scheduling, genome sequencing, kidney exchange
sequences, . . .
T-79.4202 PAT
Autumn 2015
3/32
Figure : An optimal TSP route through 100 European cities
T-79.4202 PAT
Autumn 2015
4/32
How to solve it?
I
Brute force: Try all the n! possible routes and choose the
shortest one.1
I
Not feasible in practice.
E.g.
I
I
I
I
1
n = 22 ⇒ n! ≈ 1020
1 route/ms ⇒ runtime ≈ 36 bn years
age of the universe ≈ 10-20 bn years
To be precise, there are only (n − 1)!/2 different routes.
T-79.4202 PAT
Autumn 2015
5/32
I
Hardware brute force: Get a faster machine — e.g. a
1.000.000 core parallel computer where each core can
check 1 route/ns.
I
Still not feasible.
I
20 bn years is now enough for about 30 cities.
T-79.4202 PAT
Autumn 2015
6/32
Fundamental challenge
I
The number of possible routes n! ≈ nn grows at an
exponential rate. (So called “combinatorial explosion”.)
I
Say that an increasing function f : N → N has polynomial
growth, if for some constant c, f (2n) ≤ cf (n).2
I
If an algorithm’s runtime has this characteristic, then
increasing processing power by a factor of c makes it
possible to deal with inputs of twice the size.
I
On the other hand, if the runtime is exponential, e.g. c n for
some c > 1, then a c-fold increase in processing power
only adds one to the size of feasible inputs.
2
This is true for instance if f (n) = Θ(nk ) for some constant k .
T-79.4202 PAT
Autumn 2015
7/32
I
Better idea: Think of a faster algorithm.
I
E.g. Lecture 9 presented a dynamic programming
algorithm with a runtime of “only” O(n2 2n ).3
I
Try some powerful heuristic techniques: integer
programming with cutting-plane heuristics, modern SAT
solvers, simulated annealing, genetic algorithms, . . .
I
Here the state of the art is that TSP instances of maybe a
few 1.000 cities can usually be solved, and some
instances of a few 10.000 cities have been solved.4
I
But this seems to be the limit.
3
In comparison to plain brute-force, n2 2n ms = 20 bn years for n ≈ 57.4
Current world record is a VLSI design problem with 85.900 “cities”, cf.
http://www.math.uwaterloo.ca/tsp/.
4
T-79.4202 PAT
Autumn 2015
8/32
I
Could one then prove that solving the TSP problem using
any method always requires exponential time?
I
Yes and no.
I
TSP is a member of a large class of equivalent “apparently
very difficult” problems that are either all easy or all
difficult, but it is not known which is the case.5
I
Resolving this question is the famous
P = NP?
problem.
I
5
6
“P = NP?” is one of the seven “millennium” mathematical
problems defined by the Clay Mathematics Institute6 in the
year 2000. The first person resolving this question will be
awarded 1.000.000 USD by the Institute.
Although most researchers would overwhelmingly bet on the latter.
http://www.claymath.org/
T-79.4202 PAT
Autumn 2015
9/32
2. Decision problems
I
I
The decision problem for a set S asks if a given input x
belongs to S. Inputs may be assumed to be encoded as
binary strings.
Ex. 1. ”Is a given binary number even?”
S = { binary encodings of even numbers }
= { binary strings ending in a zero }
I
Ex. 2. ”Is a given binary number x a prime?”
S = { binary encodings of primes }
I
Ex. 3. ”Does a given graph G, encoded as a binary string,
contain a Hamiltonian circuit, i.e. a cycle that visits each
vertex in G exactly once?”
S = { binary encodings of Hamiltonian graphs }
T-79.4202 PAT
Autumn 2015
10/32
3. Problem complexity class P
I
Recall the time complexity functions for an algorithm A:
I time (x) = the number of unit computation steps performed
A
by algorithm A on input x
I time (n) = max {time (x) : |x| = n}
A
A
I
One usually considers an algorithm A as “feasible” if its time
complexity function timeA (n) is bounded by some polynomial in
the input length |x| = n.
I
Correspondingly, a set S is considered as “feasibly decidable” if
its decision problem has an algorithm of this type.
I
For simplicity, we identify “the decision problem for set S” with
the set S itself.
I
Let us thus define the general class of “feasibly solvable
problems”:
P={
decision problems (sets) S that have some
polynomially time-bounded solution algorithm }
T-79.4202 PAT
Autumn 2015
11/32
4. Problem complexity class NP
I
Many decision problems S for which a polynomial-time
decision method is not known, are of the following
“guess-verify” type:
I
The membership of an input x in S could be verified easily,
if one knew (or was able to “guess”) a witness string or
certificate w that is of polynomial length in |x| and validates
that x ∈ S.
I
This kind of problem cannot be solved trivially by trying out
all the possible witness strings w, |w| ≤ p(|x|), because if
|x| = n there are 2p(n) alternatives (= many!)
T-79.4202 PAT
Autumn 2015
12/32
I
Ex. 1. The Hamiltonian circuit problem is of the
guess-verify type: if one can guess a correct ordering w for
the vertices of a graph G, then it is easy to check that the
vertices in this order indeed constitute Hamiltonian circuit.
Let us denote the (polynomial-time) verification algorithm
A(G, w).
I
Ex. 2. The problem of recognising composite numbers
(non-primes) is also of the guess-verify type: if one for a
given input number x can guess a pair of nontrivial factors
w = hy , zi, then it is easy to check that indeed 1 < y , z < x
and x = y · z. There is thus also for this problem a
polynomial-time verification algorithm A(x, w).
T-79.4202 PAT
Autumn 2015
13/32
I
I
Guess-verify type problems are also said to be of NP-type.
The abbreviation NP stands for “nondeterministic
polynomial time” and indicates that the description of an
NP-type problem consists of:
(i) a “nondeterministic” guess of a polynomial-length witness
string, and
(ii) a polynomial-time (deterministic) check of the validity of the
witness w.r.t. the given input.
I
More precisely: a decision problem S is of NP-type, if there
exist a (deterministic) polynomial-time verification
algorithm A and a polynomial p bounding the length of the
witness string such that for any input x the following holds:
x ∈ S ⇔ (∃w : |w| ≤ p(|x|) and A(x, w) = OK).
I
Let us denote this class of problems (sets) as:
NP = { NP-type decision problems }
T-79.4202 PAT
Autumn 2015
14/32
5. Examples of NP-type problems
1. The propositional satisfiability problem (SAT):
I
Input: A Boolean formula F = F (x1 , . . . , xn ) built of binary
“propositional variables” x1 , . . . , xn , constants 1 (“true”) and
0 (“false”) and connectives ∧ (“and”), ∨ (“or”) and ¬ (“not”).
I
Question: Is F satisfiable, i.e. does there exist a truth
assignment t to the variables in F ,
t : {x1 , . . . , xn } → {0, 1},
such that F evaluates to “true” under t:
F (t(x1 ), . . . , t(xn )) = 1
T-79.4202 PAT
Autumn 2015
15/32
Are these formulas satisfiable?
F = (a ∨ ¬b) ∧ (¬a ∨ ¬c) ∧ (b ∨ c)
G = (¬a ∨ ¬b) ∧ (¬a ∨ b) ∧ (a ∨ ¬b)
H = (¬a∨¬b)∧(¬a∨¬c)∧(a∨¬b)∧(a∨¬c)∧(b∨c)
T-79.4202 PAT
Autumn 2015
16/32
First two are, third is not
F = (a ∨ ¬b) ∧ (¬a ∨ ¬c) ∧ (b ∨ c)
G = (¬a ∨ ¬b) ∧ (¬a ∨ b) ∧ (a ∨ ¬b)
H = (¬a∨¬b)∧(¬a∨¬c)∧(a∨¬b)∧(a∨¬c)∧(b∨c)
T-79.4202 PAT
Autumn 2015
17/32
2. The travelling salesman decision problem (TSP):
I
Input: A complete weighted graph hG, di and an integer k ,
i.e. “a map, a distance table, and a maximum length of a
route”.
I
Question: Does G contain a Hamiltonian circuit, i.e. a
“salesman’s tour”, that is of length at most k ?
T-79.4202 PAT
Autumn 2015
18/32
3. The vertex cover problem (VC): Given a graph G and an
integer k , determine if G contains a vertex cover of size at most
k , i.e. a set S of at most k vertices that contains at least one of
the endpoints of every edge in G.
4. The independent set problem (IS): Given a graph G and
an integer k , determine if G contains an independent set of size
at least k , i.e. a set S of at least k vertices such that there are
no edges between any two vertices in S.
5. The clique problem (CLIQUE): Given a graph G and an
integer k , determine if G contains a clique of size at least k , i.e.
a set S of at least k vertices such that there are edges between
all pairs of vertices in S.
T-79.4202 PAT
Autumn 2015
19/32
Is there a vertex cover of size 5?
T-79.4202 PAT
Autumn 2015
20/32
Yes — but no cover of size 4
T-79.4202 PAT
Autumn 2015
21/32
Is there an independent set of size 5?
T-79.4202 PAT
Autumn 2015
22/32
Yes
T-79.4202 PAT
Autumn 2015
23/32
Is there a 5-clique?
T-79.4202 PAT
Autumn 2015
24/32
No — the biggest cliques are of size 4
T-79.4202 PAT
Autumn 2015
25/32
6. Problem transformations (“reductions”)
A decision problem (set) S can be (polynomial-time) reduced to
a decision problem (set) T , denoted S ≤P T , if there exists a
polynomial-time computable function f such that for any input x
to S:
x ∈ S ⇔ f (x) ∈ T .
A polynomial-time reduction function f thus transforms any
input x to problem S to a “corresponding” (= answer-equivalent)
input f (x) to problem T .
The existence of a reduction function shows that in a certain
sense problem S is just a special case of problem T and thus
“simpler” than T .
Lemma 1 If S ≤P T and T ∈ P, then S ∈ P.
T-79.4202 PAT
Autumn 2015
26/32
Example: VC ≤P IS ≤P CLIQUE
VC
IS
CLIQUE
Lemma 2 Let G = (V , E) be an undirected graph and S ⊆ V .
Then the following conditions are equivalent:
(i) S is a vertex cover in G.
(ii) V \ S is an independent set in G.
(iii) V \ S is a clique in G’s complement graph G, which has
the same vertices as G, but edges between exactly the
vertices which are not connected in G.
T-79.4202 PAT
Autumn 2015
27/32
Proof of claim VC ≤P IS.
I
Denote the number of vertices in a graph G by |G|.
I
By Lemma 2, G contains a vertex cover of size ≤ k if and
only if it contains an independent set of size ≥ |G| − k .
I
Choose as the reduction function:
f (hG, k i) = hG, |G| − k i.
I
This can clearly be computed in polynomial time and
satisfies the reduction condition:
hG, k i ∈ VC ⇔ f (hG, k i) = hG, |G| − k i ∈ IS.
T-79.4202 PAT
Autumn 2015
28/32
7. NP-complete problems
A decision problem (set) is NP-complete if:
(i) T ∈ NP and
(ii) S ≤P T for all S ∈ NP.
The NP-completeness of a problem T is a strong argument for
it being computationally difficult, because T then contains as
(reduction-encoded) special cases all other NP-type problems,
and is thus “at least as difficult as any other NP-type problem”.
Lemma 3 Let T be an NP-complete problem. If T ∈ P, then
P = NP.
T-79.4202 PAT
Autumn 2015
29/32
Theorem 4 The following problems are NP-complete:
(1) the propositional satisfiability problem (SAT),
(2) the Hamiltonian circuit problem (HC),
(3) the travelling salesman decision problem (TSP),
(4) the vertex cover problem (VC),
(5) the independent set problem (IS),
(6) the clique problem (CLIQUE),
(7) –(10.007) thousands of other known problems.7
7
After 30 years of research it was discovered in 2002 that the problem of
recognising primes, and hence also non-primes, is in fact in P (Agrawal,
Kayal & Saxena 2002). It however doesn’t follow from this that the problem of
factoring, i.e. actually finding the factors of large composite numbers, is easy.
T-79.4202 PAT
Autumn 2015
30/32
The proof of Theorem 4 is in most cases based on the following
elementary lemma:
Lemma 5 Let S be some NP-complete problem, T ∈ NP and
S ≤P T . Then also T is NP-complete.
However, to get such chains of mutual problem reductions
started, one needs to establish the NP-completeness of some
fundamental problem by “generic” reductions from arbitrary
NP-type problems, using only the basic definitions of the
problem class NP and polynomial-time reducility.
This was achieved independently by Stephen Cook and Leonid
Levin in the early 1970’s:
Theorem 6 The propositional satisfiability problem SAT is
NP-complete.
T-79.4202 PAT
Autumn 2015
31/32
Further information
I
T-79.4101 Discrete Models and Search
I
T-79.5103 Computational Complexity Theory
I
T-79.5207 Advanced Course in Algorithms
T-79.4202 PAT
Autumn 2015
32/32