The Complexity
of
Optimization Problems
Summary
- Complexity of algorithms and problems
- Complexity classes: P and NP
- Reducibility
- Karp reducibility
- Turing reducibility
- NP-complete problems
- Complexity of optimization problems
- Classes PO and NPO
- NP-hard optimization problems
Uniform and logarithmic cost
- Uniform cost: the overall number of instructions
executed by the algorithm before halting
- Logarithmic cost:each instruction has a cost
depending on the number of bits of the operands
- E.g. product of two n-bit integer costs O(nlogn)
- Same for space measure (but we will talk only of time
measure)
Example: xy
- Uniform cost: 2+3y
- Logarithmic cost: aylogy+by2logx(logy+loglogx)+c
begin
r:=1;
while y  0 do
begin
r:=r*x; y:=y-1
end;
return r
end.
Worst case analysis
- Instances of the same size may result in different
execution costs (e.g. sorting)
- Cost of applying the algorithm on the worst case
instance of a given size
- Gives certainty that the algorithm will perform its task
within the established time bound
- It is easier to determine
Input size
- Size of input: number of bits needed to present the
specific input
- Existence of encoding scheme which is used to
describe any problem instance
- For any pair of natural encoding schemes and for any
instance x, the resulting strings are polynomially related
- I.e., |ei(x)|  pi,j(|ej(x)|) and |ej(x)|  pi,j(|ei(x)|)
- Avoid unary base encoding
Asymptotic Analysis
- Let t(x) be the running time of algorithm A on input x.
The worst case running time of A is given by
t(n)=max(t(x) | x such that |x|  n)
- Upper bound: A has complexity O(f(n)) if t(n) is
O(f(n)) (that is, we ignore constants)
- Lower bound: A has complexity (f(n)) if t(n) is
(f(n))
Complexity of a problem
- A problem P
- has a complexity lower bound (f(n)) if any algorithm for
P has complexity (f(n))
- has a complexity upper bound O(f(n)) if an algorithm for P
exists with complexity O(f(n))
Decision problems
- Set of instances partitioned into a YES-subset and a
NO-subset
- Given an instance x, decide which subset x belongs to
- A decision problem P is solved by an algorithm A if,
for every instance, A halts and returns YES if and only
if the instance belongs to the YES-subset
x  YP?
A
YES
NO
Complexity Classes
- For any function f(n), TIME(f(n)) is the set of decision
problems which can be solved with a time complexity
O(f(n))
- P = the union of TIME(nk) for all k
- EXPTIME = the union of TIME (2nk) for all k
- P is contained in EXPTIME
- It is possible to prove (by diagonalization) that
EXPTIME is not contained in P
Examples
- SATISFYING TRUTH ASSIGNMENT: given a CNF
formula F and a truth assignment f, does f satisfy F?
- SATISFYING TRUTH ASSIGNMENT is in P
- SATISFIABILITY (simply, SAT): given a CNF
formula F, is F satisfiable?
- SAT is in EXPTIME.
- Open problem: SAT is in P?
Class NP
- A problem P is in class NP if there exist a polynomial
p and a polynomial-time algorithm A such that, for
any instance x, x is a YES-instance if and only if there
exists a string y with |y|  p(|x|) such that A(x,y)
returns YES
- y is said to be a certificate
- Example: SAT is in NP (the certificate is a truth assignment
that satisfies the formula)
- P is contained in NP (the certificate is the computation
of the polynomial-time algorithm)
Non-deterministic algorithms: SAT
begin
for each variable v
guess Boolean value f(v);
if f satisfies F then return YES
else return NO
v1
end.
TRUE
FALSE
v2
(v1 or v2 or (not v3))
and
((not v1) or (not v2) or v3)
v2
TRUE
FALSE
v3
TRUE
TRUE
v3
FALSE TRUE
FALSE
v3
FALSE TRUE
v3
FALSE TRUE
f1
f2
f3
f4
f5
f6
f7
YES
NO
YES
YES
YES
YES
NO
FALSE
f8
YES
Non-deterministic algorithms and NP
begin
guess string y with |y|  p(|x|);
if A(x,y) returns YES then return YES
else return NO
end.
Every problem in NP admits a
polynomial-time non-deterministic
algorithm
Each computation path, which returns
YES, is a certificate of polynomial length
that can be checked in polynomial time
Every problem that admits a polynomialtime non-deterministic algorithm is in
NP
YES
NO
YES
YES
YES
YES
NO
YES
Karp reducibility
- A decision problem P1 is Karp reducible to a decision
problem P2 (in short, P1  P2) if there exists a
polynomial-time computable function R such that, for
any x, x is a YES-instance of P1 if and only if R(x) is a
x I P
YES-instance of P2
1
R
- If P1  P2 and P2 is in P, then P1 is in P
y I P
2
A2
an swer
Example: {0,1}-Linear programming
- SAT {0,1}-LINEAR PROGRAMMING
- For each Boolean variable v of a CNF Boolean formula F,
we introduce a {0,1}-valued variable z
- For each clause l1 or l2 or … or lk of F, we introduce the
inequality z1+ z2 + … + zk  1, where
zi= z if li= v
and zi= (1-z) if li= not v
- E.g. (v1 or v2 or (not v3)) and ((not v1) or (not v2) or v3) is
transformed into the following two inequalities:
z1+z2+(1-z3)  1
and
(1-z1)+(1-z2) +z3  1
- If f is a truth assignment, let g be the natural corresponding
{0,1}-value assignment (0=FALSE,1=TRUE)
- f satisfies F if and only g satisfies all inequalities
Turing reducibility
- A decision problem P1 is Turing reducible to a
decision problem P2 if there exists a polynomial-time
algorithm R solving P1 such that R may access to an
oracle algorithm solving P2
x  IP
1
g(x)
R
y 1,y 2 ,... with yi  IP2
f(y 1),f(y 2),...
oracle
for P2
- If P1  P2 then P1 is Turing reducible to P2
Example: Equivalent formulas
- SAT is Turing reducible to EQUIVALENT
FORMULAS
- Given a CNF Boolean formula F, query the oracle with
input F and x and (not x)
- If the oracle answers YES, then F is not satisfiable,
otherwise F is satisfiable
- It is not known whether SAT is Karp reducible to
EQUIVALENT FORMULAS
NP-complete problems
- A decision problem P is NP-complete if P is in NP
and, for any decision problem P1 in NP, P1  P
- If P is NP-complete and P is in P, then P=NP
- NP-complete problems are the hardest in NP
- P versus NP question can be solved by focusing on an NPcomplete problem
- Cook’s Theorem: SAT is NP-complete
Optimization problem
- Optimization problem P characterized by
- Set of instances I
- Function SOL that associates to any instance the set of
feasible solutions
- Measure function m that, for any feasible solution, provides
its positive integer value
- Goal, that is, either MAX or MIN
- An optimal solution is a feasible solution y* such that
m(x,y*) = Goal{m(x,y) | y  SOL(x)}
- For any instance x, m*(x) denotes optimal measure
MINIMUM VERTEX COVER
- INSTANCE: Graph G=(V,E)
- SOLUTION: A subset U of V such that, for any edge
(u,v), either u is in U or v is in U
- MEASURE: Cardinality of U
Three problems in one
- Constructive problem: given an instance, compute
an optimal solution and its value
- We will study these problems
- Evaluation problem: given an instance, compute the
optimal value
- Decision problem: given an instance and an integer
k, decide whether the optimal value is at least (if
Goal=MAX) or at most (if Goal=MIN) k
Class NPO
- Optimization problems such that
- I is recognizable in polynomial time
- Solutions are polynomially bounded and recognizable in
polynomial time
- m is computable in polynomial time
- Example: MINIMUM VERTEX COVER
- Theorem : If P is in NPO, then the corresponding
decision problem is in NP
Class PO
- NPO problems solvable in polynomial time
- Example: MINIMUM PATH
- An optimization problem P is NP-hard if any problem
in NP is Turing reducible to P
- Theorem: If the decision problem corresponding to a
NPO problem P is NP-complete, then P is NP-hard
- Example: MINIMUM VERTEX COVER
- Corollary: If P  NP then PO  NPO
Evaluating versus constructing
- Decision problem is Turing reducible to evaluation
problem
- Evaluation problem is Turing reducible to constructive
problem
- Evaluation problem is Turing reducible to decision
problem
- Binary search on space of possible measure values
- Is constructive problem Turing reducible to evaluation
problem?
MAXIMUM SATISFIABILITY
- INSTANCE: CNF Boolean formula, that is, set C of
clauses over set of variables V
- SOLUTION: A truth-assignment f to V
- MEASURE: Number of satisfied clauses
Evaluating versus constructing: MAX SAT
begin
for each variable v
begin
k := MAX SATeval(x);
xTRUE:= formula obtained by setting v to TRUE in x;
xFALSE:= formula obtained by setting v to FALSE in x;
if MAX SATeval(xTRUE) = k then
begin
f(v) := TRUE; x := xTRUE
end
else
begin
f(v) := FALSE; x := xFALSE
end;
return f
end.
Theorem: if the decision problem is NP-complete, then the constructive
problem is Turing reducible to the decision problem