Problems and Their Classes
• Solved many problems using different algorithms
• For each problem - algorithm and computing time
• Problem classes based on computing time of best
known algorithm
P class
NP Class
P Class
• This group consist of all problem whose computing
time are polynomial time
• Computing time is bounded by polynomials of small
degree
Insertion sort O(n2)
Merge and Quick sort O(n logn)
Binary search O(log n)
Linear Class
Quadratic and Cubic Class
• Also called as tractable (easy) problems
NP Class
• Computing of best known algorithm are non
polynomial
• TSP using dynamic programming O(n2 2n), Knapsack
O(2 n/2)
• No one was has been able to develop a polynomial
time algorithms for this class problems
• Requires vast amount of time to execute even
moderate size problems
• Problems that can be solved using super polynomial,
exponential time algorithm are called intractable
(Hard )
NPC Theory
• Does not provide a method of obtaining polynomial
time algorithm for problems of NP Class
• Nor it say that algorithms of polynomial complexity
do not exist
• But many of the problems for which there are no
known polynomial time algorithms are
computationally related
Further Classes of Problems
• NP Complete (Non deterministic polynomial time
complete) problems – NPC Problem can be solved in
polynomial time if and only if all other NPC problems
can also be solved in polynomial time
• NP Hard problems – if NP hard problem can be
solved in polynomial time , then all NPC problems
can be solved in polynomial time
• No NPC or NPHard problem is polynomially solvable
• All NP Complete problems are NP hard but some NP
Hard are not NPC
• There are many problems of NPC and NPH , focus
should be on nondeterministic computations (to be
defined)
• Relationship of these classes to nondeterministic
computations together with apparent power of
nondeterminism leads conclusion no NPC or NPH is
polynomially solvable
Deterministic and Non Deterministic
Algorithms
• Deterministic Algorithm
Result of every operation is uniquely defined
Agree with way programs are executed on a computer
• In a theoretical framework we can remove this
restriction on the outcome of every operation
• Non Deterministic Algorithm
Allow algorithm to contain operations whose outcomes
are limited to specific set of possibilities
Machine executing such operations are allowed to
choose any one of these outcome subject to termination
condition defined
This leads to non deterministic Algorithm
Non Deterministic Algorithms
• Introduce new functions
Choice (set S) – arbitrarily choose one element of S
Failure() – signals an unsuccessful completion
Success()- signals a successful completion
X = choice(1,n) results in x being assigned any one of
integer in the range of 1 to n, but no rule how this
choice is to be made !
• Failure and success – just to define a computation of
algorithm and cannot be used to effect a return
• Whenever there is a set of choices that leads to a
successful completion, then one such set of choices is
always made and algorithm terminates successfully
Non Deterministic Algorithms
• NDA terminates unsuccessfully if and only if there is
no set of choices leading to success signal
• Hence depending upon order in which choices are
made – affects the successful / unsuccessful
termination of NDA
• Computing time of Choice(), Success(),failure() is
O(1) – Constant
• Machine capable of handling NDA is called as
nondeterministic Machine (NDM)
• Although, NDM do not exist in practice, there are
certain problems that can not be solved by
deterministic algorithm.
Non Deterministic Search Algorithms
• A decision problem gives answer either 0 or 1
• Search a element x in a given set A, write index else 0
J = choice (1,n); // Select form a set
If A[j] = x then {write (j);success();} //write index
Write(0); failure();
• Computing time of this algorithm is O(1) in both
cases , HOW ?
• Any Deterministic search algorithm requires O(n)
time – Linear
Non Deterministic Sorting
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
Algorithm Nsort(A,N)
// sort n positive numbers
{
For I = 1 to n do B[i] = 0 ; //
initialize B
For I = 1 to n do
{
J = choice(1,n);
If B[j] != 0 then failure();
B[j] = A[i];
}
For I = 1 to n-1 do // verify the order
If B[i] > B[i+1] then failure ();
Write (B);
Success();
}
•
•
•
•
•
•
•
In the for loop of 5 to 10 each A[i] is
assigned to position in B
Line 7 non deterministically
identifies this position
Line 8 checks whether this position is
already used or not !
The order of numbers in B is some
permutation of the initial order in A
For loop of line no 11 and 12 verifies
the B is in ascending order,
Since there is always a set of choices
at line no 7 for ascending order, this is
a sorting algorithm of Complexity
O(N)
All deterministic sorting algorithms
must have A complexity O(nlogn)
Non Deterministic Algorithms
• A deterministic interpretation of a non deterministic
algorithm can be made by allowing unbounded
parallelism in computation
• Each time a choice is to be made, the algorithm makes
several copies of itself
• One copy for each possible choices, many copies are
executing at the same time.
• First copy to reach successful completion terminates all
other copies
• If a copy reaches to failure completion then only that
copy of algorithm terminates
• This interpretation enable us to better understand NDA
Non Deterministic Machine
• NDM does not makes any copies of an algorithm
every time choice is made
• NDM has ability to select a correct element from set
of choices (if exists)
• A correct element is defined relative to a shortest
sequence of choices that leads to successful
completion
• If no sequence of choices leading to successful
termination then unsuccessful termination /
computation with one unit of time
• Machine is Fictitious, no concerns with how the
machine can make a correct choice at each step
Definitions
• Decision problem - a problem having the answer
either 0 or 1. Decision problem can be NPC
• Decision Algorithm – solves a decision problem.
Gives output 1 on successful completion and 0 on
unsuccessful termination
• Optimization Problem – problem involving the
identification of an optimal value (either maximum or
minimum) for a given cost function. Optimization
problem may be NP Hard
• Satisfiability problem – find out whether a given
expression is true for some assignment of truth values
to the variable in that expression
Definitions
• Clique – maximal complete subgraph for a given
graph, size of Clique is the number of vertices in it
• Reducibility – Let L1 and L2 are 2 problems. L2 can
be solved in polynomial time using a deterministic
algorithm. If the same algorithm can solve problem
L1 in deterministic polynomial time, then L1 α L2
(L1 Reduces to L2)
• α is a transitive relation
NDA
• It is possible to construct NDA for which many
different choices lead to completions.
• Ex. If numbers of Array A are equal , many different
permutations will result in sorted sequence.
• Concern with NDA that generate unique output
• Successful completion if output 1
• Output 0 if no sequence of choices leading to a
successful completion.
• Output statements are implicit in signals success and
failure
• Idea of decision algorithm may appear very restrictive
but, many optimization problems can reformed into
decision problems with property
Decision problem can be solved in polynomial time if
and only if the corresponding optimization problem can
• In other cases, we can at least make statement that if
the decision problem cannot be solved in polynomial
time then the optimization problem cannot either.
Maximal Clique
• A maximal complete subgraph of a graph G(V,E) is
clique.
• Size of clique is the number of vertices in it.
• Max Clique problem is an optimization problem that
has to determine the size of largest Clique in G
• Corresponding decision problem is to determine
whether G has a clique of size at least k for some give
k.
Maximal Clique – Decision Problem
• Let Dclique(G,k) be a deterministic decision algorithm
for the Clique decision problem
• Size of max clique can be found by different values of k
• K = n, k= n-1, k= n-2 ,…. Until Dclique output = 1
• If time complexity of Dclique = f(n) then size of max
clique can be found in time <=n.f(n)
• Also if size of max clique can be determined in time
g(n), then the decision problem can be solved in time
g(n)
• Max clique problem can be solved in polynomial if and
only if the clique decision problem can be solved in
polynomial time
Non Deterministic Clique Pseudocode
1. Algorithm DCK(G,n,k)
2. {
3. S=0; // s is initially empty set
4. For i= 1to k do
5. {
6. T = choice (1,n);
7. If t € S then failure ();
8. S = s union {t} // add t to S
9. }
10. // at this point S contains k distinct
vertex indices
11. For all pairs(I,j) such that i€S, j€S and
i!=j do
12. If (I,j) is not an edge of G then
failure();
13. Success()
14. }
•
•
Algorithm begins by trying to form a
set of k distinct vertices
Tests to see whether these vertices
form a complete subgraph.
Satisfiability problem
• Let x1,x2 variables and x1’,x2’ are negation
• Literal is either a variable or its negation
• Disjunctive normal form (CNF) – (x1^ x2)V(x3^x4’)
AKA Sum of Product
• Conjunctive normal form (CNF) –
(x3Vx4’)^(x1VX2’) - AKA Product of Sum
• Satisfiability problem is to determine whether a
formula is true for some assignment of truth values to
the variables.
Satisfiability problem
1. Algorithm eval (E,n)
2. // Determine whether the
propositional formula E is
satisfiable
3. For i=1 to n do // choose a
truth value assignment
4. X = choice(false,true);
5. If E(x1,x2,…xn)then
success();
6. Else failure();
7. }
• Algorithm terminates successfully
if and only if a given
propositional formula
E(x1,x2,….xn) is satisfiable
• Algorithm proceed by simply
choosing non deterministically
one of the 2 n possible
assignements of truth values to
x1,x2,…xn and verifying that E is
true for that assignment
• Time taken is O(n) to choose
values of x1,x2,….xn plus the
time needed to deterministically
evaluate E for that assignment
which is proportional to E
Relation between p, np, npc and nph
• Deterministic Algorithm A
and input I, halting problem
is to determine that whether
A will stop or enter in an
infinite loop.
• Deterministic algorithms are
special case of non
deterministic algorithms
np
nph
p
npc
• What we do not know and
what has become the most
famous unsolved problem in
computer science is whether
P = NP or P != NP
Cook theorem
• Is there any single problem in NP such that if we
showed it to be in P, then that would imply that P =
NP?
• Satisfiability is in P if and only if P = NP
NP Hard and NPC
• A problem L is NP Hard if and only if Satisfiability
reduces to L (Satisfiability α L)
• A problem is NPC if and only if L is NP hard and
L € NP
• Only a decision problem can be NPC
• Optimization problems may be NPH
• If L1 is decision problem and L2 is optimization
problem is possible L1 α L2
• Ex. Knapsack decision problem reduces to knapsack
optimization problem
• Clique decision problem reduces to Clique
optimization problem
NP Hard and NPC
• Optimization problems may reduces to corresponding
decision problem
• Optimization problems cannot be NP Complete where
as decision problem can
• There also exist NP Hard decision problems that are
not NPC
Halting problem of Deterministic algorithm
• Arbitrary deterministic algorithm A with input I ever
terminates (or enters into infinite loop)
• This is well known undecidable problem, No
algorithm (of any complexity), Clearly cannot be in
NP
• To show Satisfiability α halting problem, construct
algorithm A, whose input are propositional to formula
X.
• X has n variables and A tries 2n possible truth
assignments and verifies X is satisfiable
• A stops if X is satisfiable or enters in an infinite loop
Halting problem of Deterministic algorithm
• If we had a polynomial time algorithm for halting
problem then we could solve Satisfiability problem in
polynomial time using Algorithm A and input X
• Halting problem is NP HARD NOT NP
Polynomially equivalence
• Two problems L1 and L2 are said to be polynomially
equivalent if and only if L1 α L2 and L2 α L1
• Let L1 is NP Hard, To show L2 as NP Hard, show
L1 α L2
• We know α is transitive relation, it follows if
Satisfiability α L1 and L1 α L2 then Satisfiability α
L2
• To show that an NP HARD decision problem is NPC
exhibit a polynomial time nondeterministic algorithm
for it
CNF Satisfiability α Clique Decision Problem (CDP)
• Let F = ᴧ 1<=i<=k Ci be a propositional formula in
Conjunctive normal form (CNF)
• Let 1<= I <=n be the variables in F
• Construct a graph G=(V,E) from F, such that G has a
clique of size at least k if and only if F is satisfiable
• If we have a polynomial time algorithm for CDP then
we can obtain a polynomial time algorithm for CNF
• For any F, Define graph G as V = {(σ,i)|σ is literal in
clause Ci} E= {(σ,i),(δ,j)|i≠j and σ≠δ’}
Example
• F=(x1 v x2 v x3)ᴧ(x1’v x2’v x3’)
(x1,1)
(x1’,2)
(x2,1)
(x2’,2)
(x3,1)
(x3’,2)
• By setting x1 = true and x2’ = true F is satisfied
Node Cover decision problem (NCDP)
• A set S € V is a node cover for a graph G(V,E) if and
only if all edges in E are incident to at least one
vertex in S.
• Size of |S| of the cover is the number of vertices in S
• 1
2 S = {2,4} is anode cover of size 2
•
3 S = {1,3,5} size 3
• 5
4
• In NCDP , we are given a graph and integer K,
determine whether G has a node cover of size K
CDP α NCDP
• Let G (V,E) and K define in instance in CDP, |V|=n
• Construct G’ (complement of a graph) with a node
cover of size at most n-k if and only if G has a clique
of size at least k
2
2
1
5
3
1
5
3
4
4
• G
G’
• G’ has node cover of {4,5}
• G has Clique of size 5-2 = 3 with node 1-2-3
Polynomial Reductions
• Problem P is reducible to Q
P p Q
Transforming inputs of P
 to inputs of Q
• Reducibility relation is transitive.
Algorithmic Analysis
Dynamic Programming
Dynamic Programming
•
Dynamic programming is typically applied to optimization
problems. In such problem there can be many solutions. Each
solution has a value, and we wish to find a solution with the
optimal value.
Divide-and-conquer
• Divide-and-conquer method for algorithm design:
• Divide: If the input size is too large to deal with in a straightforward
manner, divide the problem into two or more disjoint subproblems
• Conquer: conquer recursively to solve the subproblems
• Combine: Take the solutions to the subproblems and “merge” these
solutions into a solution for the original problem
Dynamic programming
• Dynamic programming is a way of improving on inefficient divideand-conquer algorithms.
•
By “inefficient”, we mean that the same recursive call is made over
and over.
•
If same subproblem is solved several times, we can use table to store
result of a subproblem the first time it is computed and thus never have
to recompute it again.
• Dynamic programming is applicable when the subproblems are
dependent, that is, when subproblems share subsubproblems.
Ex1:Assembly-line scheduling
• Automobiles factory with two assembly lines.
 Each line has the same number “n” of stations. Numbered j = 1, 2, ..., n.
 We denote the jth station on line i (where i is 1 or 2) by Si,j .
 The jth station on line 1 (S1,j) performs the same function as the jth station on
line 2 (S2,j ).
 The time required at each station varies, even between stations at the same
position on the two different lines, as each assembly line has different
technology.
 time required at station Si,j is (ai,j) .
 There is also an entry time (ei) for the chassis to enter assembly line i and an exit
time (xi) for the completed auto to exit assembly line i.
Ex1:Assembly-line scheduling
• After going through station Si,j, can either
 Job stay on same line
 next station is Si,j+1
 no transfer cost , or
 job transfer to other line
 next station is Si+1,j+1
 transfer cost from Si,j to Si+1,j+1 is ti,j ( j = 1, … , n–1)
40
Ex1:Assembly-line scheduling
•(Time between adjacent stations are nearly 0).
Optimal Solution Structure
• optimal substructure : choosing the best path to Sij.
• The structure of the fastest way through the factory (from
the starting point)
• The fastest possible way to get through Si,1 (i = 1, 2)
 Only one way: from entry starting point to Si,1
 take time is entry time (ei)
Step 1: Optimal Solution Structure
• The fastest possible way to get through Si,j (i = 1, 2) (j = 2, 3, ..., n). Two
choices:
 Stay in the same line: Si,j-1  Si,j
 Time is Ti,j-1 + ai,j
 If the fastest way through Si,j is through Si,j-1, it must have taken a fastest way
through Si,j-1
 Transfer to other line: Si+1,j-1  Si,j
 Time is Ti+1,j-1 + t3-i,j-1 + ai,j
Optimal Solution Structure
• An optimal solution to a problem
 finding the fastest way to get through Si,j
•
contains within it an optimal solution to sub-problems
 finding the fastest way to get through either Si,j-1 or Si+1,j-1
•
Example
Recursive Solution
• Define the value of an optimal solution recursively in terms of the
optimal solution to sub-problems
• Sub-problem here
 finding the fastest way through station j on both lines (i=1,2)
 Let fi [j] be the fastest possible time to go from starting point through Si,j
•
The fastest time to go all the way through the factory: f*
•
x1 and x2 are the exit times from lines 1 and 2, respectively
Thank you