CPSC 320: Intermediate Algorithm
Design and Analysis
July 30, 2014
1
Course Outline
• Introduction and basic concepts
• Asymptotic notation
• Greedy algorithms
• Graph theory
• Amortized analysis
• Recursion
• Divide-and-conquer algorithms
• Randomized algorithms
• Dynamic programming algorithms
• NP-completeness
2
Schedule
• Monday: BC Day, no classes
• Wednesday:
• Quiz 5 (NP and NP completeness)
• Review: Amortized Analysis
• Friday:
• Survey (bonus marks)
• Review: probably Graph Theory and Dynamic Programming
3
NP Completeness
4
Recap
• We are working on problems, not algorithms
• Our focus now is on decision problems (yes/no), not optimization problems
• We distinguish “finding” a solution and “checking” a solution
• Classes:
• P: decision problems that are solvable in polynomial time
• NP: decision problems for which a given certificate can be checked in
polynomial time
• NP hard: at least as “hard” as any problem in NP
• NP complete (NPC): both NP and NP hard
5
NP complete
• NPC problems:
• have no known algorithm that runs in polynomial time
• There is no proof that such an algorithm doesn’t exist
• Examples:
• Hamiltonian path: path that traverses all nodes exactly once
• 3-SAT: assign values to Boolean variables that satisfy a set of clauses
• Graph coloring: assign colors to graphs, adjacent nodes have different colors
• Cook’s theorem (paraphrased): if the satisfiability problem can be solved in
polynomial time, any problem in NP can be solved in polynomial time
6
NP complete
• Why is it useful to know a problem is NP complete?
• So you can stop looking for trivial answers
• Focus on heuristics or local optimality
• Evaluate pseudo-polynomial options
7
NPC Proof
• How do we prove a problem 𝑝 is NP hard?
• Reduce any known NPC problem 𝑝′ into 𝑝 in polynomial time
• If 𝑝′ is NPC, and we can solve 𝑝′ by reducing it into 𝑝, then 𝑝 is bound in time
complexity by the problem 𝑝′
• We say that 𝑝′ ≤𝑃 𝑝
• Proof has three main parts:
• Demonstrate a polynomial-time transformation from an instance of 𝑝′ to an
instance of 𝑝
• Prove that if 𝑝′ solves as Yes, then 𝑝 solves as Yes in the transformed instance
• Prove that if 𝑝 solves as Yes in the transformed instance, then 𝑝′ solves as Yes
• How do we prove a problem is NPC?
• Prove it is NP, and prove it is NP hard
8
Clique
• Problem: Given a graph 𝐺, is there a set (“clique”) of 𝑘 nodes for which there is an
edge for every pair of nodes (a fully-connected subgraph)?
• Is this NP?
• Given a set of 𝑘 nodes, we can check if it forms a clique in 𝑂(𝑛2 )
• Is this NPC?
• Can we reduce an NPC problem into it?
9
Clique – NPC proof
• We will try to prove that 3-SAT ≤𝑃 Clique
• Given an instance 𝐼 of 3-SAT, with 𝑘 clauses, build a graph 𝐺 such that:
• For each clause 𝐶𝑟 , build one node 𝑣𝑖𝑟 for each literal in 𝐶𝑟
• Build an edge between 𝑣𝑖𝑟 and 𝑣𝑗𝑠 if 𝑟 ≠ 𝑠 and the corresponding literals are not
negations of each other (e.g., they are not 𝑥 and 𝑥)
• We can build this graph in polynomial time
10
Clique – NPC proof
• We want to prove that 𝐼 is satisfiable if and only if there is a clique with size 𝑘
• If 𝐼 is satisfiable:
• At least one of the literals in each clause is true
• The node for a true literal in each clause is connected to the true literal in all
other clauses
• There are 𝑘 clauses, so this clique has 𝑘 nodes
• If there is a clique with size 𝑘
• Each node in the clique is in a different clause
• The nodes in the clique are not contradictory
• We can assign true/false for nodes that correspond to variables/negations, and
any value for other variables
• QED
11
Hamiltonian Path
• Problem: given a graph 𝐺, is there a path that traverses all nodes exactly once?
• Variation: Hamiltonian cycle, a path that traverses all nodes and returns to the
node of origin
• Is this problem in NP?
• Is this problem NP complete?
12
Hamiltonian Path – NPC proof
• Let’s prove 3-SAT ≤𝑃 Hamiltonian Path
• Given an instance 𝐼 of 3-SAT with 𝑛 variables and 𝑚 clauses, build a directed graph
𝐺 = (𝑉, 𝐸) with:
• Two nodes, 𝑠 and 𝑡
• For each variable 𝑖, b = 3𝑚 + 3 nodes 𝑣𝑖1 . . 𝑣𝑖𝑏 , with edges in both directions (1 to
𝑏 and 𝑏 to 1)
• An edge from each variable’s ends to the next variable’s ends, and from 𝑠 to 𝑣𝑖
ends, and from 𝑣𝑛 ends to 𝑡
• For each clause 𝑗, a node 𝑐𝑗 and an edge from/to a corresponding node for
each variable in the clause
• Left-to-right path represents true, Right-to-left represents false
• See Kleinberg section 8.5 or handwritten nodes for more details
13
Hamiltonian Path – NPC proof
• If 𝐼 is satisfiable:
• There is a path from 𝑠 to 𝑡 going in the direction of the values of the variables
• For each clause, one of the variables can “diverge” to the corresponding node
• So all nodes are visited, so there is a Hamiltonian path
• If there is a Hamiltonian path:
• The path has to traverse a path in one of the directions fully
• The path has to visit each clause once, and must return to the same variable
to fully visit all nodes
• The direction the variable nodes are traversed determines the value of its
variable in 𝐼, and one of the literals in the clause has to be obeyed
• So the variables correctly solve the 3-SAT problem
• QED
14
Traveling Salesman Problem
• Problem: a salesman wants to visit a list of cities, each city once, with minimal
traveling cost
• Converting into decision problem: can a salesman visit all cities, each city
once, with cost under 𝑘?
• Is this problem NP?
• Is this problem NP complete?
15
Traveling Salesman Problem
• Let’s prove Hamiltonian Path ≤𝑃 Traveling Salesman
• Given a graph 𝐺 = (𝑉, 𝐸), build an instance of the traveling salesman problem with:
• Each node in 𝑉 corresponds to a city
• The cost between two cities is 1 if there is an edge between corresponding
nodes, or 2 otherwise
• The maximum cost to traveling salesman is the number of nodes 𝑛 minus one
16
Traveling Salesman Problem
• If there is a Hamiltonian path
• The Hamiltonian path corresponds to an itinerary for the traveling salesman,
with cost equal to the number of traversed edges, so 𝑛 − 1
• If there is an itinerary with cost 𝑛 − 1
• Since 𝑛 cities are traversed, the itinerary only used paths with cost 1
• So the itinerary only passed through pairs of nodes with an edge, so it’s a
Hamiltonian Path
• QED
17
Subset Sum
• Problem: given a set of numbers 𝑤1 . . 𝑤𝑛 and a target 𝑊, is there a subset of the
numbers that adds up to 𝑊?
• We worked with this in dynamic programming, but result was pseudopolynomial
• Is this problem NP?
• Is this problem NPC?
• http://cs.mcgill.ca/~lyepre/pdf/assignment2solutions/subsetSumNPCompleteness.pdf
18