UNIT III
DYNAMIC PROGRAMMING ALGORITHMS
Developing a Dynamic Programming Algorithm-Subtle Points- Question for
the Little Bird- Sub instances and Sub solutions-Set of SubstancesDecreasing Time and Space-Number of Solutions-Code. Reductions and NPCompleteness-Satisfiability-Proving NP-Completeness- 3-Coloring- Bipartite
Matching. Randomized Algorithms-Randomness to Hide Worst CasesOptimization Problems with a Random Structure.
Dynamic Programming
• A hard topic.
• I try to provide a unified way to think of it
and a fixed set of steps to follow.
• Even if you don’t get the details of the
algorithm correct, at least get the right structure.
• I provide analogies (little bird) to make it
hopefully more fun & easier to follow.
Non-Deterministic
Poly-Time Decision Problems (NP)
•An optimization problem
•Each solution is either valid or not
(no cost)
•The output is
•Yes, it has a valid solution.
•No, it does not
•the solution is not returned
•Eg: Given graph and integer <G,k>,
does G have a clique of size k?
•Key: Given
•an instance I
(= <G,k>)
•and a solution S (= subset of nodes)
•there is a poly-time alg Valid(I,S) to test
whether or not S is a valid solution for I.
•Poly-time in |I| not in |S|. |S| can't be too big.
Not Valid
Valid
Formal definition:
Prob  NP iff  poly time Valid
such that Prob(I) =  S Valid(I,S)
k=4
•Key:
•If the instance has a valid solution
•A non-deterministic (fairy god mother)
could prove it to you by giving you
such a solution as a witness.
•You could check that it is valid.
•You could convince your boss.
Valid
k=4
•Key:
•If the instance does not have a valid solution
•A non-deterministic (fairy god mother)
could prove it to you by giving you
????
•You have no way to convince your boss.
k=5
Non-Deterministic Poly-Time (NP)
•Example: 3-Col:
•Instance: A graph G.
•Solution: Colouring C nodes of G with
3 colours such that every edge has two
colours.
•G is a Yes instance if there is such a
colouring.
•Given an instance G and a solution C,
there is a poly-time alg Valid(G,C) to
test whether or not C is a valid 3colouring
of G.
 3-Col  NP.
Non-Deterministic Poly-Time (NP)
•Example: Airplane Wing:
•Instance: Requirements I of the wing.
•Solution: A description S of how to make the wing.
•I is a Yes instance if there is such a wing.
•Given an instance I and a proof S,
there is a poly-time alg Valid(I,S) to test
whether or not S is a valid solution for I.
Airplane Wing  NP.
I = [weight, lift, cost, …]
Reductions
Palg ≤poly Poracle
•Reduction: Design a fast algorithm for one computational problem,
using a supposedly fast algorithm for another problem as a
subroutine.
NP vs Co-NP Preserving Reductions
• Definition is asymmetric.
• There is a witness that a graph has a 3-Col.
• There is no known witness that a graph has no 3-Col.
Cook Reduction:
• Design any fast algorithm for Palg using a supposed fast
algorithm for Poracle as a subroutine.
Karp Reduction:
The algorithm for Palg calls that for Poracle only once
Yes  Yes & No  No
Karp Reduction:
Yes  Yes
& No  No
We will only consider reductions of this simple form. Because they preserve NP
vs Co-NP
• Problem Pnew is NP-Complete
• Pnew not too hard.
• Pnew  NP
NP-Complete Problems
Computable
Exp
Test in poly-time
if a given solution
is valid
Sat
NP
complete
Pnew
Poly
Known
GCD
NP-Complete Problems
Problem Pnew is NP-Complete
Computable
•Pnew not too hard.
Exp
•Pnew  NP
•Pnew sufficiently hard.
•PNP, P ≤poly Pnew
NP
Sat
•Easier: Sat ≤poly Pnew
•Cook: P ≤poly Sat complete
Pnew
Poly
Known
GCD
K-Clique vs K-Independent Set
Clique: Given <G,k>, does G contains a k-clique?
Independent Set: Given <G,k>, does G contains a k-Ind Set?
A K-independent set
is a set of K nodes
with no edges between them.
A K-clique is a set of K nodes
with all edges between them.
K-Clique vs K-Independent Set
Clique: Given <G,k>, does G contains a k-clique?
Independent Set: Given <G,k>, does G contains a k-Ind Set?
Brute Force: Try out all n choose k possible subsets.
If k = (n) then 2(n) subsets to check
only O(n3)
If k=3 then
K-Clique vs K-Independent Set
Clique
<G,k>
≤poly Indep Set
G has a k clique
or not
<G*, k>
BUILD:
Clique
Oracle
G* has a k Indep. set
or not
GIVEN:
Indep.
Set
Oracle
K-Clique vs K-Independent Set
Clique
<G,k>
≤poly Indep Set
G has a k clique
or not
<G*, k>
Proof of correctness:
G* has a k Indep. set
BUILD:
Our oracle says
yes to <G,k> or not
Clique
iff Old oracle
says yes to <G*,k>
iff G* has Oracle
a k indep. set
iff G has a k clique
GIVEN:
Indep.
Set
Oracle
K-Clique vs K-Independent Set
G
G*
G* has edge
if and only if
G does not
K-Clique vs K-Independent Set
G
This graph
contains a
clique
of size 4.
if and only if
G*
This graph
contains an
independent
set of size 4.
12 Steps
•Steps for proving that Pnew is NP-Complete
•
1) I am addicted to solving NP-Complete Problems
• 2) I trust in my higher power to help
A witness
12 Steps
•Steps for proving that Pnew is NP-Complete
•
0) Pnew  NP
•
1) What to reduce it to
2) What is what
3) Direction of reduction & Code
4) Look for similarities
5) Instance Map
6) Solution Map
7) Valid to Valid
8) Reverse Solution Map
9) Valid to Valid
10) Working Algorithm
11) Running Time
12 Steps
Step 0: Pnew  NP
Formal definition:
Pnew  NP iff  poly time Valid
such that Pnew(I) =  S Valid(I,S)
•Design poly-time algorithm Valid(I,S)
•which determines whether a given solution S
•is valid for a given instance I.
12 Steps
Step 1: What to reduce it to
Palg ≤poly Poracle
Choose a problem Pis NP-comp
that is as similar to Pnew as possible and
that is already known to be NP-Complete.
12 Steps
Step 2: What is what
Pnew
Inew
= Independent Set
Snew
Pis NP-comp
Iis NP-comp
k=3
= Clique
Sis NP-comp
k=4
12 Steps
Step 3: Direction of reduction & Code
Reduce Pnew to Pis NP-comp or Pis NP-comp to Pnew?
Pis NP-comp ≤poly Pnew
Palg ≤poly Poracle
12 Steps
Step 3: Direction of reduction & Code
12 Steps
Step 4: Look for similarities
Clique
Independent Set
Both instances are graphs: nodes & edges
A clique solution is a subset of the nodes with edges.
An Ind Set solution is a subset of the nodes without edges.
12 Steps
Step 5: Instance Map
12 Steps
Step 5: Instance Map
Ialg
Ioracle = InstanceMap(Ialg)
12 Steps
Step 5: Instance Map
Ialg
no yes
Yes mapped to Yes
No to No
Ioracle
no yes
12 Steps
Step 6: Solution Map
12 Steps
Step 6: Solution Map
Ialg
Potential Solution
= SolutionMap(S).
Ioracle
Potential Solution S.
12 Steps
Step 7: Valid to Valid
Ialg
Valid
Salg = SolutionMap(Soracle)
Ioracle = InstanceMap(Ialg)
If Soracle is valid
solution for Ioracle,
then Salg is valid
solution for Ialg
Valid
Soracle
12 Steps
Step 8: Reverse Solution Map
Ialg
Potential Solution S.
Ioracle
Not part of code,
but of proof.
Potential Solution
= ReverseSolutionMap(S).
12 Steps
Step 9: Valid to Valid
Ialg
Valid
Salg
Ioracle = InstanceMap(Ialg)
If Salg is valid
solution for Ialg,
then Soracle is valid
solution for Ioracle
Valid
Soracle = ReverseSolutionMap(Salg)
12 Steps
Step 10: Working Algorithm
Our Algalg says yes to Ialg
iff old Algoracle says yes to Ioracle
iff Ioracle has a valid solution Soracle
iff Ialg has a valid solution Salg
iff Ialg is a yes instance.
6&7
Steps 8&9
12 Steps
Step 11: Running Time
Algalg(Ialg) is poly time if
the following are poly time:
Algoracle(Ioracle)
Assumed
Ioracle = InstanceMap(Ialg)
Salg = SolutionMap(Soracle)
Your job
Palg ≤poly Poracle
or Palg
Poracle
Scheduling
Graph
Colouring
Clique
Independent Set
Circuit
Satisfiability
Any NP-Problem
This will prove that
Cir-SAT is NP-Complete.
Reduction
Any NP-problem
I
≤poly Cir-SAT
I don’t even know what
problem I am trying to solve!!!
?
BUILD:
Oracle for
arbitrary
NP Problem
GIVEN:
Oracle for
Cir-Sat
We need to solve some unknown NP-Problem.
What do we know about it?
Formal definition:
Parbitrary  NP iff  poly time Valid
such that Parbitrary(I) =  S Valid(I,S)
We have a poly-time alg Valid(I,S) to test
whether or not S is a valid solution for I.
Valid
k=4
Reduction
Any NP-problem
≤poly Cir-SAT
I
Please, give me
a solution S
such that
Valid(I,S) is true.
BUILD:
Oracle for
arbitrary
NP Problem
That looks like
a Turing Machine.
I only know about
circuits
GIVEN:
Oracle for
Cir-Sat
The Circuit Satisfiability Problem
An instance is a circuit C.
A solution is an assignment X = [F,T,F…].
C(X) evaluates to T or F.
x1 F
x2T
x3F
AND
AND
F
F
OR
F
OR
NOT
F
OR
T
One bit output
No feedback
The Circuit Satisfiability Problem
An instance is a circuit C.
A solution is an assignment X = [F,T,F…].
A valid solution has C(X) = True.
x1 F
x2F
x3F
AND
AND
F
F
OR
F
OR
NOT
T
OR
F
Given a circuit,
does it have a
satisfying assignment?
The Circuit Satisfiability Problem
Turing (and friends) prove that
any algorithm computed by a
JAVA program in poly-time
can be computed by a
Turing Machine in poly-time.
Very Powerful
Cook proves that any algorithm
computed by a Turing Machine
in time T(n) can be computed
by a family of circuits of size [T(n)]2.
But clearly, circuits compute.
Reduction
Any NP-problem
≤poly Cir-SAT
I build a circuit VI(S)
equivalent to Valid(I,S)
I
Please, give me
a solution S
such that
VI(S) is true.
BUILD:
Oracle for
arbitrary
NP Problem
Thanks for the circuit.
But what is S?
I can only find
assignments.
GIVEN:
Oracle for
Cir-Sat
Reduction
Any NP-problem
I
≤poly Cir-SAT
I build a circuit VI(S)
I decode X equivalent
into S
to Valid(I,S)
and am done.
Please, give me
an assignment X
such that
VI(X) is true.
BUILD:
Oracle for
arbitrary
NP Problem
My pleasure.
Here: X
GIVEN:
Oracle for
Cir-Sat
12 Step Program
•Let's be more formal.
•And flow the 12 steps.
Step 0: Cir-SAT  NP
•Design poly-time algorithm ValidCir-SAT(C,X)
•which determines whether a given assignment X
•is valid for a given circuit C.
• Easy: Evaluate C(X).
Step 1: What to reduce it to
Palg ≤poly Cir-Sat
Choose a problem Pis NP-comp
that is as similar to Cir-Sat as possible and
that is already known to be NP-Complete.
Have none.
Any NP-problem
≤poly Cir-SAT
We have a poly-time alg Valid(I,S) to test
whether or not S is a valid solution for I.
Step 2: What is what
Pnew
Inew
x1 F
AND
Parbitrary
= Circuit-Sat
Snew
x2F
x3F
AND
OR
NOT
OR
OR
Iarbitrary
= some NP problem
=I
Sarbitrary
=S
Step 3: Direction of reduction & Code
Given oracle for Cir-Sat,
we need to be able to solve any NP-Complete problem.
Parbitrary
Cir-Sat
Step 4: Look for similarities
x1F
AND
x2F
x3F
AND
S
I
OR
NOT
OR
?
OR
Step 5: Instance Map
Step 5: Instance Map
I  circuit
We have a poly-time alg Valid(I,S) to test
whether or not S is a valid solution for I.
Let Validn(I,S) be a circuit:
•I is a bit string representing an instance I.
•S is a bit string representing a solution S.
•Outputs T if S is an encoding of a valid solution of I.
Eg: Clique
G,k
I
S
{u.v}  E OR uS OR vS
|S|  k
And over pairs of nodes
Step 5: Instance Map
I  circuit
Given an instance I
Circuit VI(S) = Valid(I,S)
I
hard wired
Validn(I,S)VI(S)
I
S
Outputs T if
S is an encoding of
a valid solution S of I
Step 6: Solution Map
Step 6: Solution Map
S  assignment
X=[T,F,F,T,F,T]
X is viewed as a bit string S representing a solution S.
If X is not a bit string
representing a solution
then “what ever”
I
hard wired
I
VI(S)
S
Outputs T if
S is an encoding of
a valid solution S of I
Step 6: Solution Map
S  assignment
X=[T,F,F,T,F,T]
X is viewed as a bit string S representing a solution S.
solution S
I
hard wired
I
VI(S)
S
Outputs T if
S is an encoding of
a valid solution S of I
Step 7: Valid  Valid
VI(X) = T
Valid(I,S) = Validn(I,S) = VI(X) = T
S is a valid solution of I.
I
hard wired
I
VI(S)
S
Outputs T if
S is an encoding of
a valid solution S of I
Step 8: Rev. Sol. Map
solution  assignment
solution S
S bit string representing a solution S
X=[T,F,F,T,F,T]
I
hard wired
I
VI(S)
S
Outputs T if
S is an encoding of
a valid solution S of I
Step 9: Valid  Valid
S is a valid solution of I.
VI(X) = Validn(I,S) = Valid(I,S) = T
VI(X) = T
I
hard wired
I
VI(S)
S
Outputs T if
S is an encoding of
a valid solution S of I
Reduction
Any NP-problem
≤poly Cir-SAT
I Yes/No
i.e.  S, S is a valid solution for I
VI
BUILD:
Opt.
problem
satisfiable
or not
i.e.  X, VI(X)
GIVEN:
Alg for
circuit
problem
Palg ≤poly Poracle
or Palg
Poracle
3-Col
?
Scheduling
Graph
Colouring
Clique
Independent Set
Circuit
Satisfiability
Any NP-Problem
3-Col
Graph Col: Given graph G & k
can G be coloured with k colours?
3-Col:
Given graph G
can G be coloured with 3 colours?
3-Col
≤poly Graph Col
If you can decide whether you can colour with k colours,
then you can decide whether you can colour with 3 colours.
G,k
C
G
C
Palg ≤poly Poracle
or Palg
Poracle
3-Col
Scheduling
Graph
Colouring
Clique
Independent Set
3-SAT
?
Circuit
Satisfiability
Any NP-Problem
3-SAT
Cir Sat: Given circuit C
does it have a satisfying assignment X?
3-SAT: Given an expression:
•A circuit consisting of a big AND of clauses
•Each clause is the OR of at most 3 literals
•Each literal is a variable or its negation.
does it have a satisfying assignment X?
FT F F T F
FT F F T F
xoryorz AND xorwora AND …
3-SAT
Cir Sat: Given circuit C
does it have a satisfying assignment X?
3-SAT: Given an expression:
does it have a satisfying assignment X?
3-Sat
≤poly Cir Sat
If you can decide whether any circuit is satifiable,
then you can decide whether a circuit that happens to be
an expression is satisfiable.
FT F F T F
FT F F T F
xoryorz AND xorwora AND …
Palg ≤poly Poracle
or Palg
Poracle
3-Col
Scheduling
Graph
Colouring
Clique
Independent Set
3-Col
Graph Col
?
≤poly 3-SAT

≤poly Cir SAT
3-SAT
Circuit
Satisfiability
Any NP-Problem
Step 5: Instance Map
Graph G  Expression V
Given an instance G
Circuit VG(C) = Valid(G,C)
Encoding C of C:
u is a node
r is a colour
x<u,r> = T
if node u is colour r.
G
hard wired
C
Outputs T if
C is an encoding of
a valid 3-colouring C of G
Step 5: Instance Map
Graph G  Expression V
Given an instance G
Circuit VG(C) = Valid(G,C)
G
hard wired
C
Encoding C of C:
u is a node
x<u, r> = F OR x<v, r> = F
r is a colour
x<u,r> = T
if node u is colour r. And over edges <u,v> and colours r
clauses with 2 literals
Step 5: Instance Map
Graph G  Expression V
Given an instance G
Circuit VG(C) = Valid(G,C)
G
hard wired
C
Encoding C of C:
u is a node
x<u, r> =T OR x<u, g>=T OR x<u, b>=T
r is a colour
x<u,r> = T
And over node u
if node u is colour r.
clauses with 3 literals