LECTURE 11:
Dynamic programming
-I-
Algorithmics - Lecture 11
1
Outline
• What is dynamic programming ?
• Basic steps in applying dynamic programming
• Bottom-up vs. top-down approaches in developing
recurrence relations
• Applications of dynamic programming
Algorithmics - Lecture 11
2
What is dynamic programming ?
• It is a technique for solving problems which can be decomposed in
overlapping subproblems – it can be applied to problems having
the optimal substructure property
• Its particularity is that it solves each smaller subproblem only
once and it records each result in a table from which a solution to
the initial problem can be constructed.
Remarks.
• Dynamic programming was developed by Richard Bellman in
1950 as a general method for optimizing multistage decision
processes.
• In dynamic programming the word programming stands for
planning not for computer programming.
• The word dynamic is related with the manner in which the tables
used in obtaining the solution are constructed
Algorithmics - Lecture 11
3
What is dynamic programming ?
• Dynamic programming strategy is related with divide and conquer
strategy because both of them are based on dividing a problem in
subproblems. However there are some differences between these
two approaches:
– divide and conquer: the subproblems are usually independent so the
solution of one subproblem cannot be used for another subproblem
– dynamic programming: the subproblems are dependent (overlapping)
so we need the result obtained for a subproblem for several times
(therefore it is important to store it)
• Dynamic programming is also related to greedy strategy since
both of them can be applied to optimization problems which have
the property of optimal substructure
Algorithmics - Lecture 11
4
Outline
• What is dynamic programming ?
• Basic steps in applying dynamic programming
• Bottom-up vs. top-down approaches in developing
recurrence relations
• Applications of dynamic programming
Algorithmics - Lecture 11
5
Basic steps in applying dynamic programming
1.
Analyze the structure of a solution: establish how the solution of
a problem depends on the solutions of its subproblems. Usually
this step means verifying the optimal substructure property.
2.
Find a recurrence relation which relates a value (e.g. the
optimization criterion) corresponding to the problem’s solution
with the values corresponding to subproblems solutions.
3.
Develop the recurrence relation (in dynamic programming it is
common to use a bottom up approach) and construct a table
containing information useful to construct the solution.
4.
Construct the solution based on information collected in the
previous step.
Algorithmics - Lecture 11
6
Outline
•
What is dynamic programming ?
•
Basic steps in applying dynamic programming
•
Bottom-up vs. top-down approaches in developing
recurrence relations
•
Applications of dynamic programming
Algorithmics - Lecture 11
7
Developing recurrence relations
There are two approaches in developing a recurrence relation:
•
Bottom-up: start from the particular case and try to generate new
values by using the already computed values (the values which
are used to compute further values are at least temporarily
stored).
•
Top-down: try to construct the desired element by expressing it
through some previous elements (which are not known yet, thus
they also need to be computed). This approach is usually
implemented in a recursive manner. It has the main disadvantage
that it might need the (re)computation for several times of the
same value
Algorithmics - Lecture 11
8
Developing recurrence relations
Example 1. Computing the m-th element of Fibonacci’s sequence
f1=f2=1; fn=fn-1+fn-2 for n>2
Efficiency:
Dominant op.: addition
0
if m<=2
Top-down approach:
T(m) =
T(m-1)+T(m-2)+1 if m>2
fib(m)
T:
IF (m=1) OR (m=2) THEN
RETURN 1
0 0 1 2 4 7 12 20 33 54 …
ELSE
Fibonacci:
RETURN fib(m-1)+fib(m-2)
1 1 2 3 5 8 13 21 34 55 …
ENDIF
fm belongs to O(phim),
phi=(1+sqrt(5))/2
Exponential complexity !
Algorithmics - Lecture 11
9
Developing recurrence relations
Example 1. Computing the m-th element of Fibonacci’s sequence
f1=f2=1; fn=fn-1+fn-2 for n>2
Efficiency:
Bottom-up approach:
T(m)=m-2 => linear complexity
fib(m)
f[1]←1; f[2] ← 1;
FOR i ← 3,m DO
f[i] ← f[i-1]+f[i-2]
ENDFOR
RETURN f[m]
Remark: space-for-time tradeoff
situation
The use of extra space can be avoided
(limited to only two additional
variables)
fib(m)
f1 ← 1; f2 ← 1;
FOR i ← 3,m DO
f2 ← f1+f2; f1 ← f2-f1;
ENDFOR
RETURN f2
Algorithmics - Lecture 11
10
Developing recurrence relations
Example 2. Computing a binomial coefficient C(n,k)
0,
C(n,k)=
if n<k
1, if k=0 or n=k
C(n-1,k)+C(n-1,k-1),
Top-down approach:
otherwise
Efficiency:
Problem size: (n,k)
Dominant operation: addition
comb(n,k)
T(n,k) >= 2 min{k,n-k}
IF n<k THEN RETURN 0
T(n,k)  Ω(2 min{k,n-k} )
ELSE IF (k=0) OR (n=k) THEN
RETURN 1
Rmk: min{k,n-k} is the length of
ELSE
the shortest branch in the
RETURN comb(n-1,k)+comb(n-1,k-1)
tree of recursive calls
ENDIF
Algorithmics - Lecture 11
11
ENDIF
Developing recurrence relations
Example 2. Computing a binomial coefficient C(n,k)
0,
C(n,k)=
if n<k
1, if k=0 or n=k
C(n-1,k)+C(n-1,k-1),
otherwise
Bottom-up approach: constructing the Pascal’s triangle
0 1 2
… k-1
k
0
1
1
1 1
2
1 2 1
…
k
1
…
1
…
n-1 1
C(n-1,k-1)
C(n-1,k)
n
1
C(n,k)
Algorithmics - Lecture 11
12
Developing recurrence relations
Algorithm (Pascal triangle):
Comb(n,k)
C[0..n,0..k] ← 0
FOR i ← 0,n DO
FOR j ← 0,min{i,k} DO
IF (j=0) OR (j=i) THEN
C[i,j] ← 1
ELSE
C[i,j] ← C[i-1,j]+C[i-1,j-1]
ENDIF
ENDFOR
ENDFOR
RETURN C[0..n,0..k]
Efficiency:
Problem size: (n,k)
Dominant operation: addition
T(n,k)=(1+2+…+k-1) +(k+…+k)
=k(k-1)/2+k(n-k+1)
T(n,k)(nk)
Remark. If only C(n,k) have to be computed it suffices to use an
array of k elements as extra space
Algorithmics - Lecture 11
13
Outline
•
What is dynamic programming ?
•
Basic steps in applying dynamic programming
•
Bottom-up vs. top-down approaches in developing
recurrence relations
•
Applications of dynamic programming
Algorithmics - Lecture 11
14
Applications of dynamic programming
Application 1: Longest strictly increasing subsequence
Let a1,a2,…,an be a sequence. Find the longest subsequence satisfying:
aj1<aj2<…<ajk (a strictly increasing subsequence; 1<=j1<j2…<jk<=n)
k is maximal (longest subsequence)
Rmk: the elements in the subsequence do not have to be consecutive
elements in the initial sequence
Example:
a = (2,5,1,3,6,8,2,10,4)
Strictly increasing subsequences of length 5 (maximal length):
(2,5,6,8,10)
(2,3,6,8,10)
(1,3,6,8,10)
Algorithmics - Lecture 11
15
Longest strictly increasing subsequence
1.
Analysis of the solution structure.
Let s=(aj1, aj2,…,aj(k-1) ,ajk ) be an optimal solution. Then it doesn’t exist
an element in a[1..n] after ajk which is greater than ajk . Moreover it
doesn’t exist an element which is placed between the last two
elements of s (otherwise s wouldn’t be an optimal one). We have
to prove that s’=(aj1, aj2,…,aj(k-1) ) is an optimal solution for the
subproblem of finding the longest strictly increasing subsequence
which ends in aj(k-1) . If we suppose that s’ is not optimal then it
would exist an optimal s”. By adding to s” the element ajk one
would obtain a better solution than s, implying that s is not optimal.
This is in contradiction with the initial hypothesis, thus s’ should be
optimal meaning that the problem has the property of optimal
substructure.
Algorithmics - Lecture 11
16
Longest strictly increasing subsequence
1.
Analysis of the solution structure.
Thus one can consider that we have to solve a generic problem:
P(i): find the longest strictly increasing subsequence of a[1..i] having
a[i] the last element
The optimal solution s(i) of P(i) contains the optimal solution s(j) of a
subproblem P(j) (with j<i and a[j]<a[i]). The element a[j] is chosen
such that s(j) has the largest possible length.
Algorithmics - Lecture 11
17
Longest strictly increasing subsequence
2.
Find a recurrence relation
Let Bi be the length of the longest strictly increasing subsequence which
ends up in ai
1
if i=1
Bi =
1+ max{Bj | 1<=j<=i-1, aj<ai}
Example:
a = (2,5,1,3,6,8,2,10,4)
B = (1,2,1,2,3,4,2,5,3)
Rmk: If the set {j| 1<=j<=i-1, aj<ai} is empty then the maximum is
considered to be 0.
Algorithmics - Lecture 11
18
Longest strictly increasing subsequence
3. Develop the recurrence relation
1
if i=1
Bi =
1+max{Bj | 1<=j<=i-1, aj<ai}
Complexity: θ(n2)
computeB(a[1..n])
B[1] ← 1
FOR i ← 2,n DO
max ← 0
FOR j ← 1,i-1 DO
IF a[j]<a[i] AND max<B[j]
THEN max ← B[j]
ENDIF
ENDFOR
B[i] ← max+1
ENDFOR
RETURN B[1..n]
Algorithmics - Lecture 11
19
Longest strictly increasing subsequence
4.
Construction of the
solution
Find the maximum of B
Construct s successively
starting with the last
element
Complexity: θ(n)
construct(a[1..n],B[1..n])
m:=1 // m will be the index of the maximum
//of B[1..n]
FOR i ← 2,n DO
IF B[i]>B[m] THEN m ← i ENDIF
ENDFOR
k ← B[m] // length of the optimal solution
s[k] ← a[m] // last element of the solution
WHILE B[m]>1 DO
i ← m-1 // search for the previous element
WHILE a[i]>=a[m] OR B[i]<>B[m]-1 DO
i ← i-1
ENDWHILE
m ← i; k ← k-1; s[k] ← a[m]
ENDWHILE
RETURN s[1..k]
Algorithmics - Lecture 11
20
Longest strictly increasing subsequence
Remark: the construction of the
solution can be simplified if the
index of the previous element of
the subsequence is saved when
B[1..n] is constructed
construct(a[1..n],B[1..n])
m:=1
FOR i ← 2,n DO
IF B[i]>B[m] THEN m ← i ENDIF
ENDFOR
k ← B[m]
s[k] ← a[m]; i ← P[m]
WHILE i>=1 DO
k ← k-1;
s[k] ← a[i]
i ← P[i]
ENDWHILE
RETURN s[1..k]
computeB(a[1..n])
B[1] ← 1; P[1..n] ←0
FOR i ← 2,n DO
max ← 0
FOR j ← 1,i-1 DO
IF a[j]<a[i] AND max<B[j]
THEN max ← B[j]; P[i] ←j
ENDIF
ENDFOR
B[i] ← max+1
ENDFOR
Algorithmics - Lecture 11
RETURN B[1..n]
21
Applications of dynamic programming
Application 2: Longest common subsequence of two sequences
Given two sequences a1,…, an and b1,…,bm find a subsequence
c1,…ck which satisfies:
•
There exists i1,…,ik and j1,…,jk such that
c1=ai1=bj1, c2=ai2=bj2, … , ck=aik=bjk
•
k is maximal
Remark: This problem occurs frequently in bioinformatics where
sequences of nucleotides (DNA) or amino acids (proteins) are
compared in order to check if they are similar or not. Two
sequences are considered similar if they contain a long
common subsequence.
Algorithmics - Lecture 11
22
Longest common subsequence
Example:
a: 2 1 4 3 2
b: 1 3 4 2
Common subsequences:
1, 3
1, 2
4, 2
1, 3, 2
1, 4, 2
Rmk: the subsequence
does not necessarily
contain consecutive
elements
Variant of this problem: find the longest
common sequence of consecutive
elements
Example:
a: 2 1 3 4 5
b: 1 3 4 2
Common subsequences:
1, 3
3, 4
1, 3, 4
Algorithmics - Lecture 11
23
Longest common subsequence
1.
Analyzing the structure of an optimal solution
Let us consider that the optimal solution ends in a[i] = b[j]. Then the
problem consists of finding the longest common subsequence of
partial sequences a[1..i] and b[1..j].
Thus we can consider the generic problem P(i,j) as the problem of
finding the longest common subsequence of the sequences a[1..i]
and b[1..j] (even if a[i] is not equal to b[j]).
The subproblems which should be solved are:
P(i-1,j-1)
(if a[i]=b[j])
or P(i-1,j) or P(i,j-1) (if a[i]<>b[j])
Algorithmics - Lecture 11
24
Longest common subsequence
2.
L[i,j]=
Let L(i,j) be the length of the optimal solution of P(i,j)
0
1+L[i-1,j-1]
max{L[i-1,j],L[i,j-1]}
if i=0 or j=0
if a[i]=b[j]
otherwise
Algorithmics - Lecture 11
25
Longest common subsequence
Example:
a: 2 1 4 3 2
b: 1 3 4 2
Development of the recurrence
relation:
0
if i=0 or j=0
L[i,j]= 1+L[i-1,j-1]
if a[i]=b[j]
max{L[i-1,j],L[i,j-1]} otherwise
0
21
12
43
34
25
Algorithmics - Lecture 11
0
0
0
0
0
0
0
0
1
1
0
0
1
1
1
1
3
2
0
0
1
1
2
2
4
2
3 4
0
0
0
1
1
1
2 2
2
2
2
3
26
Longest common subsequence
Recurrence relation:
compute(a[1..n],b[1..m])
FOR i←0,n DO L[i,0] ← 0 ENDFOR
FOR j← 1,m DO L[0,j]← 0 ENDFOR
0
if i=0 or j=0
FOR i ← 1,n DO
L[i,j]= 1+L[i-1,j-1]
if a[i]=b[j]
FOR j ← 1,m DO
max{L[i-1,j],L[i,j-1]} otherwise
IF a[i]=b[j]
THEN L[i,j] ← L[i-1,j-1]+1
ELSE
L[i,j] ← max{L[i-1,j],L[i,j-1]}
ENDIF
ENDFOR ENDFOR
RETURN L[0..n,0..m]
Algorithmics - Lecture 11
27
Longest common subsequence
Construction of the solution:
-
-
Start from the L[m,n]
If the corresponding elements in
the sequences are equal then
include the common value in the
solution and go up-left (to the
element on the previous row and
previous column)
If the elements are different then
go up or left (to the largest
value)
0
21
12
43
34
25
Algorithmics - Lecture 11
0
0
0
0
0
0
0
0
1
1
0
0
1
1
1
1
3
4
2
2 3 4
0
0
0
0
0
1
1
1
1
1
2 2
2
2
2
2
2
3
28
Longest common subsequence
Construction of the solution
in a recursive manner:
construct(i,j)
IF i>=1 AND j>=1 THEN
IF a[i]=b[j]
THEN
construct(i-1,j-1)
k ← k+1
c[k] ← a[i]
ELSE
IF L[i-1,j]>L[i,j-1]
THEN construct (i-1,j)
ELSE construct (i,j-1)
ENDIF ENDIF ENDIF
Remarks:
•
•
•
a, b, c and k are global
arrays
Before the call of the
function, the variable k
should be initialized with 0
The initial call of the
function
construct(n,m)
Algorithmics - Lecture 11
29
Next lecture will be on …
…other applications of dynamic programming
… and memoization (memory functions)
Algorithmics - Lecture 11
30