DESIGN AND ANALYSIS OF ALGORITHMS (UNIT-V @ Dynamic Programming)
DAA 5TH UNIT DETAILS
UNIT V:
Dynamic Programming: General method, applications-Matrix chain multiplication, Optimal
binary search trees, 0/1 knapsack problem, All pairs shortest path problem, Travelling sales
person problem, Reliability design.
Important Questions for exam point of view:
1. (a) Find the solution for the knapsack problem. When n=3,
(W1, W2, W3)= (18, 15, 10,) . (P1, P2, P3)= (25, 24, 15) and m=20..
(b) Explain the general concept of Dynamic programming.
2. (a) Find the shortest paths between all pairs of nodes in the following graph
(b)What are the advantages of finding shortest paths and also explain the
application areas..
3. Find the shortest path b/w all pairs of nodes in the following graph and explain with the
suitable algorithm.
4.
(a)Discuss the dynamic programming solution for the problems of reliability
design..
(b) Define merging and purging rules in O/1 knapsack problem.
bphanikrishna.wordpress.com
1
Department of C.S.E
DESIGN AND ANALYSIS OF ALGORITHMS (UNIT-V @ Dynamic Programming)
UNIT-V
Dynamic Programming: Dynamic Programming is an algorithm design method for solving
problems with overlapping sub-problems.
 Each sub-problem solved only once and the result of each sub-problem is stored in a
table (generally implemented as an array or a hash table) for future reference.
 These sub-problems may be used to obtain the original solution and the technique of
storing the sub-problem solution is known as “memoization”
 Dynamic programming is a powerful technique that can be used to solve many
problems in time O(n2) or O(n3) for which a naïve (uninstructed) approach would take
exponential time.
 Dynamic Programming is a general approach to solving problem, much like “divideand –Conquer” method.
Comparison between “Divide & Conquer” & Dynamic Programming:
 Dynamic programming like the divide and conquer method, solves problems by
combining the solutions to sub-problems.
 In DAndC algorithm partition the problem into disjoint sub-problems, solve the subproblems recursively, then combine their solutions to solve the original problem.
In contrast (dissimilarity), dynamic Programming applies when the sub-problems
overlap that is, when sub-problem share the sub-problems.
 DAndC algorithm does more work than necessary repeatedly solving the common
sub-problems
A DP algorithm solves each sub-problem just once and then saves its answer in a
table, thereby avoiding the work re-computing the answer every time it solves each
sub-problem.
 DP typically apply to optimization problems, such problems can have many possible
solutions. Each solution has a value, and we wish to find a solution with the optimal
(min or max) value.
bphanikrishna.wordpress.com
2
Department of C.S.E
DESIGN AND ANALYSIS OF ALGORITHMS (UNIT-V @ Dynamic Programming)
DandC approach
Dynamic Programming approach
Fibonacci number: F1=F2=1;
simple idea:- memoize
fib(n){
memo={}
if n≤2 return 1;
fib{n){
else return fib(n-1)+fib(n-2);
if n in memo: return memo[n];
}
else if n≤2 return f=1;
else f=fib(n-1) + fib(n-2);
memo[n]=f;
return f;
}
Time Complexity:
Time complexity:
T(n)=T(n-1)+O(1)
T(n)=T(n-1)+T(n-2)+O(1)
=O(n)
≥2T(n-2)+)O(1)
≥ 2n/2 Exponential time.
Steps to Develop “Dynamic Programming” Algorithm:
Sequence of steps for Develop Dynamic Programming
Step1:- Characterize the structure of an optimal solution
Step2:- Recursively define the value of an optimal solution
Step3:- Compute the value of an optimal solution, typically in a bottom-up fashion.
Step4:- Construct an optimal solution from computed information.
An optimal sequence of decisions can be found by making the decisions one at a time. This is
true for all problems solved by the greedy method. For many problems it is not possible to
take a step wise decision made is optimal.
Principle Optimality: The principle of optimality states that an optimal sequence of decisions
has the property that whatever the initial state and decision are the remaining decisions must
constitute an optimal decision sequences with regards to the state resulting from the 1st
decision.
bphanikrishna.wordpress.com
3
Department of C.S.E
DESIGN AND ANALYSIS OF ALGORITHMS (UNIT-V @ Dynamic Programming)
Applications:
 Matrix chain multiplication
 Optimal binary search trees
 0/1 knapsack problem
 All pairs shortest path problem
 Travelling sales person problem
 Reliability design
Matrix Chain Multiplication:
Let A be an n×m matrix, B be an m×p matrix Then C=AB is n×p matrix. It is computed in
O(nmp) time using traditional matrix multiplication.
Example
A15×4
A24×6
A36×2
A1.A2.A3= (A1.A2).A3 (or) A1.(A2.A3)
(A1.A2).A3
A1.A2A12=[5×4].[ 4×6] then result is 5×6 (it requires 5×4×6=120 multiplications)
A13=A1.A2.A3=(A1.A2).A3=A12.A3[5×6].[ 6×2] then result is 5×2
(it requires 5×6×2=60 multiplications)
Total=120+60=180 multiplications required for computing (A1.A2).A3
A1.(A2.A3)
A2.A3A23=[4×6].[ 6×2] then result is 4×2 (it requires 4×6×2=48 multiplications)
A13=A1.A2.A3= A1.(A2.A3)=A1.A23[5×4].[ 4×2] then result is 5×2
(it requires 5×4×2=40 multiplications)
Total=48+40=88 multiplications required for computing A1.(A2.A3)
From finally we concluded that, order of operations makes a huge difference.
bphanikrishna.wordpress.com
4
Department of C.S.E
DESIGN AND ANALYSIS OF ALGORITHMS (UNIT-V @ Dynamic Programming)
Input:- n matrices A1, A2,------An of dimensions P1×P2, P2×P3, ------Pn×Pn-1 respectively.
Aim:- To compute the matrix product A1.A2.--------An.
Problem:- In what order should A1.A2.--------An multiplied. So that it would take the
minimum no of computations to derive the product.
Solution:- One way to solve this problem by suing dynamic programming method, for this
we will perform the following steps.
Step1
Let Mij denotes the cost of multiplying. Ai----Aj where the cost is measured in the
number of scalar multiplications.
Here M(i,i)=0 for all i &
M(1,n) is required solution
Step2
The sequence of decisions can be built using the principle of optimality.
Consider the process of matrix chain multiplication. Let ‘T’ be the tree
corresponding to the optimal way of multiplying Ai---Aj. ‘T’ has a left sub-tree L
and right sub-tree R
L correspond to multiplying Ai—Ak &
R correspond to multiplying Ak+1---Aj. For some integers ‘k’ such that i < k ≤ j-1
Thus we get optimal sub-chains of matrix and then multiplications are performed.
Step 3
We will apply formula for computing each sequence
Mij=min{Mi,k+Mk+1,j+Pi Pk+1 Pj+1 i≤ k ≤ j-1 }
Algorithm for Matrix chain Multiplication
Algorithm MatrixChain(d0, d1, ---dn)
// Sequence d0, d1, ---dn of integers
// for i,j=0,1----n-1, the minimum no of multiplications Mij needed to compute the product
Ai.Ai+1.-----Aj. Where AK dk×dk+1 matrix
{
for i:=0 to n-1 do
Mii=0;
for b=1 to n-1 do
for i=0 to n-b-1 do
j=i+b;
mij=+0.0;
for k=i to j-1 do Mij=min{Mi,k+Mk+1,j+di dk+1 dj+1 for all i≤ k ≤ j-1 }
}
bphanikrishna.wordpress.com
5
Department of C.S.E
DESIGN AND ANALYSIS OF ALGORITHMS (UNIT-V @ Dynamic Programming)
Example: Consider A1 5×4, A24×6, A36×2, A42×7
P1=5, P2=4, P3=6, P4=2, P5=7
A1
5×4
A2
A3
A4
4×6
6×2
2×////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
7
bphanikrishna.wordpress.com
6
Department of C.S.E