Chapter 10
Dynamic Programming
Agenda for This Week
• Dynamic Programming
–
–
–
–
–
Definition
Recursive Nature of Computations in DP
Forward and Backward Recursion
Basic Elements of DP
Applications of DP
2
Dynamic Programming
• An approach to problem solving that
decomposes a large problem into a number
of smaller problems for finding optimal
solution
• Used for problems that are in multistage in
nature
• No standard mathematical formulation
3
Stagecoach Problem
• Used to illustrate the features of DP
• Concerns a mythical fortune seeker to go to west to
join the gold rush in California during mid-19th
century
• A prudent man who was quite concerned about his
safety
• Could not go through some territories
• Starting and final destination points are fixed
• Had considerable choice as to which states to travel
4
Cost Estimates
2
1
3
4
3
5
6
7
2
10
6
-
3
9
8
4
6
4
4
5
2
10
6
4
1
3
3
8
9
5
5
9
6
6
4
8
7
7
7
5
5
5
9
7
8
6
9
4
6
4
6
4
6
4
8
9
8 6
5
10
8
7
7
9
10
4
5
5
Solving Problem
• Select routes that minimizes total cost
• One possible approach is to use trail and error method
• DP starts with a small portion of the original problem and finds the optimal
solution
• Optimizes the nest stage using the solution from preceding one
• Continues until all stages are solved
• First decompose it into stages as delineated by the vertical dashed lines
2
10
6
4
1
3
3
5
5
9
6
4
6
4
7
6
4
8 6
5
8
9
8
7
7
9
10
4
5
6
Recursive Nature of Computations in DP
• Start with the smaller problem where he has completed his journey
and has only one to go
• Computations are done recursively
• Solution of one subproblem is used as an input for next
subproblem
2
10
6
4
1
3
3
5
5
9
6
4
6
4
7
6
4
8 6
5
8
9
8
7
7
9
10
4
5
7
DP Concept
• Stages are formed by decomposing the original problem
into a number of subproblems
Input
Decision Problem
Decision Criteria
Output
• Shortest path context
–
–
–
–
Input— any node or state
Decision Problem— which arc?
Decision criteria—shortest distance
Output– any node or state
8
DP Notation
• Xn=input to stage n (output from stage n+1)
• dn=decision variable at stage n (state variable)
XN
dN
X
X
Stage N N-1 n
rN(xN,dN)
dn
Stage n
Xn-1
rn(xn,dn)
X1
d1
Stage 1
X0
r1(x1,d1)
• Stage transformation function
• Return function—return of a stage
9
Stage one
• First stage begins with nodes that are exactly one arc away from
final destination (CA)
• Nodes 8 and 9 are the input nodes or states for stage 1
• Nodes 10 is the output nodes for stage 1
2
10
6
4
1
5
3
3
8
9
7
6
4
6
4
6
4
8
9
6
5
5
8
7
7
9
10
4
5
10
Decomposing the Problem-Stage two
• Second stage begins with all nodes that are exactly two arcs away
from destination and ends with all nodes that are exactly one arc
away
• Nodes 5, 6, and 7 are the input nodes for stage 2
• Nodes 8 and 9 are the output nodes for stage 2
2
10
6
4
1
5
3
3
8
9
9
6
4
6
4
7
8
6
4
6
5
5
8
7
9
10
4
5
7
11
Decomposing the Problem-Stage three
• Third stage begins with all nodes that are exactly three arcs away
from destination and ends with all nodes that are exactly two arcs
away
• Nodes 2, 3, and 4 are the input nodes for stage 3
• Nodes 5, 6 and 7 are the output nodes for stage 3
2
10
6
4
1
5
3
3
8
9
9
6
4
6
4
7
8
6
4
6
5
5
8
7
9
10
4
5
7
12
Decomposing the Problem-Stage Four
• Forth stage begins with all nodes that are exactly four arcs away
from destination and ends with all nodes that are exactly three
arcs away
• Nodes 1 the input node for stage 4
• Nodes 2, 3 and 4 are the output nodes for stage 3
2
10
4
1
3
5
6
3
8
9
7
8
6
4
6
4
6
4
Stage 4
9
6
5
5
8
7
9
10
4
5
7
Stage 1
13
Solving the Problem-Stage one
Input Arc
8
9
8-10
9-10
Output
10
10
Shortest to 10
6
4
6
8
6
10
4
9
4
14
Solving by DP Notation-Stage 1
• X1=input to stage 1 (output from
stage 2)
• d1=decision variable at stage 1
(state variable)
• Stage transformation function
• Return function—return of a stage
6
8
Xn
10
Stage n
Xn-1
rn(xn,dn)
d1
x1
6
dn
8
9
f1(x1)= r1(x1,d1)
10
d1*
6
4
10
10
4
9
4
15
Solving the Problem- Stage two
11
Input Arc
5
6
7
5-8
6-8
7-9
Output
Shortest
8
8
9
11
10
9
5
10
5
6
8
9
6
4
10
6
8
Min [(5+6),(9+4)]=11
Min [(4+6),(8+4)]=10
Min [(7+6),(5+4)]= 9
9
7
9
7 5
4
4
16
Solving by DP Notation-Stage 2
d2
x2
5
6
7
r2(x2,d2)+ f1(x1)
8
9
11
10
13
13
12
9
f2(x2) d2*
11
10
9
8
8
9
11
5
10
5
6
8
9
6
4
10
6
8
9
9
7 5
4
4
7
17
Solving the Problem- Stage Three
Input Arc
2
3
4
2-6
3-7
4-6
Output
Shortest
6
7
6
16
15
14
2
Min [(10+11),(6+10)]=16
Min [(9+11),(8+10), (6+9)]=15
Min [(11+6),(10+4),(7+9)]= 14
10
11
5
16 6
15 9
3 8
6
10
14 6 4
4
7
9
5
6
8
9
6
4
10
6
8
7
9
7 5
4
4
18
Solving by DP Notation-Stage 3
d3
x3
2
3
4
r3(x3,d3)+ f2(x2)
5
21
20
17
6
16
18
14
7
15
16
f3(x3) d3*
16
15
14
6
7
6
11
2
10
5
16 6
10
15 9
3 8
6
8
9
6
4
10
6
8
6
14 6 4
4
5
9
7
7
9
7 5
4
4
19
Solving the Problem- Stage Four
Input Arc
1
Output
Shortest
3
18
1-3
Min [(4+16),(3+15), (5+14)]=18
16
2
11
10
5
6
4
18
1
3
5
15
3
9
8
6
14 6
4
4
7
10
5
6
8
9
6
4
10
6
8
9
7
9
7 5
4
4
20
Solving by DP Notation-Stage 4
d4
x4
r4(x4,d4)+ f3(x3)
1
f4(x4) d3*
2
3
4
20
18
19
18
3
16
2
11
10
5
6
18
1
4
15
9
8
3
3
5
6
14 6
4
4
10
5
6
8
9
6
4
10
6
8
9
7
7
9
7 5
4
4
21
Best Decision at Each stage
Stage
4
3
2
1
2
10
6
4
1
3
Arc
1-3
3-7
7-9
9-10
3
5
5
9
6
4
6
4
7
6
4
8 6
5
8
9
8
7
7
9
10
4
5
22
Dynamic EOQ Models
• Dynamic EOQ models are different,
because
– Inventory level is reviewed periodically over a
number finite periods
– Demand per period, though deterministic, is
dynamic (MRP situation)
• No shortage is allowed and setup cost is
incurred each time a production lot is
started
23
A Production and Inventory
Control Problem
• Making decision on a production quantity for each of
periods so that the demand can be met at a minimum
cost
–
–
–
–
–
–
–
–
N=no. of periods (stages)
Dn=demand during stage n
Xn= amount of inventory on hand at beginning of stage n
dn= production quantity for stage n
Pn=production capacity in stage n
Wn=storage capacity at the end of stage
Cn=production cost per unit in stage
Hn=holding cost per unit of ending inventory for stage n
24
Example
Month Demand
1
2
3
2
3
3
Capacity
Production Storage
3
2
3
2
3
2
Cost/unit
Production Holding
175
150
200
30
30
40
Beginning inventory for month 1is 1 units
25
DP Formulation
Note: Ending Inventory=
Beginning Inventory
+Production
-Demand
X3=1
X2
Month 1
Month 2
r3(x3,d3)
r2(x2,d2)
X1
Month 3
r1(x1,d1)
X0
26
Knapsack Problem
• N items can be put into a knapsack
• Each time has a certain weight and a value
• Determine how many units of each item to place
in the knapsack to maximize the total value
• Constraint is placed on the maximum weight
• Similar to a manger who must make a biweekly
selection of jobs from each of four categories to
process during a 2-week period
27
Job Data for the Manufacturing Operation
• Select certain jobs
during the next 2 weeks
such that all the jobs
selected can be
processed within 10
working days and the
total value of the jobs is
maximized
Job
Category
No. of
Jobs to be
processed
Estimated
Completion
time per job
Value
rating
per job
1
4
1
2
2
3
3
8
3
2
4
11
4
2
7
20
28
Formulation
• At stage 1, we must decide how many jobs from
category 1 to process
• At stage 2, we must decide how many jobs from
category 2 to process, etc.
• Value rating is a subjective score (scale 1 to 20)
• xn=No. of days of processing time remaining at the
beginning of stage n (state variable)
• dn=No. of jobs processed from category n (decision
variable at stage n)
29