SEEM 3470: Dynamic Optimization and Applications
2013–14 Second Term
Handout 6: More Examples on DP and Shortest Path Problem
Instructor: Shiqian Ma
February 17, 2014
Suggested Reading: Bertsekas’ Lecture Slides on Dynamic Programming; Section 2.3 of Chapter
2 of Bertsekas, Dynamic Programming and Optimal Control: Volume I (3rd Edition), Athena
Scientific, 2005.
1
The Three-Jug Puzzle
An 8-gallon jug is filled with fluid. Given two empty 5- and 3-gallon jugs, we want to derive the 8
gallons of fluid into two equal parts using the three jugs. No other measuring devices are allowed.
What is the smallest number of pourings needed to achieve this result?
The solution process can be systemarized by representing the problem as a shortest path problem. A node is defined to represent the amount of fluid in the 8-, 5-, and 3-gallon jugs, respectively.
This means that the network starts with node (8,0,0) and terminates with the desired solution node
(4,4,0). A new node is generated from the current node by pouring fluid from one jug into another.
Figure 1 shows different routs that lead from start node (8,0,0) to end node (4,4,0). The arc
between two successive nodes represents a single pouring, and hence can be assumed to have a
length of 1 unit. The problem reduces to determining the shortest path between node (8,0,0) to
node (4,4,0). The optimal solution, given by the bottom path in Figure 1, requires 7 pourings.
Figure 1: Three-jug puzzle as a shortest path problem
2
0/1 Knapsack Problem
We want to select items from a collection of n items to put into a backpack. The total capacity
of the backpack is W . The volumes of the n items are w1 , . . . , wn , and their values are v1 , . . . , vn .
Decide which items should be put into the backpack to maximize the total value.
1
A binary integer programming formulation is:
max
s.t.
n
X
i=1
n
X
vi xi
wi x i ≤ W
i=1
xi = {0, 1}, ∀i = 1, . . . , n.
Note that there are 2n possible solutions to be investigated.
A DP formulation:
• Stage i is represented by item i
• The state at stage i is the volume limit assigned to stages (items) i, i + 1, . . . , n.
Define m[i, w] recursively as

 m[i, w] = m[i − 1, w],
if wi > w
(the new item is more than the current volume limit)

m[i, w] = max{m[i − 1, w], m[i − 1, w − wi ] + vi }, if wi ≤ w.
Note that m[i, w] is the optimal value using items 1, . . . , i, when the volume limit is w. The terminal
value is m[0, w] = 0 for 0 ≤ w ≤ W , which corresponds to the situation that no item can be selected.
The solution we want to compute is m[n, W ]. To do this efficiently we can use a table to store
previous computations.
Example: Let W = 10, and
i
vi
wi
1
10
5
2
40
4
3
30
6
4
50
3
Then table based on the DP recursive formula is
V [i, w] 0 1 2 3
4
5
6
i=0 0 0 0 0
0
0
0
1
0 0 0 0
0 10 10
2
0 0 0 0 40 40 40
3
0 0 0 0 40 40 40
4
0 0 0 50 50 50 50
Clearly, total complexity is O(nW ).
A shortest path formulation. Find the
3
7
0
10
40
40
90
8
0
10
40
40
90
9
0
10
50
50
90
10
0
10
50
70
90
longest path in Figure 2.
Tower of Hanoi
(Resource: http : //en.wikipedia.org/wiki/T ower of Hanoi)
This puzzle of “Tower of Hanoi” consists of three rods, and a number of disks of different sizes
which can slide onto any rod. The puzzle starts with the disks in a neat stack in ascending order of
size on one rod, the smallest at the top, thus making a conical shape. The objective of the puzzle
is to move the entire stack to another rod, obeying the following simple rules:
2
Figure 2: Knapsack problem as a shortest path problem
• Only one disk can be moved at a time.
• Each move consists of taking the upper disk from one of the stacks and placing it on top of
another stack i.e. a disk can only be moved if it is the uppermost disk on a stack.
• No disk may be placed on top of a smaller disk.
A key to solving this puzzle is to recognize that it can be solved by breaking the problem down
into a collection of smaller problems and further breaking those problems down into even smaller
problems until a solution is reached. For example:
• label the rods A, B, C.
• let n be the total number of disks
• number the discs from 1 (smallest, topmost) to n (largest, bottommost)
3
Figure 3: The Tower of Hanoi Puzzle
To move n disks from rod A to rod C:
• move n − 1 disks from A to B. This leaves disk n alone on rod A
• move disk n from A to C
• move n − 1 disks from B to C so they sit on disk n
The above is a recursive algorithm, to carry out steps 1 and 3, apply the same algorithm again for
n − 1.
Let S(n) be the minimum number of moves. The DP recursive formula is
S(n) = 2S(n − 1) + 1.
It can be shown that S(n) = 2n − 1, an exponential number!
Figure 4: The Tower of Hanoi as a shortest path problem: 1 disk
4
Figure 5: The Tower of Hanoi as a shortest path problem: 2 disks
Figure 6: The Tower of Hanoi as a shortest path problem: 3 disks
5