Announcements
• Midterms are marked
• Assignment 2:
– Still analyzing
COSC 3101N
J. Elder
Loop Invariants (Revisited)
• Question 5(a): Design an iterative
algorithm for Parity:
p=0
for i = 1 to n do
p = xor(p,s[i])
Loop Invariant?
return p
LI: After i iterations are performed, p=Parity(s[1…i])
COSC 3101N
J. Elder
What is a loop invariant?
• An assertion about the state of one or more
variables used in the loop.
• When the exit condition is met, the LI leads
naturally to the postcondition (the goal of the
algorithm).
• Thus the LI must be a statement about the
variables that store the results of the computation.
COSC 3101N
J. Elder
Know What an LI Is Not
``the LI is NOT...''
COSC 3101N
–
code
–
The steps taken by the algorithm
–
A statement about the range of values
assumed by the loop index.
J. Elder
Dynamic Programming:
Recurrence
COSC 3101N
J. Elder
Dynamic programming
• Step 1: Describe an array of values you
want to compute.
• Step 2: Give a recurrence for computing
later values from earlier (bottom-up).
• Step 3: Give a high-level program.
• Step 4: Show how to use values in the array
to compute an optimal solution.
COSC 3101N
J. Elder
Example 1. Rock climbing
4
5
3
2
At every step our climber can reach exactly three
handholds: above, above and to the right and above
and to the left.
There is a table of “danger ratings” provided. The
“Danger” of a path is the sum of danger ratings of all
handholds on the path.
COSC 3101N
J. Elder
For every handhold, there is only
one “path” rating. Once we have
reached a hold, we don’t need to
know how we got there to move
to the next level.
This is called an “optimal substructure”
property. Once we know optimal solutions to
subproblems, we can compute an optimal
solution to the problem itself.
COSC 3101N
J. Elder
Step 2. Define a Recurrence
Let C(i,j) represent the danger of hold (i,j)
Let A(i,j) represent the cumulative danger of the safest path
from the bottom to hold (i,j)
Then
A(i,j) = C(i,j)+min{A(i-1,j-1),A(i-1,j),A(i-1,j+1)}
i.e., the safest path to hold (i,j) subsumes the safest path to
holds at level i-1 from which hold (i,j) can be reached.
COSC 3101N
J. Elder
Example 2. Activity Scheduling with Profits
g2  10
g1  1
COSC 3101N
g3  1
J. Elder
Step 2. Provide a Recurrent Solution
1. Sort activities according to finishing time: f1  f 2 
 f n (O(n log n))
2. i {1,..., n}, compute H (i)  max{l {1, 2,..., i  1} | fl  si } (O(n log n))
i.e. H (i ) is the last event that ends before event i starts.
A(0)  0
A(i )  max{ A(i  1), gi  A( H (i))}, i {1,..., n}
Decide not
to schedule
activity i
COSC 3101N
Profit from
scheduling
activity i
Optimal profit from
scheduling activities that
end before activity i begins
J. Elder
Example 3: Scheduling Jobs with
Deadlines, Profits and Durations
Input: information (di , ti , gi ) about n activities, where
di  integer deadline for job i
ti  integer duration of job i
gi  real-valued profit of job i
A schedule C is a sequence C  {C (1), C (2),..., C (n)} such that:
C (i )  scheduled start time of job i
C (i )  1 if job i is not scheduled
A schedule C is feasible if each scheduled job finishes by its deadline
and no two scheduled jobs overlapped.
Output: A feasible schedule C with maximum profit: P(C )   gi
iC
COSC 3101N
J. Elder
Dynamic Programming Solution
Precomputation:
Sort activities according to deadline: d1  d2 
COSC 3101N
 dn (O(n log n))
J. Elder
Step 2. Provide a Recurrent Solution
A(0, t )  0t  {0,..., d }
For i  {1,..., n}, t {0,..., d }, define t   min{t , di }  ti
(t  is the latest time that we can schedule job i so that it
ends both by its deadline and by time t.)
Then:
t   0  A(i, t )  A(i  1, t )
t   0  A(i, t )  max{ A(i  1, t ), gi  A(i  1, t )}
Decide not to
schedule job i
COSC 3101N
Profit from job i
Profit from scheduling activities that
end before job i begins
J. Elder
Step 2. (cntd…) Proving the Recurrent Solution
For i {1,..., n}, t {0,..., d}, define t   min{t , di }  ti
(t  is the latest time that we can schedule job i so that it
ends both by its deadline and by time t.)
Then:
t   0  A(i, t )  A(i  1, t )
t   0  A(i, t )  max{ A(i  1, t ), gi  A(i  1, t )}
We effectively schedule job i at the latest possible time.
This leaves the largest and earliest contiguous block of time for
scheduling jobs with earlier deadlines.
COSC 3101N
J. Elder
ti
event i
Case 1
d1
d2
…
di 1
COSC 3101N
di
t
ti
Case 2
d1
t
event i
d2
…
di 1
t
t di
J. Elder
Example 3. Longest Common Subsequence
COSC 3101N
J. Elder
Optimal Substructure
• Input: 2 sequences, X = x1, . . . , xm and Y = y1, . .
. , y n.
COSC 3101N
J. Elder
Proof of Optimal Substructure Part 1
Notation:
X  {x1 ,..., xm }; Y  { y1 ,..., yn }
X i  {x1 ,..., xi }; Yi  { y1 ,..., yi }
Z  {z1 ,..., zk } is an LCS of X and Y .
Theorem :
1. If xm  yn , then zk  xm  yn and Z k 1 is an LCS of X m1 and Yn 1.
Proof that zk  xm  yn :
Suppose zk  xm . Then could append xm  yn to Z  common subsequence
of X and Y of length k  1  contradiction.
Proof that Z k 1 is an LCS of X m1 and Yn1 :
Z k 1 is a length-(k  1) common subsequence of X m1 and Yn1.
Suppose there is a longer common subsequence W of X m1 and Yn1.
Appending xm  yn to W  common subsequence of length >k  contradiction
COSC 3101N
J. Elder
Proof of Optimal Substructure Part 2
Notation:
X  {x1 ,..., xm }; Y  { y1 ,..., yn }
X i  {x1 ,..., xi }; Yi  { y1 ,..., yi }
Z  {z1 ,..., zk } is an LCS of X and Y .
Theorem :
2. If xm  yn and zk  xm , then Z is an LCS of X m 1 and Y .
Proof :
zk  xm  Z is a common subsequence of X m1 and Y .
Suppose there is a common subsequence W of X m1 and Y
of length  k  W also a common subsequence of X and Y  contradiction
COSC 3101N
J. Elder
Proof of Optimal Substructure Part 3
Notation:
X  {x1 ,..., xm }; Y  { y1 ,..., yn }
X i  {x1 ,..., xi }; Yi  { y1 ,..., yi }
Z  {z1 ,..., zk } is an LCS of X and Y .
Theorem :
2. If xm  yn and zk  yn , then Z is an LCS of X and Yn 1.
Proof :
As for Part 2.
COSC 3101N
J. Elder
Step 2. Provide a Recurrent Solution
Input sequences are empty
Last elements match:
must be part of LCS
COSC 3101N
Last elements don’t match: at most
one of them is part of LCS
J. Elder
Example 6: Longest Increasing Subsequence
• Input: 1 sequence, X = x1, . . . , xn.
• Output: the longest increasing subsequence of X.
• Note: A subsequence doesn’t have to be
consecutive, but it has to be in order.
COSC 3101N
J. Elder
Step 1. Define an array of values to compute
i {0,..., n}, A(i)  length of LIS of X ending in xi .
Ultimately, we are interested in max{A(i ) |1  i  n}.
COSC 3101N
J. Elder
Step 2. Provide a Recurrent Solution
for 1  i  n,
A(i )  1  max{ A( j ) |1  j  i and x j  xi }
COSC 3101N
J. Elder
Step 3. Provide an Algorithm
function A=LIS(X)
for i=1:length(X)
Running time? O(n2)
m=0;
for j=1:i-1
if X(j) < X(i) & A(j) > m
m=A(j);
end
end
A(i)=m+1;
end
COSC 3101N
J. Elder
Step 4. Compute Optimal Solution
function lis=printLIS(X, A)
[m,mi]=max(A);
lis=printLISm(X,A,mi,'LIS: ');
lis=[lis,sprintf('%d', X(mi))];
Running time? O(n)
function lis=printLISm(X, A, mi, lis)
if A(mi) > 1
i=mi-1;
while ~(X(i) < X(mi) & A(i) == A(mi)-1)
i=i-1;
end
lis=printLISm(X, A, i, lis);
lis=[lis, sprintf('%d ', X(i))];
end
COSC 3101N
J. Elder
LIS Example
X = 96
24
61
49
90
77
46
2
83
45
A= 1
1
2
2
3
3
2
1
4
2
> printLIS(X,A)
> LIS: 24 49 77 83
COSC 3101N
J. Elder
Example 6: Optimal Binary Search Trees
Input:
Sequence K  {k1 ,..., kn } of n distinct keys, k1  k2 
 kn
pi  probability that a search is for key ki
Output:
BST with minimum expected search cost
COSC 3101N
J. Elder
Expected Search Cost
Cost = # of items examined.
For key ki , cost = depthT (ki )  1
E[search cost in T ]
n
  (depth T (ki )  1)  pi
i 1
n
 1   depthT (ki )  pi
i 1
COSC 3101N
Which BST
is more efficient?
J. Elder
Observations
COSC 3101N
J. Elder
Optimal Substructure
COSC 3101N
J. Elder
Optimal Substructure (cntd…)
Let eT  expected search cost in subtree T , starting from root(T )
j
T
  pl  depth T  (kl )
l i
Let eT  expected search cost in subtree T , starting from root(T )
j
  pl  (depthT (root(T ))  depth T  ( kl ))
T
l i
j
 depth T (root(T )) pl 
l i
j
 p  depth
l i
l
T
( kl )
j
 depth T (root(T )) pl  eT
l i
COSC 3101N
J. Elder
Recursive Solution
COSC 3101N
J. Elder
Recursive Solution (cntd…)
COSC 3101N
J. Elder
Step 2. Provide a Recurrent Solution
Expected cost of search
for left subtree
COSC 3101N
Expected cost of search
for right subtree
Added cost when subtrees
embedded under root
J. Elder
Step 3. Provide an Algorithm
n)
Running time? O(n3)
work on subtrees of increasing size l
COSC 3101N
J. Elder
Example
COSC 3101N
J. Elder
Example (cntd…)
COSC 3101N
J. Elder
Step 4. Compute Optimal Solution
Running time? O(n)
COSC 3101N
J. Elder
Elements of Dynamic Programming
• Optimal substructure:
– an optimal solution to the problem contains
within it optimal solutions to subproblems.
COSC 3101N
J. Elder
Elements of Dynamic Programming
• Cut and paste: prove optimal substructure by contradiction:
– assume an optimal solution to a problem with suboptimal solution to
subproblem
– cut out the suboptimal solution to the subproblem.
– paste in the optimal solution to the subproblem.
– show that this results in a better solution to the original problem.
– This contradicts our assertion that our original solution is optimal.
COSC 3101N
J. Elder
Elements of Dynamic Programming
• Dynamic programming uses optimal
substructure from the bottom up:
– First find optimal solutions to subproblems
– Then choose which to use in optimal solution to
problem.
COSC 3101N
J. Elder
Section V. Graph Algorithms
COSC 3101N
J. Elder
Directed and Undirected Graphs
(a) A directed graph G = (V, E), where V = {1,2,3,4,5,6} and
E = {(1,2), (2,2), (2,4), (2,5), (4,1), (4,5), (5,4), (6,3)}.
The edge (2,2) is a self-loop.
(b) An undirected graph G = (V,E), where V = {1,2,3,4,5,6} and
E = {(1,2), (1,5), (2,5), (3,6)}. The vertex 4 is isolated.
(c) The subgraph of the graph in part (a) induced by the vertex set
{1,2,3,6}.
COSC 3101N
J. Elder
Graph Isomorphism
COSC 3101N
J. Elder
Trees
COSC 3101N
J. Elder
Representations: Undirected Graphs
Adjacency List
COSC 3101N
Adjacency Matrix
J. Elder
Representations: Directed Graphs
Adjacency List
COSC 3101N
Adjacency Matrix
J. Elder