UMass Lowell Computer Science 91.503
Analysis of Algorithms
Prof. Karen Daniels
Fall, 2006
Lecture 2
Monday, 9/13/06
Design Patterns for Optimization Problems
Greedy Algorithms
Algorithmic Paradigm Context
Divide &
Conquer
View problem as collection of
subproblems
“Recursive” nature
Independent subproblems
Number of subproblems
Preprocessing
Characteristic running time
Primarily for optimization
problems
Optimal substructure:
optimal solution to problem
contains within it optimal
solutions to subproblems
Greedy choice property:
locally optimal produces
globally optimal
Heuristic version useful for
bounding optimal value
Subproblem solution order
Dynamic
Programming
overlapping
depends on
partitioning
factors
typically log
function of n
Greedy
Algorithm
typically
sequential
dependence
typically small
depends on number
and difficulty of
subproblems
typically sort
often dominated
by nlogn sort
Solve subproblem(s),
then make choice
Make choice, then solve
subproblem(s)
Greedy Algorithms
What is a Greedy Algorithm?


Solves an optimization problem
Optimal Substructure:


optimal solution contains in it optimal solutions to
subproblems
Greedy Strategy:



At each decision point, do what looks best “locally”
Choice does not depend on evaluating potential future choices
or presolving overlapping subproblems
Top-down algorithmic structure


With each step, reduce problem to a smaller problem
Greedy Choice Property:

“locally best” = globally best
Greedy Strategy Approach
1.
2.
3.
4.
5.
6.
Determine the optimal substructure of the
problem.
Develop a recursive solution.
Prove that, at any stage of the recursion, one of
the optimal choices is the greedy choice.
Show that all but one of the subproblems caused
by making the greedy choice are empty.
Develop a recursive greedy algorithm.
Convert it to an iterative algorithm.
source: 91.503 textbook Cormen, et al.
Examples
Activity Selection
 Minimum Spanning Tree
 Dijkstra Shortest Path
 Huffman Codes
 Fractional Knapsack

Activity Selection
Example Optimization Problem:
Activity Selection

Problem Instance:
 Set
S = {1,2,...,n} of n activities
 Each activity i has:
start time: si
 finish time : fi

si  f i
 Activities
i, j are compatible iff non-overlapping:
[s j  f j )
[ si  f i )
 Objective:

select a maximum-sized set of mutually compatible activities
source: 91.503 textbook Cormen, et al.
source: 91.503 textbook Cormen, et al.
Algorithmic Progression

“Brute-Force”


Dynamic Programming #1



Exponential number of subproblems
(board work)
Dynamic Programming #2


(board work)
Quadratic number of subproblems
Greedy Algorithm
Recursive Activity Selection
High-level call: RECURSIVE-ACTIVITY-SELECTOR(s,f,0,n)
Returns an optimal solution for S i , n 1
n
mn
n
mn
Errors from earlier printing are corrected in red.
source: web site accompanying 91.503 textbook Cormen, et al.
source: web site accompanying 91.503 textbook Cormen, et al.
Greedy Algorithm
source: 91.503 textbook Cormen, et al.

Algorithm:

S’ = presort activities in S by nondecreasing finish time


and renumber
GREEDY-ACTIVITY-SELECTOR(S’)




n
A
j
for i
length[S’]
{1}
1
2 to n




do if si  f j
then A  A  {i}
j
i
return A
Running time?
Streamlined Greedy Strategy
Approach
1.
2.
3.
View optimization problem as one in
which making choice leaves one
subproblem to solve.
Prove there always exists an optimal
solution that makes the greedy choice.
Show that greedy choice + optimal
solution to subproblem optimal
solution to problem.
Greedy Choice Property: “locally best” = globally best
source: 91.503 textbook Cormen, et al.
Minimum Spanning Tree
Minimum Spanning Tree
Time:
O(|E|lg|E|)
given fast
FIND-SET,
UNION
Invariant: Minimum weight
spanning forest
Produces minimum weight tree of
edges that includes every vertex.
Becomes single
tree at end
Time:
O(|E|lg|V|)
=
O(|E|lg|E|)
slightly
faster with
fast priority
queue
2
4
A
3
1
Spans all
vertices at end
G
5
6
E
Invariant: Minimum
weight tree
6
D
B
8
2
1
7
F
4
C
for Undirected, Connected,
Weighted Graph
G=(V,E)
source: 91.503 textbook Cormen et al.
Dijkstra Shortest Path
Single Source Shortest Paths:
Dijkstra’s Algorithm
for (nonnegative) weighted, directed graph G=(V,E)
2
4
A
3
1
G
5
6
D
B
6
E
8
2
1
7
F
4
C
source: 91.503 textbook Cormen et al.
Huffman Codes
Huffman Codes
source: 91.503 textbook Cormen, et al.
source: web site accompanying 91.503 textbook Cormen, et al.
source: web site accompanying 91.503 textbook Cormen, et al.
source: web site accompanying 91.503 textbook Cormen, et al.
source: web site accompanying 91.503 textbook Cormen, et al.
source: web site accompanying 91.503 textbook Cormen, et al.
Fractional Knapsack
Knapsack
Each item has value and weight.
Goal: maximize total value of items chosen,
subject to weight limit.
fractional: can take part of an item
0-1: take all or none of an item
50
30
20
10
item1 item2
Value:
$60
$100
item3
$120
“knapsack”
source: web site accompanying 91.503 textbook Cormen, et al.
Additional Examples
On course web site under Miscellaneous Docs
 Patriotic Tree
 404

review handout
Tree Vertex Cover
 91.503
midterm
Greedy Heuristic
If optimization problem does not have
“greedy choice property”, greedy approach
may still be useful as a heuristic in
bounding the optimal solution
 Example: minimization problem

Upper Bound (heuristic)
Solution
Values
Optimal (unknown value)
Lower Bound
Homework
HW# Assigned
Due
1
W 9/6
W 9/13
Content
91.404 review & Chapter 15
2
Chapter 16
W 9/13
W 9/20