Lecture 9
Greedy Strategy
Exchange Property
A problem that the greedy strategy w orks for
obtaining optimal solutions usually has the
self - reducibili ty and a certain (simple)
exchange property.
Activity Selection Problem
Mathematical Model
Given a set of intervals [ s1 , f1 ), [ s2 , f 2 ),...,
[ sn , f n ), find a subset of nonoverlap ping
intervals to maximize the cardinalit y.
nonoverlap ping : [ si , f i )  [ s j , f j )  .
Exchange Property
Suppose f1  f 2    f n . In a maximum
solution w ithout interval [ s1 , f1 ), we can
always exchange [ s1 , f1 ) with the 1st interval
in the maximum solution.
Self-Reducible Optimality
Suppose {I1* , I 2* , ..., I k*} is an optimal solution. Then
{I 2* , ..., I k*} is an optimal solution for the
Activity Selection problem on input {I i | I i  I1*  }.
Greedy Algorithm
Sort f1  f 2    f n .
S  ;
for i  1 to n
do if [ si , f i ) does not overlap with any
interval in S
then S  S  {[ si , f i )};
return S .
Optimality Proof
(1) By exchange property, we can assume
that [ s1 , f1 ) is in an optimal solution S * .
(2) S * {[ s1 , f1 )} is an optimal solution for
set {[ si , f i ) | [ si , f i )  [ s1 , f1 )  }.
Huffman Codes
Problem
Given n charactors a1 , a2 ,..., an with frequencie s
f1 , f 2 ,..., f n , respective ly, find prefix binary codes
c1 , c2 ,..., cn to minimize
| c1 |  f1  | c2 |  f 2    | cn |  f n .
prefix code : no one is a prefix of
another one.
Binary Tree Representation
Each code is represents by a path from the
root to a leaf.
So, each leaf is labeled with a charactor.
The length of a code is the length of
correspond ing path.
0
0
0
a1
1
1
1
a2
a3
1
0
0
a4
a6
1
a5
c1  000, c2  001, c3  01,
c4  100, c5  101, c6  11.
Huffman Tree Problem
Given n charactors a1 , a2 ,..., an with weig hts
f1 , f 2 ,..., f n , respective ly, find a binary tre e with
n leaves labeled by a1 , a2 ,..., an , respective ly,
to minimize
d (a1 )  f1  d (a2 )  f 2    d (an )  f n
where d (ai ) is the depth of ai , i.e ., the length
of the path from the root to ai .
Lemma
An optimal binary tre e must be complete,
i.e., each internal node has exactly tw o
children.
Exchange Property
If f i  f j and d (ai )  d (a j ), then exchnaging
ai and a j would make the objective function
value decreases in
(d (ai )  f i  d (a j )  f j )  (d ' (ai )  f i  d ' (a j )  f j )
 (d (ai )  f i  d (a j )  f j )  (d (a j )  f i  d (ai )  f j )
 (d (ai )  d (a j ))( f i  f j )
Self-reducible Optimality
If we assign the weight of each internal node
with the total weight of its decendant leaves,
then removal a subtree results still in an optimal
tree for remainer' s leaves.
0
0
0
f1
1
1
f3
1
f2
1
0
0
f4
f6
1
f5
0
0
1
1
f3
f1  f 2
0
f4
0
f1
1
f2
1
0
f6
1
f5
Idea of Construction
Sort f1  f 2    f n .
By exchange property, there exists an optimal
tree in which a1 and a2 are sibling.
By self - reducible optimality , the problem is
reduced to construct optimal tree for leaves with
weights { f1  f 2 , f 3 ,..., f n }.
This process can continue.
f1  0.1, f 2  0.2, f 3  0.3, f 4  0.3, f 5  0.1.
f1  f 5  0.2, f 2  0.2, f 3  0.3, f 4  0.3.
( f1  f 5 )  f 2  0.4, f 3  0.3, f 4  0.3.
( f1  f 5 )  f 2  0.4, f 3  f 4  0.6.
f3  f 4
( f1  f 5 )  f 2
f1  f 5
f1
f2
f5
f3
f4
Greedy Construction
Put f1 , f 2 ,..., f n into a min - preiority queque Q;
for i  1 to n  1
do allocate a new node z
left[ z ]  x  Exract - Min[ Q]
right [ z ]  y  Extract - Min[ Q]
f [ z ]  f [ x]  f [ y ]
Insert( Q, z )
return Extract - Min( Q).
Minimum Spanning Tree
What we learnt in this lecture?
• How to find greedy algorithm.
Exchange property.
Structural Property of Optimal Solution
Self-reducible optimality.
Puzzle
In activity - selection problem, suppose each interval has a nonnegativ e
weight and we want to find the maximum total weight nonoverlap ping
intervals.
Does the exchange property still hold?
If not, can you find an efficient way to find an optimal solution?
[0,1)
[0,
weigh t 1
2)
[1,
weight 2
3)
[1,
[1,
4)
weight 4
4)
weight 5
(remain) weight 1  2 - 1
2)
[1,
3)
[2,
[1,
[1,
4)
weight 2  4 - 2
[0,1)
4)
weight 4
4)
weight 5
weight 2  3 - 1
4)
weight 4
4)
weight 4  5 - 1
weigh t 1
[1,
weight 3
3)
[2,
weight 2  4 - 2
Solution 1
4)
weight 5
Solution 2
[0,
[1,
4)
weight 3
[2,
[0,
[2,
2)
[2,
weight 2
4)
weight 4