Algorithms
Sorting-part2
Heap Sort
Heaps and Priority Queues
2
5
9
6
7
Queues
Queue
• First-in, First-out (FIFO) structure
• Operations
–
–
–
–
–
enqueue: insert element at rear
dequeue: remove & return front element
front: return front element
isEmpty: check if the queue has no elements
size: return number of elements in the queue
• Sample use
– handling requests and reservations
6
Array-based Queue
• Use an array of size N in a circular fashion
• Two variables keep track of the front and rear
f index of the front element
r index immediately past the rear element
• Array location r is kept empty
normal configuration
Q
0 1 2
f
r
wrapped-around configuration
Q
0 1 2
r
f
7
Queue Operations
Algorithm size()
return (N - f + r) mod N
• We use the
modulo operator
Algorithm isEmpty()
(remainder of
return (f = r)
division)
Q
0 1 2
f
0 1 2
r
r
Q
f
8
Queue Operations (cont.)
Algorithm enqueue(o)
• Operation enqueue
if size() = N - 1 then
throws an exception if
throw FullQueueException
the array is full
else
• This exception is
Q[r]  o
implementationr  (r + 1) mod N
dependent
Q
0 1 2
f
0 1 2
r
r
Q
f
9
Queue Operations (cont.)
Algorithm dequeue()
• Operation dequeue
if isEmpty() then
throws an
throw EmptyQueueException
exception if the
else
queue is empty
o  Q[f]
• This exception is
f  (f + 1) mod N
specified in the
return o
queue ADT
Q
0 1 2
f
0 1 2
r
r
Q
f
Linked List Implementation
front
public class NodeQueue implements Queue
{
private Node front;
private Node rear;
rear
private int size;
…
}
null
Enqueue
front
rear
null
null
Dequeue
front
rear
null
return this object
Time Complexity Analysis
•
•
•
•
•
enqueue() :
dequeue() :
isEmpty() :
size() :
front():
O(1)
O(1)
O(1)
O(1)
O(1)
Trees
Make Money Fast!
Stock
Fraud
Ponzi
Scheme
Bank
Robbery
Definitions
root node
• A tree is an abstract data type
– one entry point, the root
– Each node is either a leaf or an
internal node
– An internal node has 1 or more
children, nodes that can be
reached directly from that internal
node.
– The internal node is said to be the
parent of its child nodes
internal
nodes
leaf nodes
1
6
Tree Terminology
• Root: node without parent (A)
• Internal node: node with at least
one child (A, B, C, F)
• External node (a.k.a. leaf ): node
without children (E, I, J, K, G, H, D)
• Ancestors of a node: parent,
grandparent, grand-grandparent,
etc.
• Depth of a node: number of
ancestors
• Height of a tree: maximum depth of
any node (3)
• Descendant of a node: child,
grandchild, grand-grandchild, etc.
Subtree: tree consisting of
a node and its
descendants
A
B
E
C
F
I
J
G
K
D
H
subtree
1
7
Binary Tree
• A binary tree is a tree with the
following properties:
– Each internal node has two
children left child and right child
– The children of a node are an
ordered pair
• Alternative recursive definition:
a binary tree is either
– a tree consisting of a single node,
or
D
– a tree whose root has an ordered
pair of children, each of which is a
binary tree
Applications:



arithmetic expressions
decision processes
searching
A
B
C
E
H
F
I
G
1
8
Arithmetic Expression Tree
• Binary tree associated with an arithmetic
expression
– internal nodes: operators
– external nodes: operands
• Example: arithmetic expression tree for the
expression (2  (a - 1) + (3  b))
+


-
2
a
3
1
b
1
9
Decision Tree
• Binary tree associated with a decision process
– internal nodes: questions with yes/no answer
– external nodes: decisions
• Example: dining decision
Want a fast meal?
No
Yes
How about coffee?
On expense account?
Yes
No
Yes
No
Starbucks
Spike’s
Al Forno
Café Paragon
2
0
Inorder Traversal
• In an inorder traversal a
node is visited after its left
subtree and before its
right subtree
• Application: draw a binary
tree
Algorithm inOrder(v)
if isInternal (v)
inOrder (leftChild (v))
visit(v)
if isInternal (v)
inOrder (rightChild (v))
– x(v) = inorder rank of v
– y(v) = depth of v
6
2
8
1
4
3
7
5
9
2
1
Data Structure for Binary Trees
• A node is represented by an object storing
–
–
–
–

Element
Parent node
Left child node
Right child node
B

B
A
A
D
C

D

E

C


E
Priority Queues
Sell
100
IBM
$122
Sell
300
IBM
$120
Buy 500
IBM
$119
Buy 400
IBM
$118
Priority queue
• A stack is first in, last out
• A queue is first in, first out
• A priority queue is least-first-out
– The “smallest” element is the first one removed
• (You could also define a largest-first-out priority
queue)
– If there are several “smallest” elements, the
implementer must decide which to remove first
• Remove any “smallest” element (don’t care which)
• Remove the first one added
23
The Priority Queue ADT
• A prioriy queue P supports the following methods:
-size():
Return the number of elements in P
-isEmpty():
Test whether P is empty
-insertItem(k,e): Insert a new element e with key k into P
-minElement(): Return (but don’t remove) an element of P
with smallest key; an error occurs if P is empty.
-minKey():
Return the smallest key in P; an error occurs if P
is
empty
-removeMin(): Remove from P and return an element with the
smallest key; an error condidtion occurs if P is
empty.
2
5
What is a heap
• A heap is a binary tree storing
keys at its internal nodes and
satisfying the following
properties:
– Heap-Order: for every internal
node v other than the root,
key(v)  key(parent(v))
– Complete Binary Tree: let h be
the height of the heap
• for i = 0, … , h - 1, there are 2i
nodes of depth i
• at depth h - 1, the internal nodes
are to the left of the external
nodes
• The last node of a heap
is the rightmost internal
node of depth h - 1
2
5
9
6
7
last node
Binary Heap: Definition
• Binary heap.
– Almost complete binary tree.
• filled on all levels, except last, where filled from left to right
– Min-heap ordered.
• every child greater than (or equal to) parent
06
14
78
83
26
45
18
91
81
47
77
84
53
99
64
Binary Heap: Properties
• Properties.
– Min element is in root.
– Heap with N elements has height = log2 N.
06
N = 14
Height = 3
14
78
83
27
45
18
91
81
47
77
84
53
99
64
2
8
Heaps and Priority Queues
• We can use a heap to implement a priority queue
• We store a (key, element) item at each internal
node
• We keep track of the position of the last node
• For simplicity, we show only the keys in the
pictures
(2, Ahmed)
(5, Hassan)
(9, Sara)
(6, Mona)
(7, Peter)
2
9
Insertion into a Heap
• Method insertItem of the
priority queue
corresponds to the
insertion of a key k to
the heap
• The insertion algorithm
consists of three steps
– Find the insertion node z
(the new last node)
– Store k at z and expand z
into an internal node
– Restore the heap-order
property (discussed next)
2
5
9
6
z
7
insertion node
2
5
9
6
7
z
1
3
0
Upheap
• After the insertion of a new key k, the heap-order property
may be violated
• Algorithm upheap restores the heap-order property by
swapping k along an upward path from the insertion node
• Upheap terminates when the key k reaches the root or a
node whose parent has a key smaller than or equal to k
• Since a heap has height O(log n), upheap runs in O(log n)
time
2
1
5
9
1
7
z
6
5
9
2
7
z
6
Binary Heap: Properties
• Properties.
– Min element is in root.
– Heap with N elements has height = log2 N.
06
N = 14
Height = 3
14
78
83
31
45
18
91
81
47
77
84
53
99
64
Binary Heap: Insertion
• Insert element x into heap.
– Insert into next available slot.
– Bubble up until it's heap ordered.
• Peter principle: nodes rise to level of incompetence
06
14
78
83
32
45
18
91
81
47
77
84
53
99
64
42
next free slot
Binary Heap: Insertion
• Insert element x into heap.
– Insert into next available slot.
– Bubble up until it's heap ordered.
• Peter principle: nodes rise to level of incompetence
swap with parent
06
14
78
83
33
45
18
91
81
47
77
84
53
99
64
42
Binary Heap: Insertion
• Insert element x into heap.
– Insert into next available slot.
– Bubble up until it's heap ordered.
• Peter principle: nodes rise to level of incompetence
swap with parent
06
14
78
83
34
45
18
91
81
47
77
84
42
99
64
53
42
Binary Heap: Insertion
• Insert element x into heap.
– Insert into next available slot.
– Bubble up until it's heap ordered.
• Peter principle: nodes rise to level of incompetence
– O(log N) operations.06
stop: heap ordered
14
78
83
35
42
18
91
81
47
77
84
45
99
64
53
Binary Heap: Decrease Key
• Decrease key of element x to k.
– Bubble up until it's heap ordered.
– O(log N) operations.
06
14
78
83
36
42
18
91
81
47
77
84
45
99
64
53
3
7
Removal from a Heap
• Method removeMin of
the priority queue
corresponds to the
removal of the root key
from the heap
• The removal algorithm
consists of three steps
– Replace the root key with
the key of the last node w
– Compress w and its
children into a leaf
– Restore the heap-order
property (discussed next)
2
5
9
6
7
w
last node
7
5
9
w
6
Binary Heap: Delete Min
• Delete minimum element from heap.
– Exchange root with rightmost leaf.
– Bubble root down until it's heap ordered.
• power struggle principle: better subordinate is promoted
06
14
78
83
38
42
18
91
81
47
77
84
45
99
64
53
Binary Heap: Delete Min
• Delete minimum element from heap.
– Exchange root with rightmost leaf.
– Bubble root down until it's heap ordered.
• power struggle principle: better subordinate is promoted
53
14
78
83
39
42
18
91
81
47
77
84
45
99
64
06
Binary Heap: Delete Min
• Delete minimum element from heap.
– Exchange root with rightmost leaf.
– Bubble root down until it's heap ordered.
• power struggle principle: better subordinate is promoted
exchange with left child
53
14
78
83
40
42
18
91
81
47
77
84
45
99
64
Binary Heap: Delete Min
• Delete minimum element from heap.
– Exchange root with rightmost leaf.
– Bubble root down until it's heap ordered.
• power struggle principle: better subordinate is promoted
exchange with right child
14
53
78
83
41
42
18
91
81
47
77
84
45
99
64
Binary Heap: Delete Min
• Delete minimum element from heap.
– Exchange root with rightmost leaf.
– Bubble root down until it's heap ordered.
• power struggle principle: better subordinate is promoted
– O(log N) operations.14
stop: heap ordered
18
78
83
42
42
53
91
81
47
77
84
45
99
64
4
3
Updating the Last Node
• The insertion node can be found by traversing a path of O(log
n) nodes
– Go up until a left child or the root is reached
– If a left child is reached, go to the right child
– Go down left until a leaf is reached
• Similar algorithm for updating the last node after a removal
Using a heap to sort
• Using a heap-based priority queue, sorting can
be done in O( n log n ) time
– Insert all n elements in the priority queue
– Repeatedly remove the elements from the priority
queue (they will come out sorted)
– 2n operations on the priority queue, each taking O(
log n ) time => O( n log n )
• Sorting can be done within the array by treating
the array as a heap
Heap sort example
6
10
5
12
3
9
20
2
15
8
6
10
5
12
2
9
3
15
8
18
20
18
Phase 1: build the heap
for i  1 to n-1 do
insert element s[i] into the heap consisting of the
elements s[0]…s[i-1]
- Once the heap is built, s[0] will contain the
maximum element
Phase 1: build heap
6
10
5
12
2
9
3
15
8
18
20
Phase 1: build heap
10
6
5
12
2
9
3
15
8
18
20
Phase 1: build heap
10
6
5
12
2
9
3
15
8
18
20
Phase 1: build heap
12
10
5
6
2
9
3
15
8
18
20
Phase 1: build heap
12
10
5
6
2
9
3
15
8
18
20
Phase 1: build heap
12
10
9
6
2
5
3
15
8
18
20
Phase 1: build heap
20
10
12
6
2
5
3
15
8
18
9
Phase 1: build heap
20
10
12
6
2
5
3
15
8
18
9
Phase 1: build heap
20
15
12
10
2
5
3
6
8
18
9
Phase 1: build heap
20
15
12
10
2
5
8
6
3
18
9
Phase 1: build heap
Maximum element
20
18
12
10
2
5
15
6
3
8
9
Phase 2: repeatedly select max
for i  n-1 down to 1 do
swap s[0] and s[i]
“demote” s[0] to its proper place
in the heap consisting of the elements
s[0]...s[i-1]
Phase 2: select max
20
18
12
10
2
5
15
6
3
8
9
Phase 2: select max
8
18
12
10
2
5
15
6
3
20
9
Phase 2: select max
18
15
12
10
2
5
8
6
3
20
9
Phase 2: select max
18
15
12
10
2
5
8
6
3
20
9
Phase 2: select max
3
15
12
10
2
5
8
6
18
20
9
Phase 2: select max
15
10
12
6
2
5
8
3
18
20
9
Phase 2: select max
15
10
12
6
2
5
8
3
18
20
9
Phase 2: select max
3
10
12
6
2
5
8
15
18
20
9
Phase 2: select max
12
10
9
6
2
5
8
15
18
20
3
Phase 2: select max
12
10
9
6
2
5
8
15
18
20
3
Phase 2: select max
2
10
9
6
12
5
8
15
18
20
3
Phase 2: select max
10
8
9
6
12
5
2
15
18
20
3
Phase 2: select max
3
8
9
6
12
5
2
15
18
20
10
Phase 2: select max
9
8
5
6
12
3
2
15
18
20
10
Phase 2: select max
3
8
5
6
12
9
2
15
18
20
10
Phase 2: select max
8
6
5
3
12
9
2
15
18
20
10
Phase 2: select max
2
6
5
3
12
9
8
15
18
20
10
Phase 2: select max
6
3
5
2
12
9
8
15
18
20
10
Phase 2: select max
2
3
5
6
12
9
8
15
18
20
10
Phase 2: select max
5
3
2
6
12
9
8
15
18
20
10
Phase 2: select max
2
3
5
6
12
9
8
15
18
20
10
Phase 2: select max
3
2
5
6
12
9
8
15
18
20
10
Phase 2: select max
2
3
5
6
12
9
8
15
18
20
10
Heap sort completed
2
3
5
6
8
9
10
12
15
18
2
3
5
6
12
9
8
15
18
20
10
20
Heap sort time complexity
for i  1 to n-1 do
insert element s[i] into the
heap consisting of the
elements s[0]…s[i-1]
O( n log n )
O( log n ) operations
for i  n-1 down to 1 do
swap s[0] and s[i]
“demote” s[0] to its proper place
in the heap consisting of the
elements s[0]...s[i-1]
O( n log n )
About heap sort
• Build heap phase can be improved to
O( n ) if array is rearranged “bottom-up”
– Overall complexity still O( n log n ) because of second
phase
• O( n log n ) time complexity is guaranteed
• Note that heap sort is just a more clever version
of selection sort since a maximum is repeatedly
selected and placed in its proper position
Heapify – builds the heap
• heapify(Item[]s, int p, int q) {
while(p < q) {
int c=2*p + 1; // left child
if(c > q) break;
if(c+1 <= q && s[c] < s[c+1]) ++c;
if(s[p] >= s[c]) break;
swap(s[p], s[c]);
p = c;
}
}
Heapsort
• heapsort(Item[]s) {
n = s.length;
for(j=(n-1)/2, k=n-1; j>=0; --j)
heapify(s, j, k);
for(j=n-1; j>0; --j) {
swap(s[0], s[j]);
heapify(s, 0, j-1);
}
}
Problems
• Demonstrate, step by step, the operation of
Build-Heap on the array
A=[5, 3, 17, 10, 84, 19, 6, 22, 9]
87