Lecture 1
Advanced Sorting Methods
on 1D Arrays
1
• An array is a very useful data structure provided in
programming languages. This is because we can
have a quick and direct access to a specific element
in the array by using its index.
2
• However, it has at least two limitations
1. its size has to be known at compilation time,
2. the data in the array are consecutive, which
means that inserting an item into the array and
deleting an item from the array require shifting
other data in the array backwards and forwards,
respectively.
• These limitations can be overcome by using
linked structures (i.e., linked lists).
3
Lecture Content
1. Searching Methods on Array
1.1 Linear/Sequential Search Method
1.2 Binary Search Method
2. Sorting Methods on Array
2.1 Selection Sort
2.2 Insertion Sort
4
Lecture Content
2. Sorting Methods on Array
2.3 Bubble Sort
2.4 Shaker Sort
2.5 Shell Sort
2.6 Merge Sort
2.7 Quick Sort
2.8 Heap Sort
5
Linear/Sequential Search Method
• Problem: A given array a is a[0], …, a[n 1] and a given element (also called search key or
target) x. Specify whether x exists in the array a.
• Basically, we have two methods of searching,
namely linear (also called sequential) search
method and binary search method.
6
Linear/Sequential Search Method
Main idea: We look through the array sequentially
to make the comparison between x and each
element a[i] of the given array. The search
function returns
-1 if not found
i (0 ≤ i ≤ n - 1) if x is located at position i in the
array a
• The time efficiency of linear search is in (n).
7
Linear/Sequential Search Method
Algorithm linearSearch
Input: An array a[0..n - 1] of n elements and an
element x.
Output: i if x = a[i], 0 ≤ i ≤ n - 1, and -1 otherwise.
1. i ← 0
2. while (i < n) and (x ≠ a[i])
3.
i←i+1
4. end while
5. if i < n then return i else return -1
8
Binary Search Method
Precondition: the given array must be sorted (i.e.,
elements are in order).
Main idea:
- Use a search procedure based on applying the
standard divide-and-conquer paradigm.
9
Binary Search Method
Main idea:
- Divide the set of items into two parts, determine
to which of the two parts the search key belongs,
then concentrate on that part.
- A reasonable way to divide the sets of items into
parts is to keep the items sorted, then to use indices
into the sorted array to delimit the part of the array
being worked on.
10
Binary Search Method
Main idea:
- This search technique is called binary search.
The search function returns
-1 if not found
i (0 ≤ i ≤ n - 1) if x is placed at position i in the
array a
• The time efficiency of binary search is in (log n)
(i.e., at most log n + 1 comparisons).
11
Binary Search Method
m = (L + r) / 2
If x = a[m], return m
If x < a[m], search x in left half a[0..m - 1]
If x > a[m], search x in right half a[m + 1..r]
12
Binary Search Method
Algorithm binarySearch
Input: An array a[0..n - 1] of n elements sorted in
nondecreasing order and an element x.
Output: i if x = a[i]; 0 ≤ i ≤ n - 1, and -1 otherwise.
13
Binary Search Method
1. L ← 0; r ← n - 1; i ← -1 // L = left, r = right, m =
middle
2. while (L ≤ r) and (i = -1)
3.
m ← (L + r) / 2
4. if x = a[m] then i ← m
5.
else if x < a[m] then r ← m - 1
6.
else L ← m + 1
7. end while
8. return i
14
2. Sorting Methods on 1D Array
• The efficiency of data handling can often be
substantially increased if the data are sorted
according to some criteria of order.
• Two critical properties of sorting algorithms are
the number of comparisons and the number of
data movements.
• Problem: Assume that we want to sort the array a
containing a0, a1 , … , an - 1 in ascending order.
15
Selection Sort
• Selection sort scans the array to find its smallest
element and swap it with the first element. Then,
starting with the second element, scan the elements
to the right of it to find the smallest among them
and swap it with the second elements. Generally, on
pass i (0  i  n - 2), find the smallest element in
a[i..n - 1] and swap it with a[i].
• The time complexity of selection sort is in (n2)
(i.e., exactly n(n - 1)/2 comparisons).
16
Selection Sort
Algorithm
Step 1: Find the smallest (largest) element in the
current array.
Step 2: Swap it with the first (last) element.
Step 3: Eliminate the smallest (largest) element
from the current array.
Step 4: Repeat Steps 1 - 3 until the number of
elements in the current array is equal to 1.
17
Selection Sort
Notes:
Initially (i = 0): the array has n elements.
Loop 1 (i = 1): the array remains (n - 1) elements.
...
Loop n - 1 (i = n - 1): the array remains 1 element.
Therefore, we just repeat (n - 1) times and the last
element of the array at time (n - 1) is maximal,
otherwise this element must be selected and
removed at the time (n - 2).
18
Selection Sort
Algorithm selectionSort
Input: An array a[0..n - 1] of n elements.
Output: a[0..n - 1] sorted in nondecreasing order.
19
Selection Sort
1. for i ← 0 to n - 2
2.
k←i
3.
for j ← i + 1 to n - 1 // find ith smallest
4.
if a[j] < a[k] then k ← j
5. end for
6.
if k ≠ i then interchange a[i] and a[k]
7. end for
20
Example of Selection Sort
.
21
Insertion Sort
• This sorting method is analogous to the way many
people sort their hands when playing cards: pick up
one item at a time and move it into place among the
items already picked up and sorted.
• Assume that we have (i - 1) cards sorted, now we
pick up the ith card and how to sort i cards. We can
compare the ith card with previous cards of (i - 1),
(i - 2), ..., 0 to find out the right position of the ith
card and insert it into this location.
22
Insertion Sort
• We insert ai into the sorted array a containing a0,
a1 , … , ai - 1 so that after inserting the new array a
containing a0, a1, … , ai - 1, ai remains in order.
Notes
- a0 is an ordered array so we consider from the
second element a1 (i.e., 1 ≤ i ≤ n - 1).
- Insert ai into a0, a1, …, ai - 1: successively
compare ai with ai - 1, if ai < ai - 1 then exchange the
position of two cards ai and ai - 1.
23
Insertion Sort
Algorithm insertionSort
Input: An array a[0..n - 1] of n elements.
Output: a[0..n - 1] sorted in nondecreasing order.
24
Insertion Sort
1. for i ← 1 to n - 1
2.
x ← a[i]
3.
j←i-1
4.
while (j ≥ 0) and (a[j] > x)
5.
a[j + 1] ← a[j]
6.
j←j-1
7.
end while
8.
a[j + 1] ← x
9. end for
25
Example of Insertion Sort
• The time complexity of insertion sort is in (n2)
(i.e., at most n(n - 1)/2 comparisons).
26
Bubble Sort
Main idea: move backwards from the last element
(i.e., n - 1) to the second one (i.e., a[1]). In other
words, move up from the bottom element to the
second top element.
(Alternatively, move forwards from the first
element (i.e., a[0]) to the second last element (i.e.,
n - 2). In other words, move down from the top
element to the second last element.)
27
Bubble Sort
Loop 1: move up to the 2nd element a[1]
Loop 2: move up to the 3rd element a[2]
…
Loop n - 1: move up to the nth element a[n - 1]
While moving up, compare each pair of
neighboring elements. If the below element is
lighter than the above element then interchange
them. This process is repeated (n - 1) times then the
given array will be sorted.
28
Bubble Sort
Algorithm 1 bubbleSort
Input: An array a[0..n - 1] of n elements.
Output: a[0..n - 1] sorted in nondecreasing order.
1. for i ← 1 to n - 1 // (n - 1) iterations
2. for j ← n - 1 downto i
// move up
3.
if a[j] < a[j - 1] then interchange a[j - 1]
and a[j] // push small element up
4.
end for
5. end for
29
Bubble Sort
Algorithm 2 bubbleSort
Input: An array a[0..n - 1] of n elements.
Output: a[0..n - 1] sorted in nondecreasing order.
1. for i ← 0 to n - 2 // (n - 1) iterations
2. for j ← 0 to n - 2 - i // move down
3.
if a[j] > a[j + 1] then interchange a[j] and
a[j + 1] // push large element down
4.
end for
5. end for
30
Bubble Sort
Algorithm 3 bubbleSort
Input: An array a[0..n - 1] of n elements.
Output: a[0..n - 1] sorted in nondecreasing order.
1. i ← 1; sorted ← false
2. while (i ≤ n - 1) and (not sorted)
3.
sorted ← true
4.
for j ← n - 1 downto i
5.
if a[j] < a[j - 1] then
6.
interchange a[j - 1] and a[j]
31
Bubble Sort
7.
sorted ← false
8.
end if
9.
end for
10. i ← i + 1
11. end while
• The time complexity of bubble sort is in (n2).
32
Shaker Sort
Comments on Bubble sort method:
+ Light (small) elements move up quickly, heavy
(large) ones go down slowly.
+ Still performs on a sorted array.
Thought: Two tasks are performed in one iteration
• The time complexity of shaker sort is in (n2).
33
Shaker Sort
Task 1: (move up ()): push the light element up
and store the last position where swapping is
performed to change the water surface down to this
position.
Task 2: (move down ()): push the heavy element
down and save the last position where swapping is
performed to change the bottom up to this position.
The algorithm is terminated when water surface L =
1 and bottom r = n - 1 pass through.
34
Shaker Sort
Algorithm in the form of pseudocode
while (L <= r)
{
// push the light
// store the last
position
// push the heavy
// store the last
position
element up
swapping
element down
swapping
} // while (L <= r)
35
Shaker Sort (Cocktail Sort)
Algorithm shakerSort
Input: An array a[0..n - 1] of n elements.
Output: a[0..n - 1] sorted in nondecreasing order.
36
Shaker Sort (Cocktail Sort)
1. L ← 1; r ← n - 1; k ← n - 1;
2. while (L ≤ r)
3.
for i ← r downto L
4.
if a[i] < a[i - 1] then
5.
interchange a[i - 1] and a[i]
6.
k←i
7.
end if
8.
end for
9.
L←k+1
37
Shaker Sort (Cocktail Sort)
10. for j ← L to r
11.
if a[j - 1] > a[j] then
12.
interchange a[j - 1] and a[j]
13.
k←j
14.
end if
15. end for
16. r ← k - 1
17. end while
38
Shell Sort
• Shell sort solves the sorting problem by dividing
the original array into subarrays, sorting them
separately, and then dividing them again to sort the
new subarrays until the whole array is sorted. The
goal was to reduce the original problem to
subproblems that can be solved more easily and
quickly.
39
Shell Sort
• The time complexity of shell sort is in (n3/2) for
the Hibbard’s sequence 1, 3, 7, ..., 2k - 1.
• The time complexity of shell sort is in (n4/3) for
the Sedgewick’s sequence 1, 8, 23, ..., 4k+1 + 3×2k +
1.
40
Shell Sort
Algorithm
Step 1: Divide the given array a0, …, an – 1 into
subarrays. The step (distance) between elements in
each subarray is hi (hi = 9, 5, 3, 1)
Step 2: Sort each subarray using insertion sort.
Step 3: Repeat Steps 1 and 2 for the next value of
h i.
41
Example of Shell Sort
• Sort the array a containing numbers 10 8 6 20
4 3 22 1 0 15 16
Assume that hi = 5.
42
Example of Shell Sort
10 8 6 20 4 3 22 1 0 15 16
43
Example of Shell Sort
3 8 1 0 4 10 22 6 20 15 16
44
Example of Shell Sort
0 4 1 3 6 10 15 8 20 22 16
hi = 1:
The sorted array is
0 1 3 4 6 8 10 15 16 20 22
45
Merge Sort [Divide-and-Conquer Strategy]
• Mergesort in a top-down manner sorts a given
array a[L..r] by dividing it into two halves a[L..m]
and a[m + 1..r], where L = 0, r = n - 1, and m = (L
+ r)/2, sorting each of them recursively, and then
merging the two smaller sorted arrays into a single
sorted one.
• The operation of the algorithm on the list 8, 3, 2,
9, 7, 1, 5, 4 is illustrated below.
46
Example of Merge Sort
.
47
Recursive Merge Sort
Algorithm mergeSort(a, L, r) // L = left = 0, r =
right = n - 1
//Sort array a[0..n - 1] in nondecreasing order by
recursive mergesort in a top-down manner
Input: An array a[0..n - 1] of n orderable elements
Output: Array a[0..n - 1] sorted in nondecreasing
order
48
Recursive Merge Sort
1. if L < r then
2.
m ← (L + r) / 2
// m = middle
3.
mergeSort(a, L, m) // sort first half a[L..m]
4. mergeSort(a, m + 1, r) // sort 2nd half a[m+1..r]
5. merge(a, L, m + 1, r) // (n)
6. end if
49
merge
Algorithm merge(a, lp, rp, re) // lp = left
position, rp = right position, re = right end
//Merges two sorted subarrays into a sorted array
Input: Sorted subarrays a[lp..le] and a[rp..re]
//
left end le = rp - 1
Output: Array a[L..r] sorted in nondecreasing order
50
merge
1. Create a temporary array tA[L..r]
2. le ← rp - 1; i ← lp
3. while (lp ≤ le) and (rp ≤ re)
4.
if a[lp] < a[rp] then
5.
tA[i] ← a[lp];
6.
lp ← lp + 1
7.
else
8.
tA[i] ← a[rp];
9.
rp ← rp + 1
10.
end if
51
merge
11. i ← i + 1
12. end while
13. if lp ≤ le then tA[i..r] ← a[lp..le]
14. if rp ≤ re then tA[i..r] ← a[rp..re]
15. a[L..r] ← tA[L..r]
• The time efficiency of merge sort is in (n log n)
(i.e., at most n log n - n + 1).
52
Iterative Merge Sort
Algorithm mergeSort(a, n)
//Sort array a[0..n - 1] in nondecreasing order by
iterative mergesort
Input: An array a[0..n - 1] of n orderable elements
Output: Array a[0..n - 1] sorted in nondecreasing
order
53
Iterative Merge Sort
1. for size ← 1 to n - 1
2.
lp ← 0 // left pointer
3.
while ( (lp + size) < n )
4.
rp ← lp + size // right pointer
5.
re ← Min(n - 1, rp + size - 1) // right end
6.
merge(a, lp, rp, re)
7.
lp ← re + 1
8.
end while
9.
size = size * 2
10. end for
54
Quick Sort
• Quicksort (or partition sort) is a type of recursive
divide-and-conquer strategy.
• Quicksort rearranges elements of a given array
a[L..r], where L = 0, r = n - 1, to achieve its
partition, a situation where all the elements before
some position s, called partitioning position, are
smaller than or equal to a[s], called pivot element,
and all the elements after position s are greater than
or equal to a[s].
55
Quick Sort
(a[s] is called pivot or partitioning element)
• Obviously, after a partition has been achieved,
a[s] will be in its final position in the sorted array,
and we can continue sorting the two subarrays of
the elements preceding and following a[s]
independently by the same method. The two
subarrays are a[L..s - 1] and a[s + 1..r].
56
Quick Sort
Algorithm quickSort(a, L, r)
// L = left = 0 and
r = right = n - 1
//Sort an array a[L..r] in nondecreasing order by
recursive quicksort
Input: An array a[L..r] of n orderable elements
Output: Array a[L..r] sorted in nondecreasing order
57
Quick Sort
1. if L < r then
2.
s ← partition(a, L, r) //s is split position, (n)
3.
quickSort(a, L, s - 1) // sort left subarray
4.
quickSort(a, s + 1, r) // sort right subarray
5. end if
• The average running time of quick sort is in (n
log n).
58
Partition: Lomuto’s Algorithm
Algorithm partition(a, L, r) // Lomuto’s algorithm
//Partition an array a[L..r] by using its first element
as a pivot
Input: An array a[L..r] of n orderable elements
Output: A partition of a[L..r] and the position s of
the pivot element
1. p ← a[L]
2. s ← L
59
Partition: Lomuto’s Algorithm
3. for i ← L + 1 to r
4.
if a[i] < p then
5.
s←s+1
6.
if s ≠ i then swap a[s] and a[i]
7. end if
8. end for
9. swap a[L] and a[s]
10. return s
60
Partition: Lomuto’s Algorithm
• Example: find the partition of the array 5 3 9
2 7 1 8
• Example: find the partition of the array 5 7 1
6 4 8 3 2
61
Partition: Hoare’s Algorithm
Algorithm partition(a, L, r) // Hoare’s algorithm
//Partition an array a[L..r] by using its first element
as a pivot
Input: An array a[L..r] of n orderable elements
Output: A partition of a[L..r] and the position j of
the pivot element
1. p ← a[L]
2. i ← L; j ← r + 1
62
Partition: Hoare’s Algorithm
3. while (i < j)
4.
repeat i ← i + 1 until a[i] ≥ p
5.
repeat j ← j - 1 until a[j] ≤ p
6.
if i < j then swap a[i] and a[j]
7. end while
8. swap a[L] and a[j] // place pivot at final position
9. return j
63
Partition: Hoare’s Algorithm
• Initially, a[0] is selected as the pivot element
64
Partition: Hoare’s Algorithm
.
65
Heap Sort
Definition of a heap
• The given array aL, aL + 1, ..., ar, where L = left = 0
and r = right = n - 1, is called a (binary) heap if the
array satisfies the heap property: ai ≥ a2i + 1 and ai ≥
a2i + 2, where an arbitrary i belongs to the interval
[L, n / 2 - 1].
• The above heap is called a max heap because the
largest element is placed at the first position of the
array.
66
Heap Sort
Remarks:
+ a0 is the largest element in the array.
+ The subarray an/2, an/2 + 1, ..., an - 1 (also called leaf
nodes) is considered as a heap with respect to the
array a0, a1, … , an - 1. Parent nodes are a0, ..., an/2 - 1.
+ If we remove elements on the right side of the
heap then the remaining elements are still a heap.
• The time complexity of heap sort is in (n log n).
67
Heap Sort
• n = 10
Parent nodes: a0, ..., an/2 - 1.
Leaf nodes: an/2, ..., an - 1.
68
Heap Sort
Algorithm 1 heapSort
Step 1: Adjust the given array aL, aL + 1, ..., ar to
become a heap, where L = 0, r = n - 1.
Step 2: Swap aL and ar.
Step 3: Eliminate the last element from the current
array to have a new subarray.
Step 4: Repeat Steps 1, 2, and 3 until the number of
elements in the array is equal to 1.
69
Heap Sort
Algorithm 2 heapSort(a[0..n - 1])
Input: An array a[0..n - 1] of n elements.
Output: Array a[0..n - 1] sorted in nondecreasing
order
1. makeHeap (a, n)
2. for j ← n - 1 downto 1 // (n - 1) iterations
3.
interchange a[0] and a[j]
4.
siftDown(a, 0, j - 1)
5. end for
70
makeHeap
Algorithm makeHeap(a[0..n - 1])
Input: An array a[0..n - 1] of n elements.
Output: Array a[0..n - 1] is transformed into a heap
1. r ← n - 1
2. for i ← n/2 - 1 downto 0
3.
siftDown(a, i, r)
4. end for
71
siftDown
Algorithm siftDown(a, i, r)
// push parent a[i] down the binary tree until its
proper location is found, 0 ≤ i ≤ n/2 - 1
Input: An array a[L..r] and an index i between 0
and n/2 - 1.
Output: a[i] is pushed down, if necessary, so that
a[i] is not smaller than its children.
72
siftDown
1. done ← false, j ← 2*i + 1 // left child a[j] of
parent a[i]
2. while (not done) and (j ≤ r)
3.
if (j < r) and (a[j] < a[j + 1]) then // find
index of maximum value between a[j] and a[j+1]
4.
j←j+1
73
siftDown
5.
if a[i] < a[j] then // if current parent a[i] <
left child a[j] and right child a[j + 1]
6.
interchange a[i] and a[j]
7.
i ← j // update current parent node a[i]
8.
j ← 2 * j + 1; // update left child a[j] of
current parent a[i]
9.
else
10.
done ← true
11. end if
12. end while
74
Find Both Minimum amd Maximum
• Find both the minimum and maximum of an
unsorted array a[0..n - 1] containing n distinct
integers, where n is a power of 2.
Algorithm MINMAX
Input: An array a[0..n - 1] of n integers, where n is
a power of 2.
Output: (x, y): the min and max integers in a.
1. minmax(a, 0, n - 1)
75
Find Both Minimum amd Maximum
Procedure minmax(a, L, r) // L = 0, r = n - 1
1. if L = r then
2.
return (a[L], a[r])
3. else if r - L = 1 then
4.
if a[L] < a[r] then return (a[L], a[r])
5. else return (a[r], a[L])
6.
end if
7. else
76
Find Both Minimum amd Maximum
mid ← (L + r) / 2
(x1, y1) ← minmax(a, L, mid)
// x1 is min, y1 is max
10.
(x2, y2) ← minmax(a, mid + 1, r)
// x2 is min, y2 is max
11. x ← min{x1, x2} // x is final min
12.
y ← max{y1, y2} // y is final max
13.
return (x, y)
14. end if
8.
9.
77
Compute xn
• Compute the exponentiation xn, where x is a
positive integer and n is a nonnegative integer.
• Brute-force approach, (n):
xn = x × x × … × x (n - 1 multiplications).
• Decrease-by-one approach, (n):
78
Compute xn
• Divide-and-conquer approach, (n):
79
Compute xn
• Squaring algorithm, (log2 n):
• Time complexity of the squaring algorithm is
(log2 n)
80
Exercises
1. Write programs to implement advanced sorting
methods mergeSort, quickSort, and heapSort.
2. The selection problem: Write a program to find
the kth smallest element of an n-element array of
integers.
3. Write a program to implement mergeSort without
using recursion.
4. Write a program to compute xn with time
complexity of (log2 n), where x is a positive
integer and n is a nonnegative integer.
81