Algorithms & Data Structures (M)
3
Array Data Structure
 Properties
 Insertion
 Deletion
 Searching: linear search, binary search
 Merging
 Sorting: selection sort, merge-sort, quick-sort
© 2008 David A Watt, University of Glasgow
Properties (1)
 An array is a sequence of slots. Each slot
contains an element (value or object).
 Each slot has a fixed index. The indices are
consecutive integers.
low
low+1 low+2
a element element element
high–2 high–1 high
indices
element element element
slots
3-2
Properties (2)
 An array’s length is the number of slots. The
length is fixed when the array is constructed.
 Any array slot can be efficiently accessed
(inspected or updated) using its index, in O(1)
time.
3-3
Properties: subarrays
 A subarray of an array a is a sequence of
consecutive slots in a.
 Notation:
– a[l…r] is the subarray of a that comprises slots a[l],
a[l+1], …, a[r].
– Subarray notation may be used to express algorithms,
but it is not directly supported by Java.
0
1
2
3
4
5
6
7
8
a
subarray a[1…3]
subarray a[5…8]
 Subarray a[l…r] has length r – l + 1.
3-4
Properties: sorted arrays
 A (sub)array is sorted if its elements are in
ascending order, i.e., each element is less than
or equal to the element on its right.
 The meaning of “less” must be defined for each
data type:
– For numbers:
“x is less than y” means that x is numerically smaller
than y, i.e., x < y.
– For strings:
“x is less than y” conventionally means that x
lexicographically precedes y. E.g.: “bat” is less than
“bath”, which in turn is less than “bay”.
3-5
Aside: Java’s Comparable interface (1)
 The following interface (from java.util)
captures this notion:
public interface Comparable {
public int compareTo (Object that);
// Return a negative integer if this object is less than
// that, or zero if this object is equal to that, or a
// positive integer if this object is greater than that.
}
3-6
Aside: Java’s Comparable interface (2)
 If a class Thing implements Comparable, it
must define the compareTo method in
accordance with the above contract:
public class Thing implements Comparable {
…
public int compareTo (Object that) {…}
// Return a negative integer if this thing is less than
// that, or zero if this thing is equal to that, or a
// positive integer if this thing is greater than that.
}
 This captures the notion of “less” and “greater”
for objects of class Thing.
 String, Integer, … implement Comparable.
3-7
Insertion (1)
 Problem: Given a (sub)array a[left…right], insert
an element elem in the slot a[ins]. If necessary,
shift elements to make way for it.
 Array insertion algorithm:
To insert elem at index ins in a[left…right]
(where left  ins  right):
1. Copy a[ins…right–1] into a[ins+1…right].
2. Copy elem into a[ins].
3. Terminate.
i.e., shift
elements one
slot to the right
3-8
Insertion (2)
 Analysis (counting copies):
Let n = right – left + 1 be the length of the array.
Step 1 performs between 0 and n–1 copies. On
average it performs (n–1)/2 copies.
Step 2 performs 1 copy.
Average no. of copies
= (n – 1)/2 + 1
= n/2 + 1/2
 n/2
Time complexity is O(n).
3-9
Deletion (1)
 Problem: Given a (sub)array a[left…right], delete
the element in the slot a[del]. If necessary, shift
elements to fill the gap.
 Array deletion algorithm:
To delete the element at index del in a[left…right]
(where left  del  right):
1. Copy a[del+1…right] into a[del…right–1].
2. Make a[right] unoccupied.
3. Terminate.
i.e., shift
elements one
slot to the left
3-10
Deletion (2)
 Analysis (counting copies):
Let n = right – left + 1 be the length of the array.
Step 1 performs between 0 and n–1 copies.
Average no. of copies
= (n – 1)/2
 n/2
Time complexity is O(n).
3-11
Searching
 Problem: Given a (sub)array a[left…right], find
which (if any) element equals a given target
value.
 Choice of algorithms:
– linear search (unsorted or sorted array)
– binary search (sorted array).
3-12
Linear search (1)
 Linear search algorithm:
To find which (if any) element of a[left…right] equals target:
1. For p = left, …, right, repeat:
1.1. If target equals a[p], terminate yielding p.
2. Terminate yielding none.
 Loop invariant:
left
left+1
p–1
p
right–1 right
a
known not to equal target
still to be searched
3-13
Aside: invariants
 An invariant is an assertion that always holds at
a particular step in an algorithm or program.
 In particular, a loop invariant is one that holds at
every iteration of a loop.
 Invariants can be written in predicate logic, e.g.:
p : left…right and ( i : left…p–1 . a[i]  target )
 Instead we shall use schematic diagrams, e.g.:
left
left+1
p–1
p
right–1 right
a
known not to equal target
still to be searched
3-14
Linear search (2)
 Animation (successful search):
To find which (if any) element of a[left…right] equals
target:
1. For p = left, …, right, repeat:
1.1. If target equals a[p], terminate yielding p.
2. Terminate yielding none.
left = 0
a
rat
1
2
3
4
5
cat
pig
cow
fox
lion
target cow
p
6
7
tiger goat
8 = right
dog
3210
3-15
Linear search (3)
 Analysis (counting comparisons):
Let n = right – left + 1 be the length of the array.
 If the search is unsuccessful, step 1.1 is
repeated n times.
No. of comparisons = n
 If the search is successful, step 1.1 is repeated
between 1 and n times.
Average no. of comparisons = (n + 1)/2
 n/2
 In either case, time complexity is O(n).
3-16
Linear search (4)
 Implementation in Java:
static int search (Object target,
Object[] a, int left, int right) {
// Find which (if any) element of a[left…right]
// equals target.
for (int p = left; p <= right; p++) {
if (target.equals(a[p]))
return p;
}
return NONE;
}
–1, say
3-17
Binary search (1)
 Assume now that we are searching a sorted
(sub)array.
 Consider searching a dictionary for a target word:
– Bad idea: Look at page 1, then page 2, etc., until you
find the page containing the target word.
This is linear search!
– Better idea: Choose a page near the middle. If the
target word happens to be on that middle page, you’re
finished. If the target word is less (greater) than the
words on that middle page, focus on the pages before
(after) that middle page, and repeat.
This is binary search.
3-18
Binary search (2)
 Binary search algorithm:
To find which (if any) element of the sorted (sub)array
a[left…right] equals target:
1. Set l = left, and set r = right.
2. While l  r, repeat:
2.1. Let m be an integer about midway between l and r.
2.2. If target equals a[m], terminate yielding m.
2.3. Else, if target is less than a[m], set r = m–1.
2.4. Else, if target is greater than a[m], set l = m+1.
3. Terminate yielding none.
3-19
Binary search (3)
 Loop invariant:
left
l–1
l
r
r+1
right
a
known to be
less than target
still to be searched
known to be
greater than target
3-20
Binary search (4)
 Animation (successful search):
To find which (if any) element of the sorted (sub)array
a[left…right] equals target:
1. Set l = left, and set r = right.
2. While l  r, repeat:
2.1. Let m be an integer about midway between l and r.
2.2. If target equals a[m], terminate yielding m.
2.3. Else, if target is less than a[m], set r = m–1.
2.4. Else, if target is greater than a[m], set l = m+1.
3. Terminate yielding none.
left = 0
1
2
3
4
5
6
7
8 = right
cow
dog
fox
goat
lion
pig
rat
tiger
target lion
l
a cat
50
r
58
m
564
3-21
Binary search (5)
 Analysis (counting comparisons):
Let n be the length of the array.
Assume that steps 2.2–4 perform a single
comparison.
 If the search is unsuccessful, these steps are
repeated as often as we must halve n to reach 0:
No. of comparisons = floor(log2 n) + 1
 If the search is successful, these steps are
repeated at most that many times:
Max. no. of comparisons = floor(log2 n) + 1
 In either case, the time complexity is O(log n).
3-22
Binary search (6)
 Implementation in Java:
static int search (Comparable target,
Comparable[] a, int left, int right) {
// Find which (if any) element of the sorted (sub)array
// a[left…right] equals target.
int l = left, r = right;
while (l <= r) {
int m = (l + r)/2;
int comp = target.compareTo(a[m]);
if (comp == 0)
return m;
else if (comp < 0) r = m - 1;
else
l = m + 1;
}
return NONE;
}
3-23
Comparison of searching algorithms
Algorithm
No. of comparisons
Time
complexity
Linear search (unsorted array)
~ n/2 (successful)
n (unsuccessful)
O(n)
Linear search (sorted array)
~ n/2
O(n)
Binary search (sorted array)
~ log2 n
O(log n)
3-24
Merging (1)
 Problem: Given two sorted arrays, make a third
sorted array containing copies of all elements of
the two original two arrays.
 Idea: Compare the leftmost elements of the two
arrays. Whichever element is less, copy it into
the third array, and thereafter ignore that
element.
3-25
Merging (2)
 Array merging algorithm:
To merge a1[l1…r1] and a2[l2…r2] into a3[l3…r3]
(where both a1 and a2 are sorted):
1. Set i = l1, set j = l2, and set k = l3.
2. While i  r1 and j  r2, repeat:
2.1. If a1[i] is less than or equal to a2[j]:
2.1.1. Copy a1[i] into a3[k], then increment i and k.
2.2. Else:
2.2.1. Copy a2[j] into a3[k], then increment j and k.
3. If i  r1, copy a1[i…r1] into a3[k…r3].
4. If j  r2, copy a2[j…r2] into a3[k…r3].
5. Terminate.
3-26
Merging (3)
 Loop invariant:
l1
i–1
i
r1
a1
already copied
l2
j–1
still to be copied
j
r2
a2
already copied
l3
still to be copied
k–1
k
a3
copied from a1 and/or a2
unoccupied
3-27
Merging (4)
 Animation:
To merge a1[l1…r1] and a2[l2…r2] into a3[l3…]:
1. Set i = l1, set j = l2, and set k = l3.
2. While i  r1 and j  r2, repeat:
2.1. If a1[i] is less than or equal to a2[j]:
…
2.1.1.
a1[i] into a3[k],
then increment i and k.
3. If i  r1,
copyCopy
a1[i…r1]
a3[k…r3].
2.2.
Else:
4. If
j  r2,
copy a2[j…r2] into a3[k…r3].
5. Terminate.
2.2.1. Copy a2[j] into a3[k], then increment j and k.
…
l1 = 0
a1 cow
l2 = 0
a2
cat
l3 = 0
a3
cat
1 = r1
goat
1
2
3
4 = r2
dog
fox
lion
tiger
1
2
3
4
5
6
cow
dog
fox
goat
lion
tiger
i
210
j
3210
k
532104
3-28
Merging (5)
 Analysis (counting copies):
Let n1 and n2 be the lengths of a1 and a2.
Let n = n1 + n2.
Each element of a1 is copied once, and each
element of a2 is copied once.
No. of copies = n1 + n2 = n
Time complexity is O(n).
3-29
Merging (6)
 Analysis (counting comparisons):
Let n1 and n2 be the lengths of a1 and a2.
Let n = n1 + n2.
Assume that steps 2.1–2 perform a single
comparison.
Steps 2.1–2 are repeated at most n–1 times.
Max. no. of comparisons = n – 1
Time complexity is again O(n).
3-30
Merging (7)
 Implementation in Java:
static void merge (
Comparable[] a1, int l1, int r1,
Comparable[] a2, int l2, int r2,
Comparable[] a3, int l3) {
// Merge the sorted subarrays a1[l1…r1] and
// a2[l2…r2] into a3[l3…].
int i = l1, j = l2, k = l3;
while (i <= r1 && j <= r2) {
int comp = a1[i].compareTo(a2[j]);
if (comp <= 0) a3[k++] = a1[i++];
else
a3[k++] = a2[j++];
}
3-31
Merging (8)
 Implementation (continued):
while (i <= r1)
while (j <= r2)
a3[k++] = a1[i++];
a3[k++] = a2[j++];
}
 Alternative coding style:
…
while (j <= r2) {
a3[k] = a2[j]; k++; j++;
}
}
3-32
Remedial maths: arithmetic series
 An arithmetic series is a sequence of numbers
with a fixed difference between consecutive
numbers. E.g.:
«2, 3, 4, 5»
«0, 2, 4, 6, 8, 10, 12, 14, 16, 18»
 What is the sum of the arithmetic series
«1, 2, …, n»?
Average of series = average of first and last
= (n+1)/2
Length of series
= n
Sum of series
= n(n+1)/2
3-33
Sorting
 Problem: Given an unsorted array, rearrange its
elements into ascending order.
 This is important because a sorted array can be
searched and merged efficiently.
 There is a huge choice of algorithms:
– selection sort
– insertion sort
see Watt & Brown
– merge-sort
– quick-sort
– shell-sort, radix sort, etc.
see Goodrich & Tamassia
3-34
Selection sort (1)
 Idea: Find the least element in the array, swap it
into the leftmost slot (where it belongs). Repeat
this, now ignoring the leftmost slot.
3-35
Selection sort (2)
 Selection sort algorithm:
To sort a[left…right] into ascending order:
1. For l = left, …, right–1, repeat:
1.1. Set p such that a[p] is the least of a[l…right].
1.2. If p  l, swap a[p] and a[l].
2. Terminate.
 Loop invariant:
left
left+1
l–1
l
right–1 right
a
lesser elements (sorted)
greater elements (unsorted)
3-36
Selection sort (3)
 Animation:
To sort a[left…right] into ascending order:
1. For l = left, …, right–1, repeat:
1.1. Set p such that a[p] is the least of a[l…right].
1.2. If p  l, swap a[p] and a[l].
2. Terminate.
left = 0
a fox
cat
l
7
3
08
1
2
4
5
6
1
2
3
4
cow
dog
pig
fox
cat
goat
rat
p
5
6
7
8 = right
lion tiger
pig goat
rat tiger
pig
dog
378
51
3-37
Selection sort (4)
 Analysis (counting comparisons):
Let n = right–left+1 be the length of the array to
be sorted.
Step 1.1 performs right–l comparisons.
This is repeated with l = left, …, right–2, right–1.
No. of comparisons =
=
=
=

(right–left) + … + 2 + 1
(n–1) + … + 2 + 1
(n–1)n/2
n2/2 – n/2
n2/2
Time complexity is O(n2).
3-38
Selection sort (5)
 Implementation in Java:
static void sort (Comparable[] a,
int left, int right) {
// Sort a[left…right] into ascending order.
for (int l = left; l < right; l++) {
int p = l;
Comparable least = a[p];
for (int k = l+1; k <= right; k++){
int comp = a[k].compareTo(least);
if (comp < 0) {
p = k; least = a[p];
}
}
}
}
if (p != l) { a[p] = a[l]; a[l] = least; }
3-39
Aside: divide-and-conquer
 The divide-and-conquer strategy is effective for
solving many problems.
 To solve a “hard” problem:
– Break the problem down into two or more “easier” subproblems.
– Solve these sub-problems separately.
– Combine their answers.
 The divide-and-conquer strategy naturally
suggests recursive algorithms.
3-40
Merge-sort (1)
 Idea for sorting an array:
– Divide the array into two subarrays of about equal
length.
– Sort the subarrays separately.
– Merge the sorted subarrays.
 This is an application of the divide-and-conquer
strategy.
3-41
Merge-sort (2)
 Merge-sort algorithm:
To sort a[left…right] into ascending order:
1. If left < right:
1.1. Let m be an integer about midway between left and
right.
1.2. Sort a[left…m] into ascending order.
1.3. Sort a[m+1…right] into ascending order.
1.4. Merge a[left…m] and a[m+1…right] into auxiliary
array b.
1.5. Copy all elements of b into a[left…right].
2. Terminate.
3-42
Merge-sort (3)
 Animation (treating 1.2 and 1.3 as single steps):
To sort a[left…right] into ascending order:
1. If left < right:
1.1. Let m be an integer about midway between left and right.
1.2. Sort a[left…m] into ascending order.
1.3. Sort a[m+1…right] into ascending order.
1.4. Merge a[left…m] and a[m+1…right] into auxiliary array b.
1.5. Copy all elements of b into a[left…right].
2. Terminate.
left = 0
1
2
3
4
5
6
7
8 = right
a
fox
cat
cow
dog
fox
pig
fox
pig
cat
goat
rat
lion
dog
tiger
goat
pig goat
lion
rat tiger
dog
b
cat
cow
dog
fox
goat
lion
pig
rat
tiger m
4
3-43
Merge-sort (4)
 Analysis (counting comparisons):
Let n = right–left+1 be the length of the array to
be sorted.
Let the total no. of comparisons required to sort a
(sub)array of length m be comps(m).
Step 1.2 takes about comps(n/2) comparisons to
sort the left subarray, since the subarray’s length
is about n/2.
Step 1.3 takes about comps(n/2) comparisons to
sort the right subarray, similarly.
Step 1.4 takes about n–1 comparisons to merge
the subarrays.
3-44
Merge-sort (5)
 Analysis (continued):
Therefore:
comps(n)  2 comps(n/2) + n – 1 if n > 1
comps(n) = 0
if n  1
Solution:
comps(n)  n log2n
Time complexity is O(n log n).
 Space complexity is O(n), since step 1.4 needs
an auxiliary array of length n.
3-45
Quick-sort (1)
 Idea: Choose any element (called the pivot).
Then partition the array into three subarrays
such that:
– the left subarray contains only elements less than (or
equal to) the pivot;
– the middle subarray contains only the pivot;
– the right subarray contains only elements greater than
(or equal to) the pivot.
Finally sort the left subarray and the right
subarray separately.
 This is another application of the divide-andconquer strategy.
3-46
Quick-sort (2)
 Quick-sort algorithm:
To sort a[left…right] into ascending order:
1. If left < right:
1.1. Partition a[left…right] such that
a[left…p–1] are all less than or equal to a[p], and
a[p+1…right] are all greater than or equal to a[p].
1.2. Sort a[left…p–1] into ascending order.
a[p] now
1.3. Sort a[p+1…right] into ascending order.
contains
2. Terminate.
the pivot
3-47
Quick-sort (3)
 Invariants:
After
step
1.1:
After
step
1.2:
After
step
1.3:
left
p–1
p
p+1
right
a
less than or equal to
pivot (unsorted)
left
p–1
pivot
p
greater than or equal
to pivot (unsorted)
p+1
right
a
less than or equal to
pivot (sorted)
left
p–1
pivot
p
greater than or equal
to pivot (unsorted)
p+1
right
a
less than or equal to
pivot (sorted)
pivot
greater than or equal
to pivot (sorted)
3-48
Quick-sort (4)
Assume this takes a[left] as the pivot,
and doesn’t reorder the elements put
into the left or right subarray.
 Animation:
To sort a[left…right] into ascending order:
1. If left < right:
1.1. Partition a[left…right] such that
a[left…p–1] are all less than or equal to a[p], and
a[p+1…right] are all greater than or equal to a[p].
1.2. Sort a[left…p–1] into ascending order.
1.3. Sort a[p+1…right] into ascending order.
2. Terminate.
left = 0
1
2
3
4
5
a cow
fox
cat
cow
cat
dog
pig
fox
cat
goat
pig
rat
lion
rat
6
7
8 = right
tiger
lion
pig tiger
goat
rat tiger
goat
dog
p
3
3-49
Quick-sort (5)
 Analysis (counting comparisons):
Let n be the length of the (sub)array to be sorted.
Let the total no. of comparisons required to sort a
(sub)array of length m be comps(m).
Assume that step 1.1 takes about n–1
comparisons to partition the array. (It’s not a
sort!)
3-50
Quick-sort (6)
 In the best case, the pivot always turns out to be
the median element. So the left and right
subarrays both have length about n/2. Steps 1.2
and 1.3 perform about comps(n/2) comparisons
each.
Therefore:
comps(n)  2 comps(n/2) + n – 1 if n > 1
comps(n) = 0
if n  1
Solution:
comps(n)  n log2n
Best-case time complexity is O(n log n).
3-51
Quick-sort (7)
 In the worst case, the pivot always turns out to
be the smallest element. So the right subarray
has length n–1 whilst the left subarray is empty.
Step 1.3 performs comps(n–1) comparisons, but
step 1.2 does nothing at all.
Therefore:
comps(n)  comps(n–1) + n – 1 if n > 1
comps(n) = 0
if n  1
Solution:
comps(n)  (n2 – n)/2  n2/2
Worst-case time complexity is O(n2).
 This case arises if the array is already sorted!
3-52
Quick-sort (8)
 Implementation in Java:
static void sort (Comparable[] a,
int left, int right) {
// Sort a[left…right] into ascending order.
if (left < right) {
int p = partition(a, left, right);
sort(a, left, p-1);
sort(a, p+1, right);
}
}
3-53
Quick-sort partitioning (1)
 Quick-sort partitioning algorithm:
To partition a[left…right] such that a[left…p–1] are all less than
or equal to a[p], and a[p+1…right] are all greater than or equal to
a[p]:
1. Let pivot be the element in a[left], and set p = left.
2. For r = left+1, …, right, repeat:
2.1. If a[r] is less than pivot:
2.1.1. Copy a[r] into a[p], a[p+1] into a[r],
and pivot into a[p+1].
2.1.2. Increment p.
3. Terminate yielding p.
3-54
Quick-sort partitioning (2)
 Loop invariant:
left
p–1
p
p+1
r–1
r
right
a
less than or equal to pivot
pivot (unsorted)
greater than or equal
to pivot (unsorted)
still to be partitioned
 Note that other (and better) partitioning
algorithms exist.
3-55
Comparison of sorting algorithms
Algorithm
No. of comparisons
No. of
copies
Time
complexity
Space
complexity
Selection sort
~ n2/2
~ 2n
O(n2)
O(1)
Insertion sort
~ n2/4
~ n2/4
O(n2)
O(1)
Merge-sort
~ n log2n
~ 2n log2n
O(n log n)
O(n)
Quick-sort (best)
(worst)
~ n log2n
~ n2/2
~ 2n/3 log2n
0
O(n log n)
O(n2)
O(log n)
O(n)
3-56