CHAPTER 22
ANALYSIS OF ALGORITHMS
Term




A term is a number like 34, a variable like x, or the product of numbers and variables like 3x, x2, or
4xy.
The degree of a term is the number of variables in that term. Thus, 3x is ________________, 5xy is
________________, 10 is ________________, 2x2 is ________________, 9x2y is _______________.
A zero degree term is called constant, a first degree term linear, a second degree term quadratic, a third
degree term cubic.
The numerical part of a term, usually written before the variable part, is called the coefficient.
Polynomials







A polynomial is a term or a sum of terms.
The degree of a polynomial is the largest degree of any of its terms.
e.g. 34+x
e.g. 5x+6xy
Given a positive integer n, a polynomial function in x of degree n is:
f(x)=anxn + an-1xn-1 + an-2xn-2 + … + a2x2 + a1x1 + a0 (the ai's are real coefficients, an0)
A polynomial function in x of degree 0 is called a constant function.
A polynomial function in x of degree 1 is called a linear function.
A polynomial function in x of degree 2 is called a quadratic function.
A polynomial function in x of degree 3 is called a cubic function.
Exponentials

An exponential function in x has the form f(x) = kbx, where the base b is any positive real number not
equal to 1 and k is any real number not equal to 0.
Logarithms




Let bx=N where b and N are real numbers and x is unknown.
The solution of bx=N can be phrased as: "x=expbN" or "x=logbN".
For positive b and A, b1, x= logbA means ________. e.g. 2= log525 means ___________
A logarithmic function in x has the form f(x) = a*logb[c(x+d)]+e where a, c, d and e are real constants
and b is the base of the logarithm
The growth rate
log
x f(x)=log x
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30



0.00
1.00
1.58
2.00
2.32
2.58
2.81
3.00
3.17
3.32
3.46
3.58
3.70
3.81
3.91
4.00
4.09
4.17
4.25
4.32
4.39
4.46
4.52
4.58
4.64
4.70
4.75
4.81
4.86
4.91
linear quadratic
cubic exponential
f(x)=5x f(x)=2*x*x f(x)=x*x*x f(x)=pow(2,x)
5
10
15
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95
100
105
110
115
120
125
130
135
140
145
150
2
8
18
32
50
72
98
128
162
200
242
288
338
392
450
512
578
648
722
800
882
968
1058
1152
1250
1352
1458
1568
1682
1800
1
8
27
64
125
216
343
512
729
1000
1331
1728
2197
2744
3375
4096
4913
5832
6859
8000
9261
10648
12167
13824
15625
17576
19683
21952
24389
27000
2
4
8
16
32
64
128
256
512
1024
2048
4096
8192
16384
32768
65536
131072
262144
524288
1048576
2097152
4194304
8388608
16777216
33554432
67108864
134217728
268435456
536870912
1073741824
18000.00
16000.00
14000.00
12000.00
10000.00
8000.00
6000.00
4000.00
2000.00
0.00
1
2
3
log
4
5
linear
6
7
8
quadratic
9
10
cubic
11
12
exponential
The growth rate of exponential>cubic>quadratic>linear>log
The growth rate of various functions: when x is doubled
f(x)=log(x)
f(x)=x
f(x)=x2
f(x)=x3
f(x)=2x
When x is large, f(x) of exponential>cubic>quadratic>linear>log, regardless of what the functions
actually look like.
Run time in terms of actual seconds is meaningless

BubbleSort takes 4 sec to sort a set of random integers. This is meaningless. Is it good? Is it bad?
Run time in terms of input size


13
Run BubbleSort several times with different input sizes, n=10000, 20000, 40000, 80000, on random
data
o Time it (2.5 sec, 10.22 sec, 41.1 sec, 170.2 sec)
o Plot the points: (input size, run time)
o Fit a curve
o The curve is quadratic, e.g. time=f(n)=4n2+n+3
o We say f(n)= 4n2+n+3 is the runtime function in n (input size) of the algorithm on that particular
computer.
Run BubbleSort several times on a different computer with different input sizes, n=10000, 20000,
40000, 80000, on random data
o Time it (1.1 sec, 4.22 sec, 17.1 sec, 70.2 sec)
o Plot the points: (input size, run time)
o Fit a curve
14
o
o

The curve is quadratic, e.g. 3n2+2n+3
We say f(n)= 3n2+2n+3is the runtime function in n (input size) of the algorithm on that particular
computer.
Repeat on any machine.
o While the runtime function of BubbleSort varies on different computers, it is always __________.
o We say the runtime behavior of BubbleSort is __________.
For large input size, if we double the input, the run time __________.
Comparing algorithms



Suppose the runtime function of algoA is n2 and algoB is 2n2. That is, A and B differ by a
proportionality constant.
o This means that on the same machine, if A takes 1 sec, B takes ______; if A takes 1 hour, B takes
______.
Suppose the runtime function of algoA is n2 and algoB is 100n2. That is, A and B differ by a much
larger proportionality constant.
o A is ______ times faster than B.
o Buy a new machine which runs at a speed one hundred times faster than the old, B running on the
new will be as fast as A running on the old
o Given the computer speed these days, proportionality difference between algorithms has little
practical impact.
o Thus, in terms of efficiency, A and B are in the same class.
Suppose the runtime function of algoA is n2 and algoB is n3.
o A is faster than B. If n=4, A takes 16 sec, and B takes _____ sec.
o Buy a new machine which runs at a speed ____ times faster than the old, B will be as fast as A
running on the old only when n=4. If n=8, A takes 64 sec, and B takes _______ sec. We will
need a new machine ___ times faster to have B running on the new as fast as A running on the old.
o The point is, it does not matter how much faster the new machine is, for sufficiently large n, the
new computer (say, 10000 times faster) is not going to help B to catch up with A running on the
old.
o Thus, in terms of efficiency, A and B are not in the same class.
Big-O




Computer scientists use the Big O notation to abbreviate for “order of magnitude.”
A function f(n) is O(g(n)) if  2 positive constants k and c such that f(n)<=cg(n) for all n>=k.
o Informally, if a function f(n) is O(g(n)), then f(n) is bounded by cg(n) for large n.
We can prove that any constant function is O(1), any logarithm function is O(log n), any linear
function is O(n), any quadratic function is O(n2), any cubic function is O(n3), any exponential function
is O(2n).
To find the Big-O for a function, drop all insignificant terms, and drop leading coefficient.
o 3n2+2n+3 is O(________)
o 3n3+2n+3 is O(________)
o 2x+2n+3 is O(________)
o log n+3 is O(________)
o 33nlog n+3 is O(________)
Computational complexity using BigO


We will describe the runtime behavior of a method by using Big-O. This measure is independent of
any computer system and depends solely on the design and implementation of the algorithm. We call
this the computational complexity of an algorithm.
Why can we say the computational complexity of algorithm A with runtime function 3n 2+2n+3 is
O(n2)?
o When n is small, all algorithms are practical. So we only consider large n.
o
o




When n is large, insignificant terms can be dropped.
Algorithm analysis focuses on growth rate. The multiplicative constants have no impact on growth
rates. Thus, the leading coefficient can be dropped.
When n is large, run time of O(2n) > O(n3) > O(n2) > O(n log n) > O(n) > O(log n)>O(1).
o The Big O notation estimates the execution time of an algorithm in relation to the input size n. If
the time is not related to the input size, the algorithm is said to take constant time with the notation
_______.
When we say an algorithm is O(1), its runtime function is _________. When we say an algorithm is
O(log n), its runtime function is _________. When we say an algorithm is O(n), its runtime function
is _________. When we say an algorithm is O(n2), its runtime function is _________. When we say
an algorithm is O(n3), its runtime function is _________. When we say an algorithm is O(2n), its
runtime function is _________.
When comparing algorithms that belong to different Big-O classes, the Big-O notation is good enough.
However, when comparing algorithms that belong to the same Big-O class, we need to compare the
entire runtime functions of the algorithms (derived from using the same machine).
Let us assume there are 6 algorithms, with runtime functions 2 n, n3, n2, n, n log n, log n respectively.
Since the runtime of a function is directly proportional to the number of instructions executed on
behalf of the function, let us assume that these runtime functions denote the number of instructions
executed on behalf of the functions. Let us assume an instruction takes 0.000001 second (1 microsec)
to execute on a particular computer. The following table shows the time it would take for each of these
algorithms to run to completion with a particular input size on that computer.
n=1,000
n=100,000
n=1,000,000
log n
0.00001 sec
0.000017 sec
0.00002 sec
n
0.001 sec
0.1 sec
1 sec
n log n
0.01 sec
1.7 sec
20 sec
n2
1 sec
3 hr
12 day
n3
17 min
32 centuries
30000 centuries
2n
10285 centuries
109998 centuries
1099998 centuries
Deriving Big-O by statement count




public static void f()
{
System.out.println("hello");
}
public static void f()
{
System.out.println("hello");
System.out.println("hello");
}
public static void f(int n)
{
System.out.println("hello");
for (int k=0; k<n; ++k)
System.out.println("hello");
}
public static void f(int n)
{
System.out.println("hello");
for (int k=0; k<n; ++k)
{
System.out.println("hello");
System.out.println("hello");
}
}







public static void f(int n)
{
System.out.println("hello");
for (int k=0; k<n; ++k)
for (int j=0; j<n; ++j)
System.out.println("hello");
}
public static void f(int n)
{
System.out.println("hello");
for (int k=0; k<n; ++k)
{
System.out.println("hello");
for (int j=0; j<n; ++j)
{
System.out.println("hello");
System.out.println("hello");
}
}
}
public static void f(int n)
{
System.out.println("hello");
for (int k=0; k<n/2; ++k)
System.out.println("hello");
}
public static void f(int n)
{
System.out.println("hello");
for (int k=1; k<=n; k*=2)
System.out.println("hello");
}
public static void f(int n)
{
System.out.println("hello");
for (int k=0; k<n; ++k)
for (int j=1; j<=n; j*=2)
System.out.println("hello");
}
public static void f(int n)
{
System.out.println("hello");
for (int k=0; k<n; ++k)
for (int j=1; j<=n; j*=2)
System.out.println("hello");
for (int k=0; k<n; ++k)
System.out.println("hello");
}
public static void f(int n)
{
System.out.println("hello");
for (int k=0; k<n; ++k)
for (int j=1; j<=n; j*=2)
System.out.println("hello");
for (int k=0; k<n; ++k)
for (int j=0; j<n; j++)
System.out.println("hello");





}
So far, all our algorithms' execution times do not vary on the same input size. However, for many
algorithms such as bubbleSort, their execution times do vary even on the same input size.
o An input that results in the shortest execution time is called the best-case input
o An input of the same size that results in the longest execution time is called the worst-case input.
o An average-case analysis attempts to determine the average amount of time among all possible
input of the same size
public static void bubbleSort(int[] arr)
{
int pass, i, temp;
//best
worst
average
for (pass = 1; pass<arr.length; ++pass)
for (i = 0; i< arr.length -pass; ++i)
if (arr[i] > arr[i+1]) //compares
{
temp=arr[i];
//swaps
arr[i]=arr[i+1];
arr[i+1]=temp;
}
}
public static void selectionSort(int[] arr)
{
int pass, min, i, temp;
//best
worst
average
for (pass=0; pass<arr.length-1; ++pass)
{
min=pass;
for (i=pass+1; i< arr.length; ++i)
if (arr[i]<arr[min]) //compares
min=i;
temp=arr[min];
//swaps
arr[min]=arr[pass];
arr[pass]=temp;
}
}
public static void insertionSort(int[] arr)
{
int pass, pulled, i;
//best
worst
average
for (pass=1; pass<arr.length; ++pass)
{
pulled=arr[pass];
i=pass-1;
while (i>=0)
{
if (pulled<arr[i]) //compares
{
arr[i+1]=arr[i]; //swap
i=i-1;
}
else break;
}
arr[i+1]=pulled;
}
}
public static int seqSearch (int[] arr, int key)
{
//best
worst
average
int index=0;
while (index <arr.length)
if (key==arr[index])
return index;
else index ++;
return(-1);






}
public static int binSearch (int[] arr, int key)
{
int low=0, high=arr.length-1, mid;
//best
worst
average
while (low<=high)
{
mid=(low+high)/2;
if (key==arr[mid])
return mid;
else if (key<arr[mid])
high=mid-1;
else low=mid+1;
}
return (-1);
}
We are interested in the runtime behavior of an algorithm for any large n in general, and not for a
specific n such as 1.
For any large n, if the runtime behavior of an algorithm differs depending on the distribution of the n
pieces of data, we are then interested in the best, worst, and average case analysis.
When we say the best case of an algorithm is O(1), it means that in the best case scenario the runtime
is constant regardless of what n is.
It is wrong to say that the best case of an algorithm is achieved when n is 1.
To find the best case or the worst case of an algorithm, we should always consider a nontrivial n.
Space efficiency






Algorithms use memory to store and process data. Space efficiency measures the relative amount of
internal memory used by algorithms.
The space efficiency of bubbleSort is
The space efficiency of selectionSort is
The space efficiency of insertionSort is
The space efficiency of seqSearch is
The space efficiency of binSearch is
Class StopWatch
-------------------------------------------------------------------------------------------------------------------/**
Class StopWatch supports objects representing a stop watch for measuring the time required to
execute a process. Time is measured in seconds.
*/
public class StopWatch
{
private long startTime, stopTime;
/**
Initializes this StopWatch object<p>
<b>Post:</b><br> startTime and stopTime of this StopWatch object initialized to 0<p>
*/
public StopWatch()
{
startTime = stopTime = 0;
}
/**
Starts this StopWatch <p>
<b>Post:</b><br> startTime of this StopWatch set to the current system clock time in
nanosecond <p>
*/
public void start()
{
startTime = System.nanoTime();
}
/**
Stops this StopWatch <p>
<b>Post:</b><br> stopTime of this StopWatch set to the current system clock time in
nanosecond <p>
<b>Return:</b><br>Interval of time from start to stop measured in seconds.
*/
public double stop()
{
stopTime = System.nanoTime();
return (stopTime - startTime)/1000000000.0;
}
}
------------------------------------------------------------------------------------------------------------------- Figure 22-1. Class StopWatch.

System.nanoTime returns the current value of the system clock, in nanoseconds. This method can only
be used to measure elapsed time and is not related to any other notion of system or wall-clock time.
-------------------------------------------------------------------------------------------------------------------/**
The IntArray class contains useful methods to manipulate int arrays.
*/
public class IntArray
{
/**
Sorts an int array using bubble sort<p>
<b>Post:</b><br> arr sorted in ascending order <p>
*/
public static void bubbleSort(int[] arr)
{
int pass, i, temp;
for (pass = 1; pass<arr.length; ++pass)
for (i = 0; i< arr.length -pass; ++i)
if (arr[i] > arr[i+1])
{
temp=arr[i];
arr[i]=arr[i+1];
arr[i+1]=temp;
}
}
/**
Sorts an int array using selcetion sort<p>
<b>Post:</b><br> arr sorted in ascending order <p>
*/
public static void selectionSort(int[] arr)
{
int pass, min, i, temp;
for (pass=0; pass<arr.length-1; ++pass)
{
min=pass;
for (i=pass+1; i< arr.length; ++i)
if (arr[i]<arr[min])
min=i;
temp=arr[min];
arr[min]=arr[pass];
arr[pass]=temp;
}
}
/**
Sorts an int array using insertion sort<p>
<b>Post:</b><br> arr sorted in ascending order <p>
*/
public static void insertionSort(int[] arr)
{
int pass, pulled, i;
for (pass=1; pass<arr.length; ++pass)
{
pulled=arr[pass];
i=pass-1;
while (i>=0)
{
if (pulled<arr[i])
{
arr[i+1]=arr[i];
i=i-1;
}
else break;
}
arr[i+1]=pulled;
}
}
/**
Searches an int array using sequential search for a key<p>
<b>Return:</b><br>index of key in arr if found, -1 if not found<p>
*/
public static int seqSearch (int[] arr, int key)
{
int index=0;
while (index <arr.length)
if (key==arr[index])
return index;
else index ++;
return(-1);
}
/**
Searches an int array using binary search for a key<p>
<b>Pre:</b><br> arr sorted in ascending order <p>
<b>Return:</b><br>index of key in arr if found, -1 if not found<p>
*/
public static int binSearch (int[] arr, int key)
{
int low=0, high=arr.length-1, mid;
while (low<=high)
{
mid=(low+high)/2;
if (key==arr[mid])
return mid;
else if (key<arr[mid])
high=mid-1;
else low=mid+1;
}
return (-1);
}
/**
Copies and int array to another int array<p>
<b>Pre:</b><br>source and destination have the same length<p>
<b>Post:</b><br>destination is an exact copy of source<p>
*/
public static void copyArr(int[] source, int[] destination)
{
for (int j=0; j< source.length; ++j)
destination [j]= source [j];
}
}
-------------------------------------------------------------------------------------------------------------------Figure 22-2. Class IntArray.
//Example 22-1
//Demo.java
import java.util.Random;
import java.util.Scanner;
public class Demo
{
public static void main(String[] args)
{
Scanner f=new Scanner(System.in);
final int SIZE=20000;
StopWatch sw = new StopWatch();
Random rnd = new Random();
int[] arrOriginal=new int[SIZE];
int[] arr= new int[SIZE];
for (int i = 0; i < SIZE; i++)
arrOriginal [i] = rnd.nextInt(100000);
System.out.println("Array size: "+SIZE);
IntArray.copyArr(arrOriginal, arr);
sw.start();
IntArray.bubbleSort(arr);
System.out.printf("%s%.6f\n", "bubbleSort: ", sw.stop());
IntArray.copyArr(arrOriginal, arr);
sw.start();
IntArray.selectionSort(arr);
System.out.printf("%s%.6f\n", "selectSort: ", sw.stop());
IntArray.copyArr(arrOriginal, arr);
sw.start();
IntArray.insertionSort(arr);
System.out.printf("%s%.6f\n", "insertionSort: ", sw.stop());
System.out.print("enter key:");
int key=f.nextInt();
sw.start();
int index= IntArray.seqSearch(arrOriginal, key);
System.out.printf("%s%.6f", "seqSearch: ", sw.stop());
if (index!=-1) System.out.println(" Found key at: "+index);
else System.out.println(" Key not found");
sw.start();
index= IntArray.binSearch(arr, key);
System.out.printf("%s%.6f", "binSearch: ", sw.stop());
if (index!=-1) System.out.println(" Found key at: "+index);
else System.out.println(" Key not found");
}
}
Change SIZE to 40000:
Change SIZE to 80000:
Programming Assignment
Part I
Do the following:
1. I have 3 public static sorting methods sortA, sortB, sortC in class HWSort. I have compiled class
HWSort, and HWSort.class is on my web site. Download HWSort.class and put it in your working
directory. Note that HWSort.class is generated using JDK 1.8. It might not work if your system is
using a different version of the JDK. (It should work in the computer lab.)
2. Write a simple program to time the performance of sortA, sortB, sortC (similar to Example 22-1) on
random data. The API of these 3 sorting algorithms are as follows:
// Desc : Sort an array of integers into ascending order.
// Post : The elements of arr sorted in ascending order.
public static void sortA(int[] arr)
// Desc : Sort an array of integers into ascending order.
// Post : The elements of arr sorted in ascending order.
public static void sortB(int[] arr)
3.
4.
// Desc : Sort an array of integers into ascending order.
// Post : The elements of arr sorted in ascending order.
public static void sortC (int[] arr)
Run the program at least 6 times, for SIZE=20000, 40000, 80000, 160000, 320000, 640000.
On the printout, list the observed run times, and deduce the BIG-O of these 3 sorting algorithm on
random data by filling out the following table:
sortA
sortB
sortC
20000
40000
80000
160000
320000
640000
Big-O
Hand in/email:
 BigORandom.java (class BigORandom has only 1 method, main)
Part II
1.
2.
Repeat Part I, but use an already sorted array as test data (i.e. an array whose elements are in ascending
order already).
On the printout, list the observed run times, and deduce the BIG-O of these 3 sorting algorithm on
sorted data by filling out the following table:
sortA
sortB
sortC
20000
40000
80000
160000
320000
640000
Big-O
Hand in/email:
 BigOAscending.java (class BigOAscending has only 1 method, main)
Part III
If a sorting algorithm is quadratic, explain whether it should be insertionSort, selectionSort, or bubbleSort
and why. If it is not quadratic, no comment is necessary.
Note:
 These 3 parts are all equal in points. So give your reasoning for Part III with some thoughts.