Honors CS 102
Sept. 4, 2007
Asymptotic Analysis of Algorithms
(how to evaluate your programming power tools)
based on presentation material by
Mike Scott
CS 307 Fundamentals of
Computer Science
1
Algorithmic Analysis
A technique used to characterize the
execution behavior of algorithms in a manner
independent of a particular platform,
compiler, or language.
Abstract away the minor variations and
describe the performance of algorithms in a
more theoretical, processor independent
fashion.
A method to compare speed of algorithms
against one another.
CS 307 Fundamentals of
Computer Science
2
Different Algorithms
Why are the words in a dictionary in
alphabetical order?
A brute force approach
– linear search
– worst case is (d x N)
Another way
– divide and conquer
– worst case is (c x log N)
Constants are unknown and largely
irrelevant.
CS 307 Fundamentals of
Computer Science
3
Big O
The most common method and notation for
discussing the execution time of algorithms is
"Big O”.
For the alphabetized dictionary the algorithm
requires O(log N) steps.
For the unsorted list the algorithm requires
O(N) steps.
Big O is the asymptotic execution time of the
algorithm.
CS 307 Fundamentals of
Computer Science
4
Formal Definition of Big O
T(N) is O( F(N) ) if there are positive constants c
and N0 such that T(N) < cF(N) when N > N0
– There is a point N0 such that for all values of N
that are past this point, T(N) is bounded by some
multiple of F(N).
– Thus if T(N) of the algorithm is O( N2 ) then,
ignoring constants, at some point we can bound
the running time by a quadratic function of the
input size.
– Given a linear algorithm, it is technically correct
to say the running time is O(N2). O(N) is a more
precise answer as to the Big O bound of a linear
algorithm.
CS 307 Fundamentals of
Computer Science
5
Big O Examples
3n3 = O(n3)
3n3 + 8 = O(n3)
8n2 + 10n * log(n) + 100n + 1020 = O(n2)
3log(n) + 2n1/2 = O(n1/2)
2100 = O(1)
TlinearSearch(n) = O(n)
TbinarySearch(n) = O(log(n))
CS 307 Fundamentals of
Computer Science
6
Other Algorithmic Analysis Tools
Big Omega T(N) is ( F(N) ) if there are
positive constants c and N0 such that
T(N) > cF( N )) when N > N0
– Big O is similar to less than or equal, an upper
bound.
– Big Omega is similar to greater than or equal, a
lower bound.
Big Theta T(N) is ( F(N) ) if and only if T(N)
is O( F(N) )and T( N ) is ( F(N) ).
– Big Theta is similar to equals.
CS 307 Fundamentals of
Computer Science
7
Relative Rates of Growth
Analysis
Type
Mathematical
Expression
Big O
T(N) = O( F(N) )
Relative
Rates of
Growth
T(N) < F(N)
Big 
T(N) = ( F(N) )
T(N) > F(N)
Big 
T(N) = ( F(N) )
T(N) = F(N)
"In spite of the additional precision offered by Big Theta,
Big O is more commonly used, except by researchers
in the algorithms analysis field" - Mark Weiss
CS 307 Fundamentals of
Computer Science
8
What it All Means
T(N) is the actual growth rate of the
algorithm.
F(N) is the function that bounds the growth
rate.
– may be upper or lower bound
T(N) may not equal F(N).
– constants and lesser terms ignored because it is
a bounding function
CS 307 Fundamentals of
Computer Science
9
Assumptions in Big O Analysis
Once found accessing the value of a
primitive is constant time
x = y;
mathematical operations are constant time
x = y * 5 + z % 3;
if statement: constant time if test and
maximum time for each alternative are
constants
if( iMySuit ==DIAMONDS || iMySuit == HEARTS)
return RED;
else
return BLACK;
CS 307 Fundamentals of
Computer Science
10
Fixed-Size Loops
for(int suit = Card.CLUBS; suit <= Card.SPADES; suit++)
{
for(int value = Card.TWO; value <= Card.ACE; value++)
{
myCards[cardNum] = new Card(value, suit);
cardNum++;
}
}
Loops that perform a constant number of
iteration are considered to execute in
constant time. They don't depend on the size
of some data set
CS 307 Fundamentals of
Computer Science
11
Loops That Work on a Data Set
Loops like on the previous slide are fairly rare.
Normally a loop operates on a data set which can
vary is size.
public double minimum(double[] values)
{ int n = values.length;
double minValue = values[0];
for(int i = 1; i < n; i++)
if(values[i] < minValue)
minValue = values[i];
return minValue;
}
The number of executions of the loop depends on
the length of the array, values. The actual number
of executions is (length - 1).
The run time is O(N).
CS 307 Fundamentals of
Computer Science
12
Nested Loops
public void bubbleSort(double[] data)
{ int n = data.length;
for(int i = n - 1; i > 0; i--)
for(int j = 0; j < i; j++)
if(data[j] > data[j+1])
{
double temp = data[j];
data[j] = data[j + 1];
data[j + 1] = temp;
}
}
Number of executions?
CS 307 Fundamentals of
Computer Science
13
Summing Execution Times
If an algorithm’s execution time is N2 + N
then it is said to have O(N2) execution time,
not O(N2 + N).
When adding algorithmic complexities the
larger value dominates.
Formally, a function f(N) dominates a
function g(N) if there exists a constant value
n0 such that for all values N > N0 it is the
case that g(N) < f(N).
CS 307 Fundamentals of
Computer Science
14
Example of Dominance
Suppose we go for precision and determine
how fast an algorithm executes based on the
number of items in the data set.
x2/10000 + 2x log10 x + 100000
Is it plausible to say the x2 term dominates
even though it is divided by 10000?
What if we separate the equation into
(x2/10000 ) and (2x log x + 100000)?
CS 307 Fundamentals of
Computer Science
15
Summing Execution Times
For large values the x2 term dominates so
the algorithm is O(N2).
CS 307 Fundamentals of
Computer Science
16
Ranking of Algorithmic
Behaviors
Function
Common Name
N!
factorial
2N
Exponential
N d, d > 3
Polynomial
N3
Cubic
N2
Quadratic
N
N
N log N
N
Linear
N
Root - n
log N
Logarithmic
1
Constant
CS 307 Fundamentals of
Computer Science
17
Running Times
Assume N = 100,000 and processor speed
is 1,000,000 operations per second
Function
Running Time
2N
over 100 years
N3
31.7 years
N2
2.8 hours
N
N
31.6 seconds
N log N
1.2 seconds
N
0.1 seconds
N
3.2 x 10-4 seconds
log N
1.2 x 10-5 seconds
CS 307 Fundamentals of
Computer Science
18
Graphical Results
CS 307 Fundamentals of
Computer Science
19
Determining Big O
A DNA problem
Is a number prime?
Hand actions
– inserting card
– determining if card is in Hand
– combining Hands
– copying Hand
– retrieving Card
CS 307 Fundamentals of
Computer Science
20
Putting it in Context
Why worst-case analysis? Wouldn’t
average-case analysis be more useful?
A1: No. E.g., doesn’t matter if a cashier’s
terminal handles 2 transactions per second
or 3, as long as it never takes more than 20.
A2: Well, OK, sometimes, yes. However,
this often requires much more sophisticated
mathematics. In fact, for some common
algorithms, nobody has been able to do an
average case analysis.
CS 307 Fundamentals of
Computer Science
21