Analysis of Algorithms
CS 105
Introduction to Data Structures
and Algorithms
1
What is a good algorithm?


It must be correct
It must be efficient

Implementations of an algorithm must run
as fast as possible
How is this measured?

Running time

2
Running time of a program


Amount of time required for a program
to execute for a given input
If measured experimentally,


Dependent on hardware, operating system
and software
Answer will be in milliseconds, seconds,
minutes, hours, …
3
Running time of an algorithm



Answer will not be in seconds or minutes
Instead, we count the number of operations
carried out
Result will be a formula in terms of some
input size, n


Takes into account all possible inputs
Example statement on running time:
Running time of algorithm A is T(n) = 2n + 3
4
Describing algorithms



First, we need to agree on how to describe or
specify an algorithm
Note: algorithms are intended for humans
(programs are intended for computers)
Descriptions should be high-level explanations
that combine natural language and familiar
programming structures: Pseudo-code
5
Pseudo-code example
Algorithm arrayMax(A,n):
Input: An array A storing n integers.
Output: The maximum element in A.
currentMax  A[0]
for i  1 to n - 1 do
if currentMax < A[i] then
currentMax  A[i]
return currentMax
6
Pseudo-Code conventions




General Algorithm Structure
Statements
Expressions
Control Structures
7
Algorithm Structure
Algorithm heading
Algorithm name(param1, param2,...):
Input : input elements
Output : output elements
statements…
8
Statements


Assignment: use  instead of =
Method or function call:



Return statement:


object.method(arguments)
method(arguments)
return expression
Control structures
9
Control Structures




decision structures

if ... then ... [else ...]
while loops

while ... do
repeat loops

repeat ... until ...
for loop

for ... do
10
Expressions

Standard math symbols
+

-
*
/
()
Relational operators
= > < <= >= !=

Boolean operators
and


or
not
Assignment operator 
Array indexing A[i]
11
General rules on pseudo-code

Should communicate high-level ideas
and not implementation details


Syntax is not as tight (e.g., indentation and
line-breaks take the place of ; and {} in
Java)
Clear and informative
12
Back to running time


Once an algorithm has been described
in pseudo-code, we can now count the
primitive operations carried out by
the algorithm
Primitive operations: assignment, calls,
arithmetic operations, comparisons,
array accesses, return statements, …
13
Back to arrayMax example



Suppose the input array is
A = {42, 6, 88, 53}, (n = 4)
How many primitive operations are carried
out?
Some notes


The for statement implies assignments,
comparisons, subtractions, and increments
the statement currentMax  A[i] will
sometimes not be carried out
14
What to count
currentMax  A[0]: assignment, array access
for i  1 to n - 1 do:
assignment,comparison,subtraction,increment (2 ops)
if currentMax < A[i] then: comparison, access
currentMax  A[i]: assignment, access
return currentMax: return
15
Counting operations
currentMax  A[0]
for i  1 to n - 1 do
if currentMax < A[i] then
currentMax  A[i]
return currentMax
2
15
6
2
1
OPERATIONS CARRIED OUT
26
16
Handling all possible inputs


In the example just carried out,
running time = 26 for the given input
A more useful answer is to provide a formula
that applies in general


Formula is in terms of n, the input or array size
Since there may be statements that do not
always execute, depending on input values,
there are two approaches


Assume worst-case
Provide a range
17
Measuring running time
comparison
& subtraction
increment
currentMax  A[0]
for i  1 to n - 1 do
if currentMax < A[i] then
currentMax  A[i]
return currentMax
2
1+2n+2(n-1)
2(n-1)
0 .. 2(n-1)
1
RUNNING TIME (worst-case)
8n - 2
18
Nested loops
// prints all pairs from the set {1,2,…n}
for i  1 to n do
for j  1 to n do
print i, j
How many times does the print statement
execute?
19
Nested loops
// prints all pairs from the set {1,2,…n}
for i  1 to n do
for j  1 to n do
print i, j
n2 times
How about the operations implied in the
for statement headers?
20
for statement revisited
for i  1 to n do
…
1 assignment
i1
n+1 comparisons i <= n
for each value of i in {1, 2, …, n+1}
n increments = n assignments + n adds
=> 3n+2 operations
21
Nested loops running time
for i  1 to n do
for j  1 to n do
print i, j
3n+2
n (3n+2)
n2
TOTAL:
4n2 + 5n + 2
22
Nested loops example 2
// prints all pairs from the set {1,2,…n}
// excluding redundancies (i.e., 1,1 is not a
// real pair, 1,2 and 2,1 are the same pair)
for i  1 to n-1 do
for j i+1 to n do
print i, j
How many times does the print statement
execute?
23
Nested loops example 2
for i  1 to n-1 do
for j i+1 to n do
print i, j
when
when
…
when
when
i = 1,
i = 2,
n-1 prints
n-2 prints
i = n-2,
i = n-1,
2 prints
1 print
24
Nested loops example 2
Number of executions of the print
statement:
1 + 2 + 3 + … + n-1
= n(n-1)/2 = ½n2 – ½n
25
Nested loops example 2
for i  1 to n-1 do
for j i+1 to n do
print i, j
Take-home exercise: count the
operations implied by the for statement
headers
26
Running time function
considerations



In determining the running time of an
algorithm, an exact formula is possible only
when the operations to be counted are
explicitly specified
Different answers will result if we choose not
to count particular operations
Problem: variations in the answers appear
arbitrary
27
Classifying running time functions



We want to classify running times under
particular function categories
Goal: some assessment of the running
time of the algorithm that eliminates
the arbitrariness of counting operations
Example: arrayMax runs in time
proportional to n
28
Some function categories







The
The
The
The
The
The
The
constant function:
linear function:
quadratic function:
cubic function:
exponential function:
logarithm function:
n log n function:
f(n)
f(n)
f(n)
f(n)
f(n)
f(n)
f(n)
=
=
=
=
=
=
=
c
n
n2
n3
bn
log n
n log n
29
Big-Oh notation

Consider the following (running time) functions
f1(n) = 2n + 3
f2(n) = 4n2 + 5n + 2
f3(n) = 5n


We place f1 and f3 under the same category (both are
proportional to n) and f2 under a separate category
(n2 )
We say:


2n+3 is O(n), and 5n is O(n), while 4n2 + 5n + 2 is O(n2)
2n+3 and 5n are linear functions,
while 4n2 + 5n + 2 is a quadratic function
30
Significance of Big-Oh


Example
for i  0 to n-1 do
A[i]  0
Running time is
2n, if we count the assignments & array accesses in
the loop body (A[i]  0)
4n+1, if we include implicit loop assignments/adds
5n+2, if we include implicit loop comparisons

Regardless, running time is O(n)
31
Prefix Averages Problem



Given an array X storing n numbers,
compute prefix averages or running
averages of the sequence of numbers
A[i] = average of X[0],X[1],…,X[i]
Example



Input:
Output:
X = {1,3,8,2}
A = {1,2,4,3.5}
Algorithm?
32
Algorithm 1
Algorithm prefixAverages1(A,n):
Input: An array X of n numbers.
Output: An array A containing prefix averages
for i  0 to n - 1 do
sum  0
for j  0 to i do
sum  sum + X[j]
A[i]  sum/(i+1)
return array A
33
Running time of Algorithm 1



Exercise: count primitive operations
carried out by Algorithm 1
Regardless of what you decide to count,
the answer will be of the form:
a n2 + b n + c
Observe that the answer is O(n2)
34
Algorithm 2
Algorithm prefixAverages2(A,n):
Input: An array X of n numbers.
Output: An array A containing prefix averages
sum  0
for i  0 to n - 1 do
sum  sum + X[j]
A[i]  sum/(i+1)
return array A
35
Running time of Algorithm 2



Running time of Algorithm 2 will be of
the form: a n + b
Observe that the answer is O(n)
Assessment: Algorithm 2 is more
efficient than Algorithm 1, particularly
for large input (observe the growth rate
of n and n2)
36
Summary


Algorithms are expressed in pseudo-code
The running time of an algorithm is the
number of operations it carries out in terms
of input size



Answer depends on which operation(s) we decide
to count
Big-Oh notation allows us to categorize
running times according to function
categories
Next: formal definition of Big-Oh notation
37