Analysis & Design of
Algorithms
(CSCE 321)
Prof. Amr Goneid
Department of Computer Science, AUC
Part 3. Time Complexity
Calculations
Prof. Amr Goneid, AUC
1
Time Complexity Calculations
Prof. Amr Goneid, AUC
2
Time Complexity Calculations
 Performance Measurement & Modeling
 Performance Measurement
 Performance Analysis (modeling)
 Evaluating Number of Operations T(n)
 Examples of Algorithm Analysis
Prof. Amr Goneid, AUC
3
1. Performance Measurement
and Modeling
Algorithm
Actual or Pseudo
code
Actual Code
Measurement
Runs
Math. Model
T(n) and
Bounds
Prof. Amr Goneid, AUC
4
2. Performance Measurement
We measure the performance of an algorithm by time
measurement or by counting number of operations.
 Usually called Amortized Analysis:
Repeat measuring the running time m times then
divided by m.
Important issues are:
 Choice of Bound
 Clocking or counting domains
 Choice of Data Sizes
 Choice of Data Set for the Bound
Prof. Amr Goneid, AUC
5
Time Measurement Example
Set repetition count m;
Set problem size n;
tacc = 0;
Repeat on m {
start time t1 = GetTime();
Invoke A(n);
end time t2 = GetTime();
tacc = tacc + (t2 - t1);
}
tav = tacc / m;
Prof. Amr Goneid, AUC
6
Counting Example 1
int count = 0;
float s = 0.0;
float A[n][n];
for (i=1; i<=n; i++)
for (j = 1, j<=n; j++)
{ s = s + A[i][j]; count++;}
The above code finds the no. of floating point
additions T(n) = count as a function of n.
Prof. Amr Goneid, AUC
7
Counting Example 2:
Insertion Sort Function
void insertion (int a[ ], int n)
{
int i , v , j ;
for (i =1; i < n; i++)
{
v = a[i]; j = i;
while(j > 0 && a[j-1] > v)
{ a[j] = a[j-1]; j--; count++;};
a[j] = v;
}
}
Prof. Amr Goneid, AUC
8
Insertion Sort (Performance)
T1 ordered, T2 Random, T3 Inversely ordered
n
5
10
20
50
100
200
T1(n)
0
0
0
0
0
0
T2(n)
3
18
78
586
2609 10,052
T3(n)
10
45
190
1225
4950 19,900
n(n-1)/2
10
45
190
1225
4950 19,900
Prof. Amr Goneid, AUC
9
T(n) vs n for insertsort
Prof. Amr Goneid, AUC
10
3. Performance Analysis (Modeling)
 Use a high-level Description (e.g





Pseudocode) instead of implementation
Identify problem size (n)
Identify elementary processes.
Assign a number of operations to each.
Form a mathematical model of the sum of
operations in these processes.
Solve to find T(n) independent of HW
Prof. Amr Goneid, AUC
11
Performance Analysis (Modeling)
Example: Find the sum of all elements in a 2-D array of size (nxn)
sum ← 0.0;
for (i = 0 to n-1)
for (j = 0 to n-1)
sum ← sum + aij
∑i
∑j
1
The no. of floating point additions T(n) as a function of
n is:
n 1 n 1
n1
n1
i 0 j 0
i 0
i 0
T( n )    1   n  n  1  n* n  n 2  O( n 2 )
Prof. Amr Goneid, AUC
12
4. Evaluating No. of Operations T(n)
 Usually we specify a certain type of
operations, e.g., additions, array
comparisons, etc.
 Simple Statements:
T(n) = no. of specified operations.
e.g. for z = 2*x + y T(n) = 2 arithmetic
operations
 Code blocks:
T(n) = Sum of sub-block T(n)’s
Prof. Amr Goneid, AUC
13
No. of Operations T(n)
Selection Statements:
c = condition, s = statement block
 if (c) s;
T(n) = Tc(n) for c false (best case)
= Tc(n) + Ts(n) for c true
(worst case)
 If (c) s1; else s2;
Worst case T(n) = Tc(n) + max(Ts1(n),Ts2(n))
Prof. Amr Goneid, AUC
14
No. of Operations T(n)
Example:
T(n) = no. of comparisons
Given that fun1(n) does n+2
comparisons, module1
does n2 and module2 does
n4.
Find worst case T(n) for the
segment:
if ( c == fun1(n)) call
module1;
else call module2;
Process
T(n)
==
1
fun1(n)
n+2
module1
n2
module2
n4
Worst case
total
?
Prof. Amr Goneid, AUC
15
No. of Operations T(n)
Process
T(n)
Compute
m
1
Repetition Statements:
 For Loop Example:
T(n) = No. of multiplications
Fun(i,n)
Fun(i,n) does 2i+n multiplications
Total
2i + n
T(n) = ?
m = n*n
for (i =1; i <= m; i++)
Fun(i,n);
Prof. Amr Goneid, AUC
16
No. of Operations T(n)
Repetition Statements:
 while Loop Example:
In the following code:
Fun(i,n) does 3i + log n comparisons, find
T(n) = no. of comparisons
i = 0;
while(i <= n)
{Fun(i,n); i++;}
Prof. Amr Goneid, AUC
17
Running Time
 The number of operations can be used to predict the
running time of an algorithm for a given problem size (n)
 Example:
The number of operations of a sorting algorithm is directly
proportional to (n log n). Direct time measurement gives 1
ms to sort 1000 items. Find how long it will take to sort
1,000,000 items.
Solution:
T ( N )  cN log N , c  T ( N ) / ( N log N )
6
6
10
log
10
n log n
T ( n)  cn log n 
T ( N ), T (106 )  3 10 3 T (103 )  2 sec
N log N
10 log10 10
Prof. Amr Goneid, AUC
18
5. Examples of Algorithm Analysis
(a) Uniqueness Test
Check whether all the elements in a given
array are distinct
 Input:
An array A[0…n-1]
 Output: Return “true” if all the elements in
A are distinct and “false” otherwise
Prof. Amr Goneid, AUC
19
Uniqueness Test
ALGORITHM UniqueElem ents( A[0..n  1])
for i  0 to n-2 do
for j  i  1 to n-1 do
if A[i ]  A[ j ] return false
return true
n  2 n 1
n 2
n 1
i  0 j  i 1
i 0
i 1
T( n )  
2
1

(
n

1

i
)

i

n(
n

1
)
/
2


(
n
)
 

Prof. Amr Goneid, AUC
20
Exercise
Prove the formula:
n
 i  n( n  1 ) / 2
i 1
either by mathematical induction or by following
the insight of a 10year-old school boy named
Carl Friedrich Gauss (1777–1855) who grew up
to become one of the greatest mathematicians
of all times.
Prof. Amr Goneid, AUC
21
(b) More Nested Loops
 Find the number of double arithmetic operations
done by the following piece of code assuming that n
= 2m:
for( int t = n; t > 1; t /= 2 ) {
for( int u = 1; u < n; u *= 2 ) {
for( int v = 0; v < n; v += 2 ) {
... // constant number of double arithmetic
//operations
}
}
}
Prof. Amr Goneid, AUC
22
More Nested Loops
The loop variables can be transformed to:
Outermost loop: i = log t = m … 1,
Middle loop: j = log u = 0 … m-1
innermost loop: k = v/2 = 0 .. n/2-1
m m 1 n /21
T (n)  
2
2
2
c

c
(
n
/
2)
m

O
(
nm
)

O
(
n
(log
n
)
)

i 1 j  0 k  0
Prof. Amr Goneid, AUC
23
(c) Cosine Function Evaluation
The cosine function can be evaluated using a truncated infinite
series:
(  x 2 )i
cos( x )  1  
i 1 ( 2i )!
n
The corresponding algorithm is:
float cosine (float x, int n)
{
float y = -x * x; float s = 1.0;
for (int i = 1; i <= n; i++)
s = s + pow (y,i) / fact (2*i);
return s;
}
Prof. Amr Goneid, AUC
24
Cosine Function Evaluation
The number of arithmetic operations is evaluated as follows:
 fact (2*i ) uses one mult. + 2i mult. = 2i+1
 pow(y,i) uses (i-1) mult.
 1 division and 1 addition inside loop
 1 mult. outside loop (-x * x)
Show that:
T ( n )  1  3.5n  1.5n  O( n )
2
2
i.e., Quadratic algorithm.
Prof. Amr Goneid, AUC
25
A Faster Cosine Algorithm
We express the series in the form
n
(  x 2 )i
cos( x )  
  s( x,i )
i  0 ( 2i )!
i 0
n
where S(x,i) = (-x2)i / (2i)! , and S(x,0) = 1
Expressing S(x,i+1) in terms of S(x,i), then
(  x2 )
y
s( x,i  1 )  s( x,i )
 s( x,i )
( 2i  1 )( 2i  2 )
( m  1 )( m  2 )
where y = (-x2) and m = 2i
with S(x,0) = 1
Prof. Amr Goneid, AUC
26
A Faster Cosine Algorithm
Show that in this case:
n
T ( n)  1   7  1  7 n  O ( n)
i 1
This is a linear algorithm that is much faster
than the previous quadratic algorithm
Prof. Amr Goneid, AUC
27
(d) Maximum Subsequence Sum
Problem Statement
 Given a sequence of integers (possibly
negative), a1,a2,...,an, find the maximum value
of S  j a
max
k i
k
(This is zero if all integers are negative).
Example: -2, 11, -4, 13, -5, -2,
Smax = a2+a3+a4= 20
Prof. Amr Goneid, AUC
28
Maximum Subsequence Sum
Algorithm 1
S max  0 ; im  0 ; jm  0 ;
i 1 n
for
for
jin
j
S ij   ak
k i
if ( S ij  S max ){ S max  S ij ; im  i ; jm  j ; }
Prof. Amr Goneid, AUC
29
Maximum Subsequence Sum
Algorithm 1 (Analysis)
T ( n ) # of additions
n
n
j
n
n
n
T ( n )   1   ( j  i  1 )  ( 1 / 2 ) i ( i  1 )
i 1 j  i k  i
i 1 j  i
i 1
n
n

2
 ( 1 / 2 )  i   i   n(n+1 )( 2n+1 )/ 12  n(n+1 )/ 4
i 1
 i 1

T ( n )  ( 1 / 6 )( n 3  3n 2  2n )  O( n 3 )
The algorithm is Cubic
Prof. Amr Goneid, AUC
30
Maximum Subsequence Sum
Algorithm 2
S max  0 ; im  0 ; jm  0 ;
i 1 n
for
S ij  0
for
jin
S ij  S ij  a j
if ( S ij  S max ){ S max  S ij ; im  i ; jm  j ; }
Prof. Amr Goneid, AUC
31
Maximum Subsequence Sum
Algorithm 2 (Analysis)
T ( n ) # of additions
n
n
n
n
i 1
i 1
T ( n )   1   ( n  i  1 )   i  n( n  1 ) / 2
i 1 j  i
T ( n )  ( 1 / 2 )( n 2  n )  O( n 2 )
The algorithm is Quadratic
Prof. Amr Goneid, AUC
32
Maximum Subsequence Sum
Algorithm 3
i  1; S ij  0 ; S max  0 ; im  0 ; jm  0 ;
for
j 1 n
S ij  S ij  a j
if ( S ij  S max ){ S max  S ij ; im  i ; jm  j ; }
else if ( S ij  0 ){ i  j  1; S ij  0 ; }
Prof. Amr Goneid, AUC
33
Maximum Subsequence Sum
Algorithm 3 (Analysis)
T ( n ) # of additions( worst case )
n
T ( n )   2  2n
j 1
T ( n )  O( n )
The algorithm is Linear
Prof. Amr Goneid, AUC
34
(e) Histogram Processing
A bad contrast image (F) can be enhanced to an image G by
transforming every pixel intensity fij to another intensity
gij using a method called Histogram Equalization
Prof. Amr Goneid, AUC
35
Histogram Processing
 In practice, gray levels are discrete (k = 0 .. L-1)
 A histogram is an array whose element nk = no. of
pixels with gray level k (k=0 is black, k=L-1 is white).
 Algorithm to build histogram for an image with N rows
and M columns (Ntot = NM pixels):
Let rmax  L  1
for ( k  0  rmax ) nk  0 ;
for ( i  1  N )
for ( j  1  M )
k  f ij ;
nk  nk  1
Complexity  no . of arithmetic operations
Thist ( N tot )  NM  N tot
Prof. Amr Goneid, AUC
36
Histogram Equalization
Algorithm (1)
The histogram is used in the following algorithm:
for ( i  1  N )
for ( j  1  M )
r  f ij ;
r
s  (  nk ) * rmax / N tot
k 0
Complexity  no . of arithmetic operations
Best Case ( r  0 ),
Tequ ( N tot )  3 NM  3 N tot
Worst Case ( r  rmax ),
Tequ ( N tot )  ( L  2 ) N tot
Show that the Average Case ( with equal probability pr  1 / L )
gives : Tequ ( N tot )  {( L  5 ) / 2 } N tot
Prof. Amr Goneid, AUC
37
Histogram Equalization
Algorithm (2)
A better algorithm first builds a Cumulative array:
sum  n0 ; C 0  sum ;
for ( k  1  rmax )
sum  sum  nk ; C k  sum
no . of arithmetic operations  rmax  L  1
Then we use it :
for ( i  1  N )
for ( j  1  M )
r  f ij ;
gij  C r * rmax / N tot
Tequ ( N tot )  2 N tot  ( L  1 )
Prof. Amr Goneid, AUC
38
Histogram Equalization
Summary of Total Complexity
Algorithm (1):
Best Case ( r  0 ),
T ( N tot )  4 N tot
Worst Case ( r  rmax ),
T ( N tot )  ( L  3 ) N tot
Average Case T ( N tot )  {( L  7 ) / 2 } N tot
Algorithm (2):
All Cases T ( N tot )  3 N tot  ( L  1 )
Prof. Amr Goneid, AUC
39
Histogram Equalization
Numerical Example
L = 256 gray levels
Algorithm (1):
Best Case ( r  0 ),
T ( N tot )  4 N tot
Worst Case ( r  rmax ),
T ( N tot )  259N tot
Average Case T ( N tot )  131.5 N tot
Algorithm (2):
All Cases T ( N tot )  3 N tot  255
Prof. Amr Goneid, AUC
40