Complexity Analysis (Part I)
• Introduction to Algorithms and Complexity Analysis.
• Motivations for Complexity Analysis.
• Big-O Notation: An Introduction.
• Simple Complexity Analysis Examples.
• Average, Best, and Worst Cases.
Introduction to Algorithm & Complexity Analysis
• Let’s consider the execution time of some of the sorting
algorithms.
Introduction (Contd.)
• From a theoretical point of view, the execution time of the
tested algorithms is insufficient.
• So we need to define sound complexity metrics to give us a
complete view about the algorithm performance.
• Computer scientists have defined for such purposes several
•
efficiency indices such as:
–
–
–
–
–
Machine efficiency.
User efficiency.
Maintenance efficiency.
Algorithm complexity.
Coding efficiency.
Introduction (contd.)
• We will focus on the algorithm complexity measure. The main
concern is to highlight the relation between the size of the processed
input data and the algorithm complexity.
• The curves shown below indicate the number of operations required
to implement the algorithm given the size of the data (n).
Introduction (contd.)
• Let's inspect the algorithms with 2n and 10n complexities,
respectively.
Big-O Notation: an Introduction
• With the above concept of algorithm complexity, we can cast
algorithms into different complexity classes. Some common classes
are shown below:
Complexity Class
Big-O Notation
Constant
O(1)
Logarithmic
O(log n)
Linear
O(n)
Quadratic
O(n^2)
Exponential
O(2^n)
Big-O Notation (contd.)
• Question: Why do we need this notation?
• Answer: Let’s consider the following code fragment to perform the
sum of the integer elements in an array A. The array A has n
elements.
1
2
3
4
5
6
public int sum(int[] A) {
int sm = 0;
for(int i=0; i < A.length; i++)
sm += A[i];
return(sm);
}
• The number of operations performed in this method is equal to:
1+1+n+1+n(2+2)+1=5n+4
Big-O Notation (contd.)
• Let’s consider another code fragment to perform the sum of the
integer elements in a two-dimensional array B.
1 public int sum_all(int[][] B) {
2
int sm = 0;
3
for(int i=0; i < B.length; i++)
4
for(int j=0; j < B[0].length; j++)
5
sm += B[i][j];
6
return(sm);
7 }
• The number of operations performed in this method is equal to:
1+1+n+1+n(1+n+1+n(2+2)+2)+1=5n^2+5n+4
Big-O Notation (contd.)
• Now, we will consider a case of exponential class complexity.
• We will investigate matrix multiplication operation.
Number of columns in
matrix A EQUAL to the
number of rows in matrix B.
3 2 4 5
1 0 3 2
5 7 0 8
A
x
1
2
3
6
1
-2 3
2
0
1
3
0
0
1
3
B
4
=
23
8
14 45
7
2
8
19
47 17
9
56
Big-O Notation (contd.)
1 public int[][] matrixMult(int[][] A, int[][] B) \{
2
int r_a = A.length, c_a = A[0].length;
3
int r_b = B.length, c_b = B[0].length;
4
if(c_a != r_b) \{
5
System.out.pritln("ERROR! Matrix dimensions mismatch.");
6
System.exit(-1);
7
}
8
else \{
9
int[][] C = new int[r_a][c_b];
9
for(int i = 0; i < r_a; i++) \{
10
for(int j = 0; j < c_b; j++) \{
11
C[i][j] = 0;
12
for(int k = 0; k < r_b; k++)
13
C[i][j] += A[i][k] * B[k][j];
14
}
15
}
16
return(C);
17 }
18 } // End of matrixMult()
Average, Worst, and Best Cases
• An algorithm may run faster on certain data sets than others.
• Finding the average case can be very difficult, so typically
algorithms are measured in the worst case time complexity.
• Also, in certain application domains (e.g., air traffic control, medical,
etc.) knowing the worst case time complexity is of crucial
importance.
Average, Worst, and Best Cases (contd.)
• The analysis of the linear search algorithm illustrates the difficulties
in defining the algorithm complexity.
Key Value
Counter
Cost
2
-1
2
1
1 Op
0
-1
2
2 Op
3
9
7
7 Op
11
7
7 Op & Search {FAILED}
12
-4
9
Average, Worst, and Best Cases (contd.)
• From the previous example, the linear search algorithm can have
different complexity cases.
• The best case requires 1 comparison operation.
• The worst case requires n comparison operations (with possible
failure!).
• The average case is the average of all the possible cases over the
number of possibilities, i.e.,
n
1+2+…+n
O(n) =
n
n
O(n)=
∑i
=
i -1
n
n
∑ i = 1+2+…+n =
i=1
=
n+1
n(n+1)
2
Drill Questions
• Find the number of operations performed by the following code
fragment:
sum = 0;
for(i = 1; i <= n; i++)
sum += A[i][i];
• Find the number of operations performed by the following code
fragment:
for(cnt1 = 0, i = 1; i <= n; i++)
for(j = 1; j <= n; j++)
cnt1++;
• Find the number of operations performed by the following code
fragment:
for(cnt2 = 0, i = 1; i <= n; i *= 2)
for(j = 1; j <= n; j++)
cnt2++;