CSC 201
Analysis and Design of Algorithms
Lecture 05:
Analysis of time Complexity of
Sorting Algorithms
Dr.Surasak Mungsing
E-mail: [email protected]
Jul-17
1
Sorting Algorithms









Bin Sort
Radix Sort
Insertion Sort
Shell Sort
Selection Sort
Heap Sort
Bubble Sort
Quick Sort
Merge Sort
7/13/2017
2
7/13/2017
3
Bin Sort
A2
B4
C5
D4
E3
F0
G4
H3
A
J
H
E
I
G
D
B
C
Bin 2
Bin 3
Bin 4
Bin 5
D4
G4
I 4
J 3
(a) Input chain
F
Bin 0
Bin 1
(b) Nodes in bins
F0
A2
E3
H3
J 3
B4
I 4
C5
(c) Sorted chain
7/13/2017
4
Radix Sort with r=10 and d=3
216
521
425
116
91
515
124
34
96
24
515
216
116
96
(a) Input chain
521
91
124
34
24
425
(b) Chain after sorting on least significant digit
515
216
116
521
124
24
425
34
91
96
(c) Chain after sorting on second-least significant digit
24
34
91
96
116
124
216
425
(d) Chain after sorting on most significant digit
7/13/2017
5
515
521
Insertion Sort Concept
7/13/2017
6
7/13/2017
7
Shell Sort Algorithm
7/13/2017
8
Shell Sort Algorithm
7/13/2017
9
Shell Sort Algorithm
7/13/2017
10
7/13/2017
11
Shell Sort Demo
http://e-learning.mfu.ac.th/mflu/1302251/demo/Chap03/ShellSort/ShellSort.html
7/13/2017
12
Selection Sort Concept
7/13/2017
13
7/13/2017
14
Heap Sort Algorithm
7/13/2017
15
7/13/2017
16
Bubble Sort Concept
7/13/2017
17
7/13/2017
18
Quick sort
The fastest known sorting algorithm in practice.
81
13
43
92
31
57
75
26
65
0
Select pivot
81
13
43
92
31
57
65
75
26
0
Partition
13
31
57
26 43 0
65
92
75
81
7/13/2017
19
Quick sort
13
31
57
26 43 0
65
Quick sort small
0 13 26 31 43 57
75
92
81
Quick sort large
75
65
0 13 26 31 43 57 65
75
81
81
92
92
7/13/2017
20
Quick sort
7/13/2017
21
Quick Sort Partitions
7/13/2017
22
Quick sort
7/13/2017
23
External Sort: A simple merge
7/13/2017
24
Merge Sort
7/13/2017
25
Merge Sort
7/13/2017
26
7/13/2017
27
7/13/2017
28
Analysis of Insertion Sort
for (int i = 1; i < a.length; i++)
{// insert a[i] into a[0:i-1]
int t = a[i];
int j;
for (j = i - 1; j >= 0 && t < a[j]; j--)
a[j + 1] = a[j];
a[j + 1] = t;
}
7/13/2017
29
Complexity
 Space/Memory
 Time
 Count a particular operation
 Count number of steps
 Asymptotic complexity
7/13/2017
30
Comparison Count
for (int i = 1; i < a.length; i++)
{// insert a[i] into a[0:i-1]
int t = a[i];
int j;
for (j = i - 1; j >= 0 && t < a[j]; j--)
a[j + 1] = a[j];
a[j + 1] = t;
}
7/13/2017
31
Comparison Count
for (j = i - 1; j >= 0 && t < a[j]; j--)
a[j + 1] = a[j];
How many comparisons are made?
7/13/2017
32
Comparison Count
for (j = i - 1; j >= 0 && t < a[j]; j--)
a[j + 1] = a[j];
number of compares depends on
a[]s and t as well as on i
7/13/2017
33
Comparison Count
 Worst-case count = maximum count
 Best-case count = minimum count
 Average count
7/13/2017
34
Worst-Case Comparison Count
for (j = i - 1; j >= 0 && t < a[j]; j--)
a[j + 1] = a[j];
a = [1, 2, 3, 4] and t = 0  4 compares
a = [1,2,3,…,i] and t = 0  i compares
7/13/2017
35
Worst-Case Comparison Count
for (int i = 1; i < n; i++)
for (j = i - 1; j >= 0 && t < a[j];
j--)
a[j + 1] = a[j];
total compares = 1 + 2 + 3 + … + (n-1)
n
T(n) =

i=2
(i-1)
= (n-1)n/2
= O(n2)
7/13/2017
36
Average-Case Comparison Count
for (int i = 1; i < n; i++)
for (j = i - 1; j >= 0 && t < a[j]; j--)
a[j + 1] = a[j];
n
T
=

i=2
i-1
2
= O(n2)
7/13/2017
37
Analysis of Quick Sort
81
13
43
92
31
57
75
26
65
0
Select pivot
81
13
43
92
31
57
65
75
26
0
Partition
31
57
13
26 43 0
65
92
75
81
7/13/2017
38
Main
Quick Sort
Routine
private static void quicksort( Comparable [ ] a, int left, int right )
{
/* 1*/
if( left + CUTOFF <= right )
{
/* 2*/
Comparable pivot = median3( a, left, right );
// Begin partitioning
/* 3*/
int i = left, j = right - 1;
/* 4*/
for( ; ; )
{
/* 5*/
while( a[ ++i ].compareTo( pivot ) < 0 ) { }
/* 6*/
while( a[ --j ].compareTo( pivot ) > 0 ) { }
/* 7*/
if( i < j )
/* 8*/
swapReferences( a, i, j );
else
/* 9*/
break;
}
/*10*/
swapReferences( a, i, right - 1 ); // Restore pivot
/*11*/
quicksort( a, left, i - 1 ); // Sort small elements
/*12*/
quicksort( a, i + 1, right ); // Sort large elements
}
else // Do an insertion sort on the subarray
/*13*/
insertionSort( a, left, right );
}
7/13/2017
39
Worst-Case Analysis
เวลาที่ใช้ในการ run quick sort เท่ากับเวลาที่ใช้ในการทา recursive call 2 ครั้ง + linear
time ที่ใช้ในการเลือก pivot ซึ่ งทาให้ basic quick sort relation เท่ากับ
T(n) = T(i) + T(n-i-1) + cn
ในกรณี Worst- case เช่น การที่ pivot มีค่าน้อยที่สุดเสมอ เวลาที่ใช้ในการทา
recursion คือ
รวมเวลาทั้งหมด
T(n) = T(n-1) + cn
T(n-1)= T(n-2)+c(n-1)
T(n-2)= T(n-3)+c(n-2)
…
T(2) = T(1)+c(2)
n>1
n

T(n) = T(1) + c
i = O(n2)
i=2
7/13/2017
40
Best-Case Analysis
The pivot is in the middle; T(n) = 2 T(n/2) + cn
Divide both sides by n;
T(n)
n
T(n/2)
n/2
T(n/4)
n/4
T(2)
2
Add all equations;
T(n)
n
=
T(n/2)
+ c
n/2
=
T(n/4)
+ c
n/4
=
…
=
T(n/8)
n/8
T(1)
+ c
1
T(1)
=
+ c
+ c log n
1
T(n) = c n logn + n = O(n log n)
7/13/2017
41
Average-Case Analysis (1/4)
Total time;
T(N) = T(i) + T(N-i-1) + c N …………..(1)
Average time of T(i) and T(N-i-1) is
1
N -1
N

T( j )
…………..(2)
j=0
Therefore
T(N) =
2
N
N -1

T( j )
+ cN
…………..(3)
T( j )
+ cN2
…………..(4)
+ c(N – 1)2
…………..(5)
j=0
N -1
NT(N) =
2

j=0
N -2
(N-1) T(N-1) = 2

T( j )
j=0
7/13/2017
42
Average-Case Analysis (2/4)
(5) – (4);
NT(N) – (N-1) T(N-1) = 2T(N-1) +2cN - c
…………..(6)
Rearrange terms in equation and ignore c on the right-hand side;
NT(N) = (N+1) T(N-1) +2cN
(7) divides by N(N+1);
T(N-1)
T(N)
+
=
N+1
2c
N
…………..(7)
…………..(8)
N+1
7/13/2017
43
Average-Case Analysis (3/4)
T(N-1)
T(N)
…………..(8)
+
=
N+1
N
N+1
T(N-1)
T(N-2)
2c
…………..(9)
+
=
N
N-1
N
T(N-2)
T(N-3)
2c
N-1
T(2)
=
.
.
.
+
N-2
N-1
T(1)
2c
+
=
Sun equations (8) to (11);
2c
3
2
T(N)
T(1)
N+1
2
…………..(11)
3
N +1
+ 2c
=
…………..(10)

…………..(12)
i=3
7/13/2017
44
Average-Case Analysis (4/4)
T(1)
T(N)
N +1
+ 2c
=
2
N+1

…………..(12)
i=3
Sum in equation (12) ia approximately logC(N+1)+  - 3/2,
which  is Euler’s constant 0.577
T(N)
therefore
= O(log N)
…………..(13)
N+1
and
T(N) = O(Nlog N)
…………..(14)
In summary, time complexity of Quick sort algorithm for Average-Case
is
T(n) = O(n log n)
7/13/2017
45
13-Jul-17
46