BASICS
Prof. Hsin-Mu (Michael) Tsai ( 蔡欣穆)
Department of Computer Science and Information Engineering
National Taiwan University
上課!
•
醒了沒? :P
•
Homework 1 is out 5pm today, the due date
is two weeks from yesterday (5pm).
•
Download the question sets from the
course website
•
E-mail your question to
[email protected]
•
Or, come to our office hours.
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
2
WHAT IS AN ALGORITHM?
•
A computable set of steps to achieve a desired result.
•
All algorithm must satisfy the following criteria:
• Input
• Output
• Definiteness
• Finiteness
• Effectiveness
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
3
EXAMPLE
•
Statement 1: “Is n=2 the largest value of n for which there exist positive
integers x, y, and z such that 𝑥 𝑛 + 𝑦 𝑛 = 𝑧 𝑛 has a solution?”
•
Statement 2: “Store 5 divided by zero into x and go to statement ㄆ.”
•
Which criterion do they violate?
• Input
• Output
• Definiteness
• Finiteness
• Effectiveness
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
4
WHAT IS A DATA STRUCTURE?
•
An organization of information, usually in memory, for better
algorithm efficiency.
•
Or, a way to store and organize data in order to facilitate access
and modifications.
2𝑛5 − 3𝑛2 + 11𝑛 + 27
0
1
2
3
4
5
6
27
11
-3
0
0
2
0
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
5
ALGORITHM SPECIFICATION
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
6
HOW DO WE DESCRIBE AN ALGORITHM?
•
Human language (English, Chinese, …)
•
Programming language
•
A mix of the above
菜瓜布曰:
1. 出系館以後又轉到側門，過馬路以
後到對面的Starbucks。
2. 如果人不多的話就幫我買杯拿鐵。
3. 如果人太多的話，就到旁邊的全家
買一杯罐裝伯朗咖啡就好。
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
7
EXAMPLE: SELECTION SORT
•
Integers are stored in an array, list. The i-th integer is stored in
list[i], 0<i<n.
•
Goal: Devise a program to sort a set of 𝑛 ≥ 1 integers
•
Solution: From those integers that are currently unsorted, find the
smallest and place it next in the sorted list.
ㄅ ㄆ
1
1
ㄆ
2
ㄅ
1
2
ㄆ
ㄅ
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
8
EXAMPLE: SELECTION SORT
•
First attempt:
for (i=0; i<n; ++i) {
Examine list[i] to list[n-1] and suppose that
the smallest integer is at list[min];
Task 1
Interchange list[i] and list[min];
}
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
Task 2
9
Task 2
void swap(int *x, int *y) {
int temp = *x;
*x=*y;
*y=temp;
}
Or
#define SWAP(x,y,t) ((t)=(x), (x)=(y), (y)=(t))
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
10
Task 1
min=i;
for(j=i;j<n;++j)
if (list[j]<list[min])
min=j;
Finally, see program 1.4 on p. 11 for a complete program of the selection
sort we just develop.
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
11
HOW DO WE PROVE THAT IT IS CORRECT?
• Theorem 1.1: Function sort(list,n) correctly sorts a set of
n>=1 integers. The result remains in list[0], …, list[n-1]
such that 𝑙𝑖𝑠𝑡 0 ≤ 𝑙𝑖𝑠𝑡 1 ≤ ⋯ ≤ 𝑙𝑖𝑠𝑡[𝑛 − 1].
• Proof: When the outer for loop completes its iteration for
i=q, we have 𝑙𝑖𝑠𝑡 𝑞 ≤ 𝑙𝑖𝑠𝑡[𝑟], 𝑞 < 𝑟 < 𝑛. Further, on
subsequent iterations, i>q and list[0] through list[q] are
unchanged. Hence following the last iteration of the outer
for loop (i.e., i=n-2), we have 𝑙𝑖𝑠𝑡 0 ≤ 𝑙𝑖𝑠𝑡 1 ≤ ⋯ ≤ 𝑙𝑖𝑠𝑡[𝑛 −
1].
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
12
EXAMPLE: BINARY SEARCH
•
Input:
• searchnum: the number to be found
• list: sorted array, size n, and 𝑙𝑖𝑠𝑡 0 ≤ 𝑙𝑖𝑠𝑡 1 ≤ ⋯ ≤ 𝑙𝑖𝑠𝑡[𝑛 − 1]
•
Output:
• -1 if searchnum is not found in list
• the index of searchnum in list if searchnum is found
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
13
EXAMPLE:
searchnum=13;
0
1
2
3
4
5
6
7
8
9
10
11
1
3
4
4
6
7
11
13
13
13
18
19
return middle;
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
14
EXAMPLE:
searchnum=13;
0
1
2
3
4
5
6
7
8
9
10
11
1
3
4
4
6
7
11
13
13
13
18
19
left
middl
e
middle=(left+right)/2;
right
left=middle+1;
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
15
EXAMPLE:
searchnum=5;
0
1
2
3
4
5
6
7
8
9
10
11
1
3
4
4
6
7
11
13
13
13
18
19
return -1;
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
16
int binsearch(int list[], int searchnum, int left, int right)
{
int middle;
while(left<=right) {
middle=(left+right)/2;
switch(COMPARE(list[middle], searchnum)) {
case -1: left=middle+1;
break;
case 0: return middle;
case 1: right=middle-1;
}
}
return -1;
}
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
17
RECURSIVE ALGORITHMS
• What does “recursive” mean?
• A function which calls itself (direct recursion), or
• a function which calls other functions which call the
calling function again (indirect recursion).
• Any function that we can write using assignment, if-else,
and while statements can be written recursively.
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
18
RECURSIVE ALGORITHMS
• Why do we want to use recursive functions (or
algorithms)?
• It allow us to express an otherwise complex process in
very clear terms.
• Often the recursive function is easier to understand
than its iterative counterpart.
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
19
THE STORY OF RECURSIVE FUNCTION
Yo. Do the
work ㄅ.
ㄅ is too much for me.
I’ll partition it, do part
1, and clone two copies
of myself to do the rest.
Function blah
Hey man. Do the
work ㄅ part 2.
Function blah
(clone 1)
Hi. Do the
work ㄅ part 3.
Function blah
(clone 2)
I’m ㄅ. The work to be
done.
1
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
2 ㄅ
3
20
EXAMPLE 1
•
Function nchoosek (int n, int m)
•
𝑛
𝑚
=
•
𝑛
𝑚
=
n!
m! n−m !
𝑛−1
𝑚
+
𝑛−1
𝑚−1
•
How do we implement function nchoosek (int n, int m) recursively?
•
Boundary case
•
Do some work
•
Delegation
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
21
EXAMPLE 2
int binsearch(int list[], int searchnum, int left, int right) {
int middle;
if (left<=right) {
1. Termination condition
middle=(left+right)/2;
switch (COMPARE(list[middle], searchnum)) {
case -1:
return binsearch(list, searchnum, middle+1,
right);
case
0:
return middle;
2. Recursive calls
case 1:
return binsearch(list, searchnum, left,
middle-1);
}
}
}
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
22
EXAMPLE 3
•
Output all permutations
•
Input: {a,b,c}
•
Output: (a,b,c),(a,c,b),(b,a,c),(b,c,a),(c,a,b),(c,b,a)
•
How do we write this program recursively?
•
Example: input={a,b,c,d}. Output =
• a followed by all permutations of {b,c,d}
• b followed by all permutations of {a,c,d}
• c followed by all permutations of {a,b,d}
• d followed by all permutations of {a,b,c}
•
“all permutations of {…}”  recursive calls!
•
Homework: read and understand program 1.9
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
23
DATA ABSTRACTION
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
24
DATA TYPE
•
What is a data type?
A data type is a collection of objects and a set of operations that
act on those objects.
•
Data types in C
• char, int, float, long, double (unsigned, signed, …)
• Array
• Struct
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
int iarray[16];
struct {
int a;
int b;
char str[16];
int * iptr;
} blah;
25
DATA TYPE
•
Operations
• +, -, *, /, %, ==
• =, +=, -=
• ?:
• sizeof, - (negative)
• giligulu(int a, int b)
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
26
DATA TYPE
•
Representation of the objects of the data type
•
Example: char
•
char blah=‘A’; (‘A’: ASCII code is 65(dec), or 0x41 (hex))
1 byte of
memory:
01000001
Q: The maximum number which can be represented with a char
variable?
A: 255.
•
Homework: How about char, int, long, float?
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
27
DATA TYPE
•
Q: Do we need to know about the representation of a data type?
•
A: It depends.
•
We can usually write algorithms which are more efficient if we
make use of the knowledge about the representation.
•
However!!!
If the representation of the data type is changed, the program
needs to be verified, revised, or
completely re-written.
囧
• Porting to a different platform (x86, ARM, embedded system, …)
• Changing the specification of a program or a library (ex. 16-bit
int  32-bit long)
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
28
ABSTRACT DATA TYPE
•
An “abstract data type” (ADT) is a data type that is organized in
such a way that the specification of the objects and the
specification of the operations on the objects is separated from the
representation of the objects and the implementation of the
operations.
User
Specification (Interface)
Representation and Implementation
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
29
ABSTRACT DATA TYPE
•
Specifications:
• Name of the function and the description of what the function
does
• The type of the argument(s)
• The type of the result(s) (return value)
•
Function categories:
• Creator/constructor
• Transformers
• Observer/reporter
•
Homework: Read Example 1.5 on p. 20
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
30
PERFORMANCE ANALYSIS
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
31
HOW DO YOU MAKE EVALUATIVE JUDGMENT
ABOUT PROGRAM?
1. Does the program meet the original specifications of the task?
2. Does it work correctly?
3. Does the program contain documentation that shows how to use it
and how it works?
4. Does the program effectively use functions to create logical units?
5. Is the program’s code readable?
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
32
HOW DO YOU MAKE EVALUATIVE JUDGMENT
ABOUT PROGRAM?
6. Does the program efficiently use primary and secondary storage?
Primary storage: memory?
Secondary storage: Hard drive, flash disk, etc.
7. Is the program’s running time acceptable for the task?
Example: Network intrusion detection system
(1) 99.8% detection rate, 50 minutes
to finish analysis of a minute of traffic
(2) 85% detection rate, 20 seconds to
finish analysis of a minute of traffic
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
33
HOW DO YOU MAKE EVALUATIVE JUDGMENT
ABOUT PROGRAM?
6. Does the program efficiently use primary and secondary storage?
7. Is the program’s running time acceptable for the task?
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
34
SPACE & TIME COMPLEXITY
•
Space complexity of a program:
The amount of memory that it needs to run to completion.
•
Time complexity of a program:
The amount of computer time that it needs to run to completion.
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
35
SPACE COMPLEXITY
•
The space needed by a program:
1. Fixed space requirements
•
Do not depend on the number and size of the
inputs/outputs.
2. Variable space requirements
•
Depends on some characteristics of the particular instance,
I, of the problem being solved, P.
•
Depends on the additional space required when using
recursion.
• 𝑆 𝑃 = 𝑐 + 𝑆𝑃 (𝐼)
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
36
EXAMPLE
float abc(float a, float b, float c) {
return a+b+b*c+(a+b-c)/(a+b)+4.00;
}
• 𝑆𝑎𝑏𝑐 𝐼 =?
(variable space requirements)
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
37
EXAMPLE
float sum(float list[], int n) {
float tempsum=0;
int i;
for(i=0;i<n;++i)
tempsum+=list[i];
return tempsum;
}
• 𝑆𝑠𝑢𝑚 𝑛 =?
(variable space requirements)
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
38
EXAMPLE
function rsum(float list[], int n) {
if (n) return rsum(list,n-1)+list[n-1];
return 0;
}
• 𝑆𝑟𝑠𝑢𝑚 𝑛 =?
(variable space requirements)
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
39
Type
Name
Number of bytes
parameters: array pointer list[]
4
parameter: integer
4
n
return address: (used
internally)
4
TOTAL per recursive call
12
𝑆𝑟𝑠𝑢𝑚 𝑛 = 12𝑛
Probably not for this
problem.
Is it good to use?
(recursion)
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
40
TIME COMPLEXITY
•
Time taken by a program, P:
• Compile time
• Run (execution) time
•
Compile time: fixed. (Exceptions?)
• C (and other compiled programming languages)
 One compilation  Multiple executions
•
Run time: 𝑇𝑃
• Depends on instance characteristics (input)
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
41
HOW TO WE DETERMINE 𝑇𝑃 ?
•
Method 1:
• Count all instances of all operations in the program.
Add
Subtract
Load
Store
ADD(n)
SUB(n)
LDA(n)
STA(n)
𝑐𝑎
𝑐𝑠
𝑐𝑙
𝑐𝑠𝑡
𝑇𝑃 𝑛 = 𝑐𝑎 𝐴𝐷𝐷 𝑛 + 𝑐𝑠 𝑆𝑈𝐵 𝑛 + 𝑐𝑙 𝐿𝐷𝐴 𝑛 + 𝑐𝑠𝑡 𝑆𝑇𝐴(𝑛)
Is it good to use?
(method 1)
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
42
HOW TO WE DETERMINE 𝑇𝑃 ?
•
Method 2:
• Separate the program into program steps whose execution time
is independent of instance characteristics
• Count the number of steps
•
Different program steps have different amounts of computing
• a=2;
• a=2*b+3*c/d-e+f/g/a/b/c;
•
Count the number of steps needs to solve a particular instance
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
43
EXAMPLE 1
float sum(float list[], int n) {
float tempsum = 0; count++; //assignment
int i;
for (i=0;i<n;i++) {
count++; // for loop
tempsum+=list[i]; count++; //assignment
}
count++; //last iteration of for
count++; return tempsum;
}
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
44
EXAMPLE 1
float sum(float list[], int n) {
float tempsum = 0;
int i;
for (i=0;i<n;i++) {
count+=2;
}
count+=3;
return tempsum;
}
count = 2n+3 (steps)
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
45
EXAMPLE 2
float rsum(float list[], int n) {
count++; // if
if (n) {
count++; //return
return rsum(list,n-1)+list[n-1];
}
count++; //return
return list[0];
}
count = 2n+2 (steps)
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
2n+2 < 2n+3.
Does this mean rsum
is faster than sum ?
No!
46
EXAMPLE 3
void add(int a[][MAX_SIZE], int b[][MAX_SIZE], int
c[][MAX_SIZE], int rows, int cols) {
int i,j;
for(i=0; i<rows; ++i)
for(j=0;j<cols;++j)
c[i][j]=a[i][j]+b[i][j];
}
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
47
EXAMPLE 3
void add(int a[][MAX_SIZE], int b[][MAX_SIZE], int
c[][MAX_SIZE], int rows, int cols) {
int i,j;
for(i=0; i<rows; ++i) {
count++;
for(j=0;j<cols;++j) {
count++;
c[i][j]=a[i][j]+b[i][j]; count++;
}
count++;
}
count++;
}
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
count=((2*cols)+2)*rows+1
=2cols*rows+2*rows+1
48
EXAMPLE 1 REVISITED (TABULAR METHOD)
Statement
s/e
Frequency
Total steps
float sum(float list[],
int n) {
0
0
0
float tempsum=0;
1
1
1
int i;
0
0
0
for (i=0;i<n;i++)
0
n+1
n+1
1
n
n
1
1
1
tempsum+=list[i];
return tempsum; }
Please practice using the method for the other two examples.
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
49
SUMMARY
•
Instance characteristics?
• The number of inputs
• The number of outputs
• The magnitude of the inputs
•
We then calculate the number of steps which is independent of
the characteristics we selected.
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
50
•
Sometimes, the time complexity does not depend on only these
simple numbers.
•
For example, binsearch.
•
In that case, we are interested in
• the average case,
• the worst case,
• and the best case.
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
51
ASYMPTOTIC NOTATION
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
52
MOTIVATION
•
Compare the time complexities of two programs that compute
the same function.
•
Predict the growth in run time as the instance characteristics
change.
•
“Steps” are not exact.
• “3n+3” faster than “3n+5” ?
• Usually “3n+3”, “7n+2”, or “2n+15” all runs for about the
same time.
• Let’s go for a less accurate way to describe the run time…
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
53
EXAMPLE
No.
•
Program P and Q
•
𝑇𝑃 𝑛 = 𝑐1 𝑛 2 + 𝑐2 𝑛
•
𝑇𝑄 𝑛 = 𝑐3 𝑛
•
For sufficiently large n, Q will be faster than P FOR ANY 𝑐1 , 𝑐2 , 𝑐3 .
•
Example:
• 𝑐1 = 1, 𝑐2 = 2, 𝑐3 = 100 , then 𝑐1 𝑛2 + 𝑐2 𝑛 2 > 𝑐3 𝑛 for 𝑛 > 98.
• 𝑐1 = 1, 𝑐2 = 2, 𝑐3 = 1000 , then 𝑐1 𝑛 2 + 𝑐2 𝑛 2 > 𝑐3 𝑛 for 𝑛 > 998.
•
Do we need to know the values of 𝑐1 , 𝑐2 , 𝑐3 ?
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
Break even point
54
ASYMPTOTIC NOTATION – BIG OH
•
Definition [Big “oh”]:
𝑓 𝑛 = Ο 𝑔 𝑛 if and only if there exist
positive constants 𝑐 and 𝑛0 such that 𝑓 𝑛 ≤
𝑐𝑔(𝑛) for all 𝑛, 𝑛 ≥ 𝑛0 .
•
“f of n is big oh of g of n”
•
“=“ is “is” not “equal”
•
Ο 𝑔 𝑛
= 𝑓(𝑛)
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
55
EXAMPLE
•
3𝑛 + 2 = Ο(𝑛) since 3𝑛 + 2 ≤ 4𝑛 for all 𝑛 ≥ 2.
•
3𝑛 + 3 = Ο(𝑛) since 3𝑛 + 3 ≤ 4𝑛 for all 𝑛 ≥ 3.
•
100𝑛 + 6 = Ο(𝑛) since 100𝑛 + 6 ≤ 101𝑛 for all 𝑛 ≥ 10.
•
10𝑛 2 + 4𝑛 + 2 = Ο(𝑛 2 ) since 10n2 + 4n + 2 ≤ 11𝑛 2 for all 𝑛 ≥ 5.
•
1000𝑛2 + 100𝑛 − 6 = Ο(𝑛 2 ) since 1000𝑛 2 + 100𝑛 − 6 ≤ 1001𝑛 2 for all
𝑛 ≥ 100.
•
6 ∗ 2𝑛 + 𝑛 2 = Ο(2𝑛 ) since 6 ∗ 2𝑛 + 𝑛 2 ≤ 7 ∗ 2n for all 𝑛 ≥ 4.
•
3𝑛 + 3 = Ο(𝑛2 ) since 3𝑛 + 3 ≤ 3𝑛 2 for all 𝑛 ≥ 2.
•
10𝑛 2 + 4𝑛 + 2 = Ο(𝑛 4 ) since 10𝑛 2 + 4𝑛 + 2 ≤ 10𝑛 4 for all 𝑛 ≥ 2.
•
3𝑛 + 2 ≠ Ο 1
•
10𝑛 2 + 4n + 2 ≠ Ο(𝑛)
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
56
THE WORLD OF BIG OH
•
Ο 1 constant
•
Ο 𝑛 linear
•
Ο 𝑛 2 quadratic
•
Ο 𝑛 3 cubic
•
Ο 2𝑛 exponential
• Ο 1 , Ο log 𝑛 , Ο 𝑛 , Ο 𝑛log 𝑛 , Ο 𝑛2 , Ο 𝑛3 , Ο 2𝑛
Faster
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
Slower
57
BIG OH IS AN UPPER BOUND
•
Doesn’t say how good it is.
•
𝑛 = Ο(𝑛)
•
𝑛 = Ο(𝑛2 )
•
𝑛 = Ο(𝑛2.5 )
•
𝑛 = Ο(2𝑛 )
•
But it should usually be as small a function of n as possible.
•
3𝑛 + 3 = Ο(𝑛2 )
•
3𝑛 + 3 = Ο 𝑛
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
58
A USEFUL THEOREM
•
Theorem: If 𝑓 𝑛 = 𝑎𝑚 𝑛 𝑚 + ⋯ + 𝑎1 𝑛 + 𝑎0 , then 𝑓 𝑛 = Ο(𝑛 𝑚 ).
•
Proof:
𝑚
𝑎𝑖 𝑛 𝑖
𝑓 𝑛 ≤
𝑖=0
𝑚
= 𝑛𝑚
𝑎𝑖 𝑛 𝑖−𝑚
𝑛𝑖−𝑚 ≤ 1
𝑖=0
𝑚
≤ 𝑛𝑚
𝑎𝑖
𝑖=0
, for all 𝑛 ≥ 1. So, 𝑓 𝑛 = Ο
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
𝑛𝑚
.
59
ASYMPTOTIC NOTATION – OMEGA
•
Definition [Omega]:
𝑓 𝑛 = Ω 𝑔 𝑛 if and only if there exist
positive constants 𝑐 and 𝑛0 such that 𝑓 𝑛 ≥
𝑐𝑔(𝑛) for all 𝑛, 𝑛 ≥ 𝑛0 .
•
“f of n is omega of g of n”
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
60
EXAMPLE
•
3𝑛 + 2 = Ω(𝑛) since 3𝑛 + 2 ≥ 3𝑛 for all 𝑛 ≥1.
•
3𝑛 + 3 = Ω(𝑛) since 3𝑛 + 3 ≥ 3𝑛 for all 𝑛 ≥1.
•
100𝑛 + 6 = Ω(𝑛) since 100𝑛 + 6 ≥ 100𝑛 for all 𝑛 ≥ 1.
•
10𝑛 2 + 4𝑛 + 2 = Ω(𝑛 2 ) since 10n2 + 4n + 2 ≥ 𝑛 2 for all 𝑛 ≥1.
•
6 ∗ 2𝑛 + 𝑛 2 = Ω(2𝑛 ) since 6 ∗ 2𝑛 + 𝑛 2 ≥ 2n for all 𝑛 ≥1.
•
3𝑛 + 3 = Ω(1)
•
10𝑛 2 + 4𝑛 + 2 = Ω(1)
•
6 ∗ 2𝑛 + 𝑛 2 = Ω 𝑛100
•
6 ∗ 2𝑛 + 𝑛 2 = Ω 𝑛 50.2
•
6 ∗ 2𝑛 + 𝑛 2 = Ω 𝑛 2
•
6 ∗ 2𝑛 + 𝑛 2 = Ω 𝑛
•
6 ∗ 2𝑛 + 𝑛 2 = Ω 1
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
61
DISCUSSION
•
Omega is a lower bound.
•
Should be as large a function as possible.
•
Theorem: If 𝑓 𝑛 = 𝑎𝑚 𝑛 𝑚 + ⋯ + 𝑎1 𝑛 + 𝑎0 and 𝑎𝑚 > 0, then 𝑓 𝑛 =
Ω(𝑛 𝑚 ).
Homework 1
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
62
ASYMPTOTIC NOTATION – THETA
•
Definition [Theta]:
𝑓 𝑛 = Θ 𝑔 𝑛 if and only if
there exist positive constants 𝑐1 ,
𝑐2 and 𝑛0 such that c1 g(n) ≥
𝑓 𝑛 ≥ 𝑐2 𝑔(𝑛) for all 𝑛, 𝑛 ≥ 𝑛0 .
•
Theta Platform
“f of n is theta of g of n”
GMC Terrain
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
63
EXAMPLE
•
3𝑛 + 2 = Θ(𝑛) since 3𝑛 + 2 ≥ 3𝑛 for all 𝑛 ≥ 2 and 3𝑛 + 2 ≤ 4𝑛 for all
𝑛 ≥ 2.
•
3𝑛 + 3 = Θ(𝑛)
•
10𝑛 2 + 4𝑛 + 2 = Θ(𝑛 2 )
•
6 ∗ 2𝑛 + 𝑛 2 = Θ(2𝑛 )
•
10 ∗ log 𝑛 + 4 = Θ log 𝑛
•
3𝑛 + 2 ≠ Θ 1
•
3𝑛 + 3 ≠ Θ 𝑛2
•
10𝑛 2 + 4𝑛 + 2 ≠ Θ 𝑛
•
10𝑛 2 + 4𝑛 + 2 ≠ Θ 1
•
6 ∗ 2𝑛 + 𝑛 2 ≠ Θ(𝑛 2 )
•
6 ∗ 2𝑛 + 𝑛 2 ≠ Θ 𝑛100
•
6 ∗ 2𝑛 + 𝑛 2 ≠ Θ 1
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
64
DISCUSSION
•
More precise than both the “big oh” and omega notations
•
It is true if and only g(n) is both an upper and lower bound on f(n).
•
Coefficient = 1
• Ω(6 ∗ 2𝑛 )
• Θ(𝑛3 )
•
Theorem: If 𝑓 𝑛 = 𝑎𝑚 𝑛 𝑚 + ⋯ + 𝑎1 𝑛 + 𝑎0 and 𝑎𝑚 > 0, then 𝑓 𝑛 =
Θ(𝑛 𝑚 ).
Homework 1
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
65
SO, WHAT NOW?
•
Asymptotic complexity can be determined quite easily
•
Work on each statement first
•
Then we add them up
•
No need for the silly step counts anymore
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
66
EXAMPLE 1
Statement
Asymptotic complexity
void add(int a[][MAX_SIZE]…) {
0
int i,j;
0
for(i=0;i<rows;i++)
for(j-0;j<cols;j++)
c[i][j]=a[i][j]+b[i][j];
}
Total
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
Θ(𝑟𝑜𝑤𝑠)
Θ(𝑟𝑜𝑤𝑠. 𝑐𝑜𝑙𝑠)
Θ(𝑟𝑜𝑤𝑠. 𝑐𝑜𝑙𝑠)
0
Θ(𝑟𝑜𝑤𝑠. 𝑐𝑜𝑙𝑠)
67
EXAMPLE 2
int binsearch(int list[], int searchnum, int left, int right)
{
int middle;
while(left<=right) {
middle=(left+right)/2;
switch(COMPARE(list[middle], searchnum)) {
case -1: left=middle+1;
break;
case 0: return middle;
case 1: right=middle-1;
}
}
return -1;
}
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
68
Worst case: Θ log 𝑛
Best case: Θ 1
searchnum=13;
0
1
2
3
4
5
6
7
8
9
10
11
1
3
4
4
6
7
11
13
13
13
18
19
left
middl
e
middle=(left+right)/2;
right
left=middle+1;
Half the searching window
every iteration.
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
69
MORE EXAMPLES
•
Please take a look at example 1.20 and 1.21 (p. 38-40)
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
70
PRACTICAL COMPLEXITY
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
71
兜基??
Θ(𝑛2 )
𝑛2 ms
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
Θ(𝑛)
106 𝑛 𝑚𝑠
72
WHICH ONE IS BETTER?
•
“When n is sufficiently large”, P is better than Q.
•
Is n going to be “sufficiently large”?
• 106 𝑛 v.s. 𝑛 2
•
if n is never larger than 106 , then …??
•
We need to look at the range of n as well.
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
73
ON A 1 BILLION-STEPS-PER-SEC COMPUTER
n
𝒏
𝒏 𝒍𝒐𝒈𝟐 𝒏
𝒏𝟐
𝒏𝟑
𝒏𝟒
𝒏𝟏𝟎
𝟐𝒏
10
.01𝜇𝑠
.03𝜇𝑠
.1𝜇𝑠
1𝜇𝑠
10𝜇𝑠
10𝑠
1𝜇𝑠
20
.02𝜇𝑠
.09𝜇𝑠
.4𝜇𝑠
8𝜇𝑠
160𝜇𝑠
2.84ℎ
1𝑚𝑠
30
.03𝜇𝑠
.15𝜇𝑠
.9𝜇𝑠
27𝜇𝑠
810𝜇𝑠
6.83𝑑
1𝑠
40
.04𝜇𝑠
.21𝜇𝑠
1.6𝜇𝑠
64𝜇𝑠
2.56𝑚𝑠
121𝑑
18𝑚
50
.05𝜇𝑠
.28𝜇𝑠
2.5𝜇𝑠
125𝜇𝑠
6.25𝑚𝑠
3.1𝑦
13𝑑
100
.10𝜇𝑠
.66𝜇𝑠
10𝜇𝑠
1𝑚𝑠
100𝑚𝑠
3171𝑦
4
∗ 1013 𝑦
103
1𝜇𝑠
9.96𝜇𝑠
1𝑚𝑠
1𝑠
16.67𝑚
3.17
∗ 1013 𝑦
32
∗ 10283 𝑦
104
10𝜇𝑠
130𝜇𝑠
100𝑚𝑠
16.67𝑚
115.7𝑑
3.17
∗ 1023 𝑦
105
100𝜇𝑠
1.66𝑚𝑠
10𝑠
11.57𝑑
3171𝑦
3.17
∗ 1033 𝑦
106
1𝑚𝑠
19.92𝑚𝑠
16.67𝑚
31.71𝑦
3.17
∗ 107 𝑦
3.17
∗ 1043 𝑦
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
74
下課!
•
等等~~ 再次提醒~~
•
Homework 1 is out 5pm today, the due date is two weeks from
yesterday (5pm).
•
Download the question sets from the course website
•
E-mail your question to [email protected]
•
Or, come to our office hours.
•
Have a nice weekend!
DATA STRUCTURE AND ALGORITHM I
HSIN-MU TSAI, FALL 2010
75