2: Asymptotic Analysis

2: Asymptotic Analysis
CSE326 Spring 2002
April 1, 2002
Linked List Search
7
6
4
9
2Λ
node
bool ListFind ( int k , Node ∗node)
{
}
UW CSE326 Sp ’02: 2—Analysis
1
Analyzing Algorithms
• Best case:
bool ListFind ( int k , Node ∗node)
{
while ( node) {
if ( node−>key == k)
return true ;
n = n−>next;
}
• Worst case:
return false ;
}
• Most of the time:
UW CSE326 Sp ’02: 2—Analysis
2
Array Search
7 6 4 9 2 1 3 6 7 5
array; n=10
bool ArrayFind( int k , int ∗ array , int n)
{
}
UW CSE326 Sp ’02: 2—Analysis
3
Analyzing Algorithms
• Best case:
bool ArrayFind( int k,
int ∗ key array ,
int n)
{
for ( int i = 0; i < n ; i ++)
if ( key array [ i ] == k)
return true ;
return false ;
}
• Worst case:
• Most of the time:
UW CSE326 Sp ’02: 2—Analysis
4
Binary Search
7 6 4 9 2 1 3 6 7 5
bool BinarySearch( int k , int ∗ array , int high , int low=0)
{
assert (low >= 0);
if ( low >= high)
return false ;
int mid = (high−low)/2;
if ( k == array[mid])
return true ;
else if ( k < array [ mid])
return BinarySearch(k , array , mid, low );
// k > array [ mid]
return BinarySearch(k , array , high , mid+1);
}
UW CSE326 Sp ’02: 2—Analysis
5
Analyzing Algorithms
bool BinarySearch( int
int
int
int
{
if ( low >= high)
return false ;
k,
∗ array ,
high ,
low=0)
• Best case:
int mid = (high−low)/2;
if ( k == array[mid])
return true ;
else if ( k < array [ mid])
return BinarySearch(k , array , mid, low );
• Worst case:
// k > array [ mid]
return BinarySearch(k , array , high , mid+1);
}
6
UW CSE326 Sp ’02: 2—Analysis
Solving the Recurrence Relation
7
UW CSE326 Sp ’02: 2—Analysis
Analysis Summary
List
Array
(linear search)
Binary
Best Case
Worst Case
Most of the Time
UW CSE326 Sp ’02: 2—Analysis
8
Which algorithm is best?
9
UW CSE326 Sp ’02: 2—Analysis
Criteria for chosing an algorithm:
• Speed of execution
• Ease of coding
• Preprocessing required
Speed is usually most important—easiest to quantify.
Best was to compare speeds is to measure and then graph.
The Need for Speed
80
array search
list search
binary search
70
time in ms
60
50
40
30
20
10
5
10
15
20
25
30
35
# elts to be searched
40
45
UW CSE326 Sp ’02: 2—Analysis
50
10
Fast Computer vs. Slow Computer
500
linear search on Pentium-IV
linear search on 486
450
400
time in ms
350
300
250
200
150
100
50
0
0
20
40
60
80
100
# elts to be searched
With same algorithm, the faster computer always wins.
11
UW CSE326 Sp ’02: 2—Analysis
Fast Computer vs. Smart Programmer I
350
linear search on Pentium-IV
binary search on 486
300
time in ms
250
200
150
100
50
0
0
20
40
60
80
100
# elts to be searched
Linear search beats binary search?
12
UW CSE326 Sp ’02: 2—Analysis
Fast Computer vs. Smart Programmer II
1000
linear search on Pentium-IV
binary search on 486
time in ms
800
600
400
200
0
0
200
400
600
800
1000
# elts to be searched
Binary search always wins—eventually.
UW CSE326 Sp ’02: 2—Analysis
13
Asymptotic Analysis
• Asymptotic analysis looks at the order of the running-time
of an algorithm.
– What happens when the input gets large?
– Ignore effects of different machines.
• Linear search is O(n) (whether on a list or an array!).
• Binary search is O(log n).
14
UW CSE326 Sp ’02: 2—Analysis
Order Notation: Intuition
Suppose we have 3 algorithms running at the following speeds:
• 100n log n + 5000n
• 10n2 + 1000n
• 15n2 + 2n
Which algorithm is fastest?
⇐⇒ Which function grows slowest?
15
UW CSE326 Sp ’02: 2—Analysis
Order Notation: Intuition
450000
15n^2 + 2n
10n^2 + 1000n
100 n log n + 5000x
400000
350000
time
300000
250000
200000
150000
100000
50000
0
0
10
20
30
40
50
60
70
80
n
15b2 + 2n ≺ 10n2 + 1000n ≺ 100n log n + 5000n?
UW CSE326 Sp ’02: 2—Analysis
16
Order Notation: Intuition
6e+06
15n^2 + 2n
10n^2 + 1000n
100 n log n + 5000x
5e+06
time
4e+06
3e+06
2e+06
1e+06
0
0
100
200
300
n
400
500
600
100n log n + 5000n ≺ 10n2 + 1000n ≺ 15n2 + 2n
17
UW CSE326 Sp ’02: 2—Analysis
Order Notation: Intuition
1.6e+09
15n^2 + 2n
10n^2 + 1000n
100 n log n + 5000x
1.4e+09
1.2e+09
time
1e+09
8e+08
6e+08
4e+08
2e+08
0
0
2000
4000
6000
8000
10000
n
O(n log n) vs. O(n2) is what matters
UW CSE326 Sp ’02: 2—Analysis
18
Order Notation
Precise Definition
• O(f (n)) is a set of functions
• g(n) ∈ O(f (n)) when
There exist c and n0 such that
g(n) ≤ c · f (n), for all n ≥ n0.
UW CSE326 Sp ’02: 2—Analysis
19
Order Notation: Example
• 10n2 + 1000n ≤ 1 · (15n2 + 2n) for n ≥ 250,
⇒ 10n2 + 1000n ∈ O(15n2 + 2n)
• 10n2 + 1000n vs. n2?
UW CSE326 Sp ’02: 2—Analysis
20
Order Notation: Definition
4.5e+07
4e+07
10n^2 + 1000n
11n^2
3.5e+07
time
3e+07
2.5e+07
2e+07
1.5e+07
1e+07
5e+06
1000
1200
1400
n
1600
1800
2000
10n2 + 1000n ≤ 11 · n2 for n ≥ 1200,
⇒ 10n2 + 1000n ∈ O(n2 )
UW CSE326 Sp ’02: 2—Analysis
21
Order Notation
00n
2 + 10
10n
O(n2)
00n 15n 2 +
g n + 50
100n lo
2n
O(15n2 + 2n) = O(n2)
UW CSE326 Sp ’02: 2—Analysis
22
Order Notation
15n 2
00n
2 + 10
10n
+ 2n
O(n2)
O(n log n)
00n
g n + 50
100n lo
O(n log n) ⊂ O(n2)
UW CSE326 Sp ’02: 2—Analysis
23
Hierarchy of Orders
x










bigger
O(2n)
O(nk )
(for constant k)
O(n3)
O(n2)
O(n1+)
(for constant > 0)
O(n log n)
smaller











y
O(n)
(for constant > 0)
O(logk n)
(for constant k)
O(log n)
O(log log n)
O(1)
UW CSE326 Sp ’02: 2—Analysis
24
Ω(·),Θ(·)
• O(f (n)) is all functions asymptotically less than or equal to f (n).
“Big Oh”
– 4n2 ∈ O(n2 )
– log n ∈ O(n2 )
– n3 6∈ O(n2 )
• Ω(f (n)) is all functions asymptotically greater than or equal to f (n).
“Big Omega”
– 4n2 ∈ Ω(n2 )
– log n 6∈ Ω(n2 )
– n3 ∈ Ω(n2 )
• Θ(f (n)) is all functions asymptotically equal to f (n)
“Big Theta”
– 4n2 ∈ Θ(n2 )
– log n 6∈ Θ(n2 )
– n3 6∈ Θ(n2 )
UW CSE326 Sp ’02: 2—Analysis
25
Menagerie of Symbols
Asymptotic
Notation
O
Ω
Θ
o
ω
Mathematics
Relation
≤
≥
=
<
>
UW CSE326 Sp ’02: 2—Analysis
26
Rules of Thumb
• Polynomials
Take term with highest power, drop coefficient.
45n4 + 20n2 + 45n + 60 ∈ O(n4 )
34n2
+ 16n0.1 + 784 log n + 2 ∈ O(n2 )
• Logorithms
Take log-term with highest power, drop coefficients and powers in argument.
50 log n200 ∈ O(log n)
2
32 log n +
log n4
+ log log n2 ∈ O(log2 n)
UW CSE326 Sp ’02: 2—Analysis
27

Download Report