Asymptotic Behavior
Algorithm : Design & Analysis
[2]
In the last class…





Goal of the Course
Algorithm: the Concept
Algorithm Analysis: the Contents
Average and Worst-Case Analysis
Lower Bounds and the Complexity of
Problems
Asymptotic Behavior




Asymptotic growth rate
The Sets ,  and 
Complexity Class
An Example: Searching an Ordered Array



Improved Sequential Search
Binary Search
Binary Search Is Optimal
How to Compare Two Algorithm?

Simplifying the analysis




assumption that the total number of steps is roughly
proportional to the number of basic operations counted (a
constant coefficient)
only the leading term in the formula is considered
constant coefficient is ignored
Asymptotic growth rate

large n vs. smaller n
Relative Growth Rate
Ω(g):functions that grow at
least as fast as g
g
Θ(g):functions that grow at
the same rate as g
Ο(g):functions that grow no
faster as g
The Set “Big Oh”

Definition


Giving g:N→R+, then Ο(g) is the set of f:N→R+, such that
for some cR+ and some n0N, f(n)cg(n) for all nn0.
A function fΟ(g) if limn→[f(n)/g(n)]=c<

Note: c may be zero. In that case, f(g), “little Oh”
Example

Let f(n)=n2, g(n)=nlgn, then:

fΟ(g), since limn→[f(n)/g(n)]= limn→[n2/nlgn]=
limn→[n/lgn]= limn→[1/(1/nln2)]=

gΟ(f), since limn→[g(n)/f(n)]= limn→[nlogn / n2]=
=0
Logarithmic Functions and Powers
Which grows faster?
log 2 n or
n?
For your reference: L’Hôpital’s rule
So, log2nO(n)
f ( n)
f ' ( n)
lim
 lim
n  g ( n)
n  g ' ( n )
with some constraints
log 2 e
log 2 n
n
n
lim
 lim
 (2 log 2 e) lim
0
n 
n 
n  n
1
n
2 n
The Result Generalized

The log function grows more slowly than any
positive power of n
lgn  o(n) for any >0
By the way:
The power of n grows more slowly than any exponential
function with base greater than 1
nk  o(cn) for any c>1
Factorials and Exponential Functions

n! grows faster than 2n for any positive integer n.
n
n!
lim n  lim
n  2
n 
n
2n  
n
 e   lim 2n  n   


n
n 
2
 2e 
Stirling' s formular : n!
n
2n  
e
n
The Sets  and 

Definition


A function f(g) if limn→[f(n)/g(n)]>0


Note: the limit may be infinity
Definition


Giving g:N→R+, then (g) is the set of f:N→R+, such that
for some cR+ and some n0N, f(n)cg(n) for all nn0.
Giving g:N→R+, then (g) = Ο(g) (g)
A function fΟ(g) if limn→[f(n)/g(n)]=c,0<c<
How Functions Grow
Algorithm
Time function(ms)
1
2
3
4
33n
46n lg n
13n2
3.4n3
Input size(n)
2n
Solution time
10
0.00033 sec.
0.0015
sec.
0.0013 sec.
0.0034 sec.
0.001 sec.
100
0.0033 sec.
0.03 sec.
0.13 sec.
3.4 sec.
41016 yr.
1,000
0.033 sec.
0.45 sec.
13 sec.
0.94 hr.
10,000
0.33 sec.
6.1 sec.
22 min.
39 days
100,000
3.3 sec.
1.3 min.
1.5 days
108 yr.
Time allowed
Maximum solvable input size (approx.)
1 second
30,000
2,000
280
67
20
1 minute
1,800,000
82,000
2,200
260
26
Increasing Computer Speed
Number of steps
performed
on input of size n
f(n)
Maximum feasible
input size
s
Maximum feasible
input size
in t times as much time
snew
lg n
s1
s1 t
n
s2
t s2
n2
s3
t s3
n3
s4
s4+lg n
Sorting a Array of 1 Million Numbers
Using merge sort,
taking time 50nlogn:
100 seconds!
Computer B
10 Mips
Computer A
1000 Mips
Using insertion sort, taking time 2n2:
2000 seconds!
Properties of O,  and 

Transitive property:


Symmetric properties



If fO(g) and gO(h), then fO(h)
fO(g) if and only if g(f)
f(g) if and only if g(f)
Order of sum function

O(f+g)=O(max(f,g))
Complexity Class
Let S be a set of f:NR* under consideration, define
the relation ~ on S as following: f~g iff. f(g)
then, ~ is an equivalence.
 Each set (g) is an equivalence class, called
complexity class.
 We usually use the simplest element as possible as
the representative, so, (n), (n2), etc.

Searching an Ordered Array

The Problem: Specification


Input:

an array E containing n entries of numeric type sorted in
non-decreasing order

a value K
Output:

index for which K=E[index], if K is in E, or, -1, if K is
not in E
Sequential Search: the Procedure

The Procedure







Int seqSearch(int[] E, int n, int K)
1. Int ans, index;
2. Ans=-1; // Assume failure
3. For (index=0; index<n; index++)
4.
If (K==E[index]) ans=index;//success!
5.
break;
6. return ans
Searching a Sequence

For a given K, there are two possibilities


K in E (say, with probability q), then K may be any one of
E[i] (say, with equal probability, that is 1/n)
K not in E (with probability 1-q), then K may be located in
any one of gap(i) (say, with equal probability, that is
1/(n+1))
gap(0)
E[1] E[2]
gap(i-1)
gap(i-1)
E[i-1] E[i] E[I+1]
E[n]
Improved Sequential Search

Since E is sorted, when an entrie larger than K is met, no nore
comparison is needed
 Worst-case complexity: n, unchanged
 Average complexity
Roughly n/2
A(n)  qAsucc (n)  (1  q ) A fail ( n)
n 1
1
Asucc (n)    (i  1) 
2
i 0  n 
n 1
n
n
 1 
 1 
A fail ( n)   
(i  1)  
n  
2 n 1
 n 1
i 0  n  1 
Note: A(n)(n)
n  n
 n  1
A(n)   
 q
  
2  n 1
 n 1 2
n 1
Divide and Conquer

If we compare K to every jth entry, we can locate the
small section of E that may contain K.




To locate a section, roughly n/j steps at most
To search in a section, j steps at most
So, the worst-case complexity: (n/j)+j, with j selected
properly, (n/j)+j(n)
However, we can use the same strategy in the small
sections recursively
Choose j = n
Binary Search
int binarySearch(int[] E, int first, int last, int K)
if (last<first)
index=-1;
else
int mid=(first+last)/2
if (K==E[mid])
index=mid;
else if (K<E[mid])
index=binarySearch(E, first, mid-1, K)
else if (K<E[mid])
index=binarySearch(E, mid+1, last, K)
return index;
Worst-case Complexity of Binary Search



Observation: with each call of the recursive
procedure, only at most half entries left for
further consideration.
At most lg n calls can be made if we want to
keep the range of the section left not less than 1.
So, the worst-case complexity of binary search
is lg n+1=⌈lg(n+1)⌉
Average Complexity of Binary Search


Observation:
 for most cases, the number of comparison is or is
very close to that of worst-case
 particularly, if n=2k-1, all failure position need
exactly k comparisons
Assumption:
 all success position are equally likely (1/n)
 n=2k-1
For your reference : Arithmetic - Geometric Series
Average Complexity of Binary Search
 i 2   i2  2 
k
k
i

i 1
i
i 1
i 1
Average
complexity
2
3
4
k
k 1

(2

2

2

3

2



(
k

1
)

2

k

2
)
 Note: We count the sum of st, which is the number of inputs
2
3
( k 1)if n=2k-1,
for which
the
algorithm
does
t
comparisons,
and
 (2  2  2  3  2    (k  1)  2
 k  2k )
st=2t-1
k

Aq ( n) (
k qA
2 k 11(n)2)(1 2qi )A(0k (n2)k 1  2)  (2 k 1  4)
i2
k
1 k
(
k

1
)
2
1
 st 
t 1
A1 ( n)   t  k 1  
t
2



 (k t 11) 2 n  2 n t 1
n
( k  1)( n  1)  1
 log n 
 lg( n  1)  1  O

n
 n 
A0  lg( n  1)
k
Aq ( n)  lg( n  1)  q
Decision Tree

A decision tree for A and a given input of size n is a binary tree
whose nodes are labeled with numbers between 0 and n-1
 Root: labeled with the index
first compared
4
 If the label on a particular node
7
1
is i, then the left child is labeled
8
the index next compared if K<E[i], 0
5
2
the right child the index next
3
compared if K>E[i], and no branch
6
for the case of K=E[i].
9
Binary Search Is Optimal

If the number of comparison in the worst case is p,
then the longest path from root to a leaf is p-1, so
there are at most 2p-1 node in the tree.

There are at least n node in the tree.
(We can prove that For all i{0,1,…,n-1}, exist a
node with the label i.)

Conclusion: n 2p-1, that is plg(n+1)
Binary Search Is Optimal

For all i{0,1,…,n-1}, exist a node with the label i.

Proof:

if otherwise, suppose that i doesn’t appear in the tree, make up
two inputs E1 and E2, with E1[i]=K, E2[i]=K’, K’>K, for any
ji, (0jn-1), E1[j]=E2[j]. (Arrange the values to keeping the
order in both arrays). Since i doesn’t appear in the tree, for both
K and K’, the algorithm behave alike exactly, and give the
same outputs, of which at least one is wrong, so A is not a
right algorithm.
Finding the Fake Coin


[Situation] You have 70 coins that are all supposed to
be gold coins of the same weight, but you know that
one coin is fake and weighs less than the others. You
have a balance scale; you can put any number of
coins on each side of the scale at one time, and it will
tell you if the two sides weigh the same, or which
side is lighter if they don’t weigh the same.
[Problem] Outline an algorithm for finding the fake
coin. How many weighings will you do?
Outline of an Algorithm
<
[1..8]:[9..16]
[1..23]:[24..46]
=
[47..54]:[55..62]
>
[24..31]:[32..40]
......
......
[47..49]:[50..52]
[47]:[48]
[53]:[54]
[63..65]:[66..68]
[50]:[51]
[55..57]:[58..60]
......
Done!
Home Assignment

pp.63 –






1.23
1.27
1.31
1.34
1.37
1.45