Mathematics for Computer Science
MIT 6.042J/18.062J
Asymptotic
Notation
Albert R Meyer, November 6, 2009
lec 9F.1
Closed form for n!
n
n! ::= 1  2  3  (n -1)  n =  i
i=1
Turn product into a sum taking logs:
ln(n!) = ln( 1·2·3···(n – 1)·n ) =
ln 1 + ln 2 + · · · + ln(n – 1) + ln(n)
n
=  ln(i)
i=1
Albert R Meyer, November 6, 2009
lec 9F.2
Closed form for n!
n
 ln(i)
Integral Method to bound
i=1
ln n
…
ln 5
ln 4
ln 3
ln(x)
ln(x+1)
ln 2
ln 2
1
2
ln 3 ln 4
3
ln 5
4
…
5
Albert R Meyer, November 6, 2009
ln
ln n
n-1
n–2
n–1
n
lec 9F.3
n
Closed form for n!
n

ln(x)
dx

ln(i)
n ln
+1

1  e 
i=1
i=1
nn
 n +1 
 (n
+1)ln
+
0.6
ln(x
+1)
dx


1
 e 
n
reminder:
x
ln
x
dx
=
x
ln
 

e
Albert R Meyer, November 6, 2009
lec 9F.5
Closed form for n!
1
n 
ln(i)  (n + )ln  

2 e
i=1
n
exponentiating:
n
n 
n!  n / e  
e
Albert R Meyer, November 6, 2009
lec 9F.6
Stirling’s Formula
A precise approximation:
n! ~
 n
2š n  
 e
Albert R Meyer, November 6, 2009
n
lec 9F.7
Little Oh:
o(∙) Oh
Little
Asymptotically smaller :
Def: f(n) = o(g(n))
iff
f(n)
lim
=0
n g(n)
Albert R Meyer, November 6, 2009
lec 9F.13
Little Oh:
o(∙)
n = o(n )
2
because
3
n
1
lim 3 = lim = 0
n• n
n n
2
Albert R Meyer, November 6, 2009
lec 9F.14
O(∙)
Big
Big Oh:
Oh
Asymptotic Order of Growth:
f(n) = O(g(n))
 f(n) 
limsup 

•

 g(n) 
n
a technicality -- ignore now
Albert R Meyer, November 6, 2009
lec 9F.15
Big Oh: O(∙)
3n = O(n )
2
because
2
3n
lim 2  3  
n• n
2
Albert R Meyer, November 6, 2009
lec 9F.16
Theta: Θ(∙)
Same Order of Growth:
f(n) = Θ(g(n))
Def: f(n)=O(g(n))
and
g(n)=O(f(n))
Albert R Meyer, November 6, 2009
lec 9F.17
Asymptotics: Intuitive
Summary
f ~ g:
f = o(g):
f = O(g):
f & g nearly equal
f much less than g
f roughly
g
f = Θ(g):
f & g roughly equal
Albert R Meyer, November 6, 2009
lec 9F.18
The Oh’s
lemma:
If f = o(g) or f ~ g, then f = O(g)
lim = 0 or lim = 1 IMPLIES lim
Albert R Meyer, November 6, 2009
lec 9F.19
The Oh’s
If f = o(g), then g
O(f) f
lim = 0
g
IMPLIES
g
lim =
f
Albert R Meyer, November 6, 2009
lec 9F.21
Big Oh:
O(∙)
Equivalent definition:
f(n) = O(g(n))
c,n0
f(n)
n n 0.
c·g(n)
Albert R Meyer, November 6, 2009
lec 9F.22
Big Oh:
O(∙)
f(x) = O(g(x))
↑
log
scale
↓
green stays
below purple
from here on
c· g(x)
ln c
f(x)
no
Albert R Meyer, November 6, 2009
lec 9F.23
Little Oh:
Lemma:
Proof:
o(∙)
xa = o(xb) for a < b
x
1 and b - a > 0
=
b
b-a
x x
a
so as x
1
,

0
b-a
x
Albert R Meyer, November 6, 2009
lec 9F.24
Little Oh:
Lemma:
ln x =
for ε > 0.
o(∙)
ε
o(x )
Albert R Meyer, November 6, 2009
lec 9F.25
Little Oh:
Lemma:
n
x
=
for a > 1.
o(∙)
x
o(a )
Albert R Meyer, November 6, 2009
lec 9F.29
Little Oh:
o(∙)
proofs:
L’Hopital’s Rule,
McLaurin Series
(see a Calculus text)
Albert R Meyer, November 6, 2009
lec 9F.30
Big Oh Mistakes
“∙ = O(∙)” defines a relation
Don’t write O(g) = f.
Otherwise: x = O(x), so O(x) = x.
But 2x = O(x), so
2x = O(x) = x,
therefore
2x = x.
Nonsense!
Albert R Meyer, November 6, 2009
lec 9F.31
Team Problems
Problems
1-4
Albert R Meyer, November 6, 2009
lec 9F.35