Lecture 3: More Big O, and Complexity
CS204/209 : Algorithms (and Scientific Computing)
September 21, 2011
CS204/209 — Lecture 3: More Big O, and Complexity
1/16
In today’s class
1 Recall ... The Big-O
Some properties of O
CS204/209 — Lecture 3: More Big O, and Complexity
For more details, see Sections 2.2 and
3.1 of Cormen et al’s Introduction to
Algorithms, and Sections 2.1 and 2.2 of
Rosen’s Discrete Mathematics (511
ROS).
2/16
Recall ... The Big-O
Towards the end of last week’s class, we discussed the need to
compare algorithms, with particular regard for how long they take
to run given particular inputs.
As we will see, this is often expressed a function that gives the
number of times a particular operation is carried out, e.g., a
multiplication or comparison.
For example, if f (n) is the number of comparisons required by
Bubble Sort and g (n) the number of comparisons required by
Merge Sort for a list of length n, can we say f (n) ≤ g (n), or vice
versa.
This is usually expressed using Big-O notation.
CS204/209 — Lecture 3: More Big O, and Complexity
3/16
Recall ... The Big-O
Definition (Big-O)
Given functions f : N → R and g : N → R, we say that
f is O g
or
f (x) = O(g (x))
if there are constants C and k such that
|f (x)| ≤ C |g (x)|
for all x > k.
In English, we read this as “f is big-oh of g ”.
It means that f does not grow any faster than g .
CS204/209 — Lecture 3: More Big O, and Complexity
4/16
Recall ... The Big-O
Example
Show that f (x) = x 2 + 2x + 1 is O(x 2 ).
Note that x 2 is also O x 2 + 2x + 1 .
CS204/209 — Lecture 3: More Big O, and Complexity
5/16
Recall ... The Big-O
Example
Show that f (x) = 7x 2 is O(x 3 ).
Is it the case that x 3 is O(7x 2 )?
CS204/209 — Lecture 3: More Big O, and Complexity
6/16
Recall ... The Big-O
1.
n
X
Some properties of O
ak x k = O(x n ).
k=0
CS204/209 — Lecture 3: More Big O, and Complexity
7/16
Recall ... The Big-O
2.
n
X
Some properties of O
k = 1 + 2 + 3 + · · · + n = O(n2 )
k=1
CS204/209 — Lecture 3: More Big O, and Complexity
8/16
Recall ... The Big-O
Some properties of O
3. n! is O(nn ).
4. log10 (n) = O(ln(n)) = O(log2 (n)) = O(logX (n)), for any
logk x
sensible X . (Here we use that logb x =
).
logk b
CS204/209 — Lecture 3: More Big O, and Complexity
9/16
Recall ... The Big-O
Some properties of O
5. log2 (n) is O(n). (This follows from the fact that n < 2n .)
6. log(n!) is O(n log(n)).
CS204/209 — Lecture 3: More Big O, and Complexity
10/16
Recall ... The Big-O
Some properties of O
Let f1 , f2 , g1 and g2 be functions such that
f1 (x) is O(g1 (x))
and
f2 (x) is O(g2 (x)).
Then
7. f1 + f2 )(x) is O max{|g1 (x)|, |g2 (x)|} .
8. f1 f2 )(x) is O g1 (x)g2 (x) .
CS204/209 — Lecture 3: More Big O, and Complexity
11/16
Recall ... The Big-O
Some properties of O
Example
Give a big-O estimate for f (n) = 3n log(n!) + (n2 + 3) log(n) for
positive integers n.
CS204/209 — Lecture 3: More Big O, and Complexity
12/16
Recall ... The Big-O
Some properties of O
Exercise
From lowest to highest, what is the correct order of the
complexities:
O(n2 ), O(log2 n), O(1), O(2n ), O(n!), O(n3 ), O(3n ), O(n log2 n)?
Exercise
Which of the following statements are true?
(i) if f = O(g ) and g = O(h) then f = O(h).
(ii) if f = O(g ) and g = O(f ) then f = g .
CS204/209 — Lecture 3: More Big O, and Complexity
13/16
Recall ... The Big-O
Some properties of O
Exercise
For each of the following, determine how large a problem can be
solved in 1 second using an algorithm that requires f (n)
operations, where each operation is carried out in 10−9 seconds.
(i) f (n) = n
(ii) f (n) = log n
(iii) f (n) = 2n
(iv) f (n) = n log n
(v) f (n) = n!
(vi) f (n) = n2
CS204/209 — Lecture 3: More Big O, and Complexity
14/16