Lecture 3: More Big O, and Complexity
CS204/209 : Algorithms (and Scientific Computing)
Wednesday, 23 September 2009
CS204/209 — Lecture 3: More Big O, and Complexity
1/17
In today’s class
1 Recall ... The Big-O
Some properties of O
CS204/209 — Lecture 3: More Big O, and Complexity
For more details, see Sections 2.2 and
3.1 of Cormen et al’s Introduction to
Algorithms, and Sections 2.1 and 2.2 of
Rosen’s Discrete Mathematics (511
ROS).
2/17
The Gossipers Problem
Have you solved the problem?
The following table gives the number of calls made for
n = 2, 3, 4, . . . people.
n
2
3
4
5
6
7
..
.
n
No of calls
1
3
4
6
10
8
10
..
.
???
Can you find a procedure that extends an algorithm, say, for
solving the problem with 4 people to one with 5 people?
Can you come up with a general formula for the number of calls
need for n gossipers?
CS204/209 — Lecture 3: More Big O, and Complexity
3/17
Problem 2
Here is a “mutilated chess board”:
; ; ; ; ;
::99:9 ::99:9::99:9 ::99:9 ::99:9 <;<;< <<;<; <<;<; <<;<; <<;<;
:9:99: :9:9:9 :9:9:9 :9:9:9 :9:9:9
<;<;;< <;<<;; <;<<;; <;<<;; <;<<;;
:9::99:9::99 :9::99 :9::99 :9::99
<;; <;; <;; <;; <;;
:9:9 :9 :9 :9 <<;<; <<;<; <<;<; <<;<; <<;<;
22111 22111 22111 22111 22111 44333 44333 44333 44333 44333
212212211212212211 212212211 212212211 212212211
434434433 434434433 434434433 434434433 434434433
1 1 1 1 1
3 3 3 3 3
222111221121221121221121221121 443343443343443343443343444333
**)*))*)**)*))*) **)*))*) **)*))*) **)*))*) ,,+,++,+ ,,+,++,+ ,,+,++,+ ,,+,++,+ ,,+,++,+
*)**)) *)**)) *)**)) *)**)) *)**))
,+,,++ ,+,,++ ,+,,++ ,+,,++ ,+,,++
*)*))*)*)) *)*)) *)*)) *)*))
,+ ,+ ,+ ,+ ,+
*)*)*)*)*) ,,+++,,+++,,+++,,+++,,+++
"!""!!"!""!! "!""!! "!""!! "!""!! $#$$## $#$$## $#$$## $#$$## $#$$##
! ! ! ! !
# # # # #
"""!!""!!"""!!""!! """!!""!! """!!""!! """!!""!!
$$$##$$## $$$##$$## $$$##$$## $$$##$$## $$$##$$##
"!"!!"!"!! "!"!! "!"!! "!"!!
$#$## $#$## $#$## $#$## $#$##
= = = = =
>=>=> >>=>= >>=>= >>=>= >>=>=
>=>==> >=>>== >=>>== >=>>== >=>>==
>=>== >=>== >=>== >=>== >=>==
>>= >>= >>= >>= >>=
65 65 65 65 65
65655 65655 65655 65655 65655
6566655 6566655 6566655 6566655 6566655
565 565 565 565 655
66556655665566556655
.-- .-- .-- .-- .-..-.- ..-.- ..-.- ..-.- ..-..-..-- .-..-- .-..-- .-..-- .-..-.-- .-- .-- .-- .- ..--..--..--..--..- &%&% &%&% &%&% &%&% &%&%
&%&&%% &%&&%% &%&&%% &%&&%% &%&&%%
&%&&%% &%&&%% &%&&%% &%&&%% &%&&%%
&%&%% &%&%% &%&%% &%&%% &%&%%
? ? ? ? ?
@?@?@ @@?@? @@?@? @@?@? @@?@?
@?@??@ @?@@?? @?@@?? @?@@?? @?@@??
@?@?? @?@?? @?@?? @?@?? @?@??
@@? @@? @@? @@? @@?
87 87 87 87 87
87877 87877 87877 87877 87877
8788877 8788877 8788877 8788877 8788877
787 787 787 787 787
88778877887788778877
0// 0// 0// 0// 0//
00/0/ 00/0/ 00/0/ 00/0/ 00/0/
0/00// 0/00// 0/00// 0/00// 0/00//
0// 0// 0// 0// 0//
00//00//00//00//00//
('(' ('(' ('(' ('(' ('('
('(('' ('(('' ('(('' ('(('' ('((''
('(('' ('(('' ('(('' ('(('' ('((''
('('' ('('' ('('' ('('' ('(''
Suppose you have 31 dominoes, each the size of two of the
squares. Can you cover the remaining 62 squares with the 31
dominoes? If not, can you prove its impossible?
CS204/209 — Lecture 3: More Big O, and Complexity
4/17
Recall ... The Big-O
Towards the end of last week’s class, we discussed the need to
compare algorithms, with particular regard for how long they take
to run given particular inputs.
As we will see, this is often expressed a function that gives the
number of times a particular operation is carried out, e.g., a
multiplication or comparison.
For example, if f (n) is the number of comparisons required by
Bubble Sort and g (n) the number of comparisons required by
Merge Sort for a list of length n, can we say f (n) ≤ g (n), or vice
versa.
This is usually expressed using Big-O notation.
CS204/209 — Lecture 3: More Big O, and Complexity
5/17
Recall ... The Big-O
Definition (Big-O)
Given functions f : N → R and g : N → R, we say that
f is O g
or
f (x) = O(g (x))
if there are constants C and k such that
|f (x)| ≤ C |g (x)|
for all x > k.
In English, we read this as “f is big-oh of g ”.
It means that f does not grow any faster than g .
CS204/209 — Lecture 3: More Big O, and Complexity
6/17
Recall ... The Big-O
Example
Show that f (x) = x 2 + 2x + 1 is O(x 2 ).
Note that x 2 is also O x 2 + 2x + 1 .
CS204/209 — Lecture 3: More Big O, and Complexity
7/17
Recall ... The Big-O
Example
Show that f (x) = 7x 2 is O(x 3 ).
Is it the case that x 3 is O(7x 2 )?
CS204/209 — Lecture 3: More Big O, and Complexity
8/17
Recall ... The Big-O
1.
n
X
Some properties of O
ak x k = O(x n ).
k=0
CS204/209 — Lecture 3: More Big O, and Complexity
9/17
Recall ... The Big-O
2.
n
X
Some properties of O
k = 1 + 2 + 3 + · · · + n = O(n2 )
k=1
CS204/209 — Lecture 3: More Big O, and Complexity
10/17
Recall ... The Big-O
Some properties of O
3. n! is O(nn ).
4. log10 (n) = O(ln(n)) = O(log2 (n)) = O(logX (n)), for any
logk x
sensible X . (Here we use that logb n =
).
logk b
CS204/209 — Lecture 3: More Big O, and Complexity
11/17
Recall ... The Big-O
Some properties of O
5. log2 (n) is O(n). (This follows from the fact that n < 2n .)
6. log(n!) is O(n log(n)).
CS204/209 — Lecture 3: More Big O, and Complexity
12/17
Recall ... The Big-O
Some properties of O
Let f1 , f 2, g1 and g2 be functions such that
f1 (x) is O(g1 (x))
and
f2 (x) is O(g2 (x)).
Then
7. f1 + f2 )(x) is O max{|g1 (x)|, |g2 (x)} .
8. f1 f2 )(x) is O g1 (x)g2 (x) .
CS204/209 — Lecture 3: More Big O, and Complexity
13/17
Recall ... The Big-O
Some properties of O
Example
Give a big-O estimate for f (n) = 3n log(n!) + (n2 + 3) log(n) for
positive integers n.
CS204/209 — Lecture 3: More Big O, and Complexity
14/17
Recall ... The Big-O
Some properties of O
Exercise
Give big-O estimates for the following functions
1
f (x) = (x + 1) log(x 2 + 1) + 3x 2 .
f (n) = 1 + 1/2 + +1/3 + · · · + 1/n.
Zn
n
X
1
1
Hint for part (b): first establish that
<
dx.
j
1 x
2
j=2
CS204/209 — Lecture 3: More Big O, and Complexity
15/17
Recall ... The Big-O
Some properties of O
Exercise
From lowest to highest, what is the correct order of the
complexities:
O(n2 ), O(log2 n), O(1), O(2n ), O(n!), O(n3 ), O(3n ), O(n log2 n)?
Exercise
Which of the following statements are true?
(i) if f = O(g ) and g = O(h) then f = O(h).
(ii) if f = O(g ) and g = O(f ) then f = g .
CS204/209 — Lecture 3: More Big O, and Complexity
16/17
Recall ... The Big-O
Some properties of O
Exercise
For each of the following, determine how large a problem can be
solved in 1 second using an algorithm that requires f (n)
operations, where each operation is carried out in 10−9 seconds.
(i) f (n) = n
(ii) f (n) = log n
(iii) f (n) = 2n
(iv) f (n) = n log n
(v) f (n) = n!
(vi) f (n) = n2
CS204/209 — Lecture 3: More Big O, and Complexity
17/17