Recurrences
Recurrences
Recurrence
A recurrence is an equality or inequality on a function that
uses values of the function on smaller inputs.
Examples:
(
1
if n = 1
T (n) =
T (n − 1) + 1 if n > 1
(
1
if n = 1
T (n) =
2T (bn/2c) + n if n > 1
(
0
if n ≤ 2
T (n) =
√
T (b nc) + 1 if n > 2
(
1
if n = 1
T (n) =
T (b2n/3c) + T (bn/3c) + 1 if n > 1
Recurrences
Methods for solving recurrences
The substitution method.
The iteration method.
The master method.
Recurrences
The substitution method
In the substitution method, we guess the answer to the
recurrence and then prove the correctness using induction.
Recurrences
Example 1: Guessing the exact solution
(
1
Recurrence T (n) =
2T (n/2) + n
if n = 1
if n > 1
Guess T (n) = n log n + n.
Base For n = 1, n log n + n = 1 = T (1).
Induction step Assume T (m) = m log m + m for all m < n.
T (n) = 2T (n/2) + n
n
n n
=2
+ n by the induction hypothesis
log +
2
2 2
n
= n log + n + n
2
= n log n − n log 2 + n + n
= n log n − n + n + n
= n log n + n
Recurrences
Example 2: Guessing upper and lower bounds
(
1
Recurrence T (n) =
2T (n/2) + n
if n = 1
if n > 1
Guess T (n) ≤ cn log n for n ≥ 2.
Base T (2) = 2T (1)+2 = 4 ≤ c ·2 log 2 = 2c ⇒ c ≥ 2.
Induction step Assume T (m) ≤ cm log m for all 2 ≤ m < n.
T (n) = 2T (n/2) + n
n
n
≤ 2 c log
+n
2
2
n
= cn log + n
2
= cn log n − cn + n
≤ cn log n if −cn + n ≤ 0 ⇒ c ≥ 1
⇒ T (n) ∈ O(n log n).
Recurrences
Lower bound
Guess T (n) ≥ dn log n for all n.
Base T (1) = 1 ≥ d · 1 log 1 = 0.
Induction step Assume T (m) ≥ dm log m for all 2 ≤ m < n.
T (n) = 2T (n/2) + n
n
n
≥ 2 d log
+n
2
2
n
= dn log + n
2
= dn log n − dn + n
≥ dn log n if −dn + n ≤ 0 ⇒ d ≤ 1
⇒ T (n) ∈ Ω(n log n).
SinceT (n) ∈ O(n log n), we conclude that T (n) ∈ Θ(n log n).
Recurrences
Example 3: Revising the guess
Recurrence T (n) = 8T (n/2) + n2 .
Guess T (n) ≤ cn3 .
Induction step
T (n) = 8T (n/2) + n2
n 3
≤ 8c
+ n2
2
n3
= 8c ·
+ n2
8
= cn3 + n2
6≤ cn3
Recurrences
Example 3: Revising the guess
Sometimes the guess needs to be revised by subtracting a
lower-order term from the previous guess.
Guess T (n) ≤ cn3 − dn2 .
Induction step
T (n) = 8T (n/2) + n2
n 2 n 3
≤8 c
−d
+ n2
2
2
n3
n2
= 8c ·
− 8d ·
+ n2
8
4
= cn3 − 2dn2 + n2
= cn3 − dn2 − dn2 + n2
≤ cn3 − dn2
if −dn2 + n2 ≤ 0 ⇒ d ≥ 1
Recurrences
The iteration method
Expand the recurrence into a summation.
Evaluate the summation.
Recurrences
Example 1
(
0
if n = 0
Recurrence T (n) =
T (n − 1) + 1 if n > 0
Iteration
T (n) = T (n − 1) + 1
= T (n − 2) + 1 + 1
= T (n − 3) + 3
..
.
= T (0) + n
=n
Recurrences
Example 2
(
1
Recurrence T (n) =
T (n − 1) + n
if n = 1
if n > 1
Iteration
T (n) = T (n − 1) + n
= T (n − 2) + (n − 1) + n
= T (n − 3) + (n − 2) + (n − 1) + n
..
.
= 1 + 2 + ··· + n
n(n + 1)
=
2
Recurrences
Changing free terms
If we are only interested in the asymptotic behavior, the free
terms in the recurrence can be simplified.
(
1
if n = 1
T (n) =
T (n − 1) + n if n > 1
(
3
if n = 1
T2 (n) =
T2 (n − 1) + 3n if n > 1
(
1
if n = 1
T3 (n) =
√
T3 (n − 1) + n + 2 n if n > 1
T2 (n) = 3T (n) for all n (T2 (n) = 3 + 6 + · · · + 3n).
T (n) ≤ T3 (n) ≤ 3T (n) for all n.
Recurrences
Recurrence tree
T (n) = T (n/3) + T (2n/3) + n.
Recurrences
Recurrence tree
T (n) = T (n/3) + T (2n/3) + n.
Recurrences
Recurrence tree
T (n) = T (n/3) + T (2n/3) + n.
Recurrences
Recurrence tree
T (n) = T (n/3) + T (2n/3) + n.
1
log 3
· n log n = n · log3 n ≤ T (n) ≤ n · log3/2 n =
⇒ T (n) ∈ Θ(n log n).
Recurrences
1
log(3/2)
· n log n
The master method
A method for solving recurrences of the form
T (n) = aT (n/b) + f (n)
where a ≥ 1, b > 1, and f (n) is an asymptotic positive
function.
Recurrences
The master theorem
Let T (n) = aT (n/b) + f (n) where a ≥ 1, b > 1 and f (n) is
asymptotically positive.
1
If f (n) ∈ O(nlogb a− ) for some constant > 0 then
T (n) ∈ Θ(nlogb a ).
2
If f (n) ∈ Θ(nlogb a ) then T (n) ∈ Θ(f (n) · log n).
3
f (n) ∈ Ω(nlogb a+ ) for some constant > 0 and if
af (n/b) ≤ cf (n) for some constant c < 1 and all
sufficiently large n, then T (n) ∈ Θ(f (n)).
T (n/b) means either bn/bc or dn/be.
Case 2 can be generalized: If f (n) ∈ Θ(nlogb a logk n) for
some k ≥ 0 then T (n) ∈ Θ(f (n) · logk+1 n).
Recurrences
Example 1 (case 1)
T (n) = 9T (n/3) + n.
We have a = 9, b = 3, f (n) = n.
logb a = log3 9 = 2.
All the conditions of the first case are satisfied:
a = 9 ≥ 1, b = 3 > 1, f (n) = n ≥ 0.
For = 1/2, f (n) ∈ O(nlogb a− ) = O(n3/2 ).
Hence, T (n) ∈ Θ(nlogb a ) = Θ(n2 ).
Recurrences
Example 2 (case 1)
T (n) = 5T (n/2) + n2 .
We have a = 5, b = 2, f (n) = n2 .
logb a = log2 5 ≈ 2.322.
All the conditions of the first case are satisfied:
a = 5 ≥ 1, b = 2 > 1, f (n) = n2 ≥ 0.
For = 0.3, f (n) ∈ O(nlogb a− ) = O(n2.022 ).
Hence, T (n) ∈ Θ(nlogb a ) = Θ(nlog2 5 ).
Recurrences
Example 3 (case 2)
T (n) = T (2n/3) + 1.
We have a = 1, b = 3/2, f (n) = 1.
logb a = log3/2 1 = 0.
All the conditions of the second case are satisfied:
a = 1 ≥ 1, b = 3/2 > 1, f (n) = 1 ≥ 0.
f (n) ∈ Θ(nlogb a ) = Θ(1).
Hence, T (n) ∈ Θ(f (n) · log n) = Θ(log n).
Recurrences
Example 4 (case 3)
T (n) = 3T (n/4) + n log n.
We have a = 3, b = 4, f (n) = n log n.
logb a = log4 3 ≈ 0.792.
All the conditions of the third case are satisfied:
a = 3 ≥ 1, b = 4 > 1, f (n) = n log n ≥ 0.
For = 0.1, f (n) ∈ Ω(nlogb a+ ) = Ω(n0.892 ).
For all n, af (n/b) = 3 n4 log n4 ≤ 43 n log n = 43 f (n).
Hence, T (n) ∈ Θ(f (n)) = Θ(n log n).
Recurrences
Example 5
T (n) = 2T (n/2) + n/ log n.
We have a = 2, b = 2, f (n) = n/ log n.
logb a = log2 2 = 1.
n/ log n < n, but n/ log n ∈
/ O(nlogb a− ) for every > 0.
Recurrences