Recurrence Relations
• Reading Material
– Chapter 2 as a whole, but in particular Section
2.8
– Chapter 4 from Cormen’s Book.
Recurrence Relations
• Objective: To discuss techniques for solving
recurrence relations.
– These techniques will be very important and “handy”
tools for analyzing algorithms that are recursive.
• Linear Homogenous Recurrences
– Characteristic Equations
• Inhomogeneous Recurrences
– The Master Theorem
– Recurrence Expansion
– The Change of Variable Method
Linear Homogeneous Equations
• Definition: A recurrence relation is called
linear homogeneous with constant coefficients
if it is of the form:
f (n)  a1 f (n  1)  a 2 f (n  2)  ...  a k f (n  k )
• We restrict our discussion to homogeneous
recurrence equations with k=1 or k=2
Solution of Linear Homogeneous Equations
• When k=1, f ( n)  af ( n  1)and hence the solution is
• When k=2, f (n)  a1 f (n  1)  a2 f (n  2)
the following steps are followed to solve the recurrence:
– Find r1 and r2, the solutions to the characteristic equation
x  a1 x  a2  0
2
– If r1 = r2 = r, then f ( n)  c1 r  c 2 nr
n
n
f
(
n
)

c
r

c
r
Otherwise
1 1
2 2
– Determine c1 and c2 from the initial values f(n0) and f(n0+1)
n
n
Examples
3
• f ( n)  
2 f (n  1)
n0
n 1
3

• g ( n )  4
3g (n  1)  2 g (n  2)

2

• h(n)  3
2h(n  1)  h(n  2)

n0
n 1
n2
n0
n 1
n2
Inhomogeneous Equations
• Definition: A recurrence relation is called
inhomogeneous if it is not a homogeneous
recurrence relation. In particular, we will
look at
k
f (n)   [ g i (n) f (n  i )]  g 0 (n)
i 1
where i 1  i  k  gi(n) is not a constant or
g0(n)  0.
Solution for Inhomogeneous Recurrences
• No general method for solving inhomogeneous
recurrences exists. However,
– There are cases where a formula for a class of
inhomogeneous recurrence relations exists (Master
Theorem)
– The rest depend on experience and/or trial and
error in choosing one of the following techniques
• Expansion
• Substitution
• Change of variable
Master Theorem
•
Theorem: Let a1 and b>1 be constants, let g(n)
be a function, and let f(n) be defined on the
nonnegative integers by the recurrence
n
f (n)  af ( )  g (n)
b
where we interpret n/b to mean either n/b or
n/b. Then f(n) can be bounded asymptotically
as follows
–
–
–
If >0  g(n)=O(nlogba - ), then f(n) = (nlogba).
If g(n)= (nlogba), then f(n) = (nlogba log n).
If >0  g(n)=(nlogba + ), and if c<1, n0 
ag(n/b)  cg(n) n > n0, then f(n) = (g(n)).
Examples
• f ( n)  16 f ( n / 4)  n
• f ( n)  f (3n / 4)  2
• f ( n)  3 f ( n / 4)  n log n
Gaps in the Master Theorem
• The three cases in the theorem do not cover
all the possibilities for g(n). There are gaps
between cases 1 and 2, and 2 and 3.
– For example, can we apply the Master theorem
on the following recurrence?
n
f (n)  2 f ( )  n log n
2
The Expansion Method
• When expanding few terms of some recurrences,
a solution may become more apparent.
• Example: Consider the previous recurrence
d

f ( n)  
n
2 f ( 2 )  n log n
n  1, d  0
n 1
The Change of Variables Method
• The idea is to change the domain of the
function and define a new recurrence in the
new domain whose solution may be easier
to obtain or is already known. Once the
solution is obtained, we convert the domain
of the function back to its original domain.
Example
n 1
d
• f ( n)  
 2 f ( n )  log n n  2
 