Recurrence Relation Examples
A recurrence relation recursively defines a sequence and is basically the basis for
analyzing recursive formulas and algorithms. For any recursive algorithm or formula,
we can reduce it to its essential terms and find its performance time, often shown
in Big-Oh notation. Recurrence relations are not easy for most people. It takes
time and practice while making a lot of errors to get it down. The following are
five examples that do not even scratch the surface, but will help you understand
one method of solving them.
1. T(n) = T(n-1)+n, T(1) = 1
In this relation, each recurrence step is subtracting 1 from n.
Substituting Equations
n -> n-1
T(n)
T(n)
T(n)
T(n)
T(n)
=
=
=
=
=
T(n-1)+n
[T(n-2)+(n-1)]+n = T(n-2)+(n-1)+n
T(n-3)+(n-2)+(n-1)+n
T(n-4)+(n-3)+(n-2)+(n-1)+n
T(n-5)+(n-4)+(n-3)+(n-2)+(n-1)+n
T(n-1)
T(n-2)
T(n-3)
T(n-4)
=
=
=
=
T(n-2)+n-1
T(n-3)+n-2
T(n-4)+n-3
T(n-5)+n-4
ith
1
2
3
4
5
Now we can see a pattern and the question would be, “What is the kth step,” because
we want to generalize it. In this case, the bold numbers highlight the step
relationship. We can now go to the kth step to generalize the recurrence relation:
T(n) = T(n-k)+(n-k+1)+(n-k+2)+(n-k+3)+...+(n-1)+n
But do not take my word for it. Try it out with different values of k.
We are not done because we still have the recurrence on both sides of the equation.
To find the solution, we need to get it to the “base case” of T(1) while we have
the T(n-k) in the equation. If we let n-k = 1 so that it would be T(1), we can
solve for k, i.e. k = n-1 and substitute it in the generalization because we need
it in terms of n:
T(n) = T(n-(n-1))+(n-(n-1)+1)+(n-(n-1)+2)+(n-(n-1)+3)+...+(n-1)+n
= T(1)+2+3+...(n-1)+n
And this sequence is a sum sequence! Therefore,
T(n) = (n(n+1))/2 = O(n2)
2. T(n) = 2T(n/2)+1, T(1) = 1, n=2k
In this relation, each recurrence step is dividing n in half.
Substituting Equations
n -> n/2
T(n)
T(n)
T(n)
T(n)
T(n)
= 2T(n/2)+1
= 2[2T(n/4 )+1]+ 1
= 4[2T(n/8 )+1]+ 3
= 8[2T(n/16)+1]+ 7
= 16[2T(n/32)+1]+15
= 4T(n/4 )+ 3
= 8T(n/8 )+ 7
= 16T(n/16)+15
= 32T(n/32)+31
T(n/2)
T(n/4)
T(n/8)
T(n/16)
=
=
=
=
2T(n/4)+1
2T(n/8)+1
2T(n/16)+1
2T(n/32)+1
ith
1
2
3
4
5
You can see a pattern forming here, although it may not be instantly apparent.
Identifying the pattern is not always easy and takes time and practice. In this
example, you can see the powers of 2 (bolded) and this can relate to the recursion
step number. Therefore you can generalize it as 2k for the kth step. You may also
notice that constant (red) is 1 less than the 2k, thus 2k-1. Now we can generalize
it all:
T(n) = 2kT(n/2k)+2k-1
We still have the recurrence on both sides, so we need to get it down to T(1) which
we know equals 1. This is where the n=2k comes in. This comes from the fact that,
for T(n/2k) = T(1), n must equal 2k. We can substitute this back into the equation
remembering that we are trying to put the relation into terms of n. This is a little
different than Example 1 since we did not solve for k, but it is not necessary in
this case.
T(n) =
=
=
=
=
2kT(n/2k)+2k-1
nT(2k/2k)+n-1
nT(1)+n-1
n(1)+n-1
2n-1
Therefore,
T(n) = 2n-1 = O(n)
3. T(n) = 2T(n/2)+n, T(1) = 1, n=2k
In this relation, each recurrence step is dividing n in half.
Substituting Equations
n -> n/2
T(n)
T(n)
T(n)
T(n)
T(n)
= 2T(n/2)+n
= 2[2T(n/4 )+n/2 ]+ n
= 4[2T(n/8 )+n/4 ]+2n
= 8[2T(n/16)+n/8 ]+3n
= 16[2T(n/32)+n/16]+4n
= 4T(n/4 )+2n
= 8T(n/8 )+3n
= 16T(n/16)+4n
= 32T(n/32)+5n
T(n/2)
T(n/4)
T(n/8)
T(n/16)
=
=
=
=
2T(n/4)+n/2
2T(n/8)+n/4
2T(n/16)+n/8
2T(n/32)+n/16
ith
1
2
3
4
5
You can see a pattern forming here, which is similar to example 2. Again, it is the
powers of 2 (bolded) and this can relate to the recursion step number. Therefore
you can generalize it as 2k for the kth step. You may also notice that constant
(red) is kn. Now we can generalize it all:
T(n) = 2kT(n/2k)+kn
We still have the recurrence on both sides, so we need to get it down to T(1) which
we know equals 1. This is where the n=2k comes in. This comes from the fact that,
for T(n/2k) = T(1), n must equal 2k and we can substitute this back into the equation.
But remember that we are trying to put it into terms of n, thus we solve n=2k for
k: k=log2n
T(n) =
=
=
=
2kT(n/2k)+kn
2log2nT(2k/2k)+nlog2n
nT(1)+nlog2n
n + nlog2n
Therefore,
T(n) = n + nlog2n = O(nlog2n)
4. T(n) = 3T(n/3)+n, T(1) = 1, n=3k
In this relation, each recurrence step is dividing n by three. This is very
similar to the example 3.
Substituting Equations
n -> n/3
T(n)
T(n)
T(n)
T(n)
T(n)
= 3T(n/3)+n
= 3[3T(n/9 )+n/3 ]+ n
= 9[3T(n/27 )+n/9 ]+2n
= 27[3T(n/81 )+n/27]+3n
= 81[3T(n/243)+n/81]+4n
=
9T(n/9)+2n
= 27T(n/27)+3n
= 81T(n/81)+4n
= 243T(n/243)+5n
T(n/3)
T(n/9)
T(n/27)
T(n/81)
=
=
=
=
3T(n/9)+n/3
3T(n/27)+n/9
3T(n/81)+n/27
3T(n/243)+n/81
ith
1
2
3
4
5
You can see a pattern forming here, which is similar to example 3. In this example,
you can see the powers of 3 (bolded) and this can relate to the recursion step
number. Therefore you can generalize it as 3k for the kth step. You may also notice
that constant (red) is kn. Now we can generalize it all:
T(n) = 3kT(n/3k)+kn
We still have the recurrence on both sides, so we need to get it down to T(1) which
we know equals 1. This is where the n=3k comes in. This comes from the fact that,
for T(n/3k) = T(1), n must equal 3k and now we can substitute this back into the
equation. But remember that we are trying to put it into terms of n, thus we solve
n=3k for k: k = log3n
T(n) =
=
=
=
3kT(n/3k)+kn
3log3nT(3k/2k)+nlog3n
nT(1)+nlog3n
n + nlog3n
Therefore,
T(n) = n + nlog3n = O(nlog3n)
5. T(n) = 2T(n/2)+n3, T(1) = 1, n=2k
In this relation, each recurrence step is dividing n in half.
Substituting Equations
n -> n/2
T(n) = 2T(n/2) + n3
T(n) = 2[2T(n/4)+(n/2)3] + n3
= 4T(n/4) + n3/4 + n3
T(n) = 4[2T(n/8)+(n/4)3] + n3/4 + n3
= 8T(n/8) + n3/16 + n3/4 + n3
T(n) = 8[2T(n/16)+(n/8)3]+ n3/16 + n3/4 + n3
= 16T(n/16) + n3/64 + n3/16 + n3/4 + n3
T(n) = 16[2T(n/32)+(n/16)3] + n3/64 + n3/16 + n3/4 + n3
= 32T(n/32) + n3/256 + n3/64 + n3/16 + n3/4 + n3
ith
T(n/2)
= 2T(n/4)+(n/2)3
1
2
T(n/4)
= 2T(n/8)+(n/4)3
3
T(n/8)
= 2T(n/16)+(n/8)3
4
T(n/16) = 2T(n/32)+(n/16)3
5
This is unwieldy because we are not combining the n3 terms, but it is not as
complicated as it appears. Assuming I do not have typos here, you can probably see
the pattern forming. The bolded values generalize to 2k. The red values (only shown
in the last line) are powers of 4 and generalize to start with 4k-1.
T(n) = 2kT(n/2k) + n3/4k-1 + n3/4k-2 + n3/4k-3 + n3/4k-4 + ... + n3
Notice there is also a pattern in these n3 term exponents that is a summation. We
can replace the k-1 exponent with an i and state that it goes from 0 to k-1. Then
factor out the n3 from these terms, drop in a summation sign, wave a magic want and
say “presto chango”:
1
𝑖
T(n) = 2kT(n/2k) + n3∑𝑘−1
𝑖=0 (4)
*hand to forehead* Do not believe me? Try it with k=5 and see if it works out. (Let
me know if it does not!)
We still have the recurrence on both sides, so we need to get it down to T(1) which
we know equals 1. This is where the n=2k comes in. This comes from the fact that,
for T(n/2k) = T(1), n must equal 2k and now we can substitute this back into the
equation. But remember that we are trying to put it into terms of n, thus we solve
n=2k for k: k = log2n
1
𝑖
T(n) = 2kT(n/2k) + n3∑𝑘−1
𝑖=0 (4)
𝐥𝐨𝐠 𝐧−1 1 𝑖
(4)
1
log n−1
n3∑𝑖=0 ( )𝑖
4
= 2log2nT(2k/2k) + n3∑𝑖=0
= nT(1) +
So the remaining the question is what is the Big-Oh for this one? I will leave
you to think about that...