The master theorem method
Chapter 4. Recurrences
한양대학교 정보보호 및 알고리즘 연구실
2008. 2. 26
이재준
담당교수님 : 박희진 교수님
1
2. Recurrence solution methods
• Master method
- Definition
determining asymptotic tight bounds
for many simple recurrences of the form
T (n) aT (n / b) f (n)
T (n)
( a ≥ 1, b ≥ 1, and f(n) is given function )
f(n)
a
T (n / b) ……. T (n / b) ……. T (n / b)
2
2. Recurrence solution methods
• Master theorem Intuition
log a
Comparing the function f (n) with the function n b
Case 1 :
Case 2 :
Case 3 :
f (n) n logb a
f (n) nlogb a
f (n) nlogb a
,then the solution is
,then the solution is
,then the solution is
T (n) (n logb a )
T (n) (nlogb a lg n)
T (n) ( f (n))
3
2. Recurrence solution methods
• Master theorem Technicalities
Case 1 : f (n) n
logb a
, and f(n) must be polynomially smaller than
n logb a
Case 2 : f (n) n logb a
Case 3 : f (n) n
logb a
, and f(n) must be polynomially lager than
polynomially smaller
f (n) o(n logb a )
≈ f ( n) n n log b a
f (n) n logb a n
f ( n)
n
n logb a
f (n) o(n logb a )
n logb a
polynomially lager
f (n) (n logb a )
≈ f ( n) n log b a n
f ( n)
n
n logb a
f (n) (n logb a )
4
2. Recurrence solution methods
• Master theorem Exceptionality
Gap between cases 1 and 2 : f(n) is asymptotically smaller but, not polynomially smaller
f ( n) n
logb a
, but not
f ( n)
n
n logb a
( for some positive constant ε )
Gap between cases 2 and 3 : f(n) is asymptotically larger but, not polynomially larger
f ( n) n
logb a
, but not
f ( n)
n
n logb a
( for some positive constant ε )
or regularity condition fail to hold
5
2. Recurrence solution methods
• Master theorem Example
The recurrence is
T (n) = 2T (n/2) + n lg n
Use the master theorem to making to conclude the solution.
6
2. Recurrence solution methods
• Master theorem Solution to the example
T (n) = 2T (n/2) + n lg n ( a=2, b=2, f(n) = n lg n )
f ( n)
Since f (n) n lg n
, but not log a n
n b
a
n logb
n
f (n) n lg n
log2 2
logb a
n
n
lg n
n
( for any positive constant ε , and some
)
f(n) is asymptotically larger ,but not polynomially larger
Thus, the recurrence falls into the gap between case 2 and 3
7
2. Recurrence solution methods
• Master theorem T (n) aT (n / b) f (n)
1. If f (n) O(nlog a ), for some constant ε > 0, then T (n) (nlog a )
b
b
2. If f (n) (nlog a ), then T (n) (nlog a lg. n)
b
b
3. If f (n) (nlog a ) for some constant ε > 0,
and if af (n/b) ≤ cf (n) for some constant c < 1 and all
sufficiently large n, then T (n) ( f (.n))
b
Q : Why o/ω-notation change to O/Ω-notation bound ?
8
2. Recurrence solution methods
• Master theorem Answer to the question
T (n) = 2T (n/2) + n lg n
Since f (n) n lg n , but not
n
n
logba
f ( n)
n
n logb a
f (n) n lg n
log2 2
logb a
n
n
lg n
n
( for any positive constant ε )
To prevent something happen like above, we use O-notation and Ω-notation
for define master theorem.
9
2. Recurrence solution methods
• Master theorem Example
The recurrence is
T (n) = 3T (n/4) + n lg n
Use the master theorem to making to conclude the solution.
10
2. Recurrence solution methods
• Master theorem Solution to the example
T (n) = 3T (n/4) + n lg n ( a=3, b=4, f(n) = n lg n )
Since f (n) n lg n
n
logba
n
log34
O(n 0.793 )
→ n lg n O(n0.793 )
f (n) n
logba
, and
f ( n)
n
n logb a
f (n) n lg n
0.793
logb a
n
n
n 0.207 lg n
n
( where ε=0.2 )
So, f(n) is asymptotically larger and polynomially larger than n
logba
11
2. Recurrence solution methods
• Master theorem Solution to the example
T (n) = 3T (n/4) + n lg n ( a=3, b=4, f(n) = n lg n )
Regularity condition
af(n/b)≤ cf(n)
a f(n/b)=3(n/4)lg(n/4)
≤ cf(n)
=(3/4)n lgn ( for c=3/4 )
Thus, the solution is T (n) ( f (n)) (n lg n)
12
2. Recurrence solution methods
• Master theorem Proof of the master theorem
PART1. Analyzes with assumption that T(n) is
defined only on exact powers of b>1
PART2. Shows how the analysis can be
extended to all positive integers n.
13
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Abuse asymptotic notation when proof for exact powers
T(n)=
n
n2
if n = 1,2,4,8
otherwise
→ T(n)=O( n 2 )
Making it absolutely clear from the context that we are doing so
When we use asymptotic notation over a limited domain
14
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma for proof
Lemma 1. Make an expression that contains a summation
Lemma 2. Determine asymptotic bounds on this summation
Lemma 3. Use Lemma1 and Lemma2 prove master theorem (n is exact power of b)
15
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma 1. Make an expression that contains a summation
Define T(n) on exact power of b by the recurrence
T(n)=
{
(1)
if n = 1,
aT (n / b) f (n) if n = b
i
( a≥1 and b≥1, f(n) defined on exact power of b )
Where i is a positive integer. Then,
T ( n ) ( n
logba
)
logbn1
a
j
f (n / b j )
j 0
16
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma 1. Make an expression that contains a summation Proof
f(n)
a
f ( n / b)
a
logbn
… f ( n / b) →
a
a
…
…
f ( n / b)
…
af (n / b)
logbn 1
f (n / b 2 ) f (n / b 2 ) f (n / b 2 ) f (n / b 2 ) f (n / b 2 ) f (n / b 2 ) f (n / b 2 ) f (n / b 2 ) f (n / b 2 ) →
…
… … … … … …
(1) (1) (1) (1)(1) (1) (1) (1)
n
logb a
…
… …→
(1) (1) (1) (1)
a 2 f (n / b 2 )
→
j
j
a
f
(
n
/
b
)
j 0
a j f (n / b j )
→ (a logb )
T ( n ) ( n
n
logba
)
logbn1
a
j 0
j
f (n / b j )
17
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma 2. Determine asymptotic bounds on this summation
g ( n)
logbn 1
a
j
f (n / b j )
j 0
Case 1. If f (n) O(n
logb a
) , for some constant ε > 0, then g (n) O(n logb a )
log a
Case 2. If f (n) (n b ) , then g (n) (n
logb a
lg n)
Case 3. If , af (n/b) ≤ cf (n) for some constant c < 1 and for all n≥b, then g (n) ( f (n))
18
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma 2. Determine asymptotic bounds on this summation Proof for Case1
g ( n)
logbn 1
a
j
f (n / b j )
j 0
Case 1. If f (n) O(n
Assumed fact :
Substitution :
logb a
) , for some constant ε > 0, then g (n) O(n logb a )
f (n) O(n logb a ) → f (n / b j ) O(( n / b j )logb a )
g ( n) O (
logbn 1
a
j
j logba
(n / b )
)
j 0
Let‘s factoring out terms and simplifying the summation which within the O-notation.
19
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma 2. Determine asymptotic bounds on this summation Proof for Case1
Substitute into g(n) : g ( n) O (
logbn 1
j
j logba
logbn 1
ab
a (n / b )
)
j 0
→
logbn 1
j
j
a (n / b )
logba
n
logba
j 0
(
j 0
n
logba
a
b logb
)j
logbn 1
(b )
j
j 0
logbn
1
)
b 1
logba n 1
n
(
)
b 1
n logb (
a
b
20
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma 2. Determine asymptotic bounds on this summation Proof for Case1
Substitute into g(n) : g ( n) O (
logbn 1
j
j logba
a (n / b )
)
j 0
g ( n) O ( n
O(n
logba
O(n
Case 1. If f (n) O(n
logb a
logba
logba
n 1
(
))
b 1
n )
) (Now we proved case1)
) , for some constant ε > 0, then g (n) O(n logb a )
21
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma 2. Determine asymptotic bounds on this summation Proof for Case2
g ( n)
logbn 1
a
j
f (n / b j )
j 0
log a
log a
Case 2. If f (n) (n b ) , then g (n) (n b lg n)
Assumed fact :
f (n) (n
Substitute into g(n) :
logb a
g ( n ) (
j logba
) → f (n / b ) ((n / b )
j
logbn 1
a
j
j logba
(n / b )
)
)
j 0
22
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma 2. Determine asymptotic bounds on this summation Proof for Case2
Substitute into g(n) : g (n) (
logbn 1
j logba
j
a (n / b )
)
j 0
logbn 1
→
a j (n / b j )
logba
n
logba
logbn 1
j 0
j 0
n
logba
a
(
b
logba
)j
logbn 1
(1)
j 0
a
n logb log bn
23
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma 2. Determine asymptotic bounds on this summation Proof for Case2
Substitute into g(n) : g (n) (
logbn 1
j logba
j
a (n / b )
)
j 0
→
logbn 1
j
j logba
a (n / b )
n
j 0
logba
log bn
a
logb
log bn )
→ g ( n ) ( n
a
(n logb lg n) (Now we proved case2)
log a
log a
Case 2. If f (n) (n b ) , then g (n) (n b lg n)
24
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma 2. Determine asymptotic bounds on this summation Proof for Case3
g ( n)
logbn 1
j
j
a
f
(
n
/
b
)
j 0
Case 3. If , af (n/b) ≤ cf (n) for some constant c < 1 and for all n≥b, then g (n) ( f (n))
25
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma 2. Determine asymptotic bounds on this summation Proof for Case3
g ( n)
logbn 1
a
j
f (n / b j )
j 0
→ g (n) f (n) af (n / b) ... a
logbn 1
f (n / b
logbn 1
)
f (n)
g(n) include f(n) when j=0
it means g(n) ≥ f(n)
so, we can conclude g(n) =Ω(f(n))
26
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma 2. Determine asymptotic bounds on this summation Proof for Case3
g ( n)
logbn 1
a
f (n / b j )
a
f ( n / b) f ( n)
c
substitute
2
a
f ( n / b 2 ) f ( n)
2
c
substitute
3
a
f ( n / b 3 ) f (n)
3
c
substitute
a4
4
f ( n / b ) f ( n)
4
c
→
j 0
j
→
→
a
n
n
f( 2) f( )
c b
b
a
n
n
f ( 3) f ( 2)
c b
b
a
n
n
f( ) f( 4)
c b3
b
→…→
aj
j
j
j
j
f
(
n
/
b
)
f
(
n
)
a
f
(
n
/
b
)
c
f ( n)
→
j
c
27
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma 2. Determine asymptotic bounds on this summation Proof for Case3
Case 3. If , af (n/b) ≤ cf (n) for some constant c < 1 and for all n≥b, then g (n) ( f (n))
g ( n)
logbn 1
a
j
f (n / b j )
j 0
logbn 1
c
j
f (n) ( because a j f (n / b j ) c j f (n) )
j 0
f ( n) c j
j 0
1
)
1 c
O ( f ( n)) ( Since c is constant )
f (n)(
28
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma 2. Determine asymptotic bounds on this summation Proof for Case3
g(n) =Ω(f(n)) and g(n)=O(f(n))
so g(n)= θ(g(n)) ( for extra powers of b ) (Now we proved case3)
Case 3. If , af (n/b) ≤ cf (n) for some constant c < 1 and for all n≥b, then g (n) ( f (n))
29
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma 3. Prove master theorem (n is exact power of b)
T(n)=
{
(1)
if n = 1,
aT (n / b) f (n) if n = b ( a≥1 and b≥1, f(n) defined on exact power of b )
i
1. If f (n) O(nlog a ), for some constant ε > 0, then T (n) (nlog a )
b
b
2. If f (n) (nlog a ), then T (n) (nlog a lg. n)
b
b
3. If f (n) (nlog a ) for some constant ε > 0,
and if af (n/b) ≤ cf (n) for some constant c < 1 and all
sufficiently large n, then T (n) ( f (n)) .
b
30
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma 3. Prove master theorem (n is exact power of b) Proof for Case1
(1)
if n = 1,
T(n)=
i
aT (n / b) f (n) if n = b ( a≥1 and b≥1, f(n) defined on exact power of b )
{
Lemma1)
T ( n ) ( n
logba
)
logbn1
a
j
f (n / b j )
j 0
T ( n ) ( n
logba
)
logbn1
a
j
f (n / b j ) ( From Lemma1 )
j 0
31
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma 3. Prove master theorem (n is exact power of b) Proof for Case1
T ( n ) ( n
logba
)
logbn1
a
j
f (n / b j )
j 0
log a
Case1. If f (n) O(n b ) , for some constant ε > 0, thenT (n) (n
logb a
)
Lemma2) case1
Case 1. If f (n) O(n
logb a
) , for some constant ε > 0, then g (n) O(n logb a )
T (n) (nlogb a ) O(nlogb a )
( From case1 in lemma2 )
32
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma 3. Prove master theorem (n is exact power of b) Proof for Case2
T ( n ) ( n
logba
)
logbn1
a
j
f (n / b j )
j 0
log a
log a
Case 2. If f (n) (n b ) , then T (n) (n b lg n)
Lemma2) case2
Case 2. If f (n) (n logb a ) , then g (n) (n
logb a
lg n)
T (n) (n logb a ) (n logb a lg n)
(n logb a lg n) ( From case2 in lemma2 )
33
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma 3. Prove master theorem (n is exact power of b) Proof for Case3
T ( n ) ( n
logba
)
logbn1
a
j
f (n / b j )
j 0
Case3. If f (n) (nlogb a ) for some constant ε > 0,
and if af (n/b) ≤ cf (n) for some constant c < 1
and all sufficiently large n, then T (n) ( f (n)) .
Lemma2) case3
Case 3. If , af (n/b) ≤ cf (n) for some constant c < 1 and for all n≥b, then g (n) ( f (n))
T (n) (n logb a ) ( f (n)) ( From case3 in lemma2 )
( f (n)) ( Because f (n) (nlog a ) )
b
34
2. Recurrence solution methods
• Proof of the master theorem n is exact power of b
Lemma 3. Prove master theorem (n is exact power of b)
T(n)=
{
(1)
if n = 1,
aT (n / b) f (n) if n = b ( a≥1 and b≥1, f(n) defined on exact power of b )
i
log a
Case1. If f (n) O(n b ) , for some constant ε > 0, thenT (n) (n
logb a
)
log a
log a
Case 2. If f (n) (n b ) , then T (n) (n b lg n)
Case3. If f (n) (nlogb a ) for some constant ε > 0,
and if af (n/b) ≤ cf (n) for some constant c < 1
and all sufficiently large n, then T (n) ( f (n)) .
( Now we proved master theorem when n is exact power of b )
35
2. Recurrence solution methods
• Proof of the master theorem n is all integers
The recurrence defined for all integers
Upper bounding )
T (n) aT ( n / b) f (n)
Lower bounding )
T (n) aT ( n / b) f (n)
36
2. Recurrence solution methods
• Proof of the master theorem n is all integers
Proving upper bounding in recurrence T (n) aT ( n / b) f (n)
f(n)
a
f (n1 )
a
logbn
f (n)
a
…
af (n / b)
… f (n1 )
f (n1 )
a
…
…
f (n2 ) f (n2 ) f (n2 ) f (n2 ) f (n2 ) f (n2 ) f (n2 ) f (n2 ) f (n2 )
a 2 f ( n / b / b,)
.....…
… … … … … …… … …
…
(1) (1) (1) (1)(1) (1) (1) (1)
(n
logb a
)
(1) (1) (1) (1)
(nlogb a )
T ( n ) ( n
logba
logbn1
) a j f (n / b j )
j 0
37
2. Recurrence solution methods
• Proof of the master theorem n is all integers
Proving upper bounding in recurrence T (n) aT ( n / b) f (n)
n
n / b ,
n / b / b,,
n / b / b, / b ,
nj =
n
{ n
if j=0
j 1/ b
if j>0
………
constant
Goal : determine the depth k
such that nk is a constant
38
2. Recurrence solution methods
• Proof of the master theorem n is all integers
Proving upper bounding in recurrence T (n) aT ( n / b) f (n)
Goal : determine the depth k such that nk is a constant
nj =
n0 n,
n
{ n
if j=0
j 1/ b
if j>0
n
n1 n0 / b 1,
b
n n 1
n n
n2 n1 / b 2 1,
b b
n n n
n3 n2 / b 3 2 1,
b b b
....
39
2. Recurrence solution methods
• Proof of the master theorem n is all integers
Proving upper bounding in recurrence T (n) aT ( n / b) f (n)
Goal : determine the depth k such that nk is a constant
n0 n,
n
n1 n0 / b 1,
b
n n
n2 n1 / b 2 1,
b
b
n
n n
n3 n2 / b 3 2 1,
b
b
b
n j 1 1
nj j i
b
i 0 b
n
1
j i
b
i 0 b
n
1
j
b
b 1
....
n j 1 1
n j n j 1 / b j i
b
i 0 b
40
2. Recurrence solution methods
• Proof of the master theorem n is all integers
Proving upper bounding in recurrence T (n) aT ( n / b) f (n)
Goal : determine the depth k such that nk is a constant
n
1
nj j
b
b 1
( Letting j log
Thus, at depth
n
b
)
1
n
b logb b 1
n
1
logn1
b 1 ( Because,
b b
n
1
n / b b 1
1
b
b 1
O (1)
nlogn
b
n
A A 1 )
log bn , the problem size is at most a constant
41
2. Recurrence solution methods
• Proof of the master theorem n is all integers
Proving upper bounding in recurrence T (n) aT ( n / b) f (n)
T ( n ) ( n
logba
)
logbn 1
Except that n is an arbitrary integer
j
j
a
f
(
n
/
b
)
and to be all integer
j 0
logbn 1
g ( n) a j f ( n / b j )
j 0
Case 1. If f (n j ) O(nlogb a ), for some constant ε > 0, then g (n) O(n logb a )
log a
Case 2. If f (n) (n b ) , then g (n) (n
logb a
lg n)
Case 3. If af ( n / b) cf (n) , for some constant c < 1
and for all n≥b+b/(b-1), then g (n) ( f (n))
42
2. Recurrence solution methods
• Proof of the master theorem n is all integers
Proving upper bounding in recurrence T (n) aT ( n / b) f (n) Proof for Case3
logbn 1
g ( n) a j f ( n / b j )
j 0
Case 3. If af ( n / b) cf (n) , for some constant c < 1
and for all n≥b, then g (n) ( f (n))
43
2. Recurrence solution methods
• Proof of the master theorem n is all integers
Proving upper bounding in recurrence T (n) aT ( n / b) f (n) Proof for Case3
logbn 1
g ( n)
a j f ( n / b j ) → a f ( n / b ) f ( n)
j 0
c
substitute
2
a
f ( n / b 2 ) f (n)
2
c
substitute
3
a
f ( n / b 3 ) f (n)
3
c
substitute
a4
4
f ( n / b ) f (n)
4
c
→
a n
n
f ( 2 ) f ( )
c b
b
→
a n
n
f ( 3 ) f ( 4 )
c b
b
→…→
→
a n
n
f ( 3 ) f ( 2 )
c b
b
→ a j f ( n / b j ) c j f ( n)
aj
j
f
(
n
/
b
) f (n) → a j f (n ) c j f (n)
j
j
c
44
2. Recurrence solution methods
• Proof of the master theorem n is all integers
Proving upper bounding in recurrence T (n) aT ( n / b) f (n) Proof for Case3
Case 3. If af ( n / b) cf (n) , for some constant c < 1
and for all n≥b, then g (n) ( f (n))
logbn 1
g ( n) a j f ( n / b j )
j 0
logbn 1
c j f ( n) ( because a j f (n j ) c j f (n) )
j 0
f ( n) c j
j 0
1
)
1 c
O ( f ( n)) ( Since c is constant )
f ( n)(
45
2. Recurrence solution methods
• Proof of the master theorem n is all integers
Proving upper bounding in recurrence T (n) aT ( n / b) f (n) Proof for Case3
g(n) =Ω(f(n)) and g(n)=O(f(n))
so g(n)= θ(g(n)) ( for extra powers of b ) (Now we proved case3)
Case 3. If af ( n / b) cf (n) , for some constant c < 1
and for all n≥b, then g (n) ( f (n))
46
2. Recurrence solution methods
• Proof of the master theorem n is all integers
Proving upper bounding in recurrence T (n) aT ( n / b) f (n) Proof for Case2
logbn 1
g ( n) a j f ( n / b j )
j 0
log a
log a
Case 2. If f (n) (n b ) , then g (n) (n b lg n)
47
2. Recurrence solution methods
• Proof of the master theorem n is all integers
Proving upper bounding in recurrence T (n) aT ( n / b) f (n) Proof for Case2
a j f (n j ) c j f (n)
cj
→ f (n j )
f ( n)
j
a
c j logba
f (n j ) j n
a
a
f (n) O(nlogb )
a
→ f (n j ) O(( n / b j ) logb )
log a
(because f (n) (n b )
logba
n
O( j )
a
)
logba
If we can show
n
j logba
O( j ) O(( n / b ) ) then
a
proof for case2
48
2. Recurrence solution methods
• Proof of the master theorem n is all integers
Proving upper bounding in recurrence T (n) aT ( n / b) f (n) Proof for Case2
j log bn
j
→b n
→ bj / n 1
49
2. Recurrence solution methods
• Proof of the master theorem n is all integers
Proving upper bounding in recurrence T (n) aT ( n / b) f (n) Proof for Case2
logba
n
j logba
O( j ) O(( n / b ) )
a
show
f ( n) O ( n
logba
f ( n j ) O( n j
n
b logba
)
j
b
b 1
n
bj b
logba
c( j (1
))
b
n b 1
→ f (n j ) c(
)
logba
then proof for case2
)
substitute
n
1
nj j
b
b 1
logba
bj b
n
logba
c( j )(1 (
))
n b 1
a
logba
n
b logba
c( j )(1
)
a
b 1
O(
logba
n
)
aj
(Now we proved case2)
50
2. Recurrence solution methods
• Proof of the master theorem n is all integers
Proving upper bounding in recurrence T (n) aT ( n / b) f (n) Proof for Case2
Substitute into g(n) : g (n) (
logbn 1
j logba
j
a (n / b )
)
j 0
→
logbn 1
j
j logba
a (n / b )
n
j 0
logba
log bn
a
logb
log bn )
→ g ( n ) ( n
a
(n logb lg n) (Now we proved case2)
log a
log a
Case 2. If f (n) (n b ) , then g (n) (n b lg n)
51
© Copyright 2026 Paperzz