nalisis de
lgoritmos
Tomado de la Universidad Nacional Sede Bogota
Depto. De Sistemas
OUTLINE
 Recurrences
• Examples of recurrences and the
Iteration method
• Characteristic equation method
• Substitution method
• Recursion tree method
• Master method
• Z transform
EXAMPLES OF RECURRENCES
INCREMENTAL RECURRENCES:
• Min
• Insertion Sort
DIVIDE AND CONQUER RECURRENCES:
• Chip Testing Problem
• Binary Search
• Convex Hull
INCREMENTAL RECURRENCES
Min
Insertion Sort
General form
T(n-1)+f(n)
for n  1
0
for n =0
T(n)=
Iteration Method
T(n) = T(n-1) + f(n)
= T(n-2) + f(n-1) + f(n)
:
= T(0) +f(1) +f(2)+ … + f(n-1) + f(n)
n
T(n)=  f (i )
i 1
Min
n
T(n)= T(n-1)+ 1 =
1 = n
i 1
Insertion sort
n
T(n)= T(n-1)+(n-1) =  i  1 = n(n-1)/2
i 1
Arithmetic Series
T(n) = T(n-1) + ( n-1 )
f(i)=i -1
n
S=
n
n 1
n 1
 f (i)  (i 1)   i   i 
i 1
i 1
i 0
n(n -1)/2
i |
S = 1 + 2 + 3 + ... + (n-1)
S = (n-1)+(n -2)+(n -3)+ ... + 1
2S = n + n + n + ... + n {n-1 terms}
Geometric Series
f(i)=a ri-1
S=
n
n
 f (i)  ar
i 1
i 1
i 1
r 1
  ar  a
r 1
i 0
n 1
n
i
-S
= -a - ar - ar2 - ... - arn-1
rS = + ar + ar2 + ... + arn-1 + arn
(r-1)S= -a+
arn
If r is a constant bounded away from 1 then S is
approximately a constant times the largest term in the
series.
Mixed Series
f(i)= iri
n
S =

i 1
 i
f (i )  ir   ir   r
i 1
i 0
i  0 r
n
n
i
 n i   r n1  1 

  r  
r i 0
r  r  1 

nr
n 1
 (n  1)r  1
2
r  1
n
n
i
-S
= -1r1 -2 r2 - … - nrn
rS
=
1 r2+...+ (n-1)rn + nrn+1
(r-1)S = - (r + r2 +...+ rn) + nrn+1
(r-1)S = - r(1+ r+ ...+ rn-1) + nrn+1
S
n


1
r  1 
n 1


nr  r 
r 1 
 r  1 
DIVIDE AND CONQUER
Chip Testing Problem
This problem is motivated by the VLSI industry,
where millions of VLSI chips must be tested
for defaults. The idea is to let the chips test
each other by built-in circuitry. Note that
VLSI stands for very large scale integration
which
is
the
integrated-circuit
chip
technology
used
to
fabricate
most
microprocessors today.
There are two kinds of chips - good and bad.
Good chips always tell the truth, i.e. will
report accurately whether the tested chip is
good or bad. Bad chips say anything about
the other chip, thus cannot be trusted. Our
goal is to identify all good chips. The problem
must be solved in O(n) ) time.
It is assumed that there are more good chips
than bad. In other words, |G| > |B|. Our
computational model states that each chip
tested counts for one time unit.
It suffices to define one good chip in time O(n).
This is because one good chip can identify all
good chips in O(n) time
A naive method
We are going to test each chip individually by
comparing it against all remaining chips. We
take votes from the remaining chips and let
the majority decide. In case of equality, good
wins. If a chip is declared bad, it is junked
and we continue.
Time Analysis :
(n-1)+(n-2)+(n-3)+...+(n/2) n + n + .... + n = (n2)/2 =
O(n2)
However such time is unacceptable, for it
greatly exceeds O(n) time.
Divide and conquer
Definitions :

G : set of all good chips

B : set of all bad chips

B-B : pair of chips that has been declared
"bad, bad"

B-G : pair of chips that has been declared
"bad, good"

G-G : pair of chips that has been declared
"good, good"
Iteration:
If n = 1 or n = 2, then return one chip (it
is good).
If n is odd, test odd chip.
If odd chip is good, return the chip (it is
good).
If odd chip is bad,
with remaining chips do :

pair all chips

junk all B-G, B-B pairs

junk one chip of every G-G pair

find (recursively) one good chip
among the remaining chips
Correctness
If before the iteration, |G| > |B|, then
after the iteration, |G| > |B|.
Before the pairing step, we have
|G|  |B| + 2.
Each B-B, B-G pair has at one bad chip. So we
junk at least as many bad chips as good ones
obtaining G1 and B1. So, we still have
|G1|  |B1| + 2.
Each G-G pair has either two good chips or two
bad ones; what matters is that both chips are
alike. Thus, the halving step leaves us with
G2 = G1 /2,
B2 = B1 /2.
So, we still have
|G1| /2  |B1|/2 + 2 /2 ,
|G2|  |B2| + 1.
Then before the next iteration
|G|  |B| + 1.
Time complexity
If T(n) is the worst-case time for n chips, then
T(2) = T(1) = 0 and
T(n) n - 1 + n/2 + T(n/2 )
< 3n/2 + T(n/2 )
< 3n/2 + T(n/2 )
Then
T(n) 3n/2 + T(n/2 )
Iteration Method
T(n) 3n/2 + T(n/2 )
 3n/2 + 3n/4 + T(n/4 )
 3n/2 + 3n/4 + 3n/8 + T(n/8 ) ...
So that clearly
T(n)  3n/2 (1 + 1/2 + 1/4 + 1/8 + ...) = 3n = O(n)
Applet from
http://www.cs.mcgill.ca/~cs251/OldCourses/1997
Binary Search
We are searching for a value x in a sorted array
A[1.. n] of size n.
We examine the middle value. Either this is the
value we are looking for and we are done, or
we must search for the value in either the left
or the right half of the table. Thus we obtain
a smaller problem of the same type as we had
before (which we can solve recursively).
Binary search strategy
(bisection method)
Find out if x is equal, larger or smaller than an/2
x = an/2
x < an/2
x > an/2
A
x = an/4
A
x < an/4
x > an/2
Discarded
x = an/8
A
Discarded
Discarded
:
Bisection search for roots
(bisection method)
Given: continuous function f and
two points a<b with f(a)<0 and
f(b)>0
Find: approximation to c s.t.
f(c)=0 and a<c<b
The recurrence we obtain is
T(n)  T(n/2 ) + 1
Iteration Method
n=2k, k = lg n
T(n) = T(n/2 ) + 1
= 1+1+ T(n/4 )
= (1+1+ …+1) = O(lg n)
k = lg n
for general n
T(n) T(n/2 ) + 1
 1+1+ T(n/4 )
 (1+1+ …+1)  lg n
So that clearly
T(n) = O(lg n)
Convex Hull
Convex Hull, CH(X), is the smallest convex
polygon containing all points from X, |X|=n
X
“It is similar to placing a rubber band
around the set and letting go! “
The following algorithm to find a
convexhull uses a divide and conquer
method similar to merge-sort:
ConvexHull(A)
B= ConvexHull (first half of A)
C= ConvexHull (second half of
A)
MergeConvexHull(B,C)
divide-and-conquer divides into two
subsets left and right in O(n), then
combine into one convex hull in O(n).
The recurrence we obtain is
T(n)  2T(n/2 ) + O(n)
The Towers of Hanoi
The Legend
It is said that after creating the world, God set
on Earth three rods made of diamond and
64 rings of gold.
These rings are all of different size. At the
creation they were threaded on one of the
rods in order of size, the largest at the
bottom and the smallest at the top. God also
created a monastery close to the rods.
The monk´s task in life is transfer all the rings
onto another rod. The only operation
permitted is to move one single disk form
one rod to another, in such way that no ring
is ever placed on top of another smaller one.
When the monks finish their task the
monastery will turn to dust and the world
will come to an end.
Hanoi(n,i,j)
if (n > 0) then
Hanoi(n-1,i,6-i-j)
move i j
Hanoi(n-1,6-i-j,j)
end_if
The recurrence we obtain is
T(n) = 2T(n-1 ) + 1
Applet from
http://www.mazeworks.com/hanoi/
CHARACTERISTIC EQUATION
METHOD
Linear Homogeneous Difference
Equations
antn + an-1 tn-1 + an-2 tn-2+ … + an-k tn-k =0
with k initial conditions
t0= l0 , t1= l1 , … , tk-1= lk-1
Guess that tn is of the form n therefore
we must have
(characteristic equation)
an n + an-1 n-1 + an-2 n-2+ … + an-k  n-k =0
an k + an-1 k-1+ an-2 k-2+ … + ak =0
Let be = r1 , r2 , …. , rk the roots of the
characteristic equation (all different).
Then tn = rin is a solution and a linear
combination of the rin is also a solution
k
t n   ck rk
n
i 1
The constants ci are determined by the
initial conditions.
Fact: !the difference equation has only
solutions of this form!
Examples
Fibonacci:
tn = tn-1 + tn-2
for
n 2,
t0 = 0, t1 = 1
Guess that xn is of the form  n
therefore must have n= n-1+ n-2
i.e. 2-  - 1=0 (characteristic eqn.)
1, 2
1 5

2
n
1 5 
1 5 
tn= A 
 + B 

 2 
 2 
n
Solve for n=0 and 1 to find A and B
• n=0:
A+B = 0 = t0
• n=1:
((A+B)+(A-B) 5 )/2 = 1 = t1
Therefore
A = 1/ 5
B =-1/ 5
Then
  1  5 n  1  5 n 
tn = 1 / 5  
  
 
 2   2  


Second example
tn- 3 tn-1 - 4 tn-2 = 0 for
n 2, t0 = 0, t1 = 1
2 - 3  - 4 = 0 (characteristic eqn.)
1= -1, 2= 4
tn= A(-1)n + B(4)n
Solve for n=0 and 1 to find A and B
• n=0:
A + B = t0 = 0
• n=1:
-A + 4B = t1 =1
Therefore
A = -1/5
B = 1/5
Then
tn = 1/ 5 ((4)n - (-1)n ).
Third example
tn= tn-1 - tn-2
for
n 2, t0 = 0, t1 = 1
2 -  + 1 = 0 (characteristic eqn.)
 1,2 = 1  3i =
2

n
tn= A  e 3 i  + B
 
 

 3i 
e 




n
Solve for n=0 and 1 to find A and B
• n=0:
A + B = t0 = 0
• n=1:
 1  3i 
 1  3i 
 + B 
 = t1 =1
A 
 2 
 2 
Therefore A = 1/
B = -1/
3i
3i
tn = 2 / 3 sin(n / 3)
is periodic:
0,1,1,0,-1,-1, 0,1,1,0,-1,-1, ...
Roots with multiplicity > 1
If ri is a root with multiplicity 2 of
of the characteristic equation
p() = ak k + ak-1 k-1 + … + a1 1 + a0
h() =  [n-k p() ]’
= ak nn + ak-1 (n-1)n-1 + … + a0 n-k
Let assume
p() = ( - r)2 q()
h() =  [n-k p() ]’
=  [n-k ( - r)2 q()]’
=  [ ( - r)2 n-k q()]’
=  [2 ( - r)n-k q() +( - r)2 [ n-k q()]’ ]
= ak nn + ak-1 (n-1)n-1 + … + a0 n-k
Then h(r) =0 and
tn = nrin is a solution of the diff. equ.
In general,
if ri is a root with multiplicity m of
of the characteristic equation.
Then
tn = nrin, tn = n2rin , … , tn = nm-1rin
are solutions and a linear combination of
these solutions are solutions.
example
tn = 5 tn-1 - 8 tn-2 + 4 tn-3 for n 3,
t0 = 0, t1 = 1 , t2 = 2
3 - 52 + 8  - 4 = ( - 1)( - 2) 2 = 0
(characteristic eqn.)
 1= 1,  2= 2
tn= A(1)n + B(2)n+ Cn(2)n
A+B
=0
A + 2B + 2C = 1
A + 4B + 8C = 3
A = -2
B=2
C = - 1/2
tn= -2(1)n + 2(2)n - (1/2)n(2)n
= 2n+1 - n2n-1- 2
Linear Inhomogeneous
Difference Equations
antn + an-1 tn-1 + 2+ … + ak tn-k = b np(n)
with b constant, p(n) polynomial of
degree d and k initial conditions
t0= p0 , t1= p1 , … , tk-1= pk-1.
Has the solutions obtained form the char.
equ.
(an n + an-1 n-1 + … + ak) (t-b) d+1 = 0
example
tn - 2 tn-1 = 3n
Multiplying by 3:
3 tn - 6 tn-1 = 3n+1
Replace n by n+1:
tn+1 - 2 tn- = 3n+1
Subtracting:
tn+1 - 5 tn- + 6 tn-1 = 0
Char. Equ.:  2 - 5  + 6 = ( - 2)( - 3) = 0
tn= A(2)n + B(3)n
Hanoi example
tn = 2 tn-1 + 1
char. equ.: ( - 2)( - 1) = 0
tn= A(2)n + B(1)n
• t0 = 0
• t1 = 1
A+ B = 0
2A+ B = 1
tn= 2n - 1
Change of variables
example
T(n) = 4T(n/2) + n
Replace n by 2k
T(2k) = 4T(2k-1) + 2k
call tk- = T(2k) then T(2k) = 4T(2k-1) + 2k is
tk = 4tk-1 + 2k
Char. equ.: ( - 4)( - 2) = 0
tk= C1(4)k + C2 2k
Back in n, 2k = n, 4k = (22)k = (2k)2 = n2
tk = T(2k) = T(n)
then
T(n)=C1 n2 + C2 n
Second example
T(n) = aT(n/b) + nh (cd lgdn + cd-1 lgd-1n + c0 )
Replace n by bk
T(bk) = aT(bk-1) + ( bh )k (cd kd + cd-1 kd+ c0 )
call tk- = T(2k) then
tk - atk-1 =( bh )k (cd kd + cd-1 kd+ c0 )
char. equ.: ( - a)( - bh) d+1 = 0
tk= C1(a)k + D0 (bh)k + D1 k(bh) k + D2 k2(bh) k +
D3 k3(bh) k + …. + Dd kd(bh) k
Back in n, bk = n,
ak = (bk)logba
then
T(n) = C1 n logba + D0 nh + D1 logbn nh +
D2 log2bn nh + … + Dd logdbn nh
SUBSTITUTION METHOD
Steps
• Guessing the form of the solutions, then
using mathematical induction to find the
constants and show the solution works.
• It works well when it is easy to guess.
But, there is no general way to guess the
correct solution.
Example
Solve:
Guess:
Proof:
T(n) = 3T(n/3) + n
T(n) = O(n lg n), i.e. T(n) = cn lg n
T(n) = 3T(n/3) + n
T(n)  3c n/3 lg n/3 + n
 c n lg (n/3) + n
= c n lg n - c n lg3 + n
= c n lg n - n (c lg 3 - 1)
 c n lg n
* The last step is true for c  1 / lg3.
Making a Good Guess
• Guessing a similar solution to the one
that you have seen before
– T(n) = 3T(n/3 + 5) + n similar to
– T(n) = 3T(n/3) + n
when n is large, the difference
between n/3 and (n/3 + 5) is
insignificant
• Another way is to prove loose upper
and lower bounds on recurrence and
then reduce the range of uncertainty.
Start with T(n) = (n) & T(n) = O(n2)
 T(n) =  (n log n)
Subtleties
When the math doesn’t quite work out in
the induction, try to adjust your guess
with a lower-order term. For example:
We guess T(n)  O(n) for
T(n) = 3T(n/3)+ 4,
but we have
T(n)  3c n/3 + 4 = c n + 4
New guess is
T(n)  c n - b, where b  0
 3(c n/3 - b)+4
= c n - 3b + 4
= c n - b - (2b-4)
Therefore,
T(n)
T(n)  c n - b, if 2b - 4  0 or if b  2
Change of Variables
Use algebraic manipulation to turn an
unknown recurrence similar to what you
have seen before.
Consider T(n) = 2T(n1/2) + lg n
Rename m = lg n and we have
T(2m) = 2T(2m/2) + m
S(m)
Set S(m) = T(2m)
and we have
= 2S(m/2) + m  S(m) = O(m lg m)
Changing back from
S(m) to T(n), we have
T(n) = T(2m)
= S(m)
= O(m lg m)
= O(lg n lg lg n)
Avoiding Pitfalls
• Be careful not to misuse asymptotic notation.
For example:
– We can falsely prove T(n) = O(n)
by guessing T(n)  c n for T(n) = 2T(n/2)
+n
T(n)  2c n/2 + n
 cn+n
= O(n)  Wrong!
– The err is that we haven’t proved T(n)  c n
MASTER THEOREM FOR
DIVIDE AND CONQUER
RECURRENCE
Master Theorem
Let a 1, b>1 be constants, f(n) be a
function and T(n) defined by
T(n)=aT(n/b)+f(n)
then
• if f (n)  O n log a 
for  >0

b
then
T(n) = (n logba )

• if
f(n) =  (n logba )
then
T(n) =  (n logba * lg n)
• if f(n) =  (n logba +  )
for  >0
and if a(f(n/b))  cn for c < 1 and all
sufficiently large n
then
T (n)   f (n)
Proving Master Therorem
Problem size T(n)=aT(n/b)+c cf(n)
n
a
# probs
1
n/b
a
n/b2
a2
b
1
ad
T(1)=c
Problem size T(n)=aT(n/b)+c cf(n)
n
a
# probs
1
n/b
a
n/b2
a2
b
1
ad
T(1)=c
Problem size T(n)=aT(n/b)+c cf(n)
n
a
# probs
1
cost
cf(n)
n/b
a
caf(n/b )
n/b2
a2
ca2f(n/b2 )
ad
cadf(1 )
b
logb a
1
ad =
a
logbn
=
T(1)=c
logb
na
Total:

n
logb a 1
logb a
 
j 0
n
f j 
b 
Lemma
Let a 1, b>1 be constants, f(n) be a
nonnegative defined on exact powers of
b. A function g(n) defined on exact
powers of b by
g ( n) 
logb a 1

j 0
 n
f j 
b 
Can then be bounded asymptotically for
exact powers of b as follows
• if

f ( n)  O n
then
logb a 

g ( n)  O n

logb a
for  >0

• if

f (n)  O n

logb a

then g (n)  O n log a lg n

b
logb a 


• if f (n)  O n
for  >0
and if a(f(n/b))  cn for c < 1 and all
sufficiently large n
then
g (n)   f (n)
Multiplying Matrices
 a11 a12
a
 21 a22
 a31 a32

a41 a42
 a11b11  a12b21  a13b31  a14b41
a b  a b  a b  a b
 21 11 22 21 23 31 24 41
 a31b11  a32b21  a33b31  a34b41

a41b11  a42b21  a43b31  a44b41
a13
a23
a33
a43
a14  b11
a24  b21

a34  b31
 
a44  b41
b12
b13
b22
b23
b32
b33
b42
b43
b14 
b24 

b34 

b44 
a11b12  a12b22  a13b32  a14b42  a11b14  a12b24  a13b34  a14b44 
a21b12  a22b22  a23b32  a24b42  a21b14  a22b24  a23b34  a24b44 
a31b12  a32b22  a33b32  a34b42  a31b14  a32b24  a33b34  a34b44 

a41b12  a42b22  a43b32  a44b42  a41b14  a42b24  a43b34  a44b44 
• n3 multiplications, n3-n2 additions
 a11 a12
a
 21 a22
 a31 a32

a41 a42
 a11b11  a12b21  a13b31  a14b41
a b  a b  a b  a b
  21 11 22 21 23 31 24 41
 a31b11  a32b21  a33b31  a34b41

a41b11  a42b21  a43b31  a44b41
a13
a23
a33
a43
a14  b11
a24  b21

a34  b31
 
a44  b41
b12
b13
b22
b23
b32
b33
b42
b43
b14 
b24 
b34 

b44 
a11b12  a12b22  a13b32  a14b42  a11b14  a12b24  a13b34  a14b44 
a21b12  a22b22  a23b32  a24b42  a21b14  a22b24  a23b34  a24b44 
a31b12  a32b22  a33b32  a34b42  a31b14  a32b24  a33b34  a34b44 

a41b12  a42b22  a43b32  a44b42  a41b14  a42b24  a43b34  a44b44 
 a11 a12
a
 21 a22
 a31 a32

a41 a42
 a11b11  a12b21  a13b31  a14b41
a b  a b  a b  a b
  21 11 22 21 23 31 24 41
 a31b11  a32b21  a33b31  a34b41

a41b11  a42b21  a43b31  a44b41
a13
a23
a33
a43
a14  b11
a24  b21

a34  b31
 
a44  b41
b12
b13
b22
b23
b32
b33
b42
b43
b14 
b24 
b34 

b44 
a11b12  a12b22  a13b32  a14b42  a11b14  a12b24  a13b34  a14b44 
a21b12  a22b22  a23b32  a24b42  a21b14  a22b24  a23b34  a24b44 
a31b12  a32b22  a33b32  a34b42  a31b14  a32b24  a33b34  a34b44 

a41b12  a42b22  a43b32  a44b42  a41b14  a42b24  a43b34  a44b44 
 a11 a12
a A11
 21 a22
 a31 a32
 A21
a41 a42
a14  b11 b12
A12a  bB11b
a23
24   21
22

a33 a34  b31 b32
A22a44  b41B21b42
a43
a13
 a11b11  a12b21  a13b31  a14b41 a11b12  a12b22  a13b32  a14b42
Bb11+A
Ba 21b  a b  a b
a b  a b  a A
11
12
b

a
a
b

21
11
22
21
23
31
24
41
21
12
22 22
23 32
24 42

 a31b11  a32b21  a33b31  a34b41 a31b12  a32b22  a33b32  a34b42

A21a Bb 11+A
B
22
a
b

a
b

a
b
a
b

b22  a43b32  a44b42
 41 11 42 21 43 31 44 41 41 12 a4221
b14 
b23B12
b24 
b33 b34 

b43B22
b44 
b13
 a11b14  a12b24  a13b34  a14b44 
11B12+A12B22 
 aA
21b14  a22b24  a23b34  a24b44 
 a31b14  a32b24  a33b34  a34b44 

B
+A
B
 aA
b

a
b

a
b

a
b
21 421224 4322
22
41 14
34
44 44 
=
A11
A12
B11
B12
A21
A22
B21
B22
A11B11+A12B21
A11B12+A12B22
A21B11+A22B21
A21B12+A22B22
• T(n)=8T(n/2)+4(n/2)2=8T(n/2)+n2
log a
log 8
3
2
8>2 so T(n) is (n
)  (n
)  (n )
b
2
Exercises
• Solution of T(n) = T(n/2) + 1 is O(lg n)
• Solution of T(n) = 2T(n/2 + 17) + n is
O(n lg n)
• Solve T(n) = 2T(n1/2) + 1 by making a
change of variables. Don’t worry whether
values are integral.