CSE 101
Algorithm Design and Analysis
Miles Jones and Russell Impagliazzo
[email protected]
[email protected]
Office 4208, 4248 CSE Building
Lecture 18: Divide & Conquer: FFT,
Uneven divisions
DIVIDE AND CONQUER
◼Break a problem into similar subproblems
◼Solve each subproblem recursively
◼Combine
MULTIPLICATION
◼Cook-Tooms: Integer multiplication reduces to polynomial
multiplication
◼How can we multiply polynomials as fast as possible?
FAST POLYNOMIAL MULTIPLICATION
◼Maybe we could approach this if we had a faster algorithm for
multiplying polynomials
◼𝑃 𝑧 = 𝑎
◼𝑄 𝑧 = 𝑏
𝑛−1
𝑥
𝑛−1
+ 𝑎
𝑛−1
𝑥
𝑛−1
+ …….
𝑛−2
𝑥
𝑛−2
+ … . 𝑎1 𝑥 + 𝑎 0
𝑏1 𝑥 + 𝑏 0
◼R(z)=P(z)Q(z) is a degree 2n-2 < 2n polynomial
◼Same outline: Evaluate P(z) at 2n points, Evaluate Q(z) at 2n points,
◼Pointwise multiply to get R(z) at 2n points,
◼Interpolate to recover R
PROBLEM
◼𝑃 𝑧 = 𝑎
𝑛−1
𝑥
𝑛−1
+ 𝑎
◼𝑄 𝑧 = 𝑏
𝑛−1
𝑥
𝑛−1
+ …….
𝑛−2
𝑥
𝑛−2
+ … . 𝑎1 𝑥 + 𝑎 0
𝑏1 𝑥 + 𝑏 0
◼R(z)=P(z)Q(z) is a degree 2n-2 < 2n polynomial
◼Same outline: Evaluate P(z) at 2n points,: each takes time O(n) Evaluate
Q(z) at 2n points,
◼Pointwise multiply to get R(z) at 2n points,
◼Interpolate to recover R
◼How can we beat trivial 𝑂 𝑛 2 time when each evaluation takes linear time?
HMMM…
◼𝑃 𝑧 = 𝑎
𝑛−1
𝑥
𝑛−1
+ 𝑎
𝑛−2
𝑥
𝑛−2
+ … . 𝑎 1 𝑥 + 𝑎_0
◼How can we beat trivial 𝑂 𝑛 2 time when each evaluation takes linear
time?
◼How can we compute P(1)? P(-1)? Both together?
TRICKY DIVIDE STEP
◼Remember, when we were evaluating at 1 and -1, we broke the
Co-efficients into two groups by even vs odd term, rather than high vs.
low
More
TWO CAN LIVE AS CHEAPLY AS ONE
◼𝑃 1 = 𝑎 0 + 𝑎 1 + 𝑎 2 + ⋯ 𝑎
𝑛−1
◼𝑃 −1 = 𝑎 0 − 𝑎 1 + 𝑎 2 − 𝑎 3 … . + −1
𝑛−1
𝑎
𝑛−1
◼Each takes n-1 adds and subtracts
◼But together we can do it by: add even co -efficients,
◼Add odd coefficients
◼Add to get P(1)
◼Subtract to get P(-1)
◼n/2-1+n/2-1 + 2 = n operations total, just one more
CONTINUING THE PATTERN
◼We get to choose which values of x to evaluate on
◼What values of x would continue this pattern?
◼Hint: can’t really be integers, since powers bigger than one
grow exponentially. Need numbers that stay about the same size
IMAGINARY NUMBERS
◼𝑖 2 = −1
◼𝑃 𝑖 = 𝑎 0 + 𝑖 𝑎 1 − 𝑎 2 − 𝑖 𝑎 3 + 𝑎 4 … . . 𝑖
𝑛−1
𝑎
𝑛−1
◼𝑃 −𝑖 = 𝑎 0 − 𝑖𝑎 1 − 𝑎 2 + 𝑖 𝑎 3 + 𝑎 4 … . .
◼To compute P(1),P(-1),P(i), P(-i):
◼Group into batches by co-efficient mod 4.
◼Add all batches. < n total
◼Each of 4 involves at most 3 adds, multiply by I, subtracts
◼n+ c total
COMPLEX MULTIPLICATION
◼View complex numbers as two-d vectors, a+bi. Can then represent as
magnitude (length)
◼And direction (angle)
◼Multiplying multiplies
◼Magnitudes, adds
◼Directions mod 2 Pi
ROOTS OF UNITY
◼Complex
◼Numbers of
◼Magnitude one
◼And directions
◼That are
◼1/n of the way
◼Around the
◼Circle
Have powers that are evenly distributed around the circle, and come
Back to 1 when you raise them to the n’th power
EVALUATING
◼Multiplication of polynomials is then broken into three separate
stages: Evaluate, multiply pointwise, interpolate
◼We can consider the evaluate stage by itself. We do this
independently, one polynomial at a time, so we’ll just consider one
polynomial P(x)=𝑎 0 + 𝑎 1 𝑥 + 𝑎 2 𝑥 2 + ⋯ 𝑎 𝑛−1 𝑥 𝑛−1 .
Assume n=2 𝑘 for some k around log n. (otherwise, add 0 co efficients)
◼We want to evaluate P at the 2 𝑘 n’th roots of unity, which
we can write as 1,w, 𝑤 2 , 𝑤 3 , … 𝑤 𝑛−1 where w is an angle 1/n around the
unit circle, w= sin(2 Π/𝑛) + 𝑖 cos (2 Π/𝑛)
DIVIDE STEP
◼As we did in the 1 vs -1 case, we divide up the co-efficients not
as ``high’’ vs. ``low’’, but ``even’’ vs. ``odd’’
𝑃
𝑃
𝑒𝑣𝑒𝑛
𝑜𝑑𝑑
(𝑥) = 𝑎 0 + 𝑎 2 𝑥 + 𝑎 4 𝑥 2 + ⋯ 𝑎
𝑥 = 𝑎1 + 𝑎 3 𝑥 + 𝑎 5
To get back P:
𝑃 𝑥 =𝑃
𝑒𝑣𝑒𝑛
𝑥2
𝑥2 + 𝑥 𝑃
+⋯.𝑎
𝑜𝑑𝑑
𝑥2
𝑛−2
𝑛−1
∗𝑥
𝑥
𝑛
2 −1
𝑛
−1
2
CLOSURE OF ROOTS UNDER SQUARING
◼Squaring
◼Doubles the
◼Angle.
◼So if n is
◼Even, the
◼Squares
◼Of the nth
◼Roots of unity are the n/2th roots of unity (half as many, since
◼Roots come in pairs w, -w whose square is the same)
RECURSIVE EVALUATION (FFT)
◼Given: a list of n co-ordinates of P, a degree n-1 polynomial
◼Problem: Evaluate P(w) for each w which is an nth root of unity
◼Base case: n=1, P is degree 0, P(1)=a_0
◼Recursion: Divide list into P_even, P_odd
◼Recursively evaluate P_{even} and P_{odd} at all the n/2 th roots of
unity
◼For each n’th root w, P(w)=P_{even}(w^2)+wP_{odd}(w^2)
◼(w^2 is an n/2’th root, so we know P_{even} and P_{odd} at w^2)
TIME ANALYSIS
◼Evaluate degree n
◼Two evaluates of degree n/2 on n/2 roots
◼Constant work per root
◼T(n)=2T(n/2)+O(n)
◼T(n) ∈ 𝑂(𝑛 log 𝑛)
POINTWISE MULTIPLICATION
◼Just evaluate a single product at each root,
◼O(n) time
INTERPOLATION
◼Inverse to evaluation, has a similar structure
◼From: Values of a degree n-1 polynomial P on all roots of unity
◼Compute: The n-1 co-efficients of P
◼Roots closed under multiplying by -1
◼P(-x) is similar to P, except alternates signs
◼P(x)+P(-x)= 2 P_{even}(x^2)
◼P(x)-P(-x)= 2 P_{odd} (x^2)
RECURSIVE INTERPOLATION (INVERSE FFT)
◼If n=1, 𝑎 0 = 𝑃 1
◼Otherwise, 𝑃
𝑒𝑣𝑒𝑛
◼
𝑜𝑑𝑑
𝑃
𝑥2 =
𝑥2 =
1
2
1
2
𝑃 𝑥 + 𝑃 −𝑥
𝑃 𝑥 − 𝑃 −𝑥
◼
Since the n/2’th roots of unity are exactly the squares of the n’th
roots, we use these to get all the values of the n/2 -1 degree
polynomials 𝑃 𝑒𝑣𝑒𝑛 and 𝑃 𝑜𝑑𝑑
◼
◼
Interpolate recursively to get their co -efficients
Copy into the co-efficients of P, alternating
TIME ANALYSIS
◼Again, we do a linear amount of work, and then two interpolations
of degree n/2 polynomials
T(n)= 2 T(n/2) + O(n)
T(n) ∈ 𝑂(𝑛 log 𝑛)
Total time O(n log n) + O(n) + O(n log n) = O(n log n)
INTEGER MULTIPLICATION
◼Think of bits of x as co-efficients of polynomial p_x
◼Think of bits of y as co-efficients of polynomial p_y
◼xy = p_x(2)p_y(2)
◼Use FFT to evaluate q(z)= p_x(z)p_y(z)
◼Plug in z=2
◼Problem: Using complex multiplication as single step
◼How many bits do we need to multiply?
◼Claim: O(log 2 𝑛 ) bits suffice to keep accuracy
◼Gives time 𝑂(𝑛 log 5 𝑛) time total, but could improve
BEST MULTIPLY ALGORITHMS
◼ Use FFT approach, but over modular multiplication mod 2^k+1
◼Schonhage, Strassen 71:
𝑂(𝑛 𝑙𝑜𝑔𝑛 log log 𝑛 )
∗𝑛
log
Furer, 07: 𝑂(𝑛 log 𝑛 2
)
Still open whether there’s a linear time algorithm
UNEVEN DIVIDE AND CONQUER
◼Sometimes, sizes of sub-parts not identical or depends on the input
◼Example:
◼Given a pointer to a binary tree, compute its depth
◼Depth(r: node)
◼If lc.r ≠NIL then d:= Depth(lc.r) else d:=0
◼If rc.r ≠ NIL then d:= max(d, Depth(rc.r))
◼Return d
TIME ANALYSIS
◼If tree is balanced, get the recurrence
◼T(n)= 2T(n/2)+ O(1),
◼If tree is totally unbalanced,
◼T(n)= T(n-1)+O(1)
EITHER CASE
◼T(n) = T(L) + T(R) +c
◼n = L+R+1
(T(0)=0, T(1)=c to make things fit)
Then by strong induction, we claim T(n) is at most cn
T(L) ≤ 𝑐𝐿
T(R) ≤ cR
T(n) ≤ cL+cR+c = c (L+R+1) =cn