CSE 190, Great ideas in algorithms:
Polynomial multiplication and FFT
1
Polynomial multiplication
A univariate polynomial is
f (x) =
n
X
f i xi .
i=0
The degree of a polynomial is the maximal i such that fi 6= 0. The product of two polynomials
f, g of degree n each is given by


! n
!
min(i,n)
n
n
n
2n
X
X
XX
X
X

f (x)g(x) =
f i xi
gj xj =
fi gj xi+j =
fj gi−j  xi .
i=0
j=0
i=0 j=0
i=0
j=0
P
fj gi−j for all
So, in order to compute the coefficients of f g, we need to compute min(i,n)
j=0
2
0 ≤ i ≤ 2n. This trivially takes time O(n ). We will see how to do it in time O(n log n),
using a variant of the Fast Fourier Transform (FFT).
1.1
FFT
Let ωn ∈ C be a primitive n-th root of unity, for example ωn = e2πi/n = cos(2π/n) +
i sin(2π/n). The order n Fourier matrix is given by
(Mn )i,j = (ωn )ij = (ωn )ij
mod n
What is so special about it? well, as we will see soon, we can multiply it by a vector in
time O(n log n), whereas for general matrices this takes time O(n2 ). To keep the description
simple, we assume from now on that n is a power of two.
Theorem 1.1. For any x ∈ Cn , we can compute (Mn )x using O(n log n) additions and
multiplications.
1
Proof. Decompose Mn into four n/2 × n/2 matrices as follows. First, reorder the rows to
list first all n/2 even indices, then all n/2 odd indices. Let Mn0 be the new matrix, with
re-ordered rows. Decompose
A B
0
Mn =
.
C D
What are A, B, C, D? If 1 ≤ a, b ≤ n/2 then
• Aa,b = (Mn )2a,b = (ωn )2ab = (ωn/2 )ab = (Mn/2 )a,b .
• Ba,b = (Mn )2a,b+n/2 = (ωn )2ab+an = (ωn )2ab = (Mn/2 )a,b .
• Ca,b = (Mn )2a+1,b = (ωn )2ab+b = (Mn/2 )a,b · (ωn )b .
• Da,b = (Mn )2a+1,b+n/2 = (ωn )2ab+b+an+n/2 = −(Mn/2 )a,b · (ωn )b .
So, let P be the n/2 × n/2 diagonal matrix with Pa,b = (ωn )b . Then
C = −D = Mn/2 P.
A = B = Mn/2 ,
In order to compute (Mn )x, decompose x = (x0 , x00 ) with x0 , x00 ∈ Cn/2 . Then
(Mn0 )x = (Ax + By, Cx + Dy) = (Mn/2 (x + y), Mn/2 P (x − y)).
Let T (n) be the number of additions and multiplications required to multiply Mn by a vector.
Then to compute (Mn )x we need to: compute x + y, x − y, P (x − y), which takes time 3n
since P is diagonal; multiply each of the vectors by Mn/2 , which takes time 2T (n/2); and
finally reorder the rows back to compute (Mn )x, which takes another n steps. So we obtain
the recursion
T (n) = 2T (n/2) + 4n.
This recursion solves to T (n) ≤ 4n log n:
T (n) ≤ 2 · 4(n/2) log(n/2) + 4n = 4n log n − 4n + 4n.
1.2
Inverse FFT
The inverse of the Fourier matrix is the complex conjugate of the Fourier matrix, up to
scaling.
Lemma 1.2. (Mn )−1 = n1 Mn∗ .
Proof. We have (Mn∗ )a,b = (ωn )ab = ωn−ab . So
(Mn Mn∗ )a,b
=
n−1
X
c=0
2
(ωn )(a−b)c
If a =
then the sum equals n. We claim that when a 6= b the sum is zero. To see that, let
Pbn−1
S = i=0 ωnic , where c 6= 0 mod n. Then
n
n−1
X
X
ic
(ωn ) · S =
(ωn ) =
(ωn )ic = S.
c
i=1
i=0
Since ωn has order n, we have ωnc 6= 1, and hence S = 0.
Corollary 1.3. For any x ∈ Cn , we can compute (Mn )−1 x using O(n log n) additions and
multiplications.
1.3
Fast polynomial multiplication
Let f (x) be a polynomial. Its order n Fourier transform is defined as its evaluations on the
n-th roots of unity:
fb(i) = f ((ωn )i ).
Lemma 1.4. Let f (x) be a polynomial of degree ≤ n − 1.
computed in time O(n log n).
P
i
Proof. Let f (x) = n−1
i=0 fi x . Then

f0
n−1

X
 f1
fb(j) =
fi (ωn )ij = (Mn )  ..
 .
i=0
fn−1
Its Fourier transform can be





Corollary 1.5. Let f be a polynomial of degree ≤ n − 1. Given the evaluations of f at the
n-th roots of unity, we can recover the coefficients of ’ f in time O(n log n).
Proof. Compute f = (Mn )−1 fb.
The Fourier transform of a product has a simple formula:
d
(f
g)(i) = (f g)((ωn )i ) = f ((ωn )i ) · g((ωn )i ) = fb(i) · gb(i).
So, we can multiply two polynomials as follows: compute their Fourier transform; multiply it coordinate-wise; and then perform the inverse Fourier transform. Note that if f, g
have degrees d, e, respectively, then f g has degree d + e. So, we need to choose n > d + e to
compute their product correctly.
3
Fast polynomial multiplication
Input : two p o l y n o m i a l s f, g , o f d e g r e e s d and e , r e s p e c t i v e l y
g i v e n as l i s t s o f c o e f f i c i e n t s
Output : t h e i r product f g as a l i s t o f c o e f f i c i e n t s
0.
1.
2.
3.
4.
5.
1.4
Let n be th e s m a l l e s t power o f two , such t h a t n ≥ d + e + 1
Pad f, g t o l e n g t h n i f n e c e s s a r y ( by adding z e r o s )
Compute x = (Mn )f
// x = fb
Compute y = (Mn )g
// y = gb
Compute zi = xi yi f o r 1 ≤ i ≤ n
// z = fcg
−1
Return (Mn ) z .
Multivariate polynomials
Let f, g be multivariate polynomials. For simplicity, lets consider bivariate polynomials. Let
f (x, y) =
n
X
i j
fi,j x y ,
g(x, y) =
i,j=0
n
X
gi,j xi y j .
i,j=0
Their product is
(f g)(x, y) =
n
X
0
2n
X
0
fi,j gi0 ,j 0 xi+i y j+j =
i,j,i0 ,j 0 =0

min(n,i) min(n,j)
X
X
i0 =0
j 0 =0

i,j=0

fi0 ,j 0 gi−i0 ,j−j 0  xi y j .
Our goal is to compute f g quickly. One approach is to define a two-dimensional FFT.
Instead, we would reduce the problem of multiplying two bivariate polynomials of degree n
in each variable, to the problem of multiplying two univariate polynomials of degree O(n2 ),
and then apply the algorithm using the standard FFT.
Let N be large enough to be determined later, and define the following univariate polynomials:
n
n
X
X
F (z) =
fi,j z N i+j , G(z) =
gi,j z N i+j .
i,j=0
i,j=0
We can clearly compute F, G from f, g in linear time, and as deg(F ), deg(G) ≤ (N + 1)n,
we can compute F · G in time O((N n) log(N n)). The only question is whether we can infer
f · g from F · G.
P
Lemma 1.6. Let N ≥ 2n + 1. If H(z) = F (z)G(z) = Hi z i then
(f g)(x, y) =
2n
X
i,j=0
4
HN i+j xi y j .
Proof. We have
H(z) = F (z)G(z) =
n
X
!
fi,j z N i+j
i=0
=
n
X
n
X
!
0
gi0 ,j 0 z N i +j
0
i=0
0
0
fi,j gi0 ,j 0 z N (i+i )+(j+j ) .
i,j,i0 ,j 0 =0
We need to show that the only solutions for
N (i + i0 ) + (j + j 0 ) = N i∗ + j ∗
where 0 ≤ i, i0 , j, j 0 ≤ n and 0 ≤ i∗ , j ∗ ≤ 2n, are these which satisfy i + i0 = i∗ , j + j 0 = j ∗ . As
0 ≤ j + j 0 , j ∗ ≤ 2n and N > 2n, if we compute the value modulo N we get that j + j 0 = j ∗ ,
and hence also i + i0 = i∗ .
Corollary 1.7. We can compute the product of two bivariate polynomials of degree n in
time O(n2 log n).
5