Algorithms and Data Structures 2014
Exercises and Solutions Week 13
1
Toom-Cook (continued)
During the last lecture, two polynomials A(x) = a0 + a1 x and B(x) = b0 + b1 x both of degree 1
were multiplied, first by evaluating A and B at three distinct points – we picked 0, 1 and −1 as
concrete values – then these results were multiplied pairwise, and finally the value representation
of the resulting polynomial C(x) = c0 + c1 x + c2 x2 (of degree 2) was translated back using the
inverse of the corresponding evaluation matrix.
1. Repeat this procedure, but now with 0, 1, ∞ as evaluation points. Evaluating A at ∞ means
that we just take A’s most leading coefficient a1 . Therefore, evaluating C at ∞ (being c2 )
amounts to computing the product a1 b1 .
2. Explain the relation of this procedure with Karatsuba’s multiplication algorithm.
3. Instead of using polynomials of degree 1 we repeat the whole procedure with polynomials
of degree 2. Then the multiplication result will have degree 4, so we need 5 evaluation
points for interpolation (the translation back from values to coefficients). Evaluate both
A(x) = a0 + a1 x + a2 x2 and B(x) = b0 + b1 x + b2 x2 at 0, 1, −1, 2, ∞. Express the evaluation
of C(x) = c0 +c1 x+c2 x2 +c3 x3 +c4 x4 in these points by means of a 5×5 matrix, and compute
the inverse of this matrix. Again, explain the relation with the Toom-Cook3 algorithm (see
exercise 4, week 12).
Solution
1. The evaluation matrix corresponding to these evaluation points is
 0
 

0 01 02
1 0 0
M = 10 11 12  = 1 1 1
0 0 1
0 0 1
and its inverse is given by1

M −1
1
= −1
0

0 0
1 −1 .
0 1
Thus,
  


c0
1 0 0
C(0)
c1  = −1 1 −1  C(1) 
0 0 1
C(∞)
c2



1 0 0
A(0)B(0)
= −1 1 −1  A(1)B(1) 
0 0 1
A(∞)B(∞)



1 0 0
a0 b0
= −1 1 −1 (a0 + a1 )(b0 + b1 )
0 0 1
a1 b1


a0 b0
= (a0 + a1 )(b0 + b1 ) − a0 b0 − a1 b1  .
a1 b1
1 You
were allowed to use a computer to obtain inverse matrices.
2. The resulting formulas for the coefficients of C are exactly the ones used by Karatsuba’s
multiplication algorithm.
3. The evaluation matrix is now
 0
0
01
0
 1
11

0
(−1)1
M =
(−1)
0
 2
21
0
0
with inverse
M −1
Now we have
 

c0
6
c1 
−3
  1
c2  = −6
  6
c3 
3
c4
0

6
−3
1
−6
= 
6
3
0

6
−3
1
−6
= 
6
3
0
02
12
(−1)2
22
0
03
13
(−1)3
23
0

0
6
3
−3
0
6
−3
1
−6
= 
6
3
0
0
0
6 −2
3
3
−3 −1
0
0
0
−1
0
1
0
0
0
6 −2
3
3
−3 −1
0
0
0
−1
0
1
0
0
0
6 −2
3
3
−3 −1
0
0
0
−1
0
1
0
 
1
04
1
14 
 

(−1)4 
 = 1
24  1
0
1
0
−2
3
−1
0
0
−1
0
1
0
0
1
−1
2
0
0 0
1 1
1 −1
4 8
0 0

0
1

1
,
16
1

0
12 

−6 
.
−12
6


C(0)
0


12 
  C(1) 
C(−1)
−6 


−12  C(2) 
C(∞)
6


A(0)B(0)
0


12 
  A(1)B(1) 
A(−1)B(−1)
−6 


−12  A(2)B(2) 
A(∞)B(∞)
6


a0 b0
0


12 
  (a0 + a1 + a2 )(b0 + b1 + b2 ) 
 (a0 − a1 + a2 )(b0 − b1 + b2 )  .
−6 


−12 (a0 + 2a1 + 4a2 )(b0 + 2b1 + 4b2 )
6
a2 b2
With some imagination, these are seen to be precisely the calculations performed by ToomCook3.
2
2.1
Fast Fourier Transform
Evaluation
Give an algorithm that evaluates a polynomial A(x) = a0 + a1 x + a2 x2 + · · · + an xn at point x in
O(n) time.
Solution
1: function E VAL(A, x)
2:
v←0
3:
p←1
4:
for i from 0 to n do
5:
v ← v + ai · p
6:
p←x·p
7:
return v
2.2
Basic properties of nth roots of unity
1. What is the sum of the nth roots of unity?
2. What is their product if n is odd? If n is even?
Solution
2π·0
1. For n = 1, the only nth root of unity is ω 0 = cos 2π·0
1 + i sin 1 = 1. Now suppose n > 1.
Using the formula for a geometric series, we see that
n−1
X
ωk =
k=0
ωn − 1
ω−1
2π·n
cos 2π·n
n + i sin n − 1
ω−1
cos(2π) + i sin(2π) − 1
=
ω−1
cos(2π) − 1
=
ω−1
1−1
=
ω−1
0
=
ω−1
= 0.
=
2. Using the laws shown during the lecture, we find that
n−1
Y
n−1
Y
2π · k
2π · k
+ i sin
ω =
cos
n
n
k=0
k=0
!
!
n−1
n−1
X 2π · k
X 2π · k
= cos
+ i sin
n
n
k=0
k=0
!
!
n−1
n−1
2π X
2π X
= cos
k + i sin
k
n
n
k=0
k=0
2π(n − 1)n
2π(n − 1)n
= cos
+ i sin
2n
2n
k
= cos(π(n − 1)) + i sin(π(n − 1))
= cos(π(n − 1))
−1
if n is even
=
1
otherwise.
2.3
Inverse FFT
Show that the matrices M (1, ω, ω 2 , . . . , ω n−1 ) and n1 M (1, ω −1 , ω −2 , . . . , ω −(n−1) ) are indeed inverses of each other. Hint: Show that their product is the identity matrix.
Solution We name M = M (1, ω, ω 2 , . . . , ω n−1 ) and N = M (1, ω −1 , ω −2 , . . . , ω −(n−1) ). That is,
Mij = ω ij and Nij = ω −ij if we number the indices from 0 to n − 1. We want to show that
M N = nI, where I is the identity matrix. We have
(M N )ij =
n−1
X
k=0
Mik Nkj =
n−1
X
ik
ω ω
k=0
−kj
=
n−1
X
(ω i−j )k .
k=0
Obviously, this yields n if i = j, giving us the desired diagonal. If i 6= j, we use again the
geometric series formula to obtain
n−1
X
(ω i−j )k =
k=0
(ω i−j )n − 1
ω i−j − 1
+ i sin 2π(i−j)n
−1
cos 2π(i−j)n
n
n
i−j
ω
−1
cos(2π(i − j)) + i sin(2π(i − j)) − 1
=
ω i−j − 1
1−1
= i−j
ω
−1
0
= i−j
ω
−1
= 0.
=
2.4
Practice with the fast Fourier Transform
1. What is the FFT of (1, 0, 0, 0) What is the appropriate value of ω in this case? And of which
sequence is (1, 0, 0, 0) the FFT?
2. Repeat for (1, 0, 1, −1).
Solution
2π
1. We are given 4 coefficients, so ω = cos 2π
4 + i sin 4 = i. However, in this case the value of ω
is irrelevant, since

 
1 1
1
1
1
1 ω ω 2 ω 3  0
 
FFT((1, 0, 0, 0), ω) = 
1 ω 2 ω 4 ω 6  0
0
1 ω3 ω6 ω9
 
1
1

=
1 .
1
The same holds for the inverse FFT, as
 
1
  1
−1 0
FFT   = FFT((1, 0, 0, 0), ω −1 )
0
4
0

 
1
1
1
1
1
−1
−2
−3   
1
1
ω
ω
ω
 0
= 
4 1 ω −2 ω −4 ω −6  0
0
1 ω −3 ω −6 ω −9
 
1
1
1
.
= 

4 1
1
2. Now we have (filling in i for ω as obtained in the previous part)

 
1
1 1 1 1
1 i i 2 i3   0 
 
FFT((1, 0, 1, −1), i) = 
1 i2 i4 i6   1 
−1
1 i3 i6 i9
 

1
1 1
1
1
1 i −1 −i   0 
 
=
1 −1 1 −1  1 
−1
1 −i −1 −i
 
1
i

=
3 .
i
Note that i−1 =
2.5
i
i2
=
i
−1
= −i. Therefore,
 
1

0
 1
FFT−1 
 1  = 4 FFT((1, 0, 1, −1), −i)
−1

 
1
1 1
1
1
−1
−2
−3  

1
1
i
i
i
 0 
= 
−2
−4
−6
i
i  1 
4 1 i
−1
1 i−3 i−6 i−9

 
1 1
1
1
1
 0 
1
1
−i
−1
i
 
= 
4 1 −1 1 −1  1 
1 i −1 −i
−1
 
1

1
−i
.
= 

4 3
i
Practice with polynomial multiplication by FFT
1. Suppose that you want to multiply the two polynomials x + 1 and x2 + 1 using the FFT.
Choose an appropriate power of two, find the FFT of the two sequences, multiply the results
componentwise, and compute the inverse FFT to get the final result.
2. Repeat for the pair of polynomials 1 + x + 2x2 and 2 + 3x.
Solution
1. Multiplication of a polynomial of degree 1 with a polynomial of degree 2 yields a polynomial of degree 3. Obviously, 3 < 4, and 4 is a power of 2. As in the previous exercise, ω = i.
The FFT of the first polynomial is

 
1
1 1 1 1
1 i i2 i3  1
 
FFT((1, 1, 0, 0), i) = 
1 i2 i4 i6  0
0
1 i3 i6 i9

 
1
1 1
1
1
1 i −1 −i  1
 
=
1 −1 1 −1 0
0
1 −i −1 −i


2
1 + i

=
 0 .
1−i
For the second polynomial we have

1 1
1 i
FFT((1, 0, 1, 0), i) = 
1 i2
1 i3

1 1
1 i
=
1 −1
1 −i
 
2
0

=
2 .
0
 
1
1
0
i3 
 
i6  1
0
i9
 
1
1
1
0
−1 −i 
 
1 −1 1
0
−1 −i
1
i2
i4
i6
The coefficients of the product polynomial are now given by the inverse FFT of the pointwise product of these value representations. That is,
 
4
  1
−1 0
FFT   = FFT((4, 0, 0, 0), i−1 )
0
4
0

 
1 1
1
1
4
−1
−2
−3   
1
1
i
i
i
 0
= 
4 1 i−2 i−4 i−6  0
1 i−3 i−6 i−9
0
 
1
1

=
1 .
1
We conclude that the resulting polynomial is 1 + x + x2 + x3 .
2. We still have ω = i. In this case, the FFT of the first polynomial is

 
1
1 1 1 1
1 i i2 i3  1
 
FFT((1, 1, 2, 0), i) = 
1 i2 i4 i6  2
0
1 i3 i6 i9

 
1
1 1
1
1
1 i −1 −i  1
 
=
1 −1 1 −1 2
0
1 −i −1 −i


4
 i−1 

=
 2 .
−1 − i
For the second polynomial we have

1
1
FFT((2, 3, 0, 0), i) = 
1
1

1
1
=
1
1

1
i
i2
i3
 
2
1
3
i3 
 
i6  0
0
i9
 
2
1
1
3
−1 −i 
 
1 −1 0
0
−1 −i
1
i2
i4
i6
1
i
−1
−i

5
2 + 3i

=
 −1  .
2 − 3i
Now we obtain the coefficients of the product polynomial as


20
−5 − i 1
−1

FFT−1 
 −2  = 4 FFT((20, −5 − i, −2, i − 5), i )
i−5



1 1
1
1
20


1 1 i−1 i−2 i−3 
 −5 − i
= 
−2
−4
−6  

−2 
i
i
4 1 i
−3
−6
−9
1 i
i
i
i−5



1 1
1
1
20


1 1 −i −1 i 
 −5 − i
= 



−2 
4 1 −1 1 −1
1 i −1 −i
i−5
 
2
5

=
7 ,
6
so the product polynomial is 2 + 5x + 7x2 + 6x3 .
3
FFT in modular arithmetic
This exercise illustrates how to do the Fourier Transform (FT) in modular arithmetic, for example,
modulo 7.
1. There is a number ω such that all the powers ω 0 , ω 1 , . . . , ω 5 are distinct (modulo 7). Find
this ω and show that ω 0 + ω 1 + · · · + ω 5 = 0 (modulo 7).
2. Using the matrix form of the FT, produce the transform of the sequence (0, 1, 1, 1, 5, 2)
modulo 7; that is, multiply this vector by the matrix M (ω 0 , ω 1 , . . . , ω 5 ). In the matrix multiplication, all calculations should be performed modulo 7.
3. Write down the matrix necessary to perform the inverse FT. Show that multiplying by this
matrix returns the original sequence. (Again all arithmetic should be performed modulo 7.)
4. Now show how to multiply the polynomials x2 + x + 1 and x3 + 2x − 1 using the FT modulo
7.