Chapter 5
Divide and Conquer,
Part 3
Slides by Kevin Wayne.
Copyright © 2005 Pearson-Addison Wesley.
All rights reserved.
1
Divide and Conquer: Goals
Divide et impera.
Veni, vidi, vici.
- Julius Caesar
Understand divide and conquer algorithms:
Break up problem into several parts.
Solve each part recursively.
Combine solutions to sub-problems into overall solution.



See examples of most common usage:
For example, Mergesort
– Break up problem of size n into two equal parts of size ½n.
– Solve two parts recursively.
– Combine two solutions into overall solution in linear time.
Counting inversions, closest pair, multiplication, FFT


Analysis:
Be able to establish time complexity of divide and conquer
algorithms

2
Matrix Multiplication
Matrix Multiplication
Matrix multiplication. Given two n-by-n matrices A and B, compute
C = AB.
 c11
c
 21
 

c n 1
c12
c22




cn 2

c1n   a11
c2 n   a21


 

 
cnn  an1
a12
a22




an 2

a1n  b11
a2 n  b21


 

 
ann  bn1
Brute force. (n3) arithmetic operations.
b12
b22




bn 2

b1n 
b2 n 
 

bnn 
n
cij   a ik bkj
k 1
Fundamental question. Can we improve upon brute force?

4
Matrix Multiplication: Warmup
Divide-and-conquer.
Divide: partition A and B into ½n-by-½n blocks.
Conquer: multiply 8 ½n-by-½n recursively.
Combine: add appropriate products using 4 matrix additions.



C11 C12   A11
C C    A
22 
 21
 21
A12   B11 B12 


A22   B21 B22 
T(n)  8T n /2  
recursive calls

(n2 )
C11

C12

C21

C22

A11  B11   A12  B21 
A11  B12   A12  B22 
A21  B11   A22  B21 
A21  B12   A22  B22 
 T(n)  (n 3 )
add, form submatrices

5
Matrix Multiplication: Key Idea
Key idea. multiply 2-by-2 block matrices with only 7 multiplications.
C11 C12   A11
C C    A
22 
 21
 21


A12   B11 B12 


A22   B21 B22 
P1

A11  ( B12  B22 )
P2
 ( A11  A12 )  B22
P3
 ( A21  A22 )  B11
C11

P5  P4  P2  P6
P4

A22  ( B21  B11 )
C12

P1  P2
C21

P3  P4
P5
P6
 ( A11  A22 )  ( B11  B22 )
 ( A12  A22 )  ( B21  B22 )
C22

P5  P1  P3  P7
P7
 ( A11  A21 )  ( B11  B12 )
7 multiplications.
18 = 10 + 8 additions (or subtractions).

6
Fast Matrix Multiplication
Fast matrix multiplication. (Strassen, 1969)
Divide: partition A and B into ½n-by-½n blocks.
Compute: 14 ½n-by-½n matrices via 10 matrix additions.
Conquer: multiply 7 ½n-by-½n matrices recursively.
Combine: 7 products into 4 terms using 8 matrix additions.




Analysis.
Assume n is a power of 2.
T(n) = # arithmetic operations.


T(n)  7T n /2 
recursive calls
(n 2 )
 T(n)  (n log2 7 )  O(n 2.81 )
add, subtract
7
Fast Matrix Multiplication in Practice
Implementation issues.
Sparsity.
Caching effects.
Numerical stability.
Odd matrix dimensions.
Crossover to classical algorithm around n = 128.





Common misperception: "Strassen is only a theoretical curiosity."
Advanced Computation Group at Apple Computer reports 8x
speedup on G4 Velocity Engine when n ~ 2,500.
Range of instances where it's useful is a subject of
controversy.


Remark. Can "Strassenize" Ax=b, determinant, eigenvalues, and
other matrix ops.
8
Fast Matrix Multiplication in Theory
Q. Multiply two 2-by-2 matrices with only 7 scalar multiplications?
A. Yes! [Strassen, 1969]
(n log2 7 )  O(n 2.81 )
Q. Multiply two 2-by-2 matrices with only 6 scalar multiplications?

A. Impossible. [Hopcroft and Kerr, 1971]
log2 6
2.59
(n
)  O(n
)
Q. Two 3-by-3 matrices with only 21 scalar multiplications?
A. Also impossible.

(n log3 21 )  O(n 2.77 )
Q. Two 70-by-70 matrices with only 143,640 scalar multiplications?

A. Yes! [Pan, 1980]
log70 143640
2.80
(n
Decimal wars.
December, 1979: O(n2.521813).
January, 1980: O(n2.521801).

)  O(n
)


9
Fast Matrix Multiplication in Theory
Best known. O(n2.376) [Coppersmith-Winograd, 1987.]
Conjecture. O(n2+) for any  > 0.
Caveat. Theoretical improvements to Strassen are
progressively less practical.
10