Proceedings of the International Congress of Mathematicians
Vancouver, 1974
Complexity of Product and Closure Algorithms
for MatricesT
M. S. Paterson
0. Abstract Some algorithms for matrix product and for "transitive closure" of matrices
over various structures are described, and relationships among the complexities of these
operations are demonstrated. Determining the connected components of directed graphs,
finding shortest distances in a network, and recognizing context-free languages are all
applications of closure algorithms in different structures.
1. Matrix product over rings. If A, B are n x n matrices over a ring, then their
product and sum are given by
{A-B)ik = 2 %• x bjh
i
{^ + B)ik = aik + bik.
Counting just the elementary ring operations used by the obvious algorithm, we
have w3 multiplications and n3 - n2 additions, and so 0(/?3) operations for the
product and n2 operations for the sum. We shall refer loosely to these counts as the
complexity of the corresponding algorithm.
If A, B are partitioned in a suitably consistent way as shown below, we have the
identity :
^ n I A\2 \ ( Bn j 2?12 \ / AnBn + 412^21 j ^11^12 + Ai2B
22:
^21 M22 / \ ^21 I ^22 / \ ^21^11 + ^22^21 ! ^21-^12 + ^22^22
This suggests a recursive algorithm which forms the product of two n x n matrices
{n > 1) by computing the 8 indicated products of n/2 x n/2 matrices and then summing them as shown. These smaller products are computed by the same algorithm
tThis research was supported in part by the National Science Foundation under grant GJ43634X.
© 1975, Canadian Mathematical Congress
483
484
M. S. PATERSON
applied recursively. This algorithm has a complexity P{n)9 which satisfies P{n) ^
8 -P(r«/21) + 0{n2). It is easily seen, however, that P{n) is proportional to n3.
2. Strassen's algorithm. The remarkable algorithm discovered by Strassen [Str]
provides a key motivation for the results presented in this paper. He showed that
the four submatrices A^By + Ai2B2j for i,j = 1 , 2 , can be computed using only 7
rather than 8 small matrix products. The method is most easily presented in diagrammatic form.
B.21
Blx
B12
B 22
11
*i:
/
12
•
12
L
l
l
n
21
22
c
21
= 4
1l ì "
*~P^T
LÀ
•>iJL ==<>— —
22
•
1,1
—-o 22
In this figure a • (o) in cell {Aij9 Bkl) represents the term + ( - ) AyBkh The 7
connected groups each represent a set of terms which can be produced by a single
product. For example, the large group of 9 terms comes from the product
{-An + A2X + ^22X^11 - ^12 + ^22)- The f o u r quadrants of the product matrix
C are given by
C n = a 4- b9
C2i = a + c + d 4- g9
C22 = a + c 4- d + e.
These identities are a variant of those in [Str], obtained by applying a simple
linear transformation. The complexity of Strassen's algorithm satisfies P{n) ^
7-P(«r«/21) + 0{n2) which yields P{n) = 0{nQ) where 0 = log 2 7 « 2-8.
Since recurrence relations of the above type will occur frequently, we present
here a more general solution.
LEMMA. Let G{n) = n& • h{n) where ß^O and h is a positive, nondecreasing function
satisfying h{n) = o{ne)foralle > O.Ifa ^ \9b > 1 and F is a nonnegative function
satisfying :
F{n) = a-F{Vn/b~\) + 0{G{n))
then, ifa = log6 a,
PRODUCT AND CLOSURE ALGORITHMS FOR MATRICES
485
F{n) £ 0{na)
if a > ß9
è 0{G{n))
if a < j8,
£ 0{G{n)-log n) ifa = ß.
3. Matrix inversion and determinants. For fields these immediate applications of
Strassen's product algorithm are given in [Str]. Since
An \ A12 W
^2lM22/
/ A^ +
\
-E
ArtAl2E-^A2lArt -
A£A12E-i
£-1
lA
21^U
where E = A22 — A2ìArtAì2 and we assume An and E to be nonsingular, we get
a recursive algorithm for matrix inverse with complexity I{n) satisfying
I{n) <: 2-I{Vnj2~\) + 0{P{n)) 4- 0{n2) <: 0{n6)
where 0 = log2 7 by the general solution given above. Similarly, we have for determinants that
det (AAn AÀ12) = dtt{An) - det(E)
\^21
^22/
and so the complexity D{n) satisfies
D{n) = 2-Z)(r/î/21) 4- I^n/21) 4- 0{P{Vnj21))
d
and so D{n) g 0{n ). A pivotal algorithm for inversion which does not require the
nonsingularity assumption above and which has the same order of complexity is
given in [Bun].
A converse reduction shows that inversion cannot be appreciably easier than
product,
/ / A Oy1
//
-A
A-B\
0 / 5 = 0
I
-B]
\0 0 / /
\0
0
//
where /is the identity matrix and 0 is the zero matrix. Therefore P{n) g I{3n).
4. Boolean matrices. Matrices with domain {0, 1} (for {false, true}) are used to
represent relations on a finite set, where, e.g., AtJ = 1 iff (/, j) is in the relation.
Composition of relations corresponds to matrix product with the Boolean operations A , v . We write A 0 B for this product when we want to make the operations
explicit.
{A^B)ik=
V^Aiy,
Also, A v B corresponds to matrix sum where {A v B)ik = aik v bik9 and we
write A £ B for A v B — B. A relation R is reflexive and transitive if R contains
the identity relation and also RoR.
Thus the {reflexive and) transitive closure A* of a square matrix A is the unique
minimal matrix satisfying
X=> A v I v
{X®X)
0
where / is the identity matrix. If we define A = /, An+l
= A 0 An for n _• 0, then
486
M. S. PATERSON
since the Boolean product is associative, it is easily shown that A* = Vw^0 An.
The following useful formula for the closure of a partitioned matrix is given and
proved for regular algebras in [Con] :
Mn ^i 2 y = Mfi
i2l AAoo
V Aoi
22) J
V\
v A& Al2 E*A2lAft
E*A2lA*x
Af,A12 E* '
E*
where E = ^422 v ^21 ^n^i2If P{n) is the complexity of Boolean product and C{n) the complexity of closure,
then based on the above formula we have C{n) S 2 • C(r«/21) 4- 0{P{n)) 4- 0{n2)
and so if P satisfies the conditions on G in the general solution we have C{n) ^
0{P{n)). Also, since
/O
A
0\*
0
\0
0 B)
0 0/
//
=io
\0
A
A-B\
I B
0 /
/
we have P{n) ^ C{3n) S 0{P{n)).
The obvious similarities with the results for matrix inversion over a field stem
from the analogy between (/ — A)~~l « S«§o An in a field and A* = V„^ 0 An for
the Boolean operations.
5. Reduction of Boolean product to ring product. Strassen's method provides a
subcubic algorithm for matrix products over a ring but it is not directly applicable
to Boolean product. However, we can use the homomorphism h: Z-> {0,1}
defined by
h{n) = 1 if n Ï 0,
= 0 if n = 0,
under which x , 4- correspond to A , v , respectively. If h is extended to matrices
in the obvious way, we have A®B = h{A gl?) for all 0,1-matrices. Better still
is to use Strassen's algorithm on Zn+\9 the ring of integers modulo « 4 - 1 , since
the ring product of two n x n matrices over {0, 1} in Z has entries no greater
than n. In this case the basic ring operations can be expressed as sequences of
Boolean operations acting on binary representations of the ring elements. In
this way we obtain a Boolean product algorithm, and hence a closure algorithm
of order 0{ne-(log n)2). For more details of these reductions, see [Fis], [Fur] and
[Mun].
If we regard Boolean matrices as representing directed graphs {A{j = 1 iff there
is an arc from i toy) then the closure matrix gives the connectivity relation {Afj = 1
iff there is a path from / toy). When the domain is taken to be {nonnegative reals}
(J {4- 00}, a matrix represents a directed network with distances on the arcs. The
closure of such a matrix with respect to the operations 4-, minimum, i.e., "product" is n^n, "sum" is min, gives the lengths of the shortest path between each
pair of nodes. The recursive formula for closure in terms of product is still valid
with + , min in place of A , v , and hence the close relation between the complexity of product and closure. In this case, however, no subcubic algorithm for
487
PRODUCT AND CLOSURE ALGORITHMS FOR MATRICES
product is known. Different partitions of the matrix yield a family of 0{ns) algorithms, including the well-known Floyd algorithm given in [Flo],
6. Context-free language recognition [Val], Our final example which introduces
some new difficulties concerns context-free languages. With no essential loss of
generality we can define a context free grammar (cfg) G as follows. G consists of a
finite alphabet 2 = {AÌ9 •••, Ak) and a finite set P of productions', each of the form
"Ai -* AjAk". We write W\A{w2 => W\A}Akw^ where wÌ9 w2 are arbitrary strings
over 2 and Ai -> AjAk is in P. Let "=>*" be the reflexive and transitive closure of
the "=>" relation. For each At- e 2 we define the language generated by A{ as LAi =
{w\Ai =>* w}. The recognition problem for G is to determine for a string we 2*
whether or not weLAi. For a cfg G we define the binary operator ® over the
domain of subsets of 2 by
Si® S2= {Ai\l Aj e Sh !AkeS2
and A{ -> AjAk e P}.
We can then define a ^-product and (J -sum for matrices with subsets of 2 as
elements. Then the transitive closure A+ of such a matrix A is the minimal matrix
satisfying: X 2 A U C^H^O- The ®-operator and hence the corresponding
matrix product is not associative and so a new algorithm for transitive closure is
necessary. The correspondence with the recognition problem is the following. If
w is the string X\ ••• Xn_i e 2n~l9 let Mw be the n x n matrix defined by
MitM = {Xt)
^•,;= 0
for / = 1, •••, n - 1,
if/ * / + 1-
LEMMA, i w any w, and 1 ^ u < v ^ w,
* Z W . . . *„_!}.
PROOF. By induction on v — w.
Thus, to solve the recognition problem it is sufficient to compute M+. Younger's
algorithm [You] computes this matrix in a straightforward way in 0{n3) operations.
Valiant [Val] obtains a subcubic algorithm by a reduction to Strassen's method.
He first shows that the ^-product for any cfg is of the same order as Boolean product. For a cfg with only one production the product is isomorphic with Boolean
product. The matrix product for an arbitrary cfg is obtainable as the union of the
products corresponding to each production as a singleton. The remaining step is
to show that transitive closure is of the same order as product for strictly upper
triangular (s.u.t) matrices, i.e., A(j = 0 if/ ^ /.
LEMMA
If A is s.u.t. then A+ is the unique s.u.t. matrix satisfying X = A U
If
\b
A
c
d
\ e
f
\g
is s.u.t. and
Xc
\E
D
F
488
M. S. PATERSON
then, by the last lemma, D = d U BD U CF [) DG. Thus
4-
\ ß
D
X
again by the lemma. We therefore have a recursive algorithm Q for closing a s.u.t.
matrix of the form
\
c
d
Xf
where the submatrices {B9 C, E) and {G) are already closed. We apply Q to the submatrix {E9f9 G) to compute F9 then compute the product CF9 andfinallycompute
D by applying Q to the matrix {B9 d U CF9 G) as above. If the linear dimensions of
the partition are in the ratio b:e:g = 1:1 : </2 and G{n) is the complexity of Q,
we have the recurrence relation
Q{n)^2-Q{rn/V2-\) + 0{P{n)).
For a closure algorithm C we close matrices {b9 c9 e) and {g) recursively and then
apply Q. Hence
C{n) S C{^/2.n/{l 4- J%) + C{n/{1 4- v^)) 4- Q{n) S 2-C(2«/3) 4- ß(«).
Since P{n) S 0{n%log nf)9 we derive C{n) ^ Ö(nö(log nf) with ô = log2 7 also,
thus achieving a subcubic recognition algorithm.
7. Conclusion. We have shown how the computational complexity for different
matrix operations over various structures can be related, and in particular how any
fast matrix product algorithms for rings can be applied indirectly to give correspondingly fast algorithm for finding connected components of directed graphs and for
recognizing context-free languages. Strassen's 0(nloga 7) algorithm for matrix product provides a nontrivial application of these reductions.
References
[Bun] J. Bunch and J. Hopcroft, Triangular factorization and inversion by fast matrix multiplication, Math. Comp. 28 (1974), 231-236.
[Con] J. Conway, Regular algebra andfinitemachines, Chapman and Hall, London, 1971.
[Fis] M. Fischer and A. Meyer, Boolean matrix multiplication and transitive closure, IEEE Conf.
Record of Twelfth Annual Sympos. on Switching and Automata Theory, 1971, pp. 129-131.
[Flo] R. Floyd, Algorithm 97, shortest path, CACM 5 (1962), 345.
[Fur] M. E. Furman, Application of a method of rapid multiplication of matrices to the problem of
finding the transitive closure of a graph, Dokl. Akad. Nauk SSSR 194 (1970), 524 = Soviet Math.
Dokl. 11 (1970), 1252. MR 42 #5833.
PRODUCT AND CLOSURE ALGORITHMS FOR MATRICES
489
[Mun] I. Munro, Efficient determination of the transitive closure of a directed graph, Information
Processing Letters 1 (1971), 56-58,
[Str] V. Strassen, Gaussian elimination is not optimal, Numer. Math. 13 (1969), 354-356. MR 40
# 2223.
[Val] L. Valiant, General context-free recognition in less than cubic time, J. Comput. and
System Sci. 10(1975), 308-315,
[You] D, Younger, Recognition and parsing of context-free languages in time n3, Information and
Control 10 (1967), 189-208,
UNIVERSITY OF WARWICK
COVENTRY, ENGLAND, CV4
7AL