Jenya Kirshtein
1
Group-theoretic approach
to Fast Matrix Multiplication
Jenya Kirshtein
Department of Mathematics
University of Denver
Group-theoretic approach to Fast Matrix Multiplication (DU November,7 2008)
Jenya Kirshtein
2
References
1. H. Cohn, C. Umans. Group-theoretic approach to Fast Matrix
Multiplication. Proceedings of the 44th Annual Symposium on
Foundations of Computer Science, 2003.
2. H. Cohn, R. Kleinberg, B. Szegedy, C. Umans. Group-theoretic
Algorithms for Matrix Multiplication. Proceedings of the 46th
Annual Symposium on Foundations of Computer Science, 2005.
3. James and Liebeck. Representations and Characters of Groups.
Cambridge University Press, 1993.
4. I. Martin Isaacs. Character Theory of Finite Groups. AMS
Bookstore, 2006.
Group-theoretic approach to Fast Matrix Multiplication (DU November,7 2008)
Jenya Kirshtein
3
Matrix multiplication
• Standard matrix multiplication takes 2n3 arithmetic operations
• Strassen suggested O(n2.81 ) recursive algorithm in 1969
Question: What is the smallest number w such that ∀ǫ > 0
matrix multiplication can be performed in at most O(nw+ǫ )?
Question: Is w = 2?
• The best known bound is w < 2.38 due to Coppersmith and
Winograd (1990)
Group-theoretic approach to Fast Matrix Multiplication (DU November,7 2008)
Jenya Kirshtein
4
Outline
• Embed matrices A, B into the elements A, B of the group
algebra C[G]
• Multiplication of A and B in the group algebra is carried out in
the Fourier domain after performing the Discrete Fourier
Transform (DFT) of A and B
• The product A B is found by performing the inverse DFT
• Entries of the matrix AB can be read off from the group
algebra product A B
We’ll consider square matrices and finite groups.
Group-theoretic approach to Fast Matrix Multiplication (DU November,7 2008)
Jenya Kirshtein
5
Group Algebra
Def: Let G be a finite group and F be a field. A group algebra
F [G] of G over F is the set of all linear combinations of elements of
G with coefficients in F , with the natural addition and scalar
multiplication, and multiplication defined by





X X
X
X


ag bh  f
ag g  
bh h  =
g∈G,ag ∈F
h∈G,bh ∈F
f ∈G
gh=f
Group-theoretic approach to Fast Matrix Multiplication (DU November,7 2008)
Jenya Kirshtein
6
Discrete Fourier Transform (DFT)
Embed polynomials as elements of the group algebra C[G]:
Let G =< z > be a cyclic group of order m ≥ 2n. Define
A=
n−1
X
ai z i
and
i=0
B=
n−1
X
bi z i
i=0
Discrete Fourier Transform is an invertible linear transformation
D : C[G] → C|G| , such that
!
n−1
n−1
n−1
X
X
X
i
i
D(A) =
ai x0 ,
ai x1 , ... ,
ai xin−1 ,
i=0
where xk = e
2πi
n k
i=0
i=0
is the nth root of unity. Then
A B = D−1 ( D(A)D(B) )
Fast Fourier Transform algorithm computes DFT in O(nlog(n))
arithmetic operations.
Group-theoretic approach to Fast Matrix Multiplication (DU November,7 2008)
Jenya Kirshtein
7
Matrices as elements of the group algebra
Let F be a field and S, T and U be subsets of G.
A = (as,t )s∈S,t∈T
and
B = (bt,u )t∈T,u∈U
are |S| × |T | and |T | × |U | matrices, indexed by elements of S, T
and T, U , respectively. Then embed A,B as elements A, B ∈ F [G]:
X
X
−1
A=
as,t s t and B =
bt,u t−1 u
s∈S,t∈T
t∈T,u∈U
Group-theoretic approach to Fast Matrix Multiplication (DU November,7 2008)
Jenya Kirshtein
8
Triple product property
Let S ⊆ G. Let Q(S) denote the right quotient set of S, i.e.
−1
Q(S) = s1 s2 : s1 s2 ∈ S
Def: A group G realizes hn1 , n3 , n3 i if ∃ subsets S1 , S2 , S3 ⊆ G
s.t. |Si | = ni and for qi ∈ Q(Si ),
if q1 q2 q3 = 1
(equivalently, if
then
q1 q2 = q3
q1 = q2 = q3 = 1
then
q1 = q2 = q3 = 1)
This condition on S1 , S2 , S3 is the triple product property.
Group-theoretic approach to Fast Matrix Multiplication (DU November,7 2008)
Jenya Kirshtein
9
Theorem [1] Let F be a field. If G realizes < n, n, n >, then the
number of field operations required to multiply two n×n matrices
over F is at most the number of operations required to multiply two
elements of F [G].
Proof. Index A with the sets S, T ; B with T, U and AB with S, U .
Consider the product in F [G]



X
X
′
−1

ast s t 
bt′ u t −1 u
s∈S,t∈T
t′ ∈T,u∈U
By triple
property,
′ product
′
′
s−1 t t −1 u = s −1 u iff s = s′ , t = t′ , u = u′ ,
so the coefficient of s−1 u in the product is
X
ast btu = (AB)su
t∈T
Group-theoretic approach to Fast Matrix Multiplication (DU November,7 2008)
Jenya Kirshtein
10
Characters and character degrees of G
Def: Let F be R or C. The set of all invertible n × n matrices with
entries in F , under matrix multiplication, forms a group called the
general linear group GL(n, F ).
Def: The trace of an n × n matrix A = (aij ) is trA =
Pn
i=1
aii .
Def: A representation of G over F is a homomorphism
ρ : G → GL(n, F ) for some integer n (n is the degree of ρ).
Def: Suppose ρ is a representation of G. With each n × n matrix
ρ(g) (g ∈ G) we associate the complex number tr(ρ(g)), which we
call χ(g). The function χ : G → C is called the character of the
representation ρ of G.
Def: If χ is a character of G, then χ(1) is called a degree of χ.
Group-theoretic approach to Fast Matrix Multiplication (DU November,7 2008)
Jenya Kirshtein
11
Theorem [2] Suppose G realizes hn, n, ni and the character degrees
of G are {di }. Then
X
w
n ≤
dw
i
i
Theorem [3] Let G be a group with the character degrees {di }.
P 2
Then |G| = i di
Corollary [2] Suppose G realizes hn, n, ni and has largest
character degree d. Then
nw ≤ dw−2 |G|
Question [1] Does there exist a finite group G that realizes
hn, n, ni and has character degrees {di } such that
X
3
n >
d3i ,
i
i.e. G gives a nontrivial bound on w.
Group-theoretic approach to Fast Matrix Multiplication (DU November,7 2008)
Jenya Kirshtein
12
Example
Def: If G is a group and X is a set, then a (left) group action of
G on X is a binary function G × X → X denoted (g, x) → g · x,
which satisfies the following two axioms:
1. (gh) · x = g · (h · x) ∀g, h ∈ G, ∀x ∈ X;
2. 1 · x = x ∀x ∈ X (1 is the identity element of G).
Def: If G and H are groups with a left action of G on H, then the
semidirect product H ⋊ G is the set H × G with the
multiplication law
(h1 , g1 ) (h2 , g2 ) = (h1 (g1 · h2 ) , g1 g2 ) .
Group-theoretic approach to Fast Matrix Multiplication (DU November,7 2008)
Jenya Kirshtein
13
Example
Denote Cycn the cyclic group of order n.
Let H = Cyc3n and G = H 2 ⋊Cyc2
Cyc2 = {1, z} acts on H 2 by switching the two factors, then
i
G = (a, b) z : a, b ∈ H, i ∈ {0, 1}
Let H1 , H2 , H3 ≤ H be the three factors of Cycn in H, H4 = H1 .
Define subsets S1 , S2 , S3 ⊆ G by
j
Si = (a, b) z : a ∈ Hi \ {0} , b ∈ Hi+1 , j ∈ {0, 1}
S1 , S2 , S3 satisfy the triple product property.
Group-theoretic approach to Fast Matrix Multiplication (DU November,7 2008)
Jenya Kirshtein
14
Example
H = Cyc3n and G = H 2 ⋊Cyc2
j
Si = (a, b) z : a ∈ Hi \ {0} , b ∈ Hi+1 , j ∈ {0, 1}
Theorem [4] Let G be nonabelian group and m be an integer. If
there exists abelian H ⊳ G of index m, then character degrees of G
are 1 and m.
Theorem If H ≤ G and [G : H] = 2, then H ⊳ G.
|G|
= 2,
Since H 2 is an abelian subgroup of G of index [G : H] = |H|
character degrees of G are at most 2.
P
P 3
P 2
6
Then since i d2i = |G|,
≤
max
{d
}
d
i
i i
i di ≤ 2|G| = 4n
Also, |Si | = 2n(n − 1) and |S1 ||S2 ||S3 | = 8n3 (n − 1)3 ,
P 3
so we have |S1 ||S2 ||S3 | > i di for n ≥ 5.
Group-theoretic approach to Fast Matrix Multiplication (DU November,7 2008)
Jenya Kirshtein
15
Example
By Corollary, |S1 |w ≤ dw−2 |G| or
2n(n − 1)w ≤ 2w−2 2n6 ,
6lnn−ln2
.
which leads to w ≤ ln(n(n−1))
The best bound on w (w ≤ 2.9088) is achieved by setting n = 17.
Group-theoretic approach to Fast Matrix Multiplication (DU November,7 2008)
Jenya Kirshtein
16
Other results
• Uniquely Solvable Puzzles prove w < 2.48
• Local Uniquely Solvable Puzzles prove w < 2.41 and w < 2.376
• Conjecture that a Strong Uniquely Solvable Puzzle capacity
2
equals 3/2 3 would imply that w = 2
• Conjecture about the existence for any n of an abelian group G
with n pairs of subsets Ai , Bi satisfying the simultaneous
double product property such that |G| = n2+o(1) and
|Ai ||Bi | ≥ n2−o(1) would imply that w = 2
Group-theoretic approach to Fast Matrix Multiplication (DU November,7 2008)
Jenya Kirshtein
17
Thank you!
Group-theoretic approach to Fast Matrix Multiplication (DU November,7 2008)