1.
Algebraic matrix multiplication
a.
b.
All-Pairs shortest paths via
fast matrix multiplication
2.
Uri Zwick
Tel Aviv University
3.
Min-Plus matrix multiplication
a.
b.
c.
4.
An O(n2.38) anlgorithm for unweighted graphs (Seidel)
An O(Mn2.38) algorithm for weighted graphs (Shoshan-Zwick)
APSP in directed graphs
1.
2.
3.
5.
Equivalence to the APSP problem
Expensive reduction to algebraic products
Fredman’s trick
APSP in undirected graphs
a.
b.
Summer School on Shortest Paths (PATH05)
DIKU, University of Copenhagen
Strassen’s algorithm
Rectangular matrix multiplication
An O(M0.68n2.58) algorithm (Zwick)
An O(Mn2.38) preprocessing / O(n) query answering alg. (Yuster-Z)
An O(n2.38logM) (1+ )-approximation algorithm
Summary and open problems
Matrix multiplication
1. Algebraic matrix multiplication
a.
b.
2.
i
A ( ai j )
B
An O(n2.38) anlgorithm for unweighted graphs (Seidel)
An O(Mn2.38) algorithm for weighted graphs (Shoshan-Zwick)
C
(bi j )
(ci j )
n
cij
APSP in directed graphs
1.
2.
3.
5.
Equivalence to the APSP problem
Expensive reduction to algebraic products
Fredman’s trick
APSP in undirected graphs
a.
b.
4.
j
Min-Plus matrix multiplication
a.
b.
c.
3.
Strassen’s algorithm
Rectangular matrix multiplication
An O(M0.68n2.58) algorithm (Zwick)
An O(Mn2.38) preprocessing / O(n) query answering alg. (Yuster-Z)
An O(n2.38logM) (1+ )-approximation algorithm
Summary and open problems
Matrix multiplication
Complexity
Authors
n3
n2.81
n2.38
(by definition)
aik bkj
k 1
Can be computed naively in O(n3) time.
Multiplying 2 2 matrices
C12
A11
A12
B11
B12
C21 C22
C11
A21
A22
B21
B22
C11
A11 B11
A12 B21
Strassen (1969)
C12
A11 B12
A12 B22
Coppersmith, Winograd (1990)
C21
A21 B11
A22 B21
C22
A21 B12
A22 B22
Conjecture/Open problem:
n2+o(1) ???
8 multiplications
4 additions
Strassen’s 2 2 algorithm
C11
A11 B11
A12 B21
M1
( A11
A22 )( B11 B22 )
C12
A11 B12
A12 B22
M2
( A21
A22 ) B11
C21
A21 B11
A22 B21
M3
A11 ( B12
C22
A21 B12
A22 B22
M4
A22 ( B21 B11 )
M5
( A11
A12 ) B22
M6
( A21
A11 )( B11 B12 )
M7
( A12
A22 )( B21 B22 )
C11
M1 M 4
C12
M3 M5
C21
M2
C22
M1 M 2
M5
M7
View each n n matrix as a 2 2 matrix
whose elements are n/2 n/2 matrices.
B22 )
Apply the 2 2 algorithm recursively.
T(n) = 7 T(n/2) + O(n2)
M4
M3
M6
T(n) = O(nlog7/log2)=O(n2.81)
7 multiplications
18 additions/subtractions
Rectangular Matrix multiplication
p
1.
A
p
n
a.
b.
c.
C
p
3.
aik bkj
Coppersmith (1997):
Complexity
n1.85p0.54+n2+o(1)
n0.29, complexity = n2+o(1) !!!
4.
B
20
17
C A B
30
2
10
23
5
cij
An O(M0.68n2.58) algorithm (Zwick)
An O(Mn2.38) preprocessing / O(n) query answering alg. (Yuster-Z)
An O(n2.38logM) (1+ )-approximation algorithm
Summary and open problems
An interesting special case
of the APSP problem
A
An O(n2.38) anlgorithm for unweighted graphs (Seidel)
An O(Mn2.38) algorithm for weighted graphs (Shoshan-Zwick)
APSP in directed graphs
1.
2.
3.
5.
Equivalence to the APSP problem
Expensive reduction to algebraic products
Fredman’s trick
APSP in undirected graphs
a.
b.
k 1
For p
Strassen’s algorithm
Rectangular matrix multiplication
2. Min-Plus matrix multiplication
B
cij
Algebraic matrix multiplication
a.
b.
n
n
n
Strassen’s n n algorithm
Min-Plus Products
6
3
10
2
1
5
7
2
5
min{aik bkj }
k
20
Min-Plus product
cij
1
8
3
5
2
7
5
8
3
5
C A B
min {aik bkj }
k
4
0
2
7
1
B
Solving APSP by repeated squaring
If W is an n by n matrix containing the edge weights
of a graph. Then Wn is the distance matrix.
A
B
C
D
A
X =
D
C
By induction, Wk gives the distances realized
by paths that use at most k edges.
E
D
W
for i 1 to log2n
do D D*D
F
X* =
G
APSP(n) MPP(n) log n
Actually: APSP(n) = O(MPP(n))
(A BD*C)*
EBD*
D*CE
D* GBD*
=
H
Thus:
Algebraic
Product
C
Min-Plus
Product
C A B
min{ aik bkj }
A B
cij
APSP(n)
cij
aik bkj
1.
?
O(n2.38)
c.
4.
cij
min{aik
k
x
x
c '11 c '12
c '21 c '22
a12
a22
a11
a21
x
x
aik bkj
x
c 'ij
k
An O(M0.68n2.58) algorithm (Zwick)
An O(Mn2.38) preprocessing / O(n) query answering alg. (Yuster-Z)
An O(n2.38logM) (1+ )-approximation algorithm
Summary and open problems
Using matrix multiplication
to compute min-plus products
a11
a21
An O(n2.38) anlgorithm for unweighted graphs (Seidel)
An O(Mn2.38) algorithm for weighted graphs (Shoshan-Zwick)
APSP in directed graphs
1.
2.
3.
5.
Fredman’s trick
APSP in undirected graphs
a.
b.
min operation
has no inverse
c11 c12
c21 c22
Equivalence to the APSP problem
b. Expensive reduction to algebraic products
3.
The fast algebraic
algorithms cannot
be used, as the
Strassen’s algorithm
Rectangular matrix multiplication
Min-Plus matrix multiplication
a.
k
k
Algebraic matrix multiplication
a.
b.
2.
Using matrix multiplication
to compute min-plus products
Assume: 0
b11 b12
b21 b22
c '11
c '21
bkj }
a12
x
x
a22
cij
b11
b21
x
x
2 APSP(n/2) + 6 MPP(n/2) + O(n2)
c '12
c '22
x
x
a11
a21
aij , bij M
x
x
a12
a22
x
x
b11
b21
x
x
b12
b22
b12
b22
first (c 'ij )
n2.38
polynomial
products
M
Mn2.38
operations per
polynomial
product
operations per
max-plus
product
Trying to implement the
repeated squaring algorithm
1.
2.
D
W
for i 1 to log2n
do D D*D
Algebraic matrix multiplication
a.
b.
Min-Plus matrix multiplication
a.
b.
Consider an easy case:
all weights are 1.
3.
After the i-th iteration, the finite elements in
D are in the range {1,…,2i}.
Strassen’s algorithm
Rectangular matrix multiplication
Equivalence to the APSP problem
Expensive reduction to algebraic products
c. Fredman’s trick
APSP in undirected graphs
An O(n2.38) anlgorithm for unweighted graphs (Seidel)
An O(Mn2.38) algorithm for weighted graphs (Shoshan-Zwick)
a.
b.
4.
APSP in directed graphs
An O(M0.68n2.58) algorithm (Zwick)
An O(Mn2.38) preprocessing / O(n) query answering alg. (Yuster-Z)
An O(n2.38logM) (1+ )-approximation algorithm
1.
2.
3.
The cost of the min-plus product is 2i n2.38
The cost of the last product is n3.38 !!!
5.
Summary and open problems
Breaking a square product into
several rectangular products
Fredman’s trick
m
The min-plus product of two n n
matrices can be deduced after only
O(n2.5) additions and comparisons.
B1
n
B2
A1 A2
A* B
MPP(n)
Fredman’s trick
min Ai * Bi
i
(n/m) (MPP(n,m,n) + n2)
Fredman’s trick (cont.)
m
air+brj
n
n
A
B
air+brj
ais+bsj
air - ais
bsj - brj
m
Naïve calculation requires n2m operations
Fredman observed that the result can be inferred
after performing only O(nm2) operations
ais+bsj
air - ais
• Generate all the differences air
bsj - brj
- ais and bsj - brj .
• Sort them using O(nm2) comparisons. (Non-trivial!)
• Merge the two sorted lists using O(nm2) comparisons.
The ordering of the elements in the sorted list
determines the result of the min-plus product
!!!
Decision Tree Complexity
All-Pairs Shortest Paths
in directed graphs with “real” edge weights
a17-a19 b92-b72
yes
Running time
n3
Authors
[Floyd ’62] [Warshall ’62]
n3 (log log n / log n)1/3
n3 (log log n / log n)1/2
[Fredman ’76]
[Takaoka ’92]
n3 / (log n)1/2
[Dobosiewicz ’90]
no
n 2.5
…
n3 (log
log n / log n)5/7
n3 log log n / log n
c11=a17+b71
c12=a14+b42
c11=a13+b31
c12=a15+b52
...
...
1.
…
[Han ’04]
[Takaoka ’04]
c11=a18+b81
c12=a16+b62
c11=a12+b21
c12=a13+b32
n3 (log log n)1/2 / log n
[Zwick ’04]
...
...
n3 / log n
[Chan ’05]
Algebraic matrix multiplication
a.
b.
2.
Strassen’s algorithm
Rectangular matrix multiplication
Directed versus undirected graphs
Min-Plus matrix multiplication
a.
b.
c.
Equivalence to the APSP problem
Expensive reduction to algebraic products
Fredman’s trick
y
x
z
3. APSP in undirected graphs
a.
b.
4.
An O(n2.38) anlgorithm for unweighted graphs (Seidel)
An O(Mn2.38) algorithm for weighted graphs (Shoshan-Zwick)
y
x
(x,z)
z
(x,y) + (y,z)
(x,z)
(x,z)
APSP in directed graphs
1.
2.
3.
5.
(x,y) + (y,z)
(x,y) – (y,z)
An O(M0.68n2.58) algorithm (Zwick)
An O(Mn2.38) preprocessing / O(n) query answering alg. (Yuster-Z)
An O(n2.38logM) (1+ )-approximation algorithm
Summary and open problems
Distances in G and its square G2
Distances in G and its square G2 (cont.)
Lemma: If (u,v)=2 2(u,v) then for every
2(u,v).
neighbor w of v we have 2(u,w)
Let G=(V,E). Then G2=(V,E2), where
(u,v) E2 if and only if (u,v) E or there
exists w V such that (u,w),(w,v) E
Lemma: If (u,v)=2 2(u,v)–1 then for every
2(u,v) and
neighbor w of v we have 2(u,w)
for at least one neighbor 2(u,w) < 2(u,v).
Let (u,v) be the distance from u to v in G.
Let 2(u,v) be the distance from u to v in G2.
Lemma:
2(u,v)=
Let A be the adjacency matrix of the G.
Let C be the distance matrix of G2
(u,v)/2 , for every u,v V.
Thus: (u,v)=2 2(u,v) or
(u,v)=2 2(u,v)-1
cu ,w
( v , w) E
cu,waw,v (CA)u ,v : deg(v) cu ,v
w
1.
Seidel’s algorithm
1. If A is an all one matrix,
then all distances are 1.
Algorithm APD(A)
if A=J then
return J-I
else
C APD(A2)
X CA , deg Ae–1
dij 2cij-[xij<cijdegj]
return D
end
2. Compute A2, the adjacency
matrix of the squared graph.
3. Find, recursively, the distances
in the square.
4. Decide, using matrix
multiplication, for every two
vertices u,v, whether their
distance is twice the distance in
the square, or twice minus 1.
Algebraic matrix multiplication
a.
b.
Complexity:
O(n2.38log n)
2.
Equivalence to the APSP problem
Expensive reduction to algebraic products
Fredman’s trick
APSP in undirected graphs
a.
b.
An O(n2.38) anlgorithm for unweighted graphs (Seidel)
An O(Mn2.38) algorithm for weighted graphs (Shoshan-Zwick)
4. APSP in directed graphs
1.
2.
3.
5.
Sampled Repeated Squaring (Z ’98)
Min-Plus matrix multiplication
a.
b.
c.
3.
Strassen’s algorithm
Rectangular matrix multiplicationbv
An O(M0.68n2.58) algorithm (Zwick)
An O(Mn2.38) preprocessing / O(n) query answering alg. (Yuster-Z)
An O(n2.38logM) (1+ )-approximation algorithm
Summary and open problems
Sampled Distance Products (Z ’98)
In the i-th
iteration, the set B
is of size n ln n / s,
where s = (3/2)i+1
D W
Choose a subset of V
for i 1 to log3/2n do
of size (9nlnn)/s
{
s (3/2)i+1
B rand( V , (9n ln n)/s )
D min{ D , D[V,B]*D[B,V] }
}
The matrices get
smaller and smaller
but the elements get
larger and larger
Select the columns
Select the rows
The is also
a slightly more complicated
of D With
whosehigh probability,
of D whose
deterministic
algorithm
distances
are
correct!
indicesall
are
in B
indices are in B
Sampled Repeated Squaring - Correctness
D W
for i 1 to log3/2n do
{
s (3/2)i+1
B rand(V,(9 ln n)/s)
D min{ D , D[V,B]*D[B,V] }
}
Invariant: After the i-th
iteration, distances that are
attained using at most (3/2)i
edges are correct.
Rectangular Matrix multiplication
p
n
n
n
n
p
Consider a shortest path that uses at most (3/2)i+1 edges
at most
1 3
2 2
at most
i
1 3
2 2
i
1 3
2 2
i
[Coppersmith ’97]:
Let s = (3/2)i+1
Failure
probability
: (1
9ln n
)
s
n2p
n1.85p0.54+n2+o(1)
Naïve complexity:
s/3
n
3
For p
n0.29, complexity = n2+o(1) !!!
Complexity of APSP algorithm
The i-th iteration:
1.
2.
n ln n / s
n ln n / s
n
The elements are
of absolute value
at most Ms
1.85
min{ Ms n
n
s
n3
, }
s
M
0.68
n
Equivalence to the APSP problem
Expensive reduction to algebraic products
Fredman’s trick
3. APSP in undirected graphs
a.
b.
4.
0.54
Strassen’s algorithm
Rectangular matrix multiplication
Min-Plus matrix multiplication
a.
b.
c.
s=(3/2)i+1
n
Algebraic matrix multiplication
a.
b.
APSP in directed graphs
1.
2.
3.
2.58
5.
An O(n2.38) anlgorithm for unweighted graphs (Seidel)
An O(Mn2.38) algorithm for weighted graphs (Shoshan-Zwick)
An O(M0.68n2.58) algorithm (Zwick)
An O(Mn2.38) preprocessing / O(n) query answering alg. (Yuster-Z)
An O(n2.38logM) (1+ )-approximation algorithm
Summary and open problems
The preprocessing algorithm (YZ
’05)
D W;B V
for i 1 to log3/2n do
{
s (3/2)i+1
B rand(B,(9n ln n)/s)
D[V,B]
min{D[V,B] , D[V,B]*D[B,B] }
D[B,V]
min{D[B,V] , D[B,B]*D[B,V] }
}
Twice Sampled Distance Products
The APSP algorithm
D W
for i 1 to log3/2n do
{
s (3/2)i+1
B rand(V,(9nln n)/s)
D
min{ D , D[V,B]*D[B,V] }
}
The query answering algorithm
(u,v)
D[{u},V]*D[V,{v}]
v
u
Query time: O(n)
The preprocessing algorithm: Correctness
Let Bi be the i-th sample.
B1 B2
B3
The query answering algorithm:
Correctness
…
Invariant: After the i-th iteration, if u Bi or v Bi
and there is a shortest path from u to v that uses at
most (3/2)i edges, then D(u,v)= (u,v).
Suppose that the shortest path from u to v
uses between (3/2)i and (3/2)i+1 edges
at most
1 3
2 2
Consider a shortest path that uses at most (3/2)i+1 edges
at most
1 3
2 2
1.
1 3
2 2
i
Strassen’s algorithm
Rectangular matrix multiplication
Equivalence to the APSP problem
Expensive reduction to algebraic products
Fredman’s trick
An O(n2.38) anlgorithm for unweighted graphs (Seidel)
An O(Mn2.38) algorithm for weighted graphs (Shoshan-Zwick)
APSP in directed graphs
1.
2.
An O(M0.68n2.58) algorithm (Zwick)
An O(Mn2.38) preprocessing / O(n) query answering alg. (Yuster-Z)
3. An O(n2.38logM) (1+ )-approximation algorithm
5.
1 3
2 2
i
1 3
2 2
i
u
v
Approximate min-plus products
Obvious idea: scaling
SCALE(A,M,R):
aij
APSP in undirected graphs
a.
b.
4.
i
Min-Plus matrix multiplication
a.
b.
c.
3.
1 3
2 2
Algebraic matrix multiplication
a.
b.
2.
at most
i
at most
i
!
#
APX-MPP(A,B,M,R) :
A’ SCALE(A,M,R)
B’ SCALE(B,M,R)
return MPP(A’,B’)
Raij / M
, if 0 aij
M
, otherwise
"
$
Complexity is Rn2.38,
instead of Mn2.38, but
small values can be
greatly distorted.
Summary and open problems
Addaptive Scaling
APX-MPP(A,B,M,R) :
C’
for r
A’
B’
C’
end
log2R to log2M do
SCALE(A,2r,R)
SCALE(B,2r,R)
min{C’,MPP(A’,B’)}
Rn2.38 logM
Complexity is
Stretch at most 1+4/R
1.
Algebraic matrix multiplication
a.
b.
2.
Min-Plus matrix multiplication
a.
b.
c.
3.
Equivalence to the APSP problem
Expensive reduction to algebraic products
Fredman’s trick
APSP in undirected graphs
a.
b.
4.
Strassen’s algorithm
Rectangular matrix multiplication
An O(n2.38) anlgorithm for unweighted graphs (Seidel)
An O(Mn2.38) algorithm for weighted graphs (Shoshan-Zwick)
APSP in directed graphs
1.
2.
3.
An O(M0.68n2.58) algorithm (Zwick)
An O(Mn2.38) preprocessing / O(n) query answering alg. (Yuster-Z)
An O(n2.38logM) (1+ )-approximation algorithm
5. Summary and open problems
All-Pairs Shortest Paths
All-Pairs Shortest Paths
in graphs with small integer weights
in graphs with small integer weights
Undirected graphs.
Edge weights in {0,1,…M}
Directed graphs.
Edge weights in {−M,…,0,…M}
Running time
Authors
Running time
Authors
Mn2.38
[Shoshan-Zwick ’99]
M0.68 n2.58
[Zwick ’98]
Improves results of
[Alon-Galil-Margalit ’91] [Seidel ’95]
Answering distance queries
Improves results of
[Alon-Galil-Margalit ’91] [Takaoka ’98]
Approximate All-Pairs Shortest Paths
in graphs with non-negative integer weights
Directed graphs. Edge weights in {−M,…,0,…M}
Preprocessing
time
Query
time
Authors
Directed graphs.
Edge weights in {0,1,…M}
Mn2.38
n
[Yuster-Zwick ’05]
(1+ )-approximate distances
any Mn1.38 distances
In particular,
can be computed in Mn2.38 time.
For dense enough graphs with small enough edge
weights, this improves on Goldberg’s SSSP algorithm.
Mn2.38 vs. mn0.5log M
Open problems
• An O(n2.38) algorithm for the directed
unweighted APSP problem?
• An O(n3- ) algorithm for the APSP
problem with edge weights in {1,2,…,n}?
• An O(n2.5- ) algorithm for the SSSP problem
with edge weights in {0,±1, ±2,…, ±n}?
Running time
(n2.38 log
M)/
Authors
[Zwick ’98]