Tensor Factorization via Matrix Factorization
Volodymyr Kuleshov∗
Arun Tejasvi Chaganty∗
Percy Liang
Kuleshov, Chaganty, Liang (Stanford University)
Stanford University
May 8, 2015
Tensor Factorization
May 8, 2015
1 / 27
Tensor factorization
What is tensor (CP) factorization?
I
(Kolda and Bader 2009)
Tensor analogue of matrix eigen-decomposition.
M=
k
X
πi ui ⊗ ui
.
i=1
=
Kuleshov, Chaganty, Liang (Stanford University)
+···+
+
k
Tensor Factorization
May 8, 2015
2 / 27
Tensor factorization
What is tensor (CP) factorization?
I
(Kolda and Bader 2009)
Tensor analogue of matrix eigen-decomposition.
T =
k
X
πi ui ⊗ ui ⊗ui
.
i=1
=
Kuleshov, Chaganty, Liang (Stanford University)
+···+
+
k
Tensor Factorization
May 8, 2015
2 / 27
Tensor factorization
What is tensor (CP) factorization?
I
(Kolda and Bader 2009)
Tensor analogue of matrix eigen-decomposition.
b =
T
k
X
πi ui ⊗ ui ⊗ui +R.
i=1
I
Goal: Given T with noise, R, recover factors ui .
=
+···+
+
+
k
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
2 / 27
Tensor factorization
What is tensor (CP) factorization?
(Kolda and Bader 2009)
Tensor analogue of matrix eigen-decomposition.
I
b =
T
k
X
πi ui ⊗ ui ⊗ui +R.
i=1
Goal: Given T with noise, R, recover factors ui .
=
+
+···+
+
=
+
+···+
+
No
n-o
rth
og
on
al
Or
tho
go
nal
I
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
2 / 27
Tensor factorization
Why tensor factorization?
I
To solve multi-linear algebra problems.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
3 / 27
Tensor factorization
Why tensor factorization?
I
I
To solve multi-linear algebra problems.
Parsing
I
Cohen, Satta, and Collins 2013
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
3 / 27
Tensor factorization
Why tensor factorization?
I
I
To solve multi-linear algebra problems.
Parsing
I
I
Cohen, Satta, and Collins 2013
Knowledge base completion
I
I
Chang et al. 2014
Singh, Rocktäschel, and Riedel 2015
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
3 / 27
Tensor factorization
Why tensor factorization?
I
I
To solve multi-linear algebra problems.
Parsing
I
I
Knowledge base completion
I
I
I
Cohen, Satta, and Collins 2013
Chang et al. 2014
Singh, Rocktäschel, and Riedel 2015
Topic modelling
I
Anandkumar et al. 2012
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
3 / 27
Tensor factorization
Why tensor factorization?
I
I
To solve multi-linear algebra problems.
Parsing
I
I
Knowledge base completion
I
I
I
Chang et al. 2014
Singh, Rocktäschel, and Riedel 2015
Topic modelling
I
I
Cohen, Satta, and Collins 2013
Anandkumar et al. 2012
Community detection
I
Anandkumar et al. 2013a
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
3 / 27
Tensor factorization
Why tensor factorization?
I
I
To solve multi-linear algebra problems.
Parsing
I
I
Knowledge base completion
I
I
I
Anandkumar et al. 2012
Community detection
I
I
Chang et al. 2014
Singh, Rocktäschel, and Riedel 2015
Topic modelling
I
I
Cohen, Satta, and Collins 2013
Anandkumar et al. 2013a
Learning latent variable graphical models
I
Anandkumar et al. 2013b
I TODO: crowdsourcing
I TODO: others
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
3 / 27
Tensor factorization
Existing tensor factorization algorithms
I
Tensor power method (Anandkumar et al. 2013b)
I
I
I
Analog of matrix power method.
Sensitive to noise.
Restricted to orthogonal tensors.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
4 / 27
Tensor factorization
Existing tensor factorization algorithms
I
Tensor power method (Anandkumar et al. 2013b)
I
I
I
I
Analog of matrix power method.
Sensitive to noise.
Restricted to orthogonal tensors.
Alternating least squares (Comon, Luciani, and Almeida 2009;
Anandkumar, Ge, and Janzamin 2014)
I
Sensitive to initialization.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
4 / 27
Tensor factorization
Existing tensor factorization algorithms
I
Tensor power method (Anandkumar et al. 2013b)
I
I
I
I
Alternating least squares (Comon, Luciani, and Almeida 2009;
Anandkumar, Ge, and Janzamin 2014)
I
I
Analog of matrix power method.
Sensitive to noise.
Restricted to orthogonal tensors.
Sensitive to initialization.
Both operate on the tensor directly.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
4 / 27
Tensor factorization
Our approach
I
Objective: a fast robust algorithm.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
5 / 27
Tensor factorization
Our approach
I
Objective: a fast robust algorithm.
I
Approach: use existing fast and robust matrix algorithms.
Kuleshov, Chaganty, Liang (Stanford University)
$
Tensor Factorization
May 8, 2015
5 / 27
Tensor factorization via matrix factorization
Single matrix factorizations
Outline
Tensor factorization
Tensor factorization via matrix factorization
Single matrix factorizations
Simultaneous matrix factorizations
Oracle projections
Random projections
Non-orthogonal tensor factorization
Empirical results
Conclusions
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
6 / 27
Tensor factorization via matrix factorization
Single matrix factorizations
Tensor factorization via single matrix factorization
T
=
π1 u1⊗3
Kuleshov, Chaganty, Liang (Stanford University)
+
π2 u2⊗3
Tensor Factorization
+
π3 u3⊗3
+
R
May 8, 2015
7 / 27
Tensor factorization via matrix factorization
Single matrix factorizations
Tensor factorization via single matrix factorization
T
=
u1⊗3
Kuleshov, Chaganty, Liang (Stanford University)
+
Tensor Factorization
u2⊗3
+
u3⊗3
May 8, 2015
8 / 27
Tensor factorization via matrix factorization
Single matrix factorizations
Tensor factorization via single matrix factorization
T
↓
T (I, I, w )
=
u1⊗3
+
u2⊗3
+
u3⊗3
=
(w > u1 )u1⊗2
+
(w > u2 )u2⊗2
+
(w > u3 )u3⊗3
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
8 / 27
Tensor factorization via matrix factorization
Single matrix factorizations
Tensor factorization via single matrix factorization
T
↓
T (I, I, w )
u1⊗3
+
(w > u1 ) u1 u1>
+
=
=
| {z }
λ1
I
u2⊗3
+
u3⊗3
(w > u2 ) u2 u2>
+
(w > u3 ) u3 u3>
| {z }
λ2
| {z }
λ3
Proposal: Eigen-decomposition on the projected matrix.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
8 / 27
Tensor factorization via matrix factorization
Single matrix factorizations
Sensitivity of single matrix projection
=
Kuleshov, Chaganty, Liang (Stanford University)
+
Tensor Factorization
+
May 8, 2015
9 / 27
Tensor factorization via matrix factorization
Single matrix factorizations
Sensitivity of single matrix projection
=
I
+
+
If two eigenvalues are equal, corresponding eigenvectors are arbitrary.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
9 / 27
Tensor factorization via matrix factorization
Single matrix factorizations
Sensitivity of single matrix projection
=
I
+
+
If two eigenvalues are equal, corresponding eigenvectors are arbitrary.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
9 / 27
Tensor factorization via matrix factorization
Single matrix factorizations
Sensitivity of single matrix projection
=
+
+
I
If two eigenvalues are equal, corresponding eigenvectors are arbitrary.
I
Problem: Eigendecomposition is very sensitive to the eigengap.
error in factors ∝
Kuleshov, Chaganty, Liang (Stanford University)
1
.
min(difference in eigenvalues)
Tensor Factorization
May 8, 2015
9 / 27
Tensor factorization via matrix factorization
Single matrix factorizations
Projections matter
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
10 / 27
Tensor factorization via matrix factorization
Single matrix factorizations
Projections matter
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
10 / 27
Tensor factorization via matrix factorization
Single matrix factorizations
Projections matter
I
How can we leverage multiple projections?
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
10 / 27
Tensor factorization via matrix factorization
Simultaneous matrix factorizations
Outline
Tensor factorization
Tensor factorization via matrix factorization
Single matrix factorizations
Simultaneous matrix factorizations
Oracle projections
Random projections
Non-orthogonal tensor factorization
Empirical results
Conclusions
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
11 / 27
Tensor factorization via matrix factorization
Simultaneous matrix factorizations
Enter simultaneous diagonalization
T (I, I, w1 )
|
{z
M1
}
=
>
>
(w u1 ) u1 u1
| 1{z }
λ11
Kuleshov, Chaganty, Liang (Stanford University)
+
(w1> u2 ) u2 u2>
| {z }
λ21
Tensor Factorization
+
(w1> u3 ) u3 u3>
| {z }
λ31
May 8, 2015
12 / 27
Tensor factorization via matrix factorization
Simultaneous matrix factorizations
Enter simultaneous diagonalization
T (I, I, w1 )
|
{z
M1
=
}
λ11
..
.
T (I, I, wl )
|
{z
Ml
}
>
=
>
(w u1 ) u1 u1
| 1{z }
+
>
>
λ11
Kuleshov, Chaganty, Liang (Stanford University)
+
| {z }
λ21
..
.
(w u1 ) u1 u1
| l{z }
(w1> u2 ) u2 u2>
+
| {z }
λ21
Tensor Factorization
| {z }
λ31
..
.
(wl> u2 ) u2 u2>
(w1> u3 ) u3 u3>
+
..
.
(wl> u3 ) u3 u3>
| {z }
λ31
May 8, 2015
12 / 27
Tensor factorization via matrix factorization
Simultaneous matrix factorizations
Enter simultaneous diagonalization
T (I, I, w1 )
|
{z
M1
=
}
λ11
..
.
T (I, I, wl )
|
I
{z
Ml
}
>
=
>
(w u1 ) u1 u1
| 1{z }
+
>
>
λ11
+
| {z }
λ21
..
.
(w u1 ) u1 u1
| l{z }
(w1> u2 ) u2 u2>
+
| {z }
λ21
| {z }
λ31
..
.
(wl> u2 ) u2 u2>
(w1> u3 ) u3 u3>
+
..
.
(wl> u3 ) u3 u3>
| {z }
λ31
Projections share factors.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
12 / 27
Tensor factorization via matrix factorization
Simultaneous matrix factorizations
Algorithm
I
Algorithm: Simultaneously diagonalize projected matrices.
b = arg max
U
L
X
off(U > Ml U)
b l=1
U
Kuleshov, Chaganty, Liang (Stanford University)
off(A) =
X
A2ij .
i6=j
Tensor Factorization
May 8, 2015
13 / 27
Tensor factorization via matrix factorization
Simultaneous matrix factorizations
Algorithm
I
Algorithm: Simultaneously diagonalize projected matrices.
b = arg max
U
L
X
off(U > Ml U)
b l=1
U
I
off(A) =
X
A2ij .
i6=j
Optimize using the Jacobi angles (Cardoso and Souloumiac 1996).
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
13 / 27
Tensor factorization via matrix factorization
Simultaneous matrix factorizations
Algorithm
I
Algorithm: Simultaneously diagonalize projected matrices.
b = arg max
U
L
X
off(U > Ml U)
b l=1
U
off(A) =
X
A2ij .
i6=j
I
Optimize using the Jacobi angles (Cardoso and Souloumiac 1996).
I
Multiple projections proposed in Anandkumar, Hsu, and Kakade 2012,
but didn’t use simultaneous diagonalization.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
13 / 27
Tensor factorization via matrix factorization
Simultaneous matrix factorizations
Comparison with single matrix factorization
I
Single matrix factorization depends on minimum eigengap.
error in factors ∝
Kuleshov, Chaganty, Liang (Stanford University)
1
.
mini,j difference in eigenvalues
Tensor Factorization
May 8, 2015
14 / 27
Tensor factorization via matrix factorization
Simultaneous matrix factorizations
Comparison with single matrix factorization
I
Single matrix factorization depends on minimum eigengap.
error in factors ∝
I
1
.
mini,j difference in eigenvalues
Simultaneous matrix factorization depends on average eigengap.
error in factors ∝
Kuleshov, Chaganty, Liang (Stanford University)
1
.
mini,j average difference in eigenvalues
Tensor Factorization
May 8, 2015
14 / 27
Tensor factorization via matrix factorization
Simultaneous matrix factorizations
Comparison with single matrix factorization
I
Single matrix factorization depends on minimum eigengap.
error in factors ∝
I
1
|λi − λj |
mini,j
.
Simultaneous matrix factorization depends on average eigengap.
error in factors ∝
Kuleshov, Chaganty, Liang (Stanford University)
1
mini,j
Tensor Factorization
PL
l=1 |λil
− λjl |
.
May 8, 2015
14 / 27
Tensor factorization via matrix factorization
Oracle projections
Outline
Tensor factorization
Tensor factorization via matrix factorization
Single matrix factorizations
Simultaneous matrix factorizations
Oracle projections
Random projections
Non-orthogonal tensor factorization
Empirical results
Conclusions
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
15 / 27
Tensor factorization via matrix factorization
Oracle projections
Oracle projections
Theorem
Pick k projections along the factors
(ui ). Then,
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
16 / 27
Tensor factorization via matrix factorization
Oracle projections
Oracle projections
Theorem
Pick k projections along the factors
(ui ). Then,
√
error in factors ≤ O
πmax
2
πmin
Kuleshov, Chaganty, Liang (Stanford University)
!
.
Tensor Factorization
May 8, 2015
16 / 27
Tensor factorization via matrix factorization
Oracle projections
Oracle projections
Theorem
Pick k projections along the factors
(ui ). Then,
√
error in factors ≤ O
πmax
2
πmin
Kuleshov, Chaganty, Liang (Stanford University)
!
.
Tensor Factorization
May 8, 2015
16 / 27
Tensor factorization via matrix factorization
Random projections
Outline
Tensor factorization
Tensor factorization via matrix factorization
Single matrix factorizations
Simultaneous matrix factorizations
Oracle projections
Random projections
Non-orthogonal tensor factorization
Empirical results
Conclusions
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
17 / 27
Tensor factorization via matrix factorization
Random projections
Random projections
Theorem
Pick O(k log k) projections
randomly from the unit sphere.
Then, with probability > 1 − δ,
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
18 / 27
Tensor factorization via matrix factorization
Random projections
Random projections
Theorem
Pick O(k log k) projections
randomly from the unit sphere.
Then, with probability > 1 − δ,
√
error in factors ≤ O
πmax
2
πmin
!
+ C (δ)
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
18 / 27
Tensor factorization via matrix factorization
Random projections
Random projections
Theorem
Pick O(k log k) projections
randomly from the unit sphere.
Then, with probability > 1 − δ,
√
error in factors ≤ O
πmax
2
πmin
!
+ C (δ)
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
18 / 27
Tensor factorization via matrix factorization
Random projections
Random projections
Theorem
Pick O(k log k) projections
randomly from the unit sphere.
Then, with probability > 1 − δ,
√
error in factors ≤ O
πmax
2
πmin
!
+ C (δ)
I
As good as having oracle
projections!
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
18 / 27
Tensor factorization via matrix factorization
Random projections
Final algorithm
I
Algorithm:
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
19 / 27
Tensor factorization via matrix factorization
Random projections
Final algorithm
I
Algorithm:
I
Project tensor on to O(k log k) random vectors.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
19 / 27
Tensor factorization via matrix factorization
Random projections
Final algorithm
I
Algorithm:
I
I
Project tensor on to O(k log k) random vectors.
(0)
Recover approximate factors ũi through simultaneous diagonalization.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
19 / 27
Tensor factorization via matrix factorization
Random projections
Final algorithm
I
Algorithm:
I
I
I
Project tensor on to O(k log k) random vectors.
(0)
Recover approximate factors ũi through simultaneous diagonalization.
Project tensor on to approximated factors.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
19 / 27
Tensor factorization via matrix factorization
Random projections
Final algorithm
I
Algorithm:
I
I
I
I
Project tensor on to O(k log k) random vectors.
(0)
Recover approximate factors ũi through simultaneous diagonalization.
Project tensor on to approximated factors.
Return factors ũi from simultaneous diagonalization.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
19 / 27
Non-orthogonal tensor factorization
Outline
Tensor factorization
Tensor factorization via matrix factorization
Single matrix factorizations
Simultaneous matrix factorizations
Oracle projections
Random projections
Non-orthogonal tensor factorization
Empirical results
Conclusions
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
20 / 27
Non-orthogonal tensor factorization
Naive approach: whitening non-orthogonal factors
=
Kuleshov, Chaganty, Liang (Stanford University)
+
Tensor Factorization
+···+
May 8, 2015
21 / 27
Non-orthogonal tensor factorization
Naive approach: whitening non-orthogonal factors
I
=
+
+···+
=
+
+···+
Use a whitening transformation to orthogonalize tensor (Anandkumar
et al. 2013b).
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
21 / 27
Non-orthogonal tensor factorization
Naive approach: whitening non-orthogonal factors
I
=
+
+···+
=
+
+···+
Use a whitening transformation to orthogonalize tensor (Anandkumar
et al. 2013b).
I
Is a major source of errors itself (Souloumiac 2009).
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
21 / 27
Non-orthogonal tensor factorization
Non-orthogonal simultaneous diagonalization
T (I, I, w1 )
|
I
{z
M1
}
=
>
>
(w u1 ) u1 u1
| 1{z }
λ11
+
(w1> u2 ) u2 u2>
| {z }
λ21
+
(w1> u3 ) u3 u3>
| {z }
λ31
No unique non-orthogonal factorization for a single matrix.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
22 / 27
Non-orthogonal tensor factorization
Non-orthogonal simultaneous diagonalization
T (I, I, w1 )
|
{z
M1
=
}
λ11
..
.
T (I, I, wl )
|
{z
Ml
}
>
=
>
(w u1 ) u1 u1
| 1{z }
+
>
>
λ1l
+
| {z }
λ21
..
.
(w u1 ) u1 u1
| l{z }
(w1> u2 ) u2 u2>
+
| {z }
λ2l
| {z }
λ31
..
.
(wl> u2 ) u2 u2>
(w1> u3 ) u3 u3>
+
..
.
(wl> u3 ) u3 u3>
| {z }
λ3l
I
No unique non-orthogonal factorization for a single matrix.
I
≥ 2 matrices have a unique non-orthogonal factorization.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
22 / 27
Non-orthogonal tensor factorization
Non-orthogonal simultaneous diagonalization
T (I, I, w1 )
|
{z
M1
=
}
λ11
..
.
T (I, I, wl )
|
{z
Ml
}
>
=
>
(w u1 ) u1 u1
| 1{z }
+
>
>
λ1l
+
| {z }
λ21
..
.
(w u1 ) u1 u1
| l{z }
(w1> u2 ) u2 u2>
+
| {z }
λ31
..
.
(wl> u2 ) u2 u2>
| {z }
λ2l
(w1> u3 ) u3 u3>
+
..
.
(wl> u3 ) u3 u3>
| {z }
λ3l
I
No unique non-orthogonal factorization for a single matrix.
I
≥ 2 matrices have a unique non-orthogonal factorization.
I
Note: λil are factor weights, not eigenvalues.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
22 / 27
Non-orthogonal tensor factorization
Non-orthogonal simultaneous diagonalization
I
Algorithm: Simultaneously diagonalize projected matrices.
b = arg max
U
L
X
off(U −1 Ml U −> )
b l=1
U
Kuleshov, Chaganty, Liang (Stanford University)
off(A) =
X
A2ij .
i6=j
Tensor Factorization
May 8, 2015
23 / 27
Non-orthogonal tensor factorization
Non-orthogonal simultaneous diagonalization
I
Algorithm: Simultaneously diagonalize projected matrices.
b = arg max
U
L
X
off(U −1 Ml U −> )
b l=1
U
I
off(A) =
X
A2ij .
i6=j
U are not constrained to be orthogonal.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
23 / 27
Non-orthogonal tensor factorization
Non-orthogonal simultaneous diagonalization
I
Algorithm: Simultaneously diagonalize projected matrices.
b = arg max
U
L
X
off(U −1 Ml U −> )
b l=1
U
I
I
off(A) =
X
A2ij .
i6=j
U are not constrained to be orthogonal.
Optimize using the QR1JD algorithm (Souloumiac 2009).
I
Only guaranteed to have local convergence.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
23 / 27
Non-orthogonal tensor factorization
Results: Non-orthogonal simultaneous diagonalization
Theorem (Oracle projections)
Pick k projections along the factors (ui ). Then,
√
error in factors ≤ O
kU −> k32
πmax
2
πmin
!
,
where U = [u1 | · · · |uk ].
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
24 / 27
Non-orthogonal tensor factorization
Results: Non-orthogonal simultaneous diagonalization
Theorem (Oracle projections)
Pick k projections along the factors (ui ). Then,
√
error in factors ≤ O
kU −> k32
πmax
2
πmin
!
,
where U = [u1 | · · · |uk ].
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
24 / 27
Empirical results
Outline
Tensor factorization
Tensor factorization via matrix factorization
Single matrix factorizations
Simultaneous matrix factorizations
Oracle projections
Random projections
Non-orthogonal tensor factorization
Empirical results
Conclusions
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
25 / 27
Conclusions
Outline
Tensor factorization
Tensor factorization via matrix factorization
Single matrix factorizations
Simultaneous matrix factorizations
Oracle projections
Random projections
Non-orthogonal tensor factorization
Empirical results
Conclusions
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
26 / 27
Conclusions
Conclusions
$
I
Reduce tensor problems to matrix ones with Õ(k) random projections.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
27 / 27
Conclusions
Conclusions
$
I
Reduce tensor problems to matrix ones with Õ(k) random projections.
I
Robust to noise with general support for non-orthogonal factors.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
27 / 27
Conclusions
Conclusions
$
I
Reduce tensor problems to matrix ones with Õ(k) random projections.
I
Robust to noise with general support for non-orthogonal factors.
I
Competitive empirical performance.
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
27 / 27
Conclusions
Conclusions
$
I
Reduce tensor problems to matrix ones with Õ(k) random projections.
I
Robust to noise with general support for non-orthogonal factors.
I
Competitive empirical performance.
I
Questions?
Kuleshov, Chaganty, Liang (Stanford University)
Tensor Factorization
May 8, 2015
27 / 27