Tensors: theory and applications
Shmuel Friedland
Univ. Illinois at Chicago
TTI-C, March 2, 2009
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
1 / 18
Overview
Ranks of 3-tensors
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
2 / 18
Overview
Ranks of 3-tensors
1
Basic facts.
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
2 / 18
Overview
Ranks of 3-tensors
1
Basic facts.
2
Complexity.
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
2 / 18
Overview
Ranks of 3-tensors
1
Basic facts.
2
Complexity.
3
Matrix multiplication
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
2 / 18
Overview
Ranks of 3-tensors
1
Basic facts.
2
Complexity.
3
Matrix multiplication
4
Results and conjectures
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
2 / 18
Overview
Ranks of 3-tensors
1
Basic facts.
2
Complexity.
3
Matrix multiplication
4
Results and conjectures
Approximations of tensors
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
2 / 18
Overview
Ranks of 3-tensors
1
Basic facts.
2
Complexity.
3
Matrix multiplication
4
Results and conjectures
Approximations of tensors
1
Rank one approximation.
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
2 / 18
Overview
Ranks of 3-tensors
1
Basic facts.
2
Complexity.
3
Matrix multiplication
4
Results and conjectures
Approximations of tensors
1
Rank one approximation.
2
Perron-Frobenius theorem
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
2 / 18
Overview
Ranks of 3-tensors
1
Basic facts.
2
Complexity.
3
Matrix multiplication
4
Results and conjectures
Approximations of tensors
1
Rank one approximation.
2
Perron-Frobenius theorem
3
Rank (R1 , R2 , R3 ) approximations
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
2 / 18
Overview
Ranks of 3-tensors
1
Basic facts.
2
Complexity.
3
Matrix multiplication
4
Results and conjectures
Approximations of tensors
1
Rank one approximation.
2
Perron-Frobenius theorem
3
Rank (R1 , R2 , R3 ) approximations
4
CUR approximations
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
2 / 18
Overview
Ranks of 3-tensors
1
Basic facts.
2
Complexity.
3
Matrix multiplication
4
Results and conjectures
Approximations of tensors
1
Rank one approximation.
2
Perron-Frobenius theorem
3
Rank (R1 , R2 , R3 ) approximations
4
CUR approximations
List of applications
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
2 / 18
Basic notions
scalar a ∈ F, vector x = (x1 , . . . , xn )> ∈ Fn , matrix A = [aij ] ∈ Fm×n ,
3-tensor T = [ti,j,k ] ∈ Fm×n×l , p-tensor T = [ti1 ,...,ip ] ∈ Fn1 ×...×np
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
3 / 18
Basic notions
scalar a ∈ F, vector x = (x1 , . . . , xn )> ∈ Fn , matrix A = [aij ] ∈ Fm×n ,
3-tensor T = [ti,j,k ] ∈ Fm×n×l , p-tensor T = [ti1 ,...,ip ] ∈ Fn1 ×...×np
Abstractly U := U1 ⊗ U2 ⊗ U3 dim Ui = mi , i = 1, 2, 3, dim U = m1 m2 m3
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
3 / 18
Basic notions
scalar a ∈ F, vector x = (x1 , . . . , xn )> ∈ Fn , matrix A = [aij ] ∈ Fm×n ,
3-tensor T = [ti,j,k ] ∈ Fm×n×l , p-tensor T = [ti1 ,...,ip ] ∈ Fn1 ×...×np
Abstractly U := U1 ⊗ U2 ⊗ U3 dim Ui = mi , i = 1, 2, 3, dim U = m1 m2 m3
Tensor τ ∈ U1 ⊗ U2 ⊗ U3
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
3 / 18
Basic notions
scalar a ∈ F, vector x = (x1 , . . . , xn )> ∈ Fn , matrix A = [aij ] ∈ Fm×n ,
3-tensor T = [ti,j,k ] ∈ Fm×n×l , p-tensor T = [ti1 ,...,ip ] ∈ Fn1 ×...×np
Abstractly U := U1 ⊗ U2 ⊗ U3 dim Ui = mi , i = 1, 2, 3, dim U = m1 m2 m3
Tensor τ ∈ U1 ⊗ U2 ⊗ U3
HISTORY: Tensors-as now W. Voigt 1898
Tensor calculus 1890 G. Ricci-Curbastro: absolute differential calculus,
T. Levi-Civita: 1900, A. Einstein: General relativity 1915
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
3 / 18
Basic notions
scalar a ∈ F, vector x = (x1 , . . . , xn )> ∈ Fn , matrix A = [aij ] ∈ Fm×n ,
3-tensor T = [ti,j,k ] ∈ Fm×n×l , p-tensor T = [ti1 ,...,ip ] ∈ Fn1 ×...×np
Abstractly U := U1 ⊗ U2 ⊗ U3 dim Ui = mi , i = 1, 2, 3, dim U = m1 m2 m3
Tensor τ ∈ U1 ⊗ U2 ⊗ U3
HISTORY: Tensors-as now W. Voigt 1898
Tensor calculus 1890 G. Ricci-Curbastro: absolute differential calculus,
T. Levi-Civita: 1900, A. Einstein: General relativity 1915
Rank one tensor ti,j,k = xi yj zk , (i, j, k ) = (1, 1, 1), . . . , (m1 , m2 , m3 )
or decomposable tensor x ⊗ y ⊗ z
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
3 / 18
Basic notions
scalar a ∈ F, vector x = (x1 , . . . , xn )> ∈ Fn , matrix A = [aij ] ∈ Fm×n ,
3-tensor T = [ti,j,k ] ∈ Fm×n×l , p-tensor T = [ti1 ,...,ip ] ∈ Fn1 ×...×np
Abstractly U := U1 ⊗ U2 ⊗ U3 dim Ui = mi , i = 1, 2, 3, dim U = m1 m2 m3
Tensor τ ∈ U1 ⊗ U2 ⊗ U3
HISTORY: Tensors-as now W. Voigt 1898
Tensor calculus 1890 G. Ricci-Curbastro: absolute differential calculus,
T. Levi-Civita: 1900, A. Einstein: General relativity 1915
Rank one tensor ti,j,k = xi yj zk , (i, j, k ) = (1, 1, 1), . . . , (m1 , m2 , m3 )
or decomposable tensor x ⊗ y ⊗ z
basis of Uj :
basis of U:
[u1,j , . . . , umj ,j ] j = 1, 2, 3
ui1 ,1 ⊗ ui2 ,2 ⊗ ui3 ,3 , ij = 1, . . . , mj , j = 1, 2, 3,
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
3 / 18
Basic notions
scalar a ∈ F, vector x = (x1 , . . . , xn )> ∈ Fn , matrix A = [aij ] ∈ Fm×n ,
3-tensor T = [ti,j,k ] ∈ Fm×n×l , p-tensor T = [ti1 ,...,ip ] ∈ Fn1 ×...×np
Abstractly U := U1 ⊗ U2 ⊗ U3 dim Ui = mi , i = 1, 2, 3, dim U = m1 m2 m3
Tensor τ ∈ U1 ⊗ U2 ⊗ U3
HISTORY: Tensors-as now W. Voigt 1898
Tensor calculus 1890 G. Ricci-Curbastro: absolute differential calculus,
T. Levi-Civita: 1900, A. Einstein: General relativity 1915
Rank one tensor ti,j,k = xi yj zk , (i, j, k ) = (1, 1, 1), . . . , (m1 , m2 , m3 )
or decomposable tensor x ⊗ y ⊗ z
basis of Uj : [u1,j , . . . , umj ,j ] j = 1, 2, 3
basisPof U: ui1 ,1 ⊗ ui2 ,2 ⊗ ui3 ,3 , ij = 1, . . . , mj , j = 1, 2, 3,
1 ,m2 ,m3
τ= m
i1 =i2 =i3 =1 ti1 ,i2 ,i2 ui1 ,1 ⊗ ui2 ,2 ⊗ ui3 ,3
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
3 / 18
Ranks of tensors
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
4 / 18
Ranks of tensors
Unfolding tensor: in direction 1:
T = [ti,j,k ] view as a matrix A1 = [ti,(j,k ) ] ∈ Fm1 ×(m2 ·m3 )
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
4 / 18
Ranks of tensors
Unfolding tensor: in direction 1:
T = [ti,j,k ] view as a matrix A1 = [ti,(j,k ) ] ∈ Fm1 ×(m2 ·m3 )
R1 := rank A1 :
dimension of row or column subspace spanned in direction 1
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
4 / 18
Ranks of tensors
Unfolding tensor: in direction 1:
T = [ti,j,k ] view as a matrix A1 = [ti,(j,k ) ] ∈ Fm1 ×(m2 ·m3 )
R1 := rank A1 :
dimension of row or column subspace spanned in direction 1
m2 ,m3
m2 ×m3 , i = 1, . . . , m
Ti,1 := [ti,j,k ]j,k
1
=1 ∈ F
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
4 / 18
Ranks of tensors
Unfolding tensor: in direction 1:
T = [ti,j,k ] view as a matrix A1 = [ti,(j,k ) ] ∈ Fm1 ×(m2 ·m3 )
R1 := rank A1 :
dimension of row or column subspace spanned in direction 1
m2 ,m3
m2 ×m3 , i = 1, . . . , m
Ti,1 := [ti,j,k ]j,k
1
=1 ∈ F
Pm1
T = i=1 Ti,1 ei,1 (convenient notation)
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
4 / 18
Ranks of tensors
Unfolding tensor: in direction 1:
T = [ti,j,k ] view as a matrix A1 = [ti,(j,k ) ] ∈ Fm1 ×(m2 ·m3 )
R1 := rank A1 :
dimension of row or column subspace spanned in direction 1
m2 ,m3
m2 ×m3 , i = 1, . . . , m
Ti,1 := [ti,j,k ]j,k
1
=1 ∈ F
Pm1
T = i=1 Ti,1 ei,1 (convenient notation)
R1 := dim span(T1,1 , . . . , Tm1 ,1 ).
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
4 / 18
Ranks of tensors
Unfolding tensor: in direction 1:
T = [ti,j,k ] view as a matrix A1 = [ti,(j,k ) ] ∈ Fm1 ×(m2 ·m3 )
R1 := rank A1 :
dimension of row or column subspace spanned in direction 1
m2 ,m3
m2 ×m3 , i = 1, . . . , m
Ti,1 := [ti,j,k ]j,k
1
=1 ∈ F
Pm1
T = i=1 Ti,1 ei,1 (convenient notation)
R1 := dim span(T1,1 , . . . , Tm1 ,1 ).
Similarly, unfolding in directions 2, 3
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
4 / 18
Ranks of tensors
Unfolding tensor: in direction 1:
T = [ti,j,k ] view as a matrix A1 = [ti,(j,k ) ] ∈ Fm1 ×(m2 ·m3 )
R1 := rank A1 :
dimension of row or column subspace spanned in direction 1
m2 ,m3
m2 ×m3 , i = 1, . . . , m
Ti,1 := [ti,j,k ]j,k
1
=1 ∈ F
Pm1
T = i=1 Ti,1 ei,1 (convenient notation)
R1 := dim span(T1,1 , . . . , Tm1 ,1 ).
Similarly, unfolding in directions 2, 3
rank T minimal r :
T = fr (x1 , y1 , z1 , . . . , xr , yr , zr ) :=
Shmuel Friedland Univ. Illinois at Chicago ()
Pr
i=1 xi
⊗ yi ⊗ zi ,
Tensors: theory and applications
TTI-C, March 2, 2009
4 / 18
Ranks of tensors
Unfolding tensor: in direction 1:
T = [ti,j,k ] view as a matrix A1 = [ti,(j,k ) ] ∈ Fm1 ×(m2 ·m3 )
R1 := rank A1 :
dimension of row or column subspace spanned in direction 1
m2 ,m3
m2 ×m3 , i = 1, . . . , m
Ti,1 := [ti,j,k ]j,k
1
=1 ∈ F
Pm1
T = i=1 Ti,1 ei,1 (convenient notation)
R1 := dim span(T1,1 , . . . , Tm1 ,1 ).
Similarly, unfolding in directions 2, 3
rank T minimal r :
T = fr (x1 , y1 , z1 , . . . , xr , yr , zr ) :=
(CANDEC, PARFAC)
Shmuel Friedland Univ. Illinois at Chicago ()
Pr
i=1 xi
⊗ yi ⊗ zi ,
Tensors: theory and applications
TTI-C, March 2, 2009
4 / 18
Basic facts
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
5 / 18
Basic facts
FACT I: rank T ≥ max(R1 , R2 , R3 )
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
5 / 18
Basic facts
FACT I: rank T ≥ max(R1 , R2 , R3 )
Reason U2 ⊗ U3 ∼ Fm2 ×m3 ≡ Fm2 m3
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
5 / 18
Basic facts
FACT I: rank T ≥ max(R1 , R2 , R3 )
Reason U2 ⊗ U3 ∼ Fm2 ×m3 ≡ Fm2 m3
Note:
R1 , R2 , R3 are easily computable
It is possible that R1 6= R2 6= R3
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
5 / 18
Basic facts
FACT I: rank T ≥ max(R1 , R2 , R3 )
Reason U2 ⊗ U3 ∼ Fm2 ×m3 ≡ Fm2 m3
Note:
R1 , R2 , R3 are easily computable
It is possible that R1 6= R2 6= R3
FACT II : For τ = T = [ti,j,k ] let
m1 ×m2 , k = 1, . . . , m . Then rank T =
1 ,m2
Tk ,3 := [ti,j,k ]m
3
i,j=1 ∈ F
minimal dimension of subspace L ⊂ Fm1 ×m2 spanned by rank one
matrices containing T1,3 , . . . , Tm3 ,3 .
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
5 / 18
Basic facts
FACT I: rank T ≥ max(R1 , R2 , R3 )
Reason U2 ⊗ U3 ∼ Fm2 ×m3 ≡ Fm2 m3
Note:
R1 , R2 , R3 are easily computable
It is possible that R1 6= R2 6= R3
FACT II : For τ = T = [ti,j,k ] let
m1 ×m2 , k = 1, . . . , m . Then rank T =
1 ,m2
Tk ,3 := [ti,j,k ]m
3
i,j=1 ∈ F
minimal dimension of subspace L ⊂ Fm1 ×m2 spanned by rank one
matrices containing T1,3 , . . . , Tm3 ,3 .
COR rank T ≤ min(mn, ml, nl)
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
5 / 18
Complexity of rank of 3-tensor
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
6 / 18
Complexity of rank of 3-tensor
Hastad 1990: Tensor rank is NP-complete for any finite field and
NP-hard for rational numbers
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
6 / 18
Complexity of rank of 3-tensor
Hastad 1990: Tensor rank is NP-complete for any finite field and
NP-hard for rational numbers
PRF: 3-sat with n variables m clauses
satisfiable iff rank T = 4n + 2m, T ∈ F(2n+3m)×(3n)×(3n+m) )
otherwise rank is larger
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
6 / 18
Generic rank
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
7 / 18
Generic rank
Rr (m, n, l) ⊂ Fm×n×l all tensors of rank ≤ r
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
7 / 18
Generic rank
Rr (m, n, l) ⊂ Fm×n×l all tensors of rank ≤ r
Rr (m, n, l) not closed variety for r ≥ 2
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
7 / 18
Generic rank
Rr (m, n, l) ⊂ Fm×n×l all tensors of rank ≤ r
Rr (m, n, l) not closed variety for r ≥ 2
generic rank=border rank=typical rank: grankF (m, n, l) the rank of a random tensor T ∈ Fm×n×l
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
7 / 18
Generic rank
Rr (m, n, l) ⊂ Fm×n×l all tensors of rank ≤ r
Rr (m, n, l) not closed variety for r ≥ 2
generic rank=border rank=typical rank: grankF (m, n, l) the rank of a random tensor T ∈ Fm×n×l
THM: grankC (m, n, l) = min(l, mn) for (m − 1)(n − 1) ≤ l.
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
7 / 18
Generic rank
Rr (m, n, l) ⊂ Fm×n×l all tensors of rank ≤ r
Rr (m, n, l) not closed variety for r ≥ 2
generic rank=border rank=typical rank: grankF (m, n, l) the rank of a random tensor T ∈ Fm×n×l
THM: grankC (m, n, l) = min(l, mn) for (m − 1)(n − 1) ≤ l.
COR: grankC (2, n, l) = min(l, 2n) for 2 ≤ n ≤ l
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
7 / 18
Generic rank
Rr (m, n, l) ⊂ Fm×n×l all tensors of rank ≤ r
Rr (m, n, l) not closed variety for r ≥ 2
generic rank=border rank=typical rank: grankF (m, n, l) the rank of a random tensor T ∈ Fm×n×l
THM: grankC (m, n, l) = min(l, mn) for (m − 1)(n − 1) ≤ l.
COR: grankC (2, n, l) = min(l, 2n) for 2 ≤ n ≤ l
Dimension count for F = C and 2 ≤ m ≤ n ≤ l < (m − 1)(n − 1):
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
7 / 18
Generic rank
Rr (m, n, l) ⊂ Fm×n×l all tensors of rank ≤ r
Rr (m, n, l) not closed variety for r ≥ 2
generic rank=border rank=typical rank: grankF (m, n, l) the rank of a random tensor T ∈ Fm×n×l
THM: grankC (m, n, l) = min(l, mn) for (m − 1)(n − 1) ≤ l.
COR: grankC (2, n, l) = min(l, 2n) for 2 ≤ n ≤ l
Dimension count for F = C and 2 ≤ m ≤ n ≤ l < (m − 1)(n − 1):
fr : (Cm × Cn × Cl )r → Cm×n×l , x ⊗ y ⊗ z = (ax) ⊗ (by) ⊗ ((ab)−1 z)
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
7 / 18
Generic rank
Rr (m, n, l) ⊂ Fm×n×l all tensors of rank ≤ r
Rr (m, n, l) not closed variety for r ≥ 2
generic rank=border rank=typical rank: grankF (m, n, l) the rank of a random tensor T ∈ Fm×n×l
THM: grankC (m, n, l) = min(l, mn) for (m − 1)(n − 1) ≤ l.
COR: grankC (2, n, l) = min(l, 2n) for 2 ≤ n ≤ l
Dimension count for F = C and 2 ≤ m ≤ n ≤ l < (m − 1)(n − 1):
fr : (Cm × Cn × Cl )r → Cm×n×l , x ⊗ y ⊗ z = (ax) ⊗ (by) ⊗ ((ab)−1 z)
mnl
grankC (m, n, l)(m + n + l − 2) ≥ mnl ⇒ grankC (m, n, l) ≥ d (m+n+l−2)
e
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
7 / 18
Generic rank
Rr (m, n, l) ⊂ Fm×n×l all tensors of rank ≤ r
Rr (m, n, l) not closed variety for r ≥ 2
generic rank=border rank=typical rank: grankF (m, n, l) the rank of a random tensor T ∈ Fm×n×l
THM: grankC (m, n, l) = min(l, mn) for (m − 1)(n − 1) ≤ l.
COR: grankC (2, n, l) = min(l, 2n) for 2 ≤ n ≤ l
Dimension count for F = C and 2 ≤ m ≤ n ≤ l < (m − 1)(n − 1):
fr : (Cm × Cn × Cl )r → Cm×n×l , x ⊗ y ⊗ z = (ax) ⊗ (by) ⊗ ((ab)−1 z)
mnl
grankC (m, n, l)(m + n + l − 2) ≥ mnl ⇒ grankC (m, n, l) ≥ d (m+n+l−2)
e
mnl
Conjecture grankC (m, n, l) = d (m+n+l−2)
e
for 2 ≤ m ≤ n ≤ l < (m − 1)(n − 1) and (3, n, l) 6= (3, 2p + 1, 2p + 1)
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
7 / 18
Generic rank
Rr (m, n, l) ⊂ Fm×n×l all tensors of rank ≤ r
Rr (m, n, l) not closed variety for r ≥ 2
generic rank=border rank=typical rank: grankF (m, n, l) the rank of a random tensor T ∈ Fm×n×l
THM: grankC (m, n, l) = min(l, mn) for (m − 1)(n − 1) ≤ l.
COR: grankC (2, n, l) = min(l, 2n) for 2 ≤ n ≤ l
Dimension count for F = C and 2 ≤ m ≤ n ≤ l < (m − 1)(n − 1):
fr : (Cm × Cn × Cl )r → Cm×n×l , x ⊗ y ⊗ z = (ax) ⊗ (by) ⊗ ((ab)−1 z)
mnl
grankC (m, n, l)(m + n + l − 2) ≥ mnl ⇒ grankC (m, n, l) ≥ d (m+n+l−2)
e
mnl
Conjecture grankC (m, n, l) = d (m+n+l−2)
e
for 2 ≤ m ≤ n ≤ l < (m − 1)(n − 1) and (3, n, l) 6= (3, 2p + 1, 2p + 1)
2
Fact: grankC (3, 2p + 1, 2p + 1) = d 3(2p+1)
4p+3 e + 1
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
7 / 18
Bilinear maps and product of matrices
bilinear map: φ : U × V → W
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
8 / 18
Bilinear maps and product of matrices
bilinear map: φ : U × V → W
[u1 , . . . , um ], [v1 , . . . , vn ], [w1 , . . . , wl ] bases in U, V, W
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
8 / 18
Bilinear maps and product of matrices
bilinear map: φ : U × V → W
[u1 , . . . , um ], [v1 , . . . , vn ], [w1 , . . . , wl ] bases in U, V, W
φ(ui , vj ) =
P
k =1 ti,j,k wk ,
Shmuel Friedland Univ. Illinois at Chicago ()
T := [ti,j,k ] ∈ Fm×n×l
Tensors: theory and applications
TTI-C, March 2, 2009
8 / 18
Bilinear maps and product of matrices
bilinear map: φ : U × V → W
[u1 , . . . , um ], [v1 , . . . , vn ], [w1 , . . . , wl ] bases in U, V, W
φ(ui , vj ) =
T =
Pr
P
a=1 xa
k =1 ti,j,k wk ,
T := [ti,j,k ] ∈ Fm×n×l
⊗ ya ⊗ za , r = rank T
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
8 / 18
Bilinear maps and product of matrices
bilinear map: φ : U × V → W
[u1 , . . . , um ], [v1 , . . . , vn ], [w1 , . . . , wl ] bases in U, V, W
φ(ui , vj ) =
T =
P
Pr
a=1 xa
φ(c, d) =
Pr
k =1 ti,j,k wk ,
T := [ti,j,k ] ∈ Fm×n×l
⊗ ya ⊗ za , r = rank T
a=1 (c
> x)(d> y)z ,
a
Shmuel Friedland Univ. Illinois at Chicago ()
c=
Pm
i=1 ci ui , d
Tensors: theory and applications
=
Pn
j=1 dj vj
TTI-C, March 2, 2009
8 / 18
Bilinear maps and product of matrices
bilinear map: φ : U × V → W
[u1 , . . . , um ], [v1 , . . . , vn ], [w1 , . . . , wl ] bases in U, V, W
φ(ui , vj ) =
T =
Pr
P
a=1 xa
k =1 ti,j,k wk ,
T := [ti,j,k ] ∈ Fm×n×l
⊗ ya ⊗ za , r = rank T
P
P
Pn
φ(c, d) = ra=1 (c> x)(d> y)za , c = m
i=1 ci ui , d =
j=1 dj vj
Complexity: r -products
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
8 / 18
Bilinear maps and product of matrices
bilinear map: φ : U × V → W
[u1 , . . . , um ], [v1 , . . . , vn ], [w1 , . . . , wl ] bases in U, V, W
φ(ui , vj ) =
T =
Pr
P
a=1 xa
k =1 ti,j,k wk ,
T := [ti,j,k ] ∈ Fm×n×l
⊗ ya ⊗ za , r = rank T
P
P
Pn
φ(c, d) = ra=1 (c> x)(d> y)za , c = m
i=1 ci ui , d =
j=1 dj vj
Complexity: r -products
Matrix product τ : FM×N × FN×L → FM×L , (A, B) 7→ AB
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
8 / 18
Bilinear maps and product of matrices
bilinear map: φ : U × V → W
[u1 , . . . , um ], [v1 , . . . , vn ], [w1 , . . . , wl ] bases in U, V, W
φ(ui , vj ) =
T =
Pr
P
a=1 xa
k =1 ti,j,k wk ,
T := [ti,j,k ] ∈ Fm×n×l
⊗ ya ⊗ za , r = rank T
P
P
Pn
φ(c, d) = ra=1 (c> x)(d> y)za , c = m
i=1 ci ui , d =
j=1 dj vj
Complexity: r -products
Matrix product τ : FM×N × FN×L → FM×L , (A, B) 7→ AB
4×4×4
M = N = L = 2, grankC (4, 4, 4) = d 4+4+4−2
e = d6.4e = 7
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
8 / 18
Bilinear maps and product of matrices
bilinear map: φ : U × V → W
[u1 , . . . , um ], [v1 , . . . , vn ], [w1 , . . . , wl ] bases in U, V, W
φ(ui , vj ) =
T =
Pr
P
a=1 xa
k =1 ti,j,k wk ,
T := [ti,j,k ] ∈ Fm×n×l
⊗ ya ⊗ za , r = rank T
P
P
Pn
φ(c, d) = ra=1 (c> x)(d> y)za , c = m
i=1 ci ui , d =
j=1 dj vj
Complexity: r -products
Matrix product τ : FM×N × FN×L → FM×L , (A, B) 7→ AB
4×4×4
M = N = L = 2, grankC (4, 4, 4) = d 4+4+4−2
e = d6.4e = 7
Product of two 2 × 2 matrices is done by 7 multiplications
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
8 / 18
Known cases of rank conjecture
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
9 / 18
Known cases of rank conjecture
2
2
12p
grank(3, 2p, 2p) = d 4p+1
e and grank(3, 2p − 1, 2p − 1) = d 3(2p−1)
4p−1 e + 1
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
9 / 18
Known cases of rank conjecture
2
2
12p
grank(3, 2p, 2p) = d 4p+1
e and grank(3, 2p − 1, 2p − 1) = d 3(2p−1)
4p−1 e + 1
(n, n, n + 2) if n 6= 2 (mod 3),
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
9 / 18
Known cases of rank conjecture
2
2
12p
grank(3, 2p, 2p) = d 4p+1
e and grank(3, 2p − 1, 2p − 1) = d 3(2p−1)
4p−1 e + 1
(n, n, n + 2) if n 6= 2 (mod 3),
(n − 1, n, n) if n = 0 (mod 3),
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
9 / 18
Known cases of rank conjecture
2
2
12p
grank(3, 2p, 2p) = d 4p+1
e and grank(3, 2p − 1, 2p − 1) = d 3(2p−1)
4p−1 e + 1
(n, n, n + 2) if n 6= 2 (mod 3),
(n − 1, n, n) if n = 0 (mod 3),
(4, m, m) if m ≥ 4,
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
9 / 18
Known cases of rank conjecture
2
2
12p
grank(3, 2p, 2p) = d 4p+1
e and grank(3, 2p − 1, 2p − 1) = d 3(2p−1)
4p−1 e + 1
(n, n, n + 2) if n 6= 2 (mod 3),
(n − 1, n, n) if n = 0 (mod 3),
(4, m, m) if m ≥ 4,
(n, n, n) if n ≥ 4
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
9 / 18
Known cases of rank conjecture
2
2
12p
grank(3, 2p, 2p) = d 4p+1
e and grank(3, 2p − 1, 2p − 1) = d 3(2p−1)
4p−1 e + 1
(n, n, n + 2) if n 6= 2 (mod 3),
(n − 1, n, n) if n = 0 (mod 3),
(4, m, m) if m ≥ 4,
(n, n, n) if n ≥ 4
2lp
(l, 2p, 2q) if l ≤ 2p ≤ 2q and and l+2p+2q−2
is integer
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
9 / 18
Known cases of rank conjecture
2
2
12p
grank(3, 2p, 2p) = d 4p+1
e and grank(3, 2p − 1, 2p − 1) = d 3(2p−1)
4p−1 e + 1
(n, n, n + 2) if n 6= 2 (mod 3),
(n − 1, n, n) if n = 0 (mod 3),
(4, m, m) if m ≥ 4,
(n, n, n) if n ≥ 4
2lp
(l, 2p, 2q) if l ≤ 2p ≤ 2q and and l+2p+2q−2
is integer
Easy to compute grankC (m, n, l):
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
9 / 18
Known cases of rank conjecture
2
2
12p
grank(3, 2p, 2p) = d 4p+1
e and grank(3, 2p − 1, 2p − 1) = d 3(2p−1)
4p−1 e + 1
(n, n, n + 2) if n 6= 2 (mod 3),
(n − 1, n, n) if n = 0 (mod 3),
(4, m, m) if m ≥ 4,
(n, n, n) if n ≥ 4
2lp
(l, 2p, 2q) if l ≤ 2p ≤ 2q and and l+2p+2q−2
is integer
Easy to compute grankC (m, n, l):
Pick at random wr := (x1 , y1 , z1 , . . . , xr , yr , zr ) ∈ (Rm × Rn × Rl )r
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
9 / 18
Known cases of rank conjecture
2
2
12p
grank(3, 2p, 2p) = d 4p+1
e and grank(3, 2p − 1, 2p − 1) = d 3(2p−1)
4p−1 e + 1
(n, n, n + 2) if n 6= 2 (mod 3),
(n − 1, n, n) if n = 0 (mod 3),
(4, m, m) if m ≥ 4,
(n, n, n) if n ≥ 4
2lp
(l, 2p, 2q) if l ≤ 2p ≤ 2q and and l+2p+2q−2
is integer
Easy to compute grankC (m, n, l):
Pick at random wr := (x1 , y1 , z1 , . . . , xr , yr , zr ) ∈ (Rm × Rn × Rl )r
mnl
e s.t. rank J(fr )(wr ) = mnl
The minimal r ≥ d (m+n+l−2)
is grankC (m, n, l) (Terracini Lemma 1915)
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
9 / 18
Known cases of rank conjecture
2
2
12p
grank(3, 2p, 2p) = d 4p+1
e and grank(3, 2p − 1, 2p − 1) = d 3(2p−1)
4p−1 e + 1
(n, n, n + 2) if n 6= 2 (mod 3),
(n − 1, n, n) if n = 0 (mod 3),
(4, m, m) if m ≥ 4,
(n, n, n) if n ≥ 4
2lp
(l, 2p, 2q) if l ≤ 2p ≤ 2q and and l+2p+2q−2
is integer
Easy to compute grankC (m, n, l):
Pick at random wr := (x1 , y1 , z1 , . . . , xr , yr , zr ) ∈ (Rm × Rn × Rl )r
mnl
e s.t. rank J(fr )(wr ) = mnl
The minimal r ≥ d (m+n+l−2)
is grankC (m, n, l) (Terracini Lemma 1915)
Avoid round-off error:
wr ∈ (Zm × Zn × Zl )r find rank J(fr )(wr ) exact arithmetic
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
9 / 18
Known cases of rank conjecture
2
2
12p
grank(3, 2p, 2p) = d 4p+1
e and grank(3, 2p − 1, 2p − 1) = d 3(2p−1)
4p−1 e + 1
(n, n, n + 2) if n 6= 2 (mod 3),
(n − 1, n, n) if n = 0 (mod 3),
(4, m, m) if m ≥ 4,
(n, n, n) if n ≥ 4
2lp
(l, 2p, 2q) if l ≤ 2p ≤ 2q and and l+2p+2q−2
is integer
Easy to compute grankC (m, n, l):
Pick at random wr := (x1 , y1 , z1 , . . . , xr , yr , zr ) ∈ (Rm × Rn × Rl )r
mnl
e s.t. rank J(fr )(wr ) = mnl
The minimal r ≥ d (m+n+l−2)
is grankC (m, n, l) (Terracini Lemma 1915)
Avoid round-off error:
wr ∈ (Zm × Zn × Zl )r find rank J(fr )(wr ) exact arithmetic
I checked the conjecture up to m, n, l ≤ 14
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
9 / 18
Generic rank III - the real case
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
10 / 18
Generic rank III - the real case
For mn ≤ l grankR (m, n, l) = mn.
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
10 / 18
Generic rank III - the real case
For mn ≤ l grankR (m, n, l) = mn.
For 2 ≤ m ≤ n ≤ l < mn − 1, there exist V1 , . . . , Vc(m,n,l) ⊂ Rm×n×l
pairwise distinct open connected semi-algebraic sets s.t.
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
10 / 18
Generic rank III - the real case
For mn ≤ l grankR (m, n, l) = mn.
For 2 ≤ m ≤ n ≤ l < mn − 1, there exist V1 , . . . , Vc(m,n,l) ⊂ Rm×n×l
pairwise distinct open connected semi-algebraic sets s.t.
c(m,n,l)
Closure(∪i=1
) = Rm×n×l
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
10 / 18
Generic rank III - the real case
For mn ≤ l grankR (m, n, l) = mn.
For 2 ≤ m ≤ n ≤ l < mn − 1, there exist V1 , . . . , Vc(m,n,l) ⊂ Rm×n×l
pairwise distinct open connected semi-algebraic sets s.t.
c(m,n,l)
Closure(∪i=1
) = Rm×n×l
rank T = grankC (m, n, l) for each T ∈ V1
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
10 / 18
Generic rank III - the real case
For mn ≤ l grankR (m, n, l) = mn.
For 2 ≤ m ≤ n ≤ l < mn − 1, there exist V1 , . . . , Vc(m,n,l) ⊂ Rm×n×l
pairwise distinct open connected semi-algebraic sets s.t.
c(m,n,l)
Closure(∪i=1
) = Rm×n×l
rank T = grankC (m, n, l) for each T ∈ V1
rank T = ρi for each T ∈ Vi
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
10 / 18
Generic rank III - the real case
For mn ≤ l grankR (m, n, l) = mn.
For 2 ≤ m ≤ n ≤ l < mn − 1, there exist V1 , . . . , Vc(m,n,l) ⊂ Rm×n×l
pairwise distinct open connected semi-algebraic sets s.t.
c(m,n,l)
Closure(∪i=1
) = Rm×n×l
rank T = grankC (m, n, l) for each T ∈ V1
rank T = ρi for each T ∈ Vi
ρi ≥ grankC (m, n, l) for i = 2, . . . , c(m, n, l)
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
10 / 18
Generic rank III - the real case
For mn ≤ l grankR (m, n, l) = mn.
For 2 ≤ m ≤ n ≤ l < mn − 1, there exist V1 , . . . , Vc(m,n,l) ⊂ Rm×n×l
pairwise distinct open connected semi-algebraic sets s.t.
c(m,n,l)
Closure(∪i=1
) = Rm×n×l
rank T = grankC (m, n, l) for each T ∈ V1
rank T = ρi for each T ∈ Vi
ρi ≥ grankC (m, n, l) for i = 2, . . . , c(m, n, l)
For l = (m − 1)(n − 1) ∃m, n:
c(m, n, l) > 1, ρc(m,n,l) ≥ grankC (m, n, l) + 1
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
10 / 18
Generic rank III - the real case
For mn ≤ l grankR (m, n, l) = mn.
For 2 ≤ m ≤ n ≤ l < mn − 1, there exist V1 , . . . , Vc(m,n,l) ⊂ Rm×n×l
pairwise distinct open connected semi-algebraic sets s.t.
c(m,n,l)
Closure(∪i=1
) = Rm×n×l
rank T = grankC (m, n, l) for each T ∈ V1
rank T = ρi for each T ∈ Vi
ρi ≥ grankC (m, n, l) for i = 2, . . . , c(m, n, l)
For l = (m − 1)(n − 1) ∃m, n:
c(m, n, l) > 1, ρc(m,n,l) ≥ grankC (m, n, l) + 1
Examples [1]
m = n ≥ 2, l = (m − 1)(n − 1) + 1.
m = n = 4, l = 11, 12
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
10 / 18
Rank one approximations
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
11 / 18
Rank one approximations
Rm×n×l IPS: hA, Bi =
Pm,n,l
Shmuel Friedland Univ. Illinois at Chicago ()
i=j=k ai,j,k bi,j,k , kT k =
Tensors: theory and applications
p
hT , T i
TTI-C, March 2, 2009
11 / 18
Rank one approximations
p
Pm,n,l
Rm×n×l IPS: hA, Bi = i=j=k
ai,j,k bi,j,k , kT k = hT , T i
hx ⊗ y ⊗ z, u ⊗ v ⊗ wi = (u> x)(v> y)(w> z)
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
11 / 18
Rank one approximations
p
Pm,n,l
Rm×n×l IPS: hA, Bi = i=j=k
ai,j,k bi,j,k , kT k = hT , T i
hx ⊗ y ⊗ z, u ⊗ v ⊗ wi = (u> x)(v> y)(w> z)
X subspace of Rm×n×l , X1 , . . . , Xd an orthonormal basis of X
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
11 / 18
Rank one approximations
p
Pm,n,l
Rm×n×l IPS: hA, Bi = i=j=k
ai,j,k bi,j,k , kT k = hT , T i
hx ⊗ y ⊗ z, u ⊗ v ⊗ wi = (u> x)(v> y)(w> z)
m×n×l , X , . . . , X an orthonormal basis of X
X subspace
1
d
P
Pd of R
PX (T ) = i=1 hT , Xi iXi , kPX (T )k2 = di=1 hT , Xi i2
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
11 / 18
Rank one approximations
p
Pm,n,l
Rm×n×l IPS: hA, Bi = i=j=k
ai,j,k bi,j,k , kT k = hT , T i
hx ⊗ y ⊗ z, u ⊗ v ⊗ wi = (u> x)(v> y)(w> z)
m×n×l , X , . . . , X an orthonormal basis of X
X subspace
1
d
P
Pd of R
PX (T ) = i=1 hT , Xi iXi , kPX (T )k2 = di=1 hT , Xi i2
kT k2 = kPX (T )k2 + kT − PX (T )k2
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
11 / 18
Rank one approximations
p
Pm,n,l
Rm×n×l IPS: hA, Bi = i=j=k
ai,j,k bi,j,k , kT k = hT , T i
hx ⊗ y ⊗ z, u ⊗ v ⊗ wi = (u> x)(v> y)(w> z)
m×n×l , X , . . . , X an orthonormal basis of X
X subspace
1
d
P
Pd of R
PX (T ) = i=1 hT , Xi iXi , kPX (T )k2 = di=1 hT , Xi i2
kT k2 = kPX (T )k2 + kT − PX (T )k2
Best rank one approximation of T :
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
11 / 18
Rank one approximations
p
Pm,n,l
Rm×n×l IPS: hA, Bi = i=j=k
ai,j,k bi,j,k , kT k = hT , T i
hx ⊗ y ⊗ z, u ⊗ v ⊗ wi = (u> x)(v> y)(w> z)
m×n×l , X , . . . , X an orthonormal basis of X
X subspace
1
d
P
Pd of R
PX (T ) = i=1 hT , Xi iXi , kPX (T )k2 = di=1 hT , Xi i2
kT k2 = kPX (T )k2 + kT − PX (T )k2
Best rank one approximation of T :
minx,y,z kT − x ⊗ y ⊗ zk = minkxk=kyk=kzk=1,a kT − a x ⊗ y ⊗ zk
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
11 / 18
Rank one approximations
p
Pm,n,l
Rm×n×l IPS: hA, Bi = i=j=k
ai,j,k bi,j,k , kT k = hT , T i
hx ⊗ y ⊗ z, u ⊗ v ⊗ wi = (u> x)(v> y)(w> z)
m×n×l , X , . . . , X an orthonormal basis of X
X subspace
1
d
P
Pd of R
PX (T ) = i=1 hT , Xi iXi , kPX (T )k2 = di=1 hT , Xi i2
kT k2 = kPX (T )k2 + kT − PX (T )k2
Best rank one approximation of T :
minx,y,z kT − x ⊗ y ⊗ zk = minkxk=kyk=kzk=1,a kT − a x ⊗ y ⊗ zk
Equivalent: maxkxk=kyk=kzk=1
Shmuel Friedland Univ. Illinois at Chicago ()
Pm,n,l
i=j=k ti,j,k xi yj zk
Tensors: theory and applications
TTI-C, March 2, 2009
11 / 18
Rank one approximations
p
Pm,n,l
Rm×n×l IPS: hA, Bi = i=j=k
ai,j,k bi,j,k , kT k = hT , T i
hx ⊗ y ⊗ z, u ⊗ v ⊗ wi = (u> x)(v> y)(w> z)
m×n×l , X , . . . , X an orthonormal basis of X
X subspace
1
d
P
Pd of R
PX (T ) = i=1 hT , Xi iXi , kPX (T )k2 = di=1 hT , Xi i2
kT k2 = kPX (T )k2 + kT − PX (T )k2
Best rank one approximation of T :
minx,y,z kT − x ⊗ y ⊗ zk = minkxk=kyk=kzk=1,a kT − a x ⊗ y ⊗ zk
Equivalent: maxkxk=kyk=kzk=1
Pm,n,l
i=j=k ti,j,k xi yj zk
Lagrange multipliers: T × y ⊗ z :=
T × x ⊗ z = λy, T × x ⊗ y = λz
Shmuel Friedland Univ. Illinois at Chicago ()
P
j=k =1 ti,j,k yj zk
Tensors: theory and applications
= λx
TTI-C, March 2, 2009
11 / 18
Rank one approximations
p
Pm,n,l
Rm×n×l IPS: hA, Bi = i=j=k
ai,j,k bi,j,k , kT k = hT , T i
hx ⊗ y ⊗ z, u ⊗ v ⊗ wi = (u> x)(v> y)(w> z)
m×n×l , X , . . . , X an orthonormal basis of X
X subspace
1
d
P
Pd of R
PX (T ) = i=1 hT , Xi iXi , kPX (T )k2 = di=1 hT , Xi i2
kT k2 = kPX (T )k2 + kT − PX (T )k2
Best rank one approximation of T :
minx,y,z kT − x ⊗ y ⊗ zk = minkxk=kyk=kzk=1,a kT − a x ⊗ y ⊗ zk
Equivalent: maxkxk=kyk=kzk=1
Pm,n,l
i=j=k ti,j,k xi yj zk
P
Lagrange multipliers: T × y ⊗ z := j=k =1 ti,j,k yj zk = λx
T × x ⊗ z = λy, T × x ⊗ y = λz
λ singular value, x, y, z singular vectors
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
11 / 18
Rank one approximations
p
Pm,n,l
Rm×n×l IPS: hA, Bi = i=j=k
ai,j,k bi,j,k , kT k = hT , T i
hx ⊗ y ⊗ z, u ⊗ v ⊗ wi = (u> x)(v> y)(w> z)
m×n×l , X , . . . , X an orthonormal basis of X
X subspace
1
d
P
Pd of R
PX (T ) = i=1 hT , Xi iXi , kPX (T )k2 = di=1 hT , Xi i2
kT k2 = kPX (T )k2 + kT − PX (T )k2
Best rank one approximation of T :
minx,y,z kT − x ⊗ y ⊗ zk = minkxk=kyk=kzk=1,a kT − a x ⊗ y ⊗ zk
Equivalent: maxkxk=kyk=kzk=1
Pm,n,l
i=j=k ti,j,k xi yj zk
P
Lagrange multipliers: T × y ⊗ z := j=k =1 ti,j,k yj zk = λx
T × x ⊗ z = λy, T × x ⊗ y = λz
λ singular value, x, y, z singular vectors
How many distinct singular values are for a generic tensor?
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
11 / 18
`p maximal problem and Perron-Frobenius
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
12 / 18
`p maximal problem and Perron-Frobenius
1
P
k(x1 , . . . , xn )> kp := ( ni=1 |xi |p ) p
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
12 / 18
`p maximal problem and Perron-Frobenius
1
P
k(x1 , . . . , xn )> kp := ( ni=1 |xi |p ) p
Problem: maxkxkp =kykp =kzkp =1
Shmuel Friedland Univ. Illinois at Chicago ()
Pm,n,l
i=j=k ti,j,k xi yj zk
Tensors: theory and applications
TTI-C, March 2, 2009
12 / 18
`p maximal problem and Perron-Frobenius
1
P
k(x1 , . . . , xn )> kp := ( ni=1 |xi |p ) p
Problem: maxkxkp =kykp =kzkp =1
Pm,n,l
i=j=k ti,j,k xi yj zk
Lagrange multipliers: T × y ⊗ z :=
Shmuel Friedland Univ. Illinois at Chicago ()
P
j=k =1 ti,j,k yj zk
Tensors: theory and applications
= λxp−1
TTI-C, March 2, 2009
12 / 18
`p maximal problem and Perron-Frobenius
1
P
k(x1 , . . . , xn )> kp := ( ni=1 |xi |p ) p
Problem: maxkxkp =kykp =kzkp =1
Pm,n,l
i=j=k ti,j,k xi yj zk
P
Lagrange multipliers: T × y ⊗ z := j=k =1 ti,j,k yj zk = λxp−1
2t
T × x ⊗ z = λyp−1 , T × x ⊗ y = λzp−1 (p = 2s−1
, t, s ∈ N)
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
12 / 18
`p maximal problem and Perron-Frobenius
1
P
k(x1 , . . . , xn )> kp := ( ni=1 |xi |p ) p
Problem: maxkxkp =kykp =kzkp =1
Pm,n,l
i=j=k ti,j,k xi yj zk
P
Lagrange multipliers: T × y ⊗ z := j=k =1 ti,j,k yj zk = λxp−1
2t
T × x ⊗ z = λyp−1 , T × x ⊗ y = λzp−1 (p = 2s−1
, t, s ∈ N)
p = 3 is most natural in view of homogeneity
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
12 / 18
`p maximal problem and Perron-Frobenius
1
P
k(x1 , . . . , xn )> kp := ( ni=1 |xi |p ) p
Problem: maxkxkp =kykp =kzkp =1
Pm,n,l
i=j=k ti,j,k xi yj zk
P
Lagrange multipliers: T × y ⊗ z := j=k =1 ti,j,k yj zk = λxp−1
2t
T × x ⊗ z = λyp−1 , T × x ⊗ y = λzp−1 (p = 2s−1
, t, s ∈ N)
p = 3 is most natural in view of homogeneity
Assume that T ≥ 0. Then x, y, z ≥ 0
For which values of p we have an analog of Perron-Frobenius
theorem?
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
12 / 18
`p maximal problem and Perron-Frobenius
1
P
k(x1 , . . . , xn )> kp := ( ni=1 |xi |p ) p
Problem: maxkxkp =kykp =kzkp =1
Pm,n,l
i=j=k ti,j,k xi yj zk
P
Lagrange multipliers: T × y ⊗ z := j=k =1 ti,j,k yj zk = λxp−1
2t
T × x ⊗ z = λyp−1 , T × x ⊗ y = λzp−1 (p = 2s−1
, t, s ∈ N)
p = 3 is most natural in view of homogeneity
Assume that T ≥ 0. Then x, y, z ≥ 0
For which values of p we have an analog of Perron-Frobenius
theorem?
Yes, for p = 3, and probably for p > 3
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
12 / 18
`p maximal problem and Perron-Frobenius
1
P
k(x1 , . . . , xn )> kp := ( ni=1 |xi |p ) p
Problem: maxkxkp =kykp =kzkp =1
Pm,n,l
i=j=k ti,j,k xi yj zk
P
Lagrange multipliers: T × y ⊗ z := j=k =1 ti,j,k yj zk = λxp−1
2t
T × x ⊗ z = λyp−1 , T × x ⊗ y = λzp−1 (p = 2s−1
, t, s ∈ N)
p = 3 is most natural in view of homogeneity
Assume that T ≥ 0. Then x, y, z ≥ 0
For which values of p we have an analog of Perron-Frobenius
theorem?
Yes, for p = 3, and probably for p > 3
No, for p = 2, and probably for p < 3
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
12 / 18
(R1 , R2 , R3 )-rank approximation of 3-tensors
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
13 / 18
(R1 , R2 , R3 )-rank approximation of 3-tensors
Fundamental problem in applications:
Approximate well and fast T ∈ Rm1 ×m2 ×m3 by rank (R1 , R2 , R3 )
3-tensor.
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
13 / 18
(R1 , R2 , R3 )-rank approximation of 3-tensors
Fundamental problem in applications:
Approximate well and fast T ∈ Rm1 ×m2 ×m3 by rank (R1 , R2 , R3 )
3-tensor.
Best (R1 , R2 , R3 ) approximation problem:
Find Ui ⊂ Fmi of dimension Ri for i = 1, 2, 3 with maximal
||PU1 ⊗U2 ⊗U3 (T )||.
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
13 / 18
(R1 , R2 , R3 )-rank approximation of 3-tensors
Fundamental problem in applications:
Approximate well and fast T ∈ Rm1 ×m2 ×m3 by rank (R1 , R2 , R3 )
3-tensor.
Best (R1 , R2 , R3 ) approximation problem:
Find Ui ⊂ Fmi of dimension Ri for i = 1, 2, 3 with maximal
||PU1 ⊗U2 ⊗U3 (T )||.
Relaxation method:
Optimize on U1 , U2 , U3 by fixing all variables except one at a time
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
13 / 18
(R1 , R2 , R3 )-rank approximation of 3-tensors
Fundamental problem in applications:
Approximate well and fast T ∈ Rm1 ×m2 ×m3 by rank (R1 , R2 , R3 )
3-tensor.
Best (R1 , R2 , R3 ) approximation problem:
Find Ui ⊂ Fmi of dimension Ri for i = 1, 2, 3 with maximal
||PU1 ⊗U2 ⊗U3 (T )||.
Relaxation method:
Optimize on U1 , U2 , U3 by fixing all variables except one at a time
This amounts to SVD (Singular Value Decomposition) of matrices:
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
13 / 18
(R1 , R2 , R3 )-rank approximation of 3-tensors
Fundamental problem in applications:
Approximate well and fast T ∈ Rm1 ×m2 ×m3 by rank (R1 , R2 , R3 )
3-tensor.
Best (R1 , R2 , R3 ) approximation problem:
Find Ui ⊂ Fmi of dimension Ri for i = 1, 2, 3 with maximal
||PU1 ⊗U2 ⊗U3 (T )||.
Relaxation method:
Optimize on U1 , U2 , U3 by fixing all variables except one at a time
This amounts to SVD (Singular Value Decomposition) of matrices:
Fix U2 , U3 . Then V = U1 ⊗ (U2 ⊗ U3 ) ⊂ Rm1 ×(m2 ·m3 )
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
13 / 18
(R1 , R2 , R3 )-rank approximation of 3-tensors
Fundamental problem in applications:
Approximate well and fast T ∈ Rm1 ×m2 ×m3 by rank (R1 , R2 , R3 )
3-tensor.
Best (R1 , R2 , R3 ) approximation problem:
Find Ui ⊂ Fmi of dimension Ri for i = 1, 2, 3 with maximal
||PU1 ⊗U2 ⊗U3 (T )||.
Relaxation method:
Optimize on U1 , U2 , U3 by fixing all variables except one at a time
This amounts to SVD (Singular Value Decomposition) of matrices:
Fix U2 , U3 . Then V = U1 ⊗ (U2 ⊗ U3 ) ⊂ Rm1 ×(m2 ·m3 )
maxU1 kPV (T )k is an approximation in 2-tensors=matrices
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
13 / 18
Fast low rank approximation I:
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
14 / 18
Fast low rank approximations II:
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
15 / 18
Fast low rank approximations II:
Approximate A ∈ Rm×n by CUR where C ∈ Rm×p , R ∈ Rq×n
for some submatrices of A.
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
15 / 18
Fast low rank approximations II:
Approximate A ∈ Rm×n by CUR where C ∈ Rm×p , R ∈ Rq×n
for some submatrices of A.
minU∈Cp×q kA − CURkF achieved for U = C † AR †
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
15 / 18
Fast low rank approximations II:
Approximate A ∈ Rm×n by CUR where C ∈ Rm×p , R ∈ Rq×n
for some submatrices of A.
minU∈Cp×q kA − CURkF achieved for U = C † AR †
Faster choice: U = A[I, J]†
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
15 / 18
Fast low rank approximations II:
Approximate A ∈ Rm×n by CUR where C ∈ Rm×p , R ∈ Rq×n
for some submatrices of A.
minU∈Cp×q kA − CURkF achieved for U = C † AR †
Faster choice: U = A[I, J]†
(corresponds to best CUR approximation on the entries read)
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
15 / 18
Fast low rank approximations II:
Approximate A ∈ Rm×n by CUR where C ∈ Rm×p , R ∈ Rq×n
for some submatrices of A.
minU∈Cp×q kA − CURkF achieved for U = C † AR †
Faster choice: U = A[I, J]†
(corresponds to best CUR approximation on the entries read)
For given A ∈ Rm×n×l , F ∈ Rm×p , E ∈ Rn×q , G ∈ Rl×r ,
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
15 / 18
Fast low rank approximations II:
Approximate A ∈ Rm×n by CUR where C ∈ Rm×p , R ∈ Rq×n
for some submatrices of A.
minU∈Cp×q kA − CURkF achieved for U = C † AR †
Faster choice: U = A[I, J]†
(corresponds to best CUR approximation on the entries read)
For given A ∈ Rm×n×l , F ∈ Rm×p , E ∈ Rn×q , G ∈ Rl×r ,
where hpi ⊂ hni × hli, hqi ⊂ hmi × hli, hr i ⊂ hmi × hli
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
15 / 18
Fast low rank approximations II:
Approximate A ∈ Rm×n by CUR where C ∈ Rm×p , R ∈ Rq×n
for some submatrices of A.
minU∈Cp×q kA − CURkF achieved for U = C † AR †
Faster choice: U = A[I, J]†
(corresponds to best CUR approximation on the entries read)
For given A ∈ Rm×n×l , F ∈ Rm×p , E ∈ Rn×q , G ∈ Rl×r ,
where hpi ⊂ hni × hli, hqi ⊂ hmi × hli, hr i ⊂ hmi × hli
minU ∈Cp×q×r kA − U × F × E × GkF achieved for U = A × E † × F † × G†
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
15 / 18
Fast low rank approximations II:
Approximate A ∈ Rm×n by CUR where C ∈ Rm×p , R ∈ Rq×n
for some submatrices of A.
minU∈Cp×q kA − CURkF achieved for U = C † AR †
Faster choice: U = A[I, J]†
(corresponds to best CUR approximation on the entries read)
For given A ∈ Rm×n×l , F ∈ Rm×p , E ∈ Rn×q , G ∈ Rl×r ,
where hpi ⊂ hni × hli, hqi ⊂ hmi × hli, hr i ⊂ hmi × hli
minU ∈Cp×q×r kA − U × F × E × GkF achieved for U = A × E † × F † × G†
CUR approximation of A obtained by choosing E, F , G submatrices of
unfolded A in the mode 1, 2, 3.
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
15 / 18
List of applications
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
16 / 18
List of applications
Face recognition
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
16 / 18
List of applications
Face recognition
Video tracking
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
16 / 18
List of applications
Face recognition
Video tracking
Factor analysis
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
16 / 18
References I
S. Friedland, On the generic rank of 3-tensors, arXiv:
0805.3777v2.
S. Friedland, V. Mehrmann, A. Miedlar, and M. Nkengla, Fast low
rank approximations of matrices and tensors, submitted,
www.matheon.de/preprints/4903.
S.A. Goreinov, E.E. Tyrtyshnikov, N.L. Zmarashkin,
Pseudo-skeleton approximations of matrices, Reports of the
Russian Academy of Sciences 343(2) (1995), 151-152.
S.A. Goreinov, E.E. Tyrtyshnikov, N.L. Zmarashkin, A theory of
pseudo-skeleton approximations of matrices, Linear Algebra Appl.
261 (1997), 1-21.
L.H. Lim, Singular values and eigenvalues of tensors: a variational
approach, CAMSAP 05, 1 (2005), 129-132.
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
17 / 18
References II
M.W. Mahoney and P. Drineas, CUR matrix decompositions for
improved data analysis, PNAS 106, (2009), 697-702.
Shmuel Friedland Univ. Illinois at Chicago ()
Tensors: theory and applications
TTI-C, March 2, 2009
18 / 18