Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
Eigenpairs of homogeneous polynomial systems
Paul Breiding
Institut für Mathematik - Technische Universität Berlin
Berkeley, December 12, 2015
1 Motivation
2 An adaptive linear homotopy method to approximate eigenpairs of
homogeneous polynomial systems
1 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
Eigenpairs of homogeneous polynomial systems
Definition
Let n, d ≥ 1 and Hn,d denote the space of homogeneous polynomials of
degree d in n variables.
We call (v, λ) ∈ (Cn \ {0}) × C an eigenpair of f = (f1 , . . . , fn ) ∈ (Hn,d )n , if
f (v) = λv.
In this case we call v eigenvector and λ eigenvalue.
Observe: ∀s ∈ C× : f (v) = λv ⇔ f (sv) = (sd−1 λ)sv.
We call (v, λ) and (sv, sd−1 λ) equivalent.
2 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
Eigenpairs of homogeneous polynomial systems
Definition
Let n, d ≥ 1 and Hn,d denote the space of homogeneous polynomials of
degree d in n variables.
We call (v, λ) ∈ (Cn \ {0}) × C an eigenpair of f = (f1 , . . . , fn ) ∈ (Hn,d )n , if
f (v) = λv.
In this case we call v eigenvector and λ eigenvalue.
Observe: ∀s ∈ C× : f (v) = λv ⇔ f (sv) = (sd−1 λ)sv.
We call (v, λ) and (sv, sd−1 λ) equivalent.
2 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
Motivation
3 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
Eigenpairs of tensors
1
Definition for tensors in [Qi, 2005]:
Let A = (ai1 ,...,id ), ai1 ,...,id ∈ R, be an order d tensor of format
n × n × . . . × n. The number λ ∈ C is called E-eigenvalue of A, if there
exists some v ∈ Cn \ {0} with
Av d−1 :=
n
X
i2 =1
2
...
n
X
id =1
aji2 ...id
n
Y
vij = λv,
v T v = 1.
j=2
Cartwright and Sturmfels [2013] allow complex tensors.
The theory of eigenvalues of tensors is called spectral theory of tensors.
4 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
Spectral theory of tensors - Applications
Many applications. Some examples are
Best rank-one approximation of symmetric tensors.
Waring decomposition of hom. polynomials.
Optimization.
Higher order Markov chains.
Diffusion tensor imaging/Magnetic resonance imaging.
For a detailed survey see The spectral theory of tensors and its applications
[Lim, Ng, and Qi, 2013].
5 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
Example: Eigenpairs in optimization
The following is due to [Lim, 2006].
Suppose that
F : Rn → R, X 7→ F (X)
is a hom. polynomial in n variables X = (X1 , . . . , Xn ) of degree d + 1.
To optimize F on the sphere {kxk = 1}, one can consider the Lagrangian of
F (X), that is
L(X, `) := F (X) − ` (kXk2 − 1).
where ` is an auxiliary variable. Then the equation ∇L = 0 gives
∇F (X) = 2`X,
kXk = 1.
6 hom. pol. system of degree d
in n variables and n equations
6 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
Example: Eigenpairs in optimization
The following is due to [Lim, 2006].
Suppose that
F : Rn → R, X 7→ F (X)
is a hom. polynomial in n variables X = (X1 , . . . , Xn ) of degree d + 1.
To optimize F on the sphere {kxk = 1}, one can consider the Lagrangian of
F (X), that is
L(X, `) := F (X) − ` (kXk2 − 1).
where ` is an auxiliary variable. Then the equation ∇L = 0 gives
∇F (X) = 2`X,
kXk = 1.
6 hom. pol. system of degree d
in n variables and n equations
6 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
An adaptive linear homotopy method to approximate
eigenpairs of homogeneous polynomial systems
The following are recent results from the paper An adaptive linear homotopy
method to approximate eigenpairs of homogeneous polynomial systems, 2015.
7 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
The homotopy method
Input: f ∈ (Hn+1,d )n and (g, ζ0 ) ∈ (Hn+1,d )n × Pn with g(ζ0 ) = 0 .
Output: An approximation of ζ1 ∈ Pn with f (ζ1 ) = 0.
The path from f to g is
discretized using a condition
number.
The zi are approximate zeros à
la Smale of the ζi .
(Projective Newton method
lets zi converge quadratically
towards ζi .)
See [Bürgisser and Cucker, 2013, p. 284] for the picture (here Y = Pn )
8 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
Homotopy method for matrix eigenpairs
Armentano et al. [2015] describe a homotopy method for the matrix
case (d = 1). See their paper A stable, polynomial-time algorithm for
the eigenpair problem.
In the following we assume d ≥ 2.
One can view equivalence classes of eigenpairs as elements in the
weighted projective space P(1, . . . , 1, d − 1). But we understand
P(Cn × C) much better!
9 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
Definition of h-eigenpairs
Definition
We call [ζ : η] ∈ P(Cn × C)\ {[0 : 1]} an h-eigenpair of f , if
f (ζ) = η d−1 ζ.
We define
F : (Hn,d )n → (Hn+1,d )n , f 7→ Ff ,
where
Ff (X, `) = f (X) − `d−1 X.
10 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
Why not use existing algorithms?
In their Bezout series (I-V) Shub and Smale present an adaptive
homotopy method for arbitrary polynomials in (Hn+1,d )n .
Paths between two systems f, g ∈ (Hn+1,d )n are parametrized using
the angle between f and g.
n
Problem: The space F(Hn,d
) ⊂ (Hn+1,d )n is an affine linear subspace.
n
Small angles in F(Hn,d
) may lead to large angles in (Hn+1,d )n .
n
F(Hn,d
)
0
n
Hn+1,d
11 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
The tool box for designing a homotopy method
There is a tool box to design homotopy methods.
(See for instance the book Condition by Bürgisser and Cucker.)
1
Define a Newton Operator.
We apply the (usual) projective Newton method to F (X, `).
2
Given two systems f, g define a path from Ff to Fg .
3
Define the condition number via the following procedure.
Define a solution map G : (Hn,d )n → P(Cn × C).
The condition number is defined as kDGk.
12 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
The geometry in the output space
Main problem: Newton’s method does not distinguish between
eigenvectors and eigenvalues and sees eigenpairs as points in P(Cn × C).
Applying Newton’s method close to the to the point [0 : 1] ∈ P(Cn × C)
yields the trivial solution f (0) = 1 · 0.
η
[0 : 1]
[ζbad : ηbad ]
[ζgood : ηgood ]
ζ
13 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
The geometry in the output space
Lemma
Let f ∈ (Hn,d )n with kf k = 1. Suppose that [ζ : η] ∈ P(Cn × C)\ {[0 : 1]} is
an h-eigenpair of f . Then |η| ≤ kζk .
η
[0 : 1]
[ζgood : ηgood ]
ζ
Proof of the lemma.
|η|d−1 kζk = η d−1 ζ = kf (ζ)k ≤ kf k kζkd .
14 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
The geometry in the output space
Lemma
Let f ∈ (Hn,d )n with kf k = 1. Suppose that [ζ : η] ∈ P(Cn × C)\ {[0 : 1]} is
an h-eigenpair of f . Then |η| ≤ kζk .
η
[0 : 1]
[ζgood : ηgood ]
ζ
Proof of the lemma.
|η|d−1 kζk = η d−1 ζ = kf (ζ)k ≤ kf k kζkd .
14 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
We need to construct a homotopy method that respects the
geometry of the h-eigenpair problem
Given two systems f, g ∈ S ((Hn,d )n ) we define Ef,g to be the geodesic
path in S ((Hn,d )n ) from f to g.
The path of our choice in the homotopy method is {Fq | q ∈ Ef,g }.
It remains to define a condition number to control the step size of the
homotopy method.
In fact, we will need two condition numbers.
15 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
Two solution maps
Definition
Let
V := { (f, [ζ : η]) ∈ (Hn,d )n × P(Cn × C)\ {[0 : 1]} | Ff (ζ, η) = 0 }
and
b := { (f, (ζ, η)) ∈ (Hn,d )n × (S(Cn ) × C) | Ff (ζ, η) = 0 } .
V
b satisfies the need of controlling the ratio
V
|η|
.
kζk
Newton’s method only works in V.
The two condition numbers come from the two possibilities to define the
ouput.
16 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
Two solution maps
b
V
V
π1
π2
(
(Hn,d )n
P(Cn × C)\ {[0 : 1]}
&
S(Cn ) × C
and
G
]
(Hn,d )n
G := π2 ◦ G.
]
[
π
b2
Generically, π1 and π
b1 are locally invertible. Denote
V
[
π
b1
(Hn,d )n
b
b := π
G
b2 ◦ G.
f
17 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
Two solution maps
b
V
V
π1
π2
(Hn,d )n
G
(
/ P(Cn × C)\ {[0 : 1]}
b
G
&
/ S(Cn ) × C
and
G
]
(Hn,d )n
G := π2 ◦ G.
]
[
π
b2
Generically, π1 and π
b1 are locally invertible. Denote
V
[
π
b1
(Hn,d )n
b
b := π
G
b2 ◦ G.
f
17 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
Two condition numbers
Definition
1
The condition number of the h-eigenpair problem with output
in P(Cn × C)\ {[0 : 1]} is
kDG(f )k
2
The condition number of the h-eigenpair problem with output
in S(Cn ) × C is
b DG(f ) .
The first condition number is used to set the step size in the algorithm.
The second one is used the analyze the algorithm.
18 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
Two condition numbers
Definition
1
The condition number of the h-eigenpair problem with output
in P(Cn × C)\ {[0 : 1]} is
kDG(f )k
2
The condition number of the h-eigenpair problem with output
in S(Cn ) × C is
b DG(f ) .
The first condition number is used to set the step size in the algorithm.
The second one is used the analyze the algorithm.
18 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
Main Theorem
Theorem (B., 2015)
1
There is an algorithm that on input f ∈ (Hn,d )n , kf k = 1, almost
surely returns an approximate zero [v : λ] ∈ P(Cn × C) of the system
Ff (X, `) = f (X) − `d−1 X.
The zero of Ff associated to [v : λ] is an h-eigenpair of f (and not the
trivial solution).
2
There is a method to choose the starting system randomly and this
method can be implemented efficiently. From this we get a randomized
algorithm. The average number of arithmetic operations of this
algorithm is
O(dnN + n5/2 N 2 ),
where N = dimC (Hn,d )n = n n+d−1
.
n−1
19 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
Ideas to prove the main theorem
1
2
n
As a starting system one can use the odeco tensor f (X) = Xid
i=1
and one if its eigenpairs, see Robeva [2014].
The sampling method is an adaption from an idea found in
A stable, polynomial-time algorithm for the eigenpair problem by
Armentano et al. [2015].
For the probabilistic analysis we use results from our paper
Distribution of the eigenvalues of a random system of homogeneous
polynomials, 2015. This is joint work with Peter Bürgisser.
20 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
Ideas to prove the main theorem
1
2
n
As a starting system one can use the odeco tensor f (X) = Xid
i=1
and one if its eigenpairs, see Robeva [2014].
The sampling method is an adaption from an idea found in
A stable, polynomial-time algorithm for the eigenpair problem by
Armentano et al. [2015].
For the probabilistic analysis we use results from our paper
Distribution of the eigenvalues of a random system of homogeneous
polynomials, 2015. This is joint work with Peter Bürgisser.
20 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
Open questions/Further work
Provide an analysis for the non-randomized algorithm.
Provide a smoothed analysis for the algorithm.
Provide an analysis for structured systems (gradients for example).
21 / 22
Motivation An adaptive linear homotopy method to approximate eigenpairs of homogeneous polynomial systems Refe
References
D. Armentano, C. Beltrán, P. Bürgisser, F. Cucker, and M. Shub. A stable,
polynomial-time algorithm for the eigenpair problem. ArXiv e-print
1505.03290, May 2015.
P. Breiding. An adaptive linear homotopy method to approximate eigenpairs of
homogeneous polynomial systems. ArXiv e-prints, December 2015.
Paul Breiding and Peter Bürgisser. Distribution of the eigenvalues of a random
system of homogeneous polynomials. ArXiv e-print 1507.02539, July 2015.
Peter Bürgisser and Felipe Cucker. Condition, volume 349 of Grundlehren der
Mathematischen Wissenschaften. Springer, Heidelberg, 2013. ISBN
978-3-642-38895-8; 978-3-642-38896-5.
Dustin Cartwright and Bernd Sturmfels. The number of eigenvalues of a tensor.
Linear Algebra Appl., 438(2):942–952, 2013. ISSN 0024-3795.
L.-H. Lim. Singular Values and Eigenvalues of Tensors: A Variational Approach.
ArXiv Mathematics e-print 0607648, July 2006.
Lek-Heng Lim, Michael K. Ng, and Liqun Qi. The spectral theory of tensors and
its applications. Numer. Linear Algebra Appl., 20(6):889–890, 2013. ISSN
1070-5325.
Liqun Qi. Eigenvalues of a real supersymmetric tensor. J. Symbolic Comput., 40
(6):1302–1324, 2005. ISSN 0747-7171.
E. Robeva. Orthogonal Decomposition of Symmetric Tensors. ArXiv e-print
1409.6685, September 2014.
22 / 22