Optimising polynomials over a simplex: PTAS proofs
revisited
Etienne de Klerk, Monique Laurent, and Zhao Sun
Nanyang Technological University, Singapore (On leave from Tilburg University, The Netherlands)
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
1 / 15
Introduction
Outline
Polynomial optimisation over a simplex: example and complexity;
Known PTAS approximation results;
New PTAS proofs via Bernstein approximation.
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
2 / 15
Introduction
Polynomial optimization on a simplex
Standard simplex in Rn
(
∆n =
)
n
x ∈R |
X
xi = 1, ∀ i : xi ≥ 0 .
i
Given f ∈ Hn,d (homogeneous n-variate polynomial of degree d), we consider the
following.
Optimization problems:
Find:
f = min f (x),
x∈∆n
Etienne de Klerk (NTU)
f = max f (x).
x∈∆n
Optimising polynomials over a simplex
3 / 15
Introduction
Polynomial optimization on a simplex
Standard simplex in Rn
(
∆n =
)
n
x ∈R |
X
xi = 1, ∀ i : xi ≥ 0 .
i
Given f ∈ Hn,d (homogeneous n-variate polynomial of degree d), we consider the
following.
Optimization problems:
Find:
f = min f (x),
x∈∆n
f = max f (x).
x∈∆n
Famous example for quadratic f : computing the stability number of a graph ...
(next slide)
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
3 / 15
Introduction
Stable sets
The stability number
Given a graph G = (V , E ), a stable set is a subset of V of pairwise non-adjacent
vertices.
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
4 / 15
Introduction
Stable sets
The stability number
Given a graph G = (V , E ), a stable set is a subset of V of pairwise non-adjacent
vertices. The cardinality of a maximum stable set is called the stability number
and denoted by α(G ).
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
4 / 15
Introduction
Stable sets
The stability number
Given a graph G = (V , E ), a stable set is a subset of V of pairwise non-adjacent
vertices. The cardinality of a maximum stable set is called the stability number
and denoted by α(G ).
For the Petersen graph, α(G ) = 4.
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
4 / 15
Introduction
The Motzkin-Straus theorem
Theorem (Motzkin-Straus ’65)
1
= min x T (A + I )x,
α(G ) x∈∆V
where A is the adjacency matrix of G = (V , E ) and I the identity matrix.
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
5 / 15
Introduction
The Motzkin-Straus theorem
Theorem (Motzkin-Straus ’65)
1
= min x T (A + I )x,
α(G ) x∈∆V
where A is the adjacency matrix of G = (V , E ) and I the identity matrix.
Theorem (Håstad (2001))
Unless NP=ZPP, one cannot approximate α(G ) to within a factor |V |(1−) in
polynomial time for any > 0.
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
5 / 15
Introduction
The Motzkin-Straus theorem
Theorem (Motzkin-Straus ’65)
1
= min x T (A + I )x,
α(G ) x∈∆V
where A is the adjacency matrix of G = (V , E ) and I the identity matrix.
Theorem (Håstad (2001))
Unless NP=ZPP, one cannot approximate α(G ) to within a factor |V |(1−) in
polynomial time for any > 0.
Corollary:
Computing f := minx∈∆ f (x) is NP-hard, even for quadratic f , and allows no
FPTAS, unless NP=ZPP.
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
5 / 15
Introduction
Approximation results
Notation
Given an integer r ≥ 0, denote
∆(n, r ) = {x ∈ ∆n | rx ∈ Nn0 }
and
f∆(n,r ) =
Etienne de Klerk (NTU)
min f (x).
x∈∆(n,r )
Optimising polynomials over a simplex
6 / 15
Introduction
Approximation results
Notation
Given an integer r ≥ 0, denote
∆(n, r ) = {x ∈ ∆n | rx ∈ Nn0 }
and
f∆(n,r ) =
min f (x).
x∈∆(n,r )
Known PTAS results:
Theorem (Bomze-De Klerk (2002), De Klerk-Laurent-Parrilo (2006))
If f quadratic
f∆(n,r ) − f ≤
Etienne de Klerk (NTU)
1
f −f .
r
Optimising polynomials over a simplex
6 / 15
Introduction
Approximation results
Notation
Given an integer r ≥ 0, denote
∆(n, r ) = {x ∈ ∆n | rx ∈ Nn0 }
and
f∆(n,r ) =
min f (x).
x∈∆(n,r )
Known PTAS results:
Theorem (Bomze-De Klerk (2002), De Klerk-Laurent-Parrilo (2006))
If f quadratic
f∆(n,r ) − f ≤
1
f −f .
r
If f ∈ Hn,d :
f∆(n,r ) − f ≤
Etienne de Klerk (NTU)
1
r +d
dd
2d − 1 d
f −f .
d
2
Optimising polynomials over a simplex
6 / 15
Introduction
PTAS results (ctd)
The PTAS proofs are essentially by examination of the proof of Pólya’s theorem:
Theorem (Pólya (1928))
If f ∈ Hn,d and f (x) > 0 for all x ∈ ∆n , then
f (x)
n
X
!r
xi
i=1
has nonnegative coefficients for some r ∈ N0 .
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
7 / 15
Introduction
PTAS results (ctd)
The PTAS proofs are essentially by examination of the proof of Pólya’s theorem:
Theorem (Pólya (1928))
If f ∈ Hn,d and f (x) > 0 for all x ∈ ∆n , then
f (x)
n
X
!r
xi
i=1
has nonnegative coefficients for some r ∈ N0 .
Background reading
V. Powers and B. Reznick. A new bound for Pólyas theorem with applications to polynomials
positive on polyhedra. Journal of Pure and Applied Algebra, 164:221–229, 2001.
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
7 / 15
Introduction
PTAS results (ctd)
The PTAS proofs are essentially by examination of the proof of Pólya’s theorem:
Theorem (Pólya (1928))
If f ∈ Hn,d and f (x) > 0 for all x ∈ ∆n , then
f (x)
n
X
!r
xi
i=1
has nonnegative coefficients for some r ∈ N0 .
Background reading
V. Powers and B. Reznick. A new bound for Pólyas theorem with applications to polynomials
positive on polyhedra. Journal of Pure and Applied Algebra, 164:221–229, 2001.
One can also do PTAS proofs via Bernstein approximation ... (next slide)
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
7 / 15
Introduction
Bernstein approximation
Definition:
I (n, r ) = {α ∈ Nn0 : |α| = r } = r ∆(n, r ).
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
8 / 15
Introduction
Bernstein approximation
Definition:
I (n, r ) = {α ∈ Nn0 : |α| = r } = r ∆(n, r ).
Bernstein approximation of f of order r ∈ N:
α r!
X
Br (f )(x) =
f
x α.
r α!
α∈I (n,r )
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
8 / 15
Introduction
Bernstein approximation
Definition:
I (n, r ) = {α ∈ Nn0 : |α| = r } = r ∆(n, r ).
Bernstein approximation of f of order r ∈ N:
α r!
X
Br (f )(x) =
f
x α.
r α!
α∈I (n,r )
Lemma
min Br (f )(x) ≥ f∆(n,r ) ≡
x∈∆n
min f (x).
x∈∆(n,r )
Follows directly from the multinomial theorem:
!r
n
X
X r!
xi
=
x α (= 1 if x ∈ ∆n ).
α!
i=1
Etienne de Klerk (NTU)
α∈I (n,r )
Optimising polynomials over a simplex
8 / 15
Introduction
Bernstein approximation and multinomial moments
Let β ∈ I (n, d) and consider the monomial
φβ (x) = x β ≡ x1β1 · · · xnβn .
We want to analyse its Bernstein approximation:
Br (φβ )(x) =
X
α∈I (n,r )
Etienne de Klerk (NTU)
r!
(α/r )β x α .
α!
Optimising polynomials over a simplex
9 / 15
Introduction
Bernstein approximation and multinomial moments
Let β ∈ I (n, d) and consider the monomial
φβ (x) = x β ≡ x1β1 · · · xnβn .
We want to analyse its Bernstein approximation:
Br (φβ )(x) =
X
α∈I (n,r )
r!
(α/r )β x α .
α!
Multinomial distribution
Consider a loaded dice with n sides, and assume the probability of seeing i is xi .
The probability of observing an outcome α ∈ I (n, r ) after rolling the dice r times
r! α
is α!
x ,
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
9 / 15
Introduction
Bernstein approximation and multinomial moments
Let β ∈ I (n, d) and consider the monomial
φβ (x) = x β ≡ x1β1 · · · xnβn .
We want to analyse its Bernstein approximation:
Br (φβ )(x) =
X
α∈I (n,r )
r!
(α/r )β x α .
α!
Multinomial distribution
Consider a loaded dice with n sides, and assume the probability of seeing i is xi .
The probability of observing an outcome α ∈ I (n, r ) after rolling the dice r times
r! α
is α!
x , and the moment of order β ∈ I (n, d) is therefore
X
α∈I (n,r )
Etienne de Klerk (NTU)
r!
(α)β x α = r d Br (φβ )(x).
α!
Optimising polynomials over a simplex
9 / 15
Introduction
Multinomial moments
The moment generating function of the multinomial distribution is:
!r
n
X
ti
mgf (t) :=
xi e
i=1
so that
Etienne de Klerk (NTU)
|β|
∂
mgf
(t)
r d Br (φβ )(x) =
β1
βn ∂t1 · · · ∂tn
Optimising polynomials over a simplex
.
t=0
10 / 15
Introduction
Multinomial moments
The moment generating function of the multinomial distribution is:
!r
n
X
ti
mgf (t) :=
xi e
i=1
so that
|β|
∂
mgf
(t)
r d Br (φβ )(x) =
β1
βn ∂t1 · · · ∂tn
.
t=0
Corollary
n
1 X |α| α Y
Br (φβ )(x) = d
r x
S (βi , αi ) ,
r
0≤α≤β
i=1
where S (βi , αi ) is the Stirling number of the second kind,
r |α| = r (r − 1) · · · (r − |α| + 1).
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
10 / 15
Introduction
Example: PTAS for quadratic f
φ(x) = xi xj
φ(x) = xi2
r −1
xi xj
r
1
=⇒ Br (φ)(x) = xi (1 − xi ) + xi2
r
=⇒ Br (φ)(x) =
Bernstein approximation of quadratic f
Pn
If f = i,j=1 Qij xi xj :
Br (f )(x) =
n
1X
1
Qii xi + 1 −
f (x).
r
r
i=1
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
11 / 15
Introduction
Example: PTAS for quadratic f (ctd.)
Theorem (Bomze-De Klerk (2002))
If f quadratic
f∆(n,r ) − f ≤
Etienne de Klerk (NTU)
1
f −f .
r
Optimising polynomials over a simplex
12 / 15
Introduction
Example: PTAS for quadratic f (ctd.)
Theorem (Bomze-De Klerk (2002))
If f quadratic
f∆(n,r ) − f ≤
1
f −f .
r
Proof:
min Br (f )(x)
x∈∆n
Etienne de Klerk (NTU)
=
min
x∈∆n
!
n
1X
1
Qii xi + 1 −
f (x)
r
r
i=1
Optimising polynomials over a simplex
12 / 15
Introduction
Example: PTAS for quadratic f (ctd.)
Theorem (Bomze-De Klerk (2002))
If f quadratic
f∆(n,r ) − f ≤
1
f −f .
r
Proof:
!
n
1X
1
min Br (f )(x) = min
Qii xi + 1 −
f (x)
x∈∆n
x∈∆n
r
r
i=1
n
X
1
1
≤
max
Qii xi + min 1 −
f (x)
x∈∆n
r x∈∆n
r
i=1
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
12 / 15
Introduction
Example: PTAS for quadratic f (ctd.)
Theorem (Bomze-De Klerk (2002))
If f quadratic
f∆(n,r ) − f ≤
1
f −f .
r
Proof:
!
n
1X
1
min Br (f )(x) = min
Qii xi + 1 −
f (x)
x∈∆n
x∈∆n
r
r
i=1
n
X
1
1
≤
max
Qii xi + min 1 −
f (x)
x∈∆n
r x∈∆n
r
i=1
1
1
=
max Qii + 1 −
f
r i
r
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
12 / 15
Introduction
Example: PTAS for quadratic f (ctd.)
Theorem (Bomze-De Klerk (2002))
If f quadratic
f∆(n,r ) − f ≤
1
f −f .
r
Proof:
min Br (f )(x)
x∈∆n
=
≤
=
≤
Etienne de Klerk (NTU)
!
n
1X
1
min
Qii xi + 1 −
f (x)
x∈∆n
r
r
i=1
n
X
1
1
max
Qii xi + min 1 −
f (x)
x∈∆n
r x∈∆n
r
i=1
1
1
max Qii + 1 −
f
r i
r
1
1
f + 1−
f.
r
r
Optimising polynomials over a simplex
12 / 15
Introduction
Example: PTAS for quadratic f (ctd.)
Theorem (Bomze-De Klerk (2002))
If f quadratic
f∆(n,r ) − f ≤
1
f −f .
r
Proof:
min Br (f )(x)
x∈∆n
=
≤
=
≤
!
n
1X
1
min
Qii xi + 1 −
f (x)
x∈∆n
r
r
i=1
n
X
1
1
max
Qii xi + min 1 −
f (x)
x∈∆n
r x∈∆n
r
i=1
1
1
max Qii + 1 −
f
r i
r
1
1
f + 1−
f.
r
r
Now use f∆(n,r ) ≤ minx∈∆n Br (f )(x).
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
12 / 15
Introduction
Small improvements
Theorem
If f ∈ H(n, 3), then for any integer r ≥ 3,
min Br (f )(x) − f ≤
x∈∆n
Etienne de Klerk (NTU)
4
4
− 2
r
r
Optimising polynomials over a simplex
(f − f ).
13 / 15
Introduction
Small improvements
Theorem
If f ∈ H(n, 3), then for any integer r ≥ 3,
min Br (f )(x) − f ≤
x∈∆n
4
4
− 2
r
r
(f − f ).
This is a small improvement on DK-Laurent-Parrilo (2006).
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
13 / 15
Introduction
Small improvements
Theorem
If f ∈ H(n, 3), then for any integer r ≥ 3,
min Br (f )(x) − f ≤
x∈∆n
4
4
− 2
r
r
(f − f ).
This is a small improvement on DK-Laurent-Parrilo (2006).
Theorem
If f ∈ H(n, d) is square free, then
rd
f∆(n,r ) − f ≤ 1 − d (f − f ) for any r ≥ d.
r
This is a small improvement on Nesterov (2003) and DK-Laurent-Parrilo (2006).
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
13 / 15
Introduction
Conclusion
We presented a short proof the PTAS for quadratic optimization over the
simplex via Bernstein approximation ...
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
14 / 15
Introduction
Conclusion
We presented a short proof the PTAS for quadratic optimization over the
simplex via Bernstein approximation ...
... and refined some error bounds for the degree 3 and square-free cases.
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
14 / 15
Introduction
Conclusion
We presented a short proof the PTAS for quadratic optimization over the
simplex via Bernstein approximation ...
... and refined some error bounds for the degree 3 and square-free cases.
Long term project: error bounds for approximation hierarchies via
constructive approximation, as started in:
E. de Klerk and M. Laurent. Error bounds for some semidefinite programming approaches to
polynomial minimization on the hypercube. SIAM Journal on Optimization, 20(6), 3104-3120,
2010.
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
14 / 15
Introduction
The End
THANK YOU!
(especially to Adam, Joerg, Markus, and Jean-Bernard)
Etienne de Klerk (NTU)
Optimising polynomials over a simplex
15 / 15