The Expressive Power of Binary
Submodular Functions
S. Živný, D. Cohen, P. Jeavons
12 May 2009, BAD, Bristol
Problem
Which submodular polynomials can be
expressed by (or decomposed into) quadratic
submodular polynomials?
Problem
Which submodular polynomials can be
expressed by (or decomposed into) quadratic
submodular polynomials?
Not all.
Problem
Which submodular polynomials can be
expressed by (or decomposed into) quadratic
submodular polynomials?
Not all.
New big class.
Problem
Which submodular polynomials can be
expressed by (or decomposed into) quadratic
submodular polynomials?
Not all.
New big class.
Degree-4 polynomials.
Motivation
Problem considered in:
I
I
I
artificial intelligence
computer vision
optimisation
Importance
Quadratic submodular polynomials
=
minimisation via (s, t)-Min-Cut.
Minimisation of submodular polynomials
I
PTIME
[ellipsoid method, GLS’81]
Minimisation of submodular polynomials
I
I
PTIME
combinatorial algorithm
[ellipsoid method, GLS’81]
[Schrijver’00, FFI’01]
Minimisation of submodular polynomials
I
I
I
PTIME
combinatorial algorithm
degree k in O(n6 + n5 L)
[ellipsoid method, GLS’81]
[Schrijver’00, FFI’01]
[Orlin ’09]
Minimisation of submodular polynomials
I
I
I
I
PTIME
[ellipsoid method, GLS’81]
combinatorial algorithm
[Schrijver’00, FFI’01]
6
5
degree k in O(n + n L)
[Orlin ’09]
3
quadratic in O(n ) using (s, t)-Min-Cut [Hammer ’65]
Minimisation of submodular polynomials
I
I
I
I
I
PTIME
[ellipsoid method, GLS’81]
combinatorial algorithm
[Schrijver’00, FFI’01]
6
5
degree k in O(n + n L)
[Orlin ’09]
3
quadratic in O(n ) using (s, t)-Min-Cut [Hammer ’65]
cubic expressible by quadratic [Billionet, Minoux ’85]
Minimisation of submodular polynomials
I
I
I
I
I
I
PTIME
[ellipsoid method, GLS’81]
combinatorial algorithm
[Schrijver’00, FFI’01]
6
5
degree k in O(n + n L)
[Orlin ’09]
3
quadratic in O(n ) using (s, t)-Min-Cut [Hammer ’65]
cubic expressible by quadratic [Billionet, Minoux ’85]
0/1-valued
[Creignou, Khanna, Sudan ’01]
Minimisation of submodular polynomials
I
I
I
I
I
I
I
PTIME
[ellipsoid method, GLS’81]
combinatorial algorithm
[Schrijver’00, FFI’01]
6
5
degree k in O(n + n L)
[Orlin ’09]
3
quadratic in O(n ) using (s, t)-Min-Cut [Hammer ’65]
cubic expressible by quadratic [Billionet, Minoux ’85]
0/1-valued
[Creignou, Khanna, Sudan ’01]
other non-Boolean results
Problem
Which submodular polynomials can be
expressed by quadratic submodular
polynomials?
Problem
Which submodular polynomials can be
expressed by quadratic submodular
polynomials?
pseudo-Boolean: {0, 1}, coefficients in R
Problem
Which submodular polynomials can be
expressed by quadratic submodular
polynomials?
Submodularity
I
key concept in combinatorial optimisation
Submodularity
I
I
key concept in combinatorial optimisation
discrete analogue of convexity
Submodularity
I
I
I
key concept in combinatorial optimisation
discrete analogue of convexity
given finite V and ψ : 2V → R,
Submodularity
I
I
I
key concept in combinatorial optimisation
discrete analogue of convexity
given finite V and ψ : 2V → R,
ψ(A ∪ B) + ψ(A ∩ B) ≤ ψ(A) + ψ(B) for all A, B ⊆ V
Submodularity
I
I
I
I
key concept in combinatorial optimisation
discrete analogue of convexity
given finite V and ψ : 2V → R,
ψ(A ∪ B) + ψ(A ∩ B) ≤ ψ(A) + ψ(B) for all A, B ⊆ V
examples: cut fns, matroid rank fns, entropy fns
Submodularity
I
I
I
I
I
key concept in combinatorial optimisation
discrete analogue of convexity
given finite V and ψ : 2V → R,
ψ(A ∪ B) + ψ(A ∩ B) ≤ ψ(A) + ψ(B) for all A, B ⊆ V
examples: cut fns, matroid rank fns, entropy fns
D IGRAPH M IN -C OST-H OM: min-max ordering
Submodularity
I
I
I
I
I
I
key concept in combinatorial optimisation
discrete analogue of convexity
given finite V and ψ : 2V → R,
ψ(A ∪ B) + ψ(A ∩ B) ≤ ψ(A) + ψ(B) for all A, B ⊆ V
examples: cut fns, matroid rank fns, entropy fns
D IGRAPH M IN -C OST-H OM: min-max ordering
VCSP, M AX -CSP: tractability
Submodularity
p(x1 , . . . , xn ) = a0 +
n
X
i=1
ai x i +
X
1≤i<j≤n
aij xi xj
Submodularity
p(x1 , . . . , xn ) = a0 +
n
X
i=1
ai x i +
X
aij xi xj
1≤i<j≤n
p is submodular ⇔ ∀i, j : aij ≤ 0
Submodularity
p(x1 , . . . , xn ) = a0 +
n
X
i=1
ai x i +
X
aij xi xj
1≤i<j≤n
p is submodular ⇔ ∀i, j : aij ≤ 0
Recognition easy.
Submodularity, cont’d
p(x1 , . . . , pn ) =a0 +
n
X
ai x i +
i=1
X
1≤i<j<k≤n
X
aij xi xj +
1≤i<j≤n
aijk xi xj xk + . . . +
X
|I|=k
aI x I
Submodularity, cont’d
p(x1 , . . . , pn ) =a0 +
n
X
ai x i +
i=1
X
1≤i<j<k≤n
X
aij xi xj +
1≤i<j≤n
aijk xi xj xk + . . . +
X
|I|=k
p is submodular ⇔ ∀i, j : δij (x) ≤ 0
aI x I
Submodularity, cont’d
p(x1 , . . . , pn ) =a0 +
n
X
ai x i +
i=1
X
X
aij xi xj +
1≤i<j≤n
aijk xi xj xk + . . . +
1≤i<j<k≤n
X
|I|=k
p is submodular ⇔ ∀i, j : δij (x) ≤ 0
δ1 (x) = p(1, x) − p(0, x)
aI x I
Submodularity, cont’d
p(x1 , . . . , pn ) =a0 +
n
X
ai x i +
i=1
X
1≤i<j<k≤n
X
aij xi xj +
1≤i<j≤n
aijk xi xj xk + . . . +
X
aI x I
|I|=k
p is submodular ⇔ ∀i, j : δij (x) ≤ 0
δ1,2 (x) = p(1, 1, x) − p(1, 0, x) − p(0, 1, x) + p(0, 0, x)
Submodularity, cont’d
p(x1 , . . . , pn ) =a0 +
n
X
ai x i +
i=1
X
1≤i<j<k≤n
X
aij xi xj +
1≤i<j≤n
aijk xi xj xk + . . . +
X
|I|=k
p is submodular ⇔ ∀i, j : δij (x) ≤ 0
p(1, 1, x) + p(0, 0, x) ≤ p(0, 1, x) + p(1, 0, x)
aI x I
Submodularity, cont’d
p(x1 , . . . , pn ) =a0 +
n
X
ai x i +
i=1
X
X
aij xi xj +
1≤i<j≤n
aijk xi xj xk + . . . +
1≤i<j<k≤n
X
|I|=k
p is submodular ⇔ ∀i, j : δij (x) ≤ 0
Recognition hard even for k=4.
aI x I
Problem
Which submodular polynomials can be
expressed by quadratic submodular
polynomials?
Problem
Which submodular polynomials can be
expressed by quadratic submodular
polynomials?
Expressibility
p ∈ hLi ⇔ p(x) = min
z
X
i
pi (x, z) + c (pi ∈ L, c ∈ R)
Example 1
Let
p(x1 , . . . , x100 ) = −x2 x5 x6 x9 .
Example 1
Let
p(x1 , . . . , x100 ) = −x2 x5 x6 x9 .
Then,
p(x1 , . . . , x100 ) = min y(3 − x2 − x5 − x6 − x9 ).
y∈{0,1}
Example 1, more generally
Let I ⊆ [n], |I| = k, and
Y
p(x1 , . . . , xn ) = −
xi .
i∈I
Example 1, more generally
Let I ⊆ [n], |I| = k, and
Y
p(x1 , . . . , xn ) = −
xi .
i∈I
Then,
p(x1 , . . . , xn ) = min y(k − 1 −
y∈{0,1}
X
i∈I
xi ).
Example 2
p(x1 , x2 , x3 , x4 ) = x1 x2 x3 x4 − x1 x2 − x1 x3 x4 − x2 x3 x4
Example 2
p(x1 , x2 , x3 , x4 ) = x1 x2 x3 x4 − x1 x2 − x1 x3 x4 − x2 x3 x4
p(x1 , x2 , x3 , x4 ) = min (y1 + 2y2 − y1 y2
y1 ,y2
− y1 x 1 − y1 x 2 − y2 x 3 − y2 x 4 )
Example 2
p(x1 , x2 , x3 , x4 ) = x1 x2 x3 x4 − x1 x2 − x1 x3 x4 − x2 x3 x4
p(x1 , x2 , x3 , x4 ) = min (y1 + 2y2 − y1 y2
y1 ,y2
− y1 x 1 − y1 x 2 − y2 x 3 − y2 x 4 )
2 extra variables, 1 is provably not enough
Example 3
p(x1 , x2 , x3 , x4 ) = − x1 x2 x3 x4 + x1 x3 x4 + x2 x3 x4
− x1 x3 − x1 x4 − x2 x3 − x2 x4 − x3 x4
Example 3
p(x1 , x2 , x3 , x4 ) = − x1 x2 x3 x4 + x1 x3 x4 + x2 x3 x4
− x1 x3 − x1 x4 − x2 x3 − x2 x4 − x3 x4
Not expressible with any number of extra variables!
More on expressibility
Theorem [Cooper, Cohen, Jeavons ’06]
1. p ∈ hLi ⇔ fPol(L) ⊆ fPol({p})
k
2. p ∈ hLi ⇒ p ∈ hLi with at most 22 extra variables
More on expressibility
Theorem [Cooper, Cohen, Jeavons ’06]
1. p ∈ hLi ⇔ fPol(L) ⊆ fPol({p})
k
2. p ∈ hLi ⇒ p ∈ hLi with at most 22 extra variables
fPol = weighted Mul
More on expressibility
Theorem [Cooper, Cohen, Jeavons ’06]
1. p ∈ hLi ⇔ fPol(L) ⊆ fPol({p})
k
2. p ∈ hLi ⇒ p ∈ hLi with at most 22 extra variables
fPol = weighted Mul
Mul = mappings satisfying many linear inequalities
More on expressibility
Theorem [Cooper, Cohen, Jeavons ’06]
1. p ∈ hLi ⇔ fPol(L) ⊆ fPol({p})
k
2. p ∈ hLi ⇒ p ∈ hLi with at most 22 extra variables
fPol = weighted Mul
Mul = mappings satisfying many linear inequalities
Mul = Multimorphisms: {0, 1}k → {0, 1}k
Notation
Subk
Sub
= submodular polynomials of degree k
= all submodular polynomials
Original plan
fPol(Sub)
=
fPol(Sub2 )
Original plan
= fPol(Sub2 )
⇓
Sub = hSub2 i
fPol(Sub)
Original plan
= fPol(Sub2 )
⇓
Sub = hSub2 i
⇓
World is nice.
fPol(Sub)
Original plan
6= fPol(Sub2 )
⇓
Sub = hSub2 i
⇓
World is still nice.
fPol(Sub)
Result on Sub4
Theorem 1 [Ž., Cohen, Jeavons ’08]
Let p ∈ Sub4 . Then the following are equivalent:
1. p ∈ hSub2 i
2. ∀{i, j}, {k, l} ⊆ [n] distinct : aij + akl + aijk + aijl ≤ 0
Result on Sub4
Theorem 1 [Ž., Cohen, Jeavons ’08]
Let p ∈ Sub4 . Then the following are equivalent:
1. p ∈ hSub2 i
2. ∀{i, j}, {k, l} ⊆ [n] distinct : aij + akl + aijk + aijl ≤ 0
3. p ∈ Cone(Fans4 )
Result on Sub4
Theorem 1 [Ž., Cohen, Jeavons ’08]
Let p ∈ Sub4 . Then the following are equivalent:
1. p ∈ hSub2 i
2. ∀{i, j}, {k, l} ⊆ [n] distinct : aij + akl + aijk + aijl ≤ 0
3. p ∈ Cone(Fans4 )
4. Fsep ∈ Mul({p})
Proof
I
(3) ⇒ (1) : p ∈ Cone(Fans4 ) ⇒ p ∈ hSub2 i
Theorem 2: All Fans are expressible.
Proof
I
(3) ⇒ (1) : p ∈ Cone(Fans4 ) ⇒ p ∈ hSub2 i
Theorem 2: All Fans are expressible.
I
(1) ⇒ (4) : p ∈ hSub2 i ⇒ Fsep ∈ Mul({p})
Theorem 3: Characterisation of Mul(Sub2 ).
Proof
I
(3) ⇒ (1) : p ∈ Cone(Fans4 ) ⇒ p ∈ hSub2 i
Theorem 2: All Fans are expressible.
I
(1) ⇒ (4) : p ∈ hSub2 i ⇒ Fsep ∈ Mul({p})
Theorem 3: Characterisation of Mul(Sub2 ).
I
(4) ⇒ (3) : Fsep ∈ Mul({p}) ⇒ p ∈ Cone(Fans4 )
The same polyhedra.
Proof
I
(3) ⇒ (1) : p ∈ Cone(Fans4 ) ⇒ p ∈ hSub2 i
Theorem 2: All Fans are expressible.
I
(1) ⇒ (4) : p ∈ hSub2 i ⇒ Fsep ∈ Mul({p})
Theorem 3: Characterisation of Mul(Sub2 ).
I
(4) ⇒ (3) : Fsep ∈ Mul({p}) ⇒ p ∈ Cone(Fans4 )
The same polyhedra.
I
(4) ⇔ (2) : Fsep ∈ Mul({p}) ⇔ aij + akl + aijk + aijl ≤ 4
Translation of different representations.
Non-expressibility over Sub4
Essentially one reason:
Non-expressibility over Sub4
Essentially one reason:
qin12


−1 0000, 1111
= +1 1100


0 o/w
Expressibility of Fans
Theorem 2 [Ž., Cohen, Jeavons ’08]
Fans of all arities are expressible over Sub2 .
Expressibility of Fans
Theorem 2 [Ž., Cohen, Jeavons ’08]
Fans of all arities are expressible over Sub2 .
Generalises previous expressibility results.
Expressibility of Fans
Theorem 2 [Ž., Cohen, Jeavons ’08]
Fans of all arities are expressible over Sub2 .
Generalises previous expressibility results.
Arity k fan expressible with O(k) extra variables.
Expressibility of Fans
Theorem 2 [Ž., Cohen, Jeavons ’08]
Fans of all arities are expressible over Sub2 .
Generalises previous expressibility results.
Arity k fan expressible with O(k) extra variables.
Optimal number of variables.
Characterisation of Mul(Sub2)
Theorem 3 [Ž., Cohen, Jeavons ’08]
F ∈ Mul(Sub2 ) ⇔
F is conservative Hamming distance non-increasing.
Characterisation of Mul(Sub2)
Theorem 3 [Ž., Cohen, Jeavons ’08]
F ∈ Mul(Sub2 ) ⇔
F is conservative Hamming distance non-increasing.
Helped to identify Fsep .
Recognition
Recognition p ∈ hSub2 i easy assuming p ∈ Sub4 .
Recognition
Recognition p ∈ hSub2 i easy assuming p ∈ Sub4 .
Recognition Sub4 co-NP-C. [Gallo, Simeone; Crama ’88]
Recognition
Recognition p ∈ hSub2 i easy assuming p ∈ Sub4 .
Recognition Sub4 co-NP-C. [Gallo, Simeone; Crama ’88]
Theorem 4 [Ž., Cohen, Jeavons ’08]
1. Testing p ∈ Sub4 ∩ hSub2 i is co-NP-C.
Recognition
Recognition p ∈ hSub2 i easy assuming p ∈ Sub4 .
Recognition Sub4 co-NP-C. [Gallo, Simeone; Crama ’88]
Theorem 4 [Ž., Cohen, Jeavons ’08]
1. Testing p ∈ Sub4 ∩ hSub2 i is co-NP-C.
2. Not all extreme rays are Fans (Promislow & Young).
Submodular polynomials
Submodular polynomials
Expressible
Submodular polynomials
Expressible
Submodular polynomials
?
Expressible
Non-expressible
Submodular polynomials
?
Expressible
Non-expressible
Submodular polynomials
CLASSIFICATION
Expressible
Non-expressible
Submodular polynomials
CLASSIFICATION
Expressible
ALGORITHM
Non-expressible
Submodular polynomials
CLASSIFICATION
Expressible
ALGORITHM
Non-expressible
RECOGNITION
Submodular polynomials
CLASSIFICATION
Expressible
ALGORITHM
Non-expressible
RECOGNITION
CONJECTURE
Multimorphism definition
F = hf1 , . . . , fk i : Dk → Dk is a multimorphism of p in m
variables if and oly if for all t1 , . . . , tk ∈ Dm ,
k
X
i=1
p(ti ) ≥
k
X
p(fi (t1 , . . . , tk )).
i=1
(where fi is applied coordinatewise)
0
1234
0
0
0
0
123
124
134
234
0
0
0
0
0
0
12
13
14
23
24
34
0
0
0
0
1
2
3
4
0
∅
0
1234
0
0
0
0
123
124
134
234
−1
−1
0
−1
0
0
12
13
14
23
24
34
0
0
0
0
1
2
3
4
0
∅
0
1234
−2
0
0
0
123
124
134
234
−1
−1
0
−1
0
0
12
13
14
23
24
34
0
0
0
0
1
2
3
4
0
∅
0
1234
−2
−1
−1
−1
123
124
134
234
−1
−1
0
−1
0
0
12
13
14
23
24
34
0
0
0
0
1
2
3
4
0
∅
−2
1234
−2
−1
−1
−1
123
124
134
234
−1
−1
0
−1
0
0
12
13
14
23
24
34
0
0
0
0
1
2
3
4
0
∅