Maximum vanishing subspace problem,
CAT(0)-space relaxation, and
block-triangularization of partitioned matrix
Hiroshi Hirai
University of Tokyo
[email protected]
Joint work with Masaki Hamada
10th Japanese-Hungarian Symposium on
Discrete Mathematics and Its Applications
May 23, 2017, Budapest, Hungary
1
DM-decomposition of matrix
(Dulmage-Mendelsohn 1958)
• A canonical form under row/column permutation
• Matching/stable set problem in bipartite graph
∗
∗
𝐴 ↦ 𝑃𝐴𝑄 =
𝑃, 𝑄:permutation matrix
∗
∗
0
∗
∗
∗
∗
∗
2
edge 𝑖𝑗 ⟺ 𝑎𝑖𝑗 ≠ 0
′
′
′
1 2 3
1
2
𝑋3
𝑋2
𝑋1
𝑋𝑋
0
*
*
*
*
0
*
𝑌 𝑌3 𝑌2 𝑌1
𝐴 = (𝑎𝑖𝑗 )
1
2′
2
3′
𝑌
3
*
*
*
𝑋
𝑌0
zero submatrix
(max.)
1′
bipartite graph
stable set
(max.)
• 𝑋, 𝑌 , 𝑋 ′ , 𝑌 ′ : stable ⇒ 𝑋 ∪ 𝑋 ′ , 𝑌 ∩ 𝑌 ′ , 𝑋 ∩ 𝑋 ′ , 𝑌 ∪ 𝑌 ′ : stable
• The family of maximum stable sets ⇒ distributive lattice
• A maximal chain ⇒ DM-decomposition
• A polynomial time algorithm via bipartite matching
3
DM-decomposition of partitioned matrix
𝐴=
𝐴11
𝐴21
↦𝑃
𝐸1
0
0
𝐸2
⋮
0 0
0
0
⋱ ⋮
⋯ 𝐸𝜇
⋯
𝑃, 𝑄: permutation
𝐸𝛼 , 𝐹𝛽 :nonsingular
𝐴1𝜈
⋯
𝐴2𝜈
⋱
⋮
⋯ 𝐴𝜇𝜈
𝐴12
𝐴22
⋮
𝐴𝜇1
𝐴𝜇2
𝐴11
𝐴21
𝐴12
𝐴22
⋮
𝐴𝜇1
(Ito-Iwata-Murota 1994)
𝐴𝜇2
∗
𝐴𝛼𝛽 : 𝑚𝛼 × 𝑛𝛽 matrix
𝐴1𝜈
𝐴2𝜈
⋱
⋮
⋯ 𝐴𝜇𝜈
𝐹1
0
⋯
⋮
0 0
∗
0
⋯
⋱
⋯
0
0
𝑄
⋮
𝐹𝜈
∗
∗
=
0
𝐹2
∗
∗
∗
∗
∗
4
Example: (2,2,2;2,2,2) matrix over GF(2)
0
1
1
0
1
0
1
0
1
1
1 0
0 1
0
1
=
0
0
1
0
0
0
0
1
1
0
1
0
0
0
0
0
0
0
1
1
1
1
1
0
1
0
1
0
0
0
0
0
1
0
1
1
0
1
0
0
1
0
0
0
1
1
1
0
1
1
1
1
1
0
1
1
0
1
1
1
1
0
0 1
1 0
0
1
0
0
0
0
1 1
1 0
0
1
1
row/column
permutation
0
1
1
1
1
1
0
1
1
1
1
0
1
1
0
0
1
1
5
Maximum Vanishing Subspace Problem
(H.2016, Implicit in Ito-Iwata-Murota 1994)
𝐴=
𝐴11
𝐴21
𝐴12
𝐴22
⋮
𝐴𝜇1
𝐴𝜇2
𝐴1𝜈
⋯
𝐴2𝜈
⋱
⋮
⋯ 𝐴𝜇𝜈
𝑋𝛼 +
𝐴𝛼𝛽 : 𝑚𝛼 × 𝑛𝛽 matrix over field 𝔽
Max.
𝛼 dim
s.t.
𝑋𝛼 ⊆ 𝔽𝑚𝛼 , 𝑌𝛽 ⊆ ॲ𝑛𝛽 ,
subsp.
𝛽 dim 𝑌𝛽
subsp.
𝐴𝛼𝛽 𝑋𝛼 , 𝑌𝛽 = 0
∀𝛼, 𝛽 ,
where 𝐴𝛼𝛽 : ॲ𝑚𝛼 × ॲ𝑛𝛽 → ॲ
𝑢, 𝑣
↦ 𝑢𝑇 𝐴𝛼𝛽 𝑣
Rem: 𝑚𝛼 = 𝑛𝛽 = 1 ---> bipartite stable set prob.
6
𝑋 = 𝑋1 ⊕ 𝑋2 ⊕ ⋯ ⊕ 𝑋𝜇
Vanishing subspace (𝑋, 𝑌) ⇔ 𝑌 = 𝑌1 ⊕ 𝑌2 ⊕ ⋯ ⊕ 𝑌𝜈
𝐴𝛼𝛽 𝑋𝛼 , 𝑌𝛽 = 0
∀𝛼, 𝛽
row/col operation “within blocks”
+ row/col permutation
𝑋
← dim 𝑋 →
𝐴~
0
⟵ dim 𝑌 ⟶
(max.)
𝑌
𝑋, 𝑌 , 𝑋 ′ , 𝑌 ′ : vanishing
(max.)
⇒ 𝑋 + 𝑋 ′ , 𝑌 ∩ 𝑌 ′ , 𝑋 ∩ 𝑋 ′ , 𝑌 + 𝑌 ′ : vanishing
• The family of max. vanishing subspace ⇒ modular lattice
• A maximal chain ⇒ DM-decomposition
7
•
Polytime algorithm for DM-decom.
is not known in general
Special cases:
• one submatrix
• each submatrix is 1x1
• each submatrix is *x1
---> Gaussian elimination
---> original DM-decom.
---> CCF of mixed matrix
Murota-Iri-Nakamura 87
• each submatrix is rank-1 (H. 16)
•
𝐴11
𝐴21
matroid
intersection
𝐴12
, 𝐴𝛼𝛽 : nonsingular ---> eigenvalue comp.
𝐴22
• sizes of submatrices & base field are fixed
(H-Nakashima 2017)
VCSP
8
Why are MVSP & DM-decomposition
interesting?
• Numerical computation vs. combinatorial optimization
• Submodular optimization on (infinite) modular lattice
Finite case: Kuivinen 2009,2011, Fujishige et al. 2015, ( ⋯ )𝑛
• Suggest “vector-space version”
of combinatorial optimization ??
𝑆 ⊆ 𝑉,
subset
subspace
finite set
|𝑆|
cardinality
finite-dimensional
vector space
dim S
dimension
9
Result (answering the problem of Ito-Iwata-Murota 1994)
Q1. Is MVSP solvable in polynomial time ?
YES (Hamada-H. 2017)
Q2. Is DM-decom. obtained in polynomial time ?
Still we do not know the complete answer but...
• DM-decom. ⊇ eigenvalue problem,
more difficult numerical problem
• A reasonable coarse decomposition
--- quasi DM-decomposition --is obtained in polytime by solving weighted-MVSP.
--- generalizes original-DM & CCF
10
In the rest of the talk, we describe the outline of the proof
Theorem (Hamada-H. 2017)
MVSP is solvable in polynomial time
⊇ polynomial number
of arithmetic operations in 𝔽
Max.
𝛼 dim
𝑋𝛼 +
s.t.
𝑋𝛼 ⊆ 𝔽𝑚𝛼 , 𝑌𝛽 ⊆ ॲ𝑛𝛽
subsp.
𝛽 dim 𝑌𝛽
subsp.
𝐴𝛼𝛽 𝑋𝛼 , 𝑌𝛽 = 0
∀𝛼, 𝛽
Difficulty:
Duality & LP/convex relaxation are not known
11
MVSP is submodular optimization
𝑚+𝑛+1
Min. −
s.t.
𝛼 dim
𝑋𝛼 −
𝛽 dim 𝑌𝛽
+𝑀
𝛼,𝛽 rank 𝐴𝛼𝛽 |𝑋𝛼 ×𝑌𝛽
(𝑋1 , 𝑋2 , … , 𝑋𝜇 , 𝑌1 , 𝑌2 , … , 𝑌𝜈 )
∈ ℒ1 × ℒ2 × ⋯ × ℒ𝜇 × ℳ1 × ℳ2 × ⋯ × ℳ𝜈
ℒ𝛼 /ℳ𝛽 : modular lattice of all vector subsp. in 𝔽𝑚𝛼 /𝔽𝑛𝛽
w.r.t.
inclusion
w.r.t. reverse
inclusion
𝑚≔
𝛼 𝑚𝛼
,𝑛 ≔
𝛽 𝑛𝛽
Lemma (cf. Iwata-Murota 1995)
𝑋𝛼 × 𝑌𝛽 ⟼ rank 𝐴𝛼𝛽 |𝑋𝛼 ×𝑌𝛽 is submodular on ℒ𝛼 × ℳ𝛽
12
Outline : Beyond Euclidean convex relaxation
modular lattice
lattice
Submodular optimization on distributive
Convex optimization on
↪
↪
Lovász extension
Euclidean
space
CAT(0)-space
•
MVSP is submodular optimization on modular lattice.
•
Splitting Proximal Point Algorithm (Bačák, Ohta-Pálfia)
to minimize sum of convex function in CAT(0)-space.
•
Apply SPPA to CAT(0)-space relaxation of MVSP.
13
Cartan-Alexandrov-Topogonov
curvature ≤ 0
CAT(0)-space (Gromov 87)
⇔ geodesic metric space (𝑆, 𝑑) in which
every triangle is “slimmer”
𝑥
𝑥
𝑆
𝑦
𝑦
𝑝
ℝ2
𝑧
𝑑 𝑥, 𝑝 ≤ 𝑥 − 𝑝
2
𝑝
𝑧
𝑑 𝑥, 𝑦 = 𝑥 − 𝑦
𝑑 𝑦, 𝑧 = 𝑦 − 𝑧
𝑑 𝑧, 𝑥 = 𝑧 − 𝑥
FACT: CAT(0)-space is uniquely geodesic
---> convex function
14
2
2
2
Modular lattice
↪
CAT(0)-space
othoscheme complex
ℒ: modular lattice of finite rank
𝐾 ℒ := the set of convex combinations 𝑝∈ℒ 𝜆 𝑝 𝑝 of ℒ
s.t. supp λ is a chain (Brady-McCammond 2012)
𝑝3
111
(ℝ𝑛 , 𝑙2 )
𝑝2
ℒ
110
𝑝1
𝑝0
000
100
Theorem (Haettel-Kielak-Schwer 2017, Chalopin-Chepoi-H-Osajda 2014)
𝐾 ℒ is a complete CAT(0)-space.
15
Example
...
ℒ
ℒ = 2{1,2,…,𝑛}
𝐾 ℒ
𝐾 ℒ ≃ 0,1
𝑛
16
Submodular function
↪ convex function
ℒ: modular lattice of finite rank
Lovasz extension 𝑓: 𝐾(ℒ) → ℝ of 𝑓: ℒ → ℝ
𝑓 𝑥 ≔
𝑖 𝜆𝑖 𝑓(𝑝𝑖 )
(𝑥 =
𝑖 𝜆𝑖 𝑝𝑖
∈ 𝐾(ℒ))
Theorem (H. 2016)
𝑓 is submodular on ℒ
⇔ 𝑓 is convex on 𝐾(ℒ) (w.r.t. CAT(0)-metric)
The original version: ℒ = 2 1,2,…,𝑛 , 𝐾 ℒ ≃ [0,1]𝑛
17
MVSP ↪ convex optimization on
CAT(0)-space
Min. −
s.t.
𝛼 dim (𝑥𝛼 )
−
𝛽 dim(𝑦𝛽 )
+𝑀
𝛼,𝛽 𝑅𝛼𝛽 (𝑥𝛼 , 𝑦𝛽 )
(𝑥1 , 𝑥2 , … , 𝑥𝜇 , 𝑦1 , 𝑦2 , … , 𝑦𝜈 )
∈ 𝐾(ℒ1 × ℒ2 × ⋯ × ℒ𝜇 × ℳ1 × ℳ2 × ⋯ × ℳ𝜈 )
≅
𝐾(ℒ1 ) × 𝐾(ℒ2 ) × ⋯ × 𝐾(ℒ𝜇 ) × 𝐾(ℳ1 ) × 𝐾(ℳ2 ) × ⋯ × 𝐾(ℳ𝜈 )
𝑅𝛼𝛽 𝑋𝛼 , 𝑌𝛽 ≔ rank 𝐴𝛼𝛽 |𝑋𝛼 ×𝑌𝛽
We apply continuous optimization algorithm
to this problem
18
To recover a minimizer of f from 𝑓, we use:
Lemma 𝑓: ℒ → ℤ, 𝑥 =
𝜆𝑖 𝑝𝑖 ∈ 𝐾 ℒ
If 𝑓 𝑥 − min 𝑓 < 1
⇒ some 𝑝𝑖 is a minimizer of 𝑓
19
Splitting Proximal Point Algorithm
𝑆, 𝑑 : complete CAT(0)-space
(Bačák 14)
𝑓1 , 𝑓2 ,..., 𝑓𝑁 : convex functions on 𝑆
Goal: minimize 𝑓 ≔
𝑁
𝑖=1 𝑓𝑖
SPPA: 𝑧𝑘+1 ≔ argmin𝑧 ∈ 𝑆 𝑓𝑘 mod 𝑁 𝑧 +
Theorem (Ohta-Pálfia 15)
𝑓𝑖 : 𝐿-Lipschitz, 𝑓: 𝜖-strongly-convex, 𝜆𝑘 ≔
⇒
𝑓 𝑧𝑘 − min 𝑓 ≤
1
𝑑(𝑧, 𝑧𝑘 )2
2𝜆𝑘
1
2𝜖 𝑘+1
poly(𝐿, 𝑁, 𝜖 −1 , diam 𝑆)
𝑘
20
Apply SPPA to perturbed objective
2
−
dim
𝑥
+
𝜖𝑑(0,
𝑥
)
𝛼
𝛼
𝛼
+
+
2
−dim(𝑦
)
+
𝜖𝑑(0,
𝑦
)
𝛽
𝛽
𝛽
ensure
𝜖-strong-convexity
𝜖 −1 = 𝑂(𝑛 + 𝑚)
𝛼,𝛽 𝑀𝑅𝛼𝛽 (𝑥𝛼 , 𝑦𝛽 )
𝑧 = (𝑥1 , 𝑥2 , … , 𝑥𝜇 , 𝑦1 , 𝑦2 , … , 𝑦𝜈 )
• 0 ≤ obj 𝑧 − obj(𝑧) ≤ 1/2
• each term is 𝑂 poly 𝑚, 𝑛 -Lipschitz
• 𝑁 = 𝑂(𝑚𝑛)
𝑧𝑘 = generated by SPPA for obj
𝑘 = Ω poly 𝑚, 𝑛
⇒ obj 𝑧𝑘 − opt ≤ obj 𝑧𝑘 − opt
⇒ supp(𝑧𝑘 ) ∋ minimizer
1
+
2
<1
21
In each iteration of SPPA, we need to solve:
2
+
𝑑 𝑥0, 𝑥
2
P1: Min. −dim 𝑥 + 𝜖𝑑 0, 𝑥
1
𝑑
2𝜆
𝑥0, 𝑥
2
+ 𝑑 𝑦0, 𝑦
2
s.t. 𝑥 ∈ 𝐾 ℒ
P2: Min. 𝑅𝐴 𝑥, 𝑦 +
s.t.
1
2𝜆
𝑥, 𝑦 ∈ 𝐾 ℒ × ℳ ≅ 𝐾 ℒ) × 𝐾(ℳ
rev. inclusion
ℒ /ℳ: modular lattice of all vector subsp. in 𝔽𝑚 / 𝔽𝑛
𝐴: 𝔽𝑚 × 𝔽𝑛 → 𝔽: bilinear form
𝑅𝐴 𝑋, 𝑌 ≔ rank 𝐴|𝑋×𝑌
P1 and P2
are solvable in
polynomial time
22
− 𝑥
𝜖 𝑥
1
P1: Min. −dim 𝑥 + 𝜖𝑑 0, 𝑥
s.t. 𝑥 ∈ 𝐾 ℒ
1
2𝜆
2
2
2
+
1
𝑑
2𝜆
𝑥0, 𝑥
2
2
2
𝑝3
𝑥0 =
𝑥∗
𝑝0
𝑥0 − 𝑥
𝑖 𝜆𝑖 𝑝𝑖
𝑝2
𝑝1
Minimizer 𝑥 ∗ exists in the simplex ∋ 𝑥 0
---> easy convex quadratic program
23
P2: Min. 𝑅𝐴 𝑥, 𝑦 +
1
2𝜆
𝑑 𝑥0, 𝑥
2
+ 𝑑 𝑦0, 𝑦
2
𝑥, 𝑦 ∈ 𝐾 ℒ × ℳ
s.t.
ℒ
ℳ
0
X
Y
0⊥
0⊥
X
Y
0
∙ ⊥ : orthogonal space w.r.t. 𝐴
distributive lattice
Minimizer (𝑥 ∗ , 𝑦 ∗ ) exists
0
0⊥
0⊥
0
in 𝐾 X , Y
× 𝐾 X ,Y
↪ 0,1
---> easy convex quadratic program
𝑚+𝑛
24
Concluding remarks
• Weighted-MVSP: Max. 𝛼 𝐶𝛼 dim 𝑋𝛼 +
is solvable in pseudo-polynomial time
𝛽 𝐷𝛽 dim 𝑌𝛽
• Quasi DM-decomposition
⟺ a chain of max. vanishing subsp. detectable by WMVSP
• Duality theory + better algorithm for MVSP ?
• Vector-space generalization of other problems ?
• CAT(0)-space relaxation ---> new paradigm
in combinatorial optimization ?
25
Thank you for your attention!
M. Hamada and H. Hirai:
Maximum vanishing subspace problem, CAT(0)-space relaxation,
and block-triangularization of partitioned matrix, 2017, arXiv.
26