Improved Moves for
Truncated Convex Models
M. Pawan Kumar
Philip Torr
Aim
Efficient, accurate MAP for truncated convex models
V1
V2
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
Vn
Random Variables V = { V1, V2, …, Vn}
Edges E define neighbourhood
Aim
Accurate, efficient MAP for truncated convex models
ab;ik
lk
li
a;i
ab;ik = wab min{ d(i-k), M }
wab is non-negative
Vb b;k
Va
d(.) is convex
Truncated Linear
ab;ik
Truncated Quadratic
ab;ik
i-k
i-k
Motivation
Low-level Vision
min{ |i-k|, M}
• Smoothly varying regions
• Sharp edges between regions
Boykov, Veksler & Zabih 1998
Well-researched !!
Things We Know
• NP-hard problem - Can only get approximation
• Best possible integrality gap - LP relaxation
Manokaran et al., 2008
• Solve using TRW-S, DD, PP
Slower than graph-cuts
• Use Range Move - Veksler, 2007
None of the guarantees of LP
Real Motivation
Gaps in Move-Making Literature
Chekuri et al., 2001
LP
Potts
2
Truncated
Linear
2 + √2
Truncated
Quadratic
O(√M)
Multiplicative Bounds
MoveMaking
Real Motivation
Gaps in Move-Making Literature
Boykov, Veksler and Zabih, 1999
LP
MoveMaking
Potts
2
2
Truncated
Linear
2 + √2
2M
Truncated
Quadratic
O(√M)
-
Multiplicative Bounds
Real Motivation
Gaps in Move-Making Literature
Gupta and Tardos, 2000
LP
MoveMaking
Potts
2
2
Truncated
Linear
2 + √2
4
Truncated
Quadratic
O(√M)
-
Multiplicative Bounds
Real Motivation
Gaps in Move-Making Literature
Komodakis and Tziritas, 2005
LP
MoveMaking
Potts
2
2
Truncated
Linear
2 + √2
4
Truncated
Quadratic
O(√M)
2M
Multiplicative Bounds
Real Motivation
Gaps in Move-Making Literature
LP
MoveMaking
Potts
2
2
Truncated
Linear
2 + √2
2 + √2
Truncated
Quadratic
O(√M)
O(√M)
Multiplicative Bounds
Outline
• Move Space
• Graph Construction
• Sketch of the Analysis
• Results
Move Space
• Initialize the labelling
• Choose interval I of L’ labels
• Each variable can
• Retain old label
• Choose a label from I
• Choose best labelling
Va
Vb
Iterate over intervals
Outline
• Move Space
• Graph Construction
• Sketch of the Analysis
• Results
Two Problems
• Choose interval I of L’ labels
• Each variable can
• Retain old label
• Choose a label from I
• Choose best labelling
Large L’ => Non-submodular
Va
Vb
Non-submodular
First Problem
Va
Vb
Submodular problem
Ishikawa, 2003; Veksler, 2007
First Problem
Va
Vb
Non-submodular
Problem
First Problem
Va
Vb
Submodular problem
Veksler, 2007
First Problem
Va
Vb
am+1
bm+1
am+2
bm+2
am+2
bm+2
an
bn
t
First Problem
Va
Vb
am+1
bm+1
am+2
bm+2
am+2
bm+2
an
bn
t
First Problem
Va
Vb
am+1
bm+1
am+2
bm+2
am+2
bm+2
an
bn
t
First Problem
Va
Vb
am+1
bm+1
am+2
bm+2
am+2
bm+2
an
bn
t
First Problem
Va
Vb
am+1
bm+1
am+2
bm+2
am+2
bm+2
an
bn
t
Model unary potentials exactly
First Problem
Va
am+1
bm+1
am+2
bm+2
am+2
bm+2
an
bn
Vb
t
Similarly for Vb
First Problem
Va
Vb
am+1
bm+1
am+2
bm+2
am+2
bm+2
an
bn
t
Model convex pairwise costs
First Problem
Wanted to model
ab;ik = wab min{ d(i-k), M }
For all li, lk  I
Have modelled
ab;ik = wab d(i-k)
Va
Vb
For all li, lk  I
Overestimated pairwise potentials
Second Problem
• Choose interval I of L’ labels
• Each variable can
• Retain old label
• Choose a label from I
• Choose best labelling
Va
Vb
Non-submodular problem !!
Second Problem - Case 1
∞
Va
Vb
s
∞
am+1
bm+1
am+2
bm+2
an
bn
t
Both previous labels lie in interval
Second Problem - Case 1
∞
Va
s
∞
am+1
bm+1
am+2
bm+2
an
bn
Vb
t
wab d(i-k)
Second Problem - Case 2
∞
Va
Vb
s
ub
am+1
bm+1
am+2
bm+2
an
bn
t
Only previous label of Va lies in interval
Second Problem - Case 2
∞
am+1
Va
Vb
s
M
ub
bm+1
am+2
bm+2
an
bn
t
ub : unary potential of previous label of Vb
Second Problem - Case 2
∞
am+1
Va
s
M
ub
bm+1
am+2
bm+2
an
bn
Vb
t
wab d(i-k)
Second Problem - Case 2
∞
am+1
Va
Vb
s
M
ub
bm+1
am+2
bm+2
an
bn
t
wab ( d(i-m-1) + M )
Second Problem - Case 3
Va
Vb
am+1
bm+1
am+2
bm+2
an
bn
t
Only previous label of Vb lies in interval
Second Problem - Case 3
ua
am+1
Va
Vb
s
M
∞
bm+1
am+2
bm+2
an
bn
t
ua : unary potential of previous label of Va
Second Problem - Case 4
Va
Vb
am+1
bm+1
am+2
bm+2
an
bn
t
Both previous labels do not lie in interval
Second Problem - Case 4
ua
am+1
Va
Vb
s
Pab
M
M
ab
ub
bm+1
am+2
bm+2
an
bn
t
Pab : pairwise potential for previous labels
Second Problem - Case 4
ua
am+1
Va
s
Pab
M
M
ab
ub
bm+1
am+2
bm+2
an
bn
Vb
t
wab d(i-k)
Second Problem - Case 4
ua
am+1
Va
s
Pab
M
M
ab
ub
bm+1
am+2
bm+2
an
bn
Vb
wab ( d(i-m-1) + M )
t
Second Problem - Case 4
ua
am+1
Va
Vb
s
Pab
M
M
ab
bm+1
am+2
bm+2
an
bn
t
Pab
ub
Graph Construction
Find st-MINCUT.
Retain old labelling
if energy increases.
Va
Vb
am+1
bm+1
am+2
bm+2
an
bn
t
ITERATE
Outline
• Move Space
• Graph Construction
• Sketch of the Analysis
• Results
Analysis
Va
Vb
Current labelling f(.)
QC ≤ Q’C
Va
Vb
Previous labelling f’(.)
QP
Va
Vb
Global Optimum f*(.)
Analysis
Va
Vb
Current labelling f(.)
QC ≤ Q’C
Va
Vb
Va
Vb
Previous labelling f’(.) Partially Optimal f’’(.)
≤
Q’0
Analysis
Va
Vb
Current labelling f(.)
QP - Q’C
Va
Vb
Va
Vb
Previous labelling f’(.) Partially Optimal f’’(.)
≥
QP- Q’0
Analysis
Va
Vb
Current labelling f(.)
QP - Q’C ≤ 0
Va
Vb
Local Optimal f’(.)
Va
Vb
Partially Optimal f’’(.)
QP- Q’0 ≤ 0
Analysis
Va
Vb
Current labelling f(.)
Va
Vb
Local Optimal f’(.)
Take expectation over all intervals
Va
Vb
Partially Optimal f’’(.)
QP- Q’0 ≤ 0
Analysis
Truncated Linear
QP ≤ 2 + max 2M , L’
L’
M
Q*
Gupta and Tardos, 2000
L’ = M
L’ = √2M
Truncated Quadratic
L’ = √M
QP ≤ O(√M)
Q*
4
2 + √2
Outline
• Move Space
• Graph Construction
• Sketch of the Analysis
• Results
Synthetic Data - Truncated Linear
Energy
Time (sec)
Faster than TRW-S
Comparable to Range Moves
With LP Relaxation guarantees
Synthetic Data - Truncated Quadratic
Energy
Time (sec)
Faster than TRW-S
Comparable to Range Moves
With LP Relaxation guarantees
Stereo Correspondence
Disparity Map
Unary Potential: Similarity of pixel colour
Pairwise Potential: Truncated convex
Stereo Correspondence
Algo
Energy1
Time1
Energy2
Time2
Swap
3678200
18.48
3707268
20.25
Exp
3677950
11.73
3687874
8.79
TRW-S 3677578 131.65 3679563 332.94
BP
Range
Our
3789486 272.06 5180705 331.36
3686844
97.23
3679552 141.78
3613003 120.14 3679552 191.20
Teddy
Stereo Correspondence
Algo
Energy1
Time1
Energy2
Time2
Swap
3678200
18.48
3707268
20.25
Exp
3677950
11.73
3687874
8.79
TRW-S 3677578 131.65 3679563 332.94
BP
Range
Our
3789486 272.06 5180705 331.36
3686844
97.23
3679552 141.78
3613003 120.14 3679552 191.20
Teddy
Stereo Correspondence
Algo
Energy1
Time1
Energy2
Time2
Swap
645227
28.86
709120
20.04
Exp
634931
9.52
723360
9.78
TRW-S
634720
94.86
651696
226.07
BP
662108
170.67 2155759 244.71
Range
634720
39.75
651696
80.40
Our
634720
66.13
651696
80.70
Tsukuba
Summary
• Moves that give LP guarantees
• Similar results to TRW-S
• Faster than TRW-S because of graph cuts
Questions Not Yet Answered
• Move-making gives LP guarantees
– True for all MAP estimation problems?
• Huber function? Parallel Imaging Problem?
• Primal-dual method?
• Solving more complex relaxations?
Improved Moves for Truncated Convex Models
Kumar and Torr, NIPS 2008
http://www.robots.ox.ac.uk/~pawan/
Questions?