Markov Random Fields
with Efficient Approximations
Yuri Boykov, Olga Veksler, Ramin Zabih
Computer Science Department
CORNELL UNIVERSITY
1
Introduction
MAP-MRF approach
(Maximum Aposteriori Probability estimation of MRF)
• Bayesian framework suitable for problems in
Computer Vision (Geman and Geman, 1984)
• Problem: High computational cost. Standard
methods (simulated annealing) are very slow.
2
Outline of the talk

Models where MAP-MRF estimation is
equivalent to min-cut problem on a graph
• generalized Potts model
• linear clique potential model

Efficient methods for solving the
corresponding graph problems

Experimental results
• stereo, image restoration
3
MRF framework in the context of stereo
• image pixels (vertices)
• neighborhood relationships (n-links)
fp - disparity at pixel p
f  ( f 1 ,..., fm) - configuration
MRF defining property:
Pr ( fp | fq, q  p)  Pr ( fp | fq, q  Np)

Hammersley-Clifford
Theorem:


Pr ( f ) ~ exp    V ( p , q )( fp, fq) 
 ( p ,q )

4
MAP estimation of MRF configuration
fˆ
Observed data
 arg max Pr ( f | O)
f
Bayes rule
fˆ  arg max Pr(O | f )  Pr( f )
f
Likelihood
function
(sensor noise)
Prior
(MRF model)

ˆf  arg max exp 
  ln g p (O | fp)   V ( p , q )( fp, fq)
f
( p ,q )
 p

5
Energy minimization
Find
f
that minimizes the Posterior Energy Function :
E ( f )    ln g p (O | fp )   V ( p , q ) ( fp, fq)
p
Data term
(sensor noise)
( p ,q )
Smoothness term
(MRF prior)
6
Generalized Potts model
Clique potential
V ( p, q )( fp, fq)  u{ p, q}   ( fp  fq)
Penalty for discontinuity at (p,q)
Energy function
E ( f )    ln g p (O | fp )  2  u{ p , q}   ( fp  fq )
p
{ p , q}
7
Static clues
-
selecting
u{ p, q}
Stereo Image: White Rectangle in front of
the black background
Disparity configurations minimizing energy E( f ):
u{ p , q}  const
u{ p , q}  const
8
Minimization of E(f) via graph cuts
Terminals (possible disparity labels)
Cost of n-link
 { p , q}  2 u { p , q}
Cost of t-link
p-vertices
(pixels)
 { p , l}  ln g p (O | l )  K p
0
9
Multiway cut
vertices
edges
V = pixels + terminals
E = n-links + t-links
Graph G = <V,E>
Remove a subset of edges
C
Graph G(C) = <V, E-C >
• C is a multiway cut if terminals are separated in G(C)
• A multiway cut
C
yields some disparity configuration
f
C
10
Main Result (generalized Potts model)

Under some technical conditions on Kp
C
the multiway min-cut C on G gives___
f
that minimizes E( f ) - the posterior energy
function for the generalized Potts model.
• Multiway cut Problem: find minimum cost
multiway cut C graph G
11
Solving multiway cut problem

Case of two terminals:
• max-flow algorithm (Ford, Fulkerson 1964)
• polinomial time (almost linear in practice).

NP-complete if the number of labels >2
• (Dahlhaus et al., 1992)

Efficient approximation algorithms that are
optimal within a factor of 2
12
Our algorithm
Initialize at arbitrary multiway cut C
1. Choose a pair of terminals
2. Consider connected pixels
13
Our algorithm
Initialize at arbitrary multiway cut C
1. Choose a pair of terminals
2. Consider connected pixels
3. Reallocate pixels between
two terminals by running
max-flow algorithm
14
Our algorithm
Initialize at arbitrary multiway cut C
1. Choose a pair of terminals
2. Consider connected pixels
3. Reallocate pixels between
two terminals by running
max-flow algorithm
4. New multiway cut C’ is obtained
Iterate until no pair of terminals improves the cost of the cut
15
Experimental results (generalized Potts model)

Extensive benchmarking on synthetic
images and on real imagery with dense
ground truth
• From University of Tsukuba
• Comparisons with other algorithms
16
Synthetic example
Image
Correlation
Multiway cut
17
Real imagery with ground truth
Ground truth
Our results
18
Comparison with ground truth
19
Gross errors (> 1 disparity)
20
Comparative results: normalized correlation
Gross errors
Data
21
Statistics
40
35
30
25
Gross errors
20
Errors
15
10
5
0
Multiway cut LOG-filtered
L1
MLMHV
Census
Normalized
correlation
22
Related work (generalized Potts model)
Greig et al., 1986 is a special case of our
method (two labels)
 Two solutions with sensor noise (function g)
highly restricted

• Ferrari et al., 1995, 1997
23
Linear clique potential model
Clique potential
V ( p , q )( fp, fq)  u{ p , q}  | fp  fq |
Penalty for discontinuity at (p,q)
Energy function
Eˆ ( f )    ln g p (O | fp)  2  u{ p , q}  | fp  fq |
p
{ p , q}
24
Minimization of Eˆ ( f ) via graph cuts
Cost of t-link
 { p , l}   ln g p (O | l )  K p
cut
C
Cost of n-link
 { p , q}  2 u { p , q}
{p,q} part of graph
Ĝ
a cut C yields some
configuration f C
25
Main Result (linear clique potential model)

Under some technical conditions on Kp
C
Ĝ
the min-cut C on
gives f
that
minimizes Eˆ ( f ) - the posterior energy
function for the linear clique potential
model.
26
Related work (linear clique potential model)

Ishikawa and Geiger, 1998
• earlier independently obtained a very similar
result on a directed graph

Roy and Cox, 1998
• undirected graph with the same structure
• no optimality properties since edge weights are
not theoretically justified
27
Experimental results

(linear clique potential model)
Benchmarking on real imagery with dense
ground truth
• From University of Tsukuba

Image restoration of synthetic data
28
Ground truth stereo image
ground truth
Generalized
Potts model
Linear clique
potential model
29
Image restoration
Noisy diamond
image
Generalized Potts
model
Linear clique
potential model
30