Interactive Image Segmentation
using Graph Cuts
PRASA 2009
Mayuresh Kulkarni and Fred Nicolls
Digital Image Processing Group
University of Cape Town
Outline
•
•
•
•
•
Image Segmentation Problem
Our Approach
Graph cuts and Gaussian Mixture Models
Results and Discussion
Future Research
What is foreground?
Image Segmentation
Our Approach
Image properties
eg. colour, texture
Difference between
adjacent pixels
Region information
Boundary information
Graph Cuts Segmentation
Cost Function : E(A) = λ R(A) + B(A)
8 – pixel neighbourhood
Pixel connectivity
Graph Cuts
Source (foreground)
Pixel connectivity (boundaries)
Inter-pixel weights (boundaries)
Source and Sink weights (regions)
Cost Function : E(A) = λ R(A) + B(A)
Sink (background)
Gaussian Mixture Models
Background GMM
Foreground GMM
Gaussian Mixture Models
Foreground
GMM
pf
pb
Background
GMM
Log Likelihood Ratio = log(K *pf/pb)
GMM components
• Greyscale images
– Intensity values
– Intensity values and MR8
filters
• Colour images
–
–
–
–
RGB values
G, (G-R), (G-B) values
Luv values
MR8 filters
Boundary information
• Inter-pixel weights
– Edge detection
– Difference between
adjacent pixels
– Gradient
• Pixel connectivity
Results
Κ = 0.01
Κ = 0.1
Κ=1
Results
Original Image
RGB, Luv and MR8 (Fscore = 0.916)
Luv and MR8 (Fscore = 0.921)
Luv (Fscore = 0.934)
Results
Original Image
Luv (Fscore = 0.945)
RGB, Luv and MR8 (Fscore = 0.906)
RGB (Fscore = 0.951)
Analysis of Results
• Accurate segmentation achieved
• Components in the GMM depend on image
• Segmentation can be controlled using K and λ
Future Research
•
•
•
•
Different grid (non-pixel grid)
Ratio cuts
Exploring other statistical models
ObjCut – segmenting particular objects
References
•
•
•
•
•
•
Y. Boykov and M. P. Jolly. Interactive graph cuts for optimal boundary and region
segmentation of objects in N-D images. In ICCV, volume 1, pages 105–112, July 2001.
Yuri Boykov and Vladimir Kolmogorov. An experimental comparison of min-cut/max-flow
algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell.,
26(9):1124–1137, 2004.
Pushmeet Kohli, Jonathan Rihan, Matthieu Bray, and Philip H. S. Torr. Simultaneous
segmentation and pose estimation of humans using dynamic graph cuts. International
Journal of Computer Vision, 79(3):285–298, 2008.
H. Permuter, J. Francos, and I. Jermyn. Gaussian mixture models of texture and colour for
image database. In ICASSP, pages 25–88, 2003.
D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human segmented natural images
and its application to evaluating segmentation algorithms and measuring ecological statistics.
In Proc. 8th Int’l Conf. Computer Vision, volume 2, pages 416–423, July 2001.
Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. “GrabCut”: interactive foreground
extraction using iterated graph cuts. ACM Trans. Graph., 23(3):309–314, August 2004.