Ort
1
ML A: 18.1.2010
Applications of Topic Models
in Computer Vision
Wolfgang Maass
Institut für Grundlagen der Informationsverarbeitung
Technische Universität Graz, Austria
Institute for Theoretical Computer Science
http://www.igi.tugraz.at/maass/
Ort
Application of topic models in
computer vision: scene classification
L. Fei-Fei, R. Fergus, and P. Perona. A Bayesian approach to unsupervised oneshot learning of object categories. In Proceedings of the 9th International
Conference on Computer Vision, Nice, France, volume 2, pages 1134-1141, 2003.
2
Ort
Topic model for categorization of visual scences
Note the additional observable variable c
(classification of scene/image)
For comparison:
graphical model
for text analysis
3
Ort
Structure of the learning algorithm
L. Fei-Fei, P. Perona. A Bayesian hierarchical
model for learning natural scene categories, 2005
4
Ort
Outcome of learning:
Results for the category „forest“
5
Ort
L. Fei-Fei, R. Fergus, and P. Perona. A Bayesian approach to unsupervised oneshot learning of object categories. In Proceedings of the 9th International
Conference on Computer Vision, Nice, France, volume 2, pages 1134-1141, 2003.
6
Ort
7
Ort
Some patches are marked for each
image that belong to the most significant
set of codewords for the category.
L. Fei-Fei, R. Fergus, and P. Perona. A Bayesian approach to unsupervised oneshot learning of object categories. In Proceedings of the 9th International
Conference on Computer Vision, Nice, France, volume 2, pages 1134-1141, 2003.
8
Ort
9
Ort
Application to video categorization
J. C. Niebles, H. Wang, and L. Fei-Fei. Unsupervised
Learning of Human Action Categories Using SpatialTemporalWords. Int J Comput Vis. vol. 79: 299–318.
10
Ort
11
Patches (codewords) in images are replaced by
„Video words“
Shown are some which frequently occur in particular action categories
J. C. Niebles, H. Wang, and L. Fei-Fei. Unsupervised Learning of Human Action Categories Using SpatialTemporalWords. Int J Comput Vis. vol. 79: 299–318.
Ort
12
Snapshots from videos, with currently active
„Video-words“ marked by boxes (their color indicates for
which action category they have highest probability)
J. C. Niebles, H. Wang, and L. Fei-Fei. Unsupervised Learning of Human Action Categories Using SpatialTemporalWords. Int J Comput Vis. vol. 79: 299–318.
Ort
13
Structure of the learning algorithm
Each action category is associated with a distribution of video-words
J. C. Niebles, H. Wang, and L. Fei-Fei. Unsupervised Learning of Human Action Categories Using SpatialTemporalWords. Int J Comput Vis. vol. 79: 299–318.