First Law Of Geography:
"Everything is related to everything else, but near
things are more related to each other.”
-Waldo Tobler (1970)
Pyramids of Features
For Categorization
Presented by Greg Griffin
Project Partner Will Coulter
Pyramids of Features
For Categorization
Presented by Greg Griffin
Project Partner Will Coulter
Buckets of Features
For Categorization
Presented by Greg Griffin
Project Partner Will Coulter
This talk is mostly about this paper:
With a little bit about benchmarking:
CALTECH 256
Images
Features
X1
A Number
X Nx
K(X,Y)

Y1
YNY

Images
Features
X1
A Number
X Nx
K(X,Y)

Y1
YNY

“How well do
they match”
“Weak Features”
“Weak Features”
(I think?)
“…points whose gradient magnitude in a given
direction exceeds a minimum threshold.”
This is just their toy example
They use SIFT descriptors as “Strong Features”.
But you could use any features you want!
Images
Features
A Number
1
2
3
4
5
6
7
8
Images
Features
X1 X5
Y1 Y5
Images
Features
D I
X1 X5

Y1 Y5
Features
D I
X1 X5

L
K (X,Y)
Y1 Y5
Features
Start By Matching Reds
D
I   min( H X (i),HY (i))
i1
D I

L
K (X,Y)
Features
Start By Matching Reds
D
I   min( H X (i),HY (i))
i1
D I
0

L
K (X,Y)
1
Features
Start By Matching Reds
D
I   min( H X (i),HY (i))
i1
10
D I
0

10
L
K (X,Y)
1
10
Features
Start By Matching Reds
D
I   min( H X (i),HY (i))
i1
D I

L
K (X,Y)
0
1
1
4
10
Features
Start By Matching Reds
D
I   min( H X (i),HY (i))
7
1
i1
D I
2
0

6
1
1
2
L
K (X,Y)
0
1
1
4
10
Features
Start By Matching Reds
D
I   min( H X (i),HY (i))
7
1
i1
D I
2
0

6
1
1
2
L
K (X,Y)
0
1
10
1
4
8
Features
Start By Matching Reds
D
I   min( H X (i),HY (i))
i1
D I

L
K (X,Y)
0
1
10
1
4
8
2
16
Features
Start By Matching Reds
D
I   min( H X (i),HY (i))
i1
5
2
1
D I
2

1
3
L
K (X,Y)
2
1
1
1
1

0
1
10
1
4
8
2
16
6
L2
Start By Matching Reds
D
I   min( H X (i),HY (i))
i1
L1
 (X ,Y )  I  
L
1
1
1
L
0
2
L
8 10
I 6 1  2
2 2
D I

0
1
10
1
4
8
2
16
6
L2
L
K (X,Y)

Start By Matching Reds
D
I   min( H X (i),HY (i))
i1
L1
 (X ,Y )  I  
L
1
1
1
L
0
2
L
8 10
I 6 1  2
2 2
D I
A compact set of matches

is preferable to widely
dispersed matches
0
1
10
1
4
8
2
16
6
L2
L
K (X,Y)

Start By Matching Reds
D
I   min( H X (i),HY (i))
i1
L1
 (X ,Y )  I  
L
1
1
1
L
0
2
L
8 10
I 6 1  2
2 2
D I

0
1
10
1
4
8
2
16
6
L2
L
K (X,Y)

Start By Matching Reds
D
I   min( H X (i),HY (i))
i1
L1
 (X ,Y )  I  
L
1
1
1
L
0
2
L1
 L (X 1,Y 1 )  I L  
L
1
L
2
0
8 10
I 6 1  2
2 2
(I I
1
)6
D I
8  6 10  8

 7.5
2
2
2

0
1
10
1
4
8
2
16
6
L2
L
K (X,Y)

D
I   min( H X (i),HY (i))
i1
L1
 (X ,Y )  I  
L
1
1
1
L
0
2
L
L1
 L (X 1,Y 1 )  I L  
2
0
1
L
8 10
I 6 1  2
2 2
(I I
1
D I
)  5.4 < 7.5

A Sanity Check:
Features X vs. Purely Isotropic Y
L
K (X,Y)

0
1
10
1
4
5.5
2
16
1.9
L2
Start By Matching Reds, Then The Blues, Then…
D
I   min( H X (i),HY (i))
i1
L1
 (X ,Y )  I  
L
1
1
1
L
0
2
L1
 L (X 1,Y 1 )  I L  
L
1
L
2
0
8 10
I 6 1  2
2 2
(I I
1
)6
D I
8  6 10  8

 7.5
2
2
2

M
 L (X,Y )   L (X m ,Y m )  7.5  ...
1
10
1
4
8
2
16
6
L2
m1
L
K (X,Y)
0

1
2
3
4
5
M=8
6
7
8
Features
D
I   min( H X (i),HY (i))
i1
D I
0

1
10
Features
D
I   min( H X (i),HY (i))
i1
D I

0
1
10
1
4
8
Features
D
I   min( H X (i),HY (i))
i1
D I

0
1
10
1
4
8
2
16
6
Foreach feature M=1…m
Foreach level = 0…L
Foreach cell i=1…D

D
I   min( H X (i),HY (i))
i1
L1
 L (X m ,Y m )  I L  
M
  
 L (X,Y)
0
1
2
L
1
L
m
m

(X
,Y
)
m m
N Ny
m1 x


(I I
1
)
Training Set
Test Set
SL(X,Y)
3.4
15
Categories
5.6
7.8
office
1.5
5.4
store
coast
street
100 Images
per Category
100-300 Images
per Category
suburb
Confusion Matrix
Train on 100
Test on 100-300
(per category)
Scene Database
Caltech 101
Hypothesis
Pyramid Matching works well when:
• Objects are aligned and localized
– ie. certain Caltech 101 categories
– biased by different values of Ntest?
• A few common features that define the
category get randomly permuted through
many positions, thanks to a large dataset
– ie. scene database
– now and then pyramid matching gets lucky
Test
How well will Pyramid matching work?
• Objects are not well aligned, or cluttered
– Caltech 256 is more challenging in this respect
Scene Database
Caltech 101
Cluster Matching
k-means of SIFT positions
More Flexible Than A Grid?
k-means of SIFT positions + color
Position Invariance & Clutter
1. Any cluster can match any cluster
2. Clusters respect duck / water boundaries
(sort of)
Tackles Alignment Problem
But… how to match efficiently?
How many clusters? How big?
Summary
Spatial Pyramid Matching is efficient and
handles a range of scales, but seems to be
sensitive to translation and clutter.
Cluster Matching has the potential to
improve translational invariance and tolerance
of clutter. But inefficient. Less principled: how
many clusters are optimal? How big should they be?
No scale invariance.
Can we have the best of both worlds?
Try Sliding Spatial Pyramids?
Slide puzzle photo from:
http://www.tenfootpolesoftware.com/products/slidepuzzle/

Test Score Matching

Lecture - NTTコミュニケーション科学基礎研究所

Laboratory Centrifuge Equipment Market