Significant points
Objects from point segmentation
•
The simplest form to detect meaningful objects is point segmentation Objects can be
described using few significant points:
– Corners
– Salient points
•
Corners are easy to extract but:
– Visual features are not corners
– Everything can be a corner
(consider f.e a textured surface)
– They are not spread in images
•
Salient points are located in any
interesting part of the image.
visible
visually
In principle they should be
at any resolution.
Corners
Corners distinguish from flat regions and edges from the properties of change of intensity in their
neighbours, i.e. considering the local auto-correlation function of the signal.
In fact the change of intensity can be computed considering the shift u,v in the window w(x,y) over
the image:
C (u,v) =
x,y
w(x,y) [ I(x+u,y+v) - I(x,y)
Sum over a small region (the hypothetical corner)
0
1
1
w(x,y) =
Flat region:
Edge region:
Corner region:
no change in all directions
change in the edge direction only
change in all directions
0
Given a shift ( x,
y)
C(x, y) =
and a point (x,y) the autocorrelation function C at (x,y) is defined as:
W
[I(x ,y ) − I(x +
i
i
i
x,
yi+ y)]2
where I( ., .) denotes the image function and (xi,yi) are the points in w centered on (x,y).
For small shifts C can be approximated with a bilinear approximation:
C ≈ [u,v] M [u,v]’
where M is a 2x2 matrix computed from image derivatives
M=
x,y
w(x,y)
Ix
IxIy
IxIy
Iy
Gradient with respect to x times
gradient with respect to y
In fact, the shifted image can be approximated by a Taylor expansion truncated to the first order terms:
I(xi+ x, yi+ y) ≈ I(xi, yi) + Ix(xi,yi) Iy(xi,yi) [ x,
y]’
where Ix(., .) Iy(.,. ) denote the partial derivatives in x and y, respectively.
Substituting the approximation into the autocorrelation equation yields:
C(x, y) =
W
W
[I(xi,yi ) − I(xi, yi) − Ix(xi,yi) Iy(xi,yi) [
[− Ix(xi,yi) Iy(xi,yi) [
[
x, y]
W
x, y]’ ]2 =
W
(Ix(xi,yi ))2
I
[
x, y]
M [
[ Ix(xi,yi) Iy(xi,yi) [
W
W x(xi,yi) Iy(xi,yi)
x, y]’ ]2 =
Ix(xi,yi) Iy(xi,yi)
W
x, y]’ ]2 =
[
x, y]’
=
(Iy(xi,yi))2
x, y]’
where the matrix M captures the intensity structure of the local neighborhood (characterizes the
structure of the image gray level patterns). In fact, the geometric interpretation of the gray levels is
encoded in the eigenvectors and eigenvalues of the matrix.
In that the matrix is symmetric, it has two nonnegative eigenvalues. By a matrix transformation rotation of the coordinate axes - it can be expressed as:
1
0
0
2
Intensity changes can therefore be analysed considering the two eigenvalues of M that give
the direction of the changes. There are three cases to be considered:
1.
If both λ1 , λ2 are small, so that the local auto-correlation function is flat (i.e., little change in c(x,
y) in any direction), the windowed image region is of approximately constant intensity.
2.
If one eigenvalue is high and the other low, so the the local auto-correlation function is ridge
shaped, then only local shifts in one direction (along the ridge) cause little change in c(x, y) and
significant change in the orthogonal direction; this indicates an edge.
3.
If both eigenvalues are high, so the local auto-correlation function is sharply peaked, then shifts in
any direction will result in a significant increase; this indicates a corner with two principal
directions. The eigenvectors of C encode the directions, the eigenvalues of C encode the
variational strength. Thus, a corner is detected if the smaller eigenvalue λ2 of C is large enough.
Harris corner detection
An alternative for detecting corners is: if λ1 λ2 is larger than a threshold, where k is a small
number (0.04, suggested by Harris).
2
Corners must have :
M
=
Ix
Ix Iy
Ix Iy
Iy
C
2
1
0
0
2
i.e. all gradients in neighborhood are (k,0) or (0, c) or (0, 0) (off-diagonals cancel)
The Harris measure to detect corner existence is given by the following function that depends
only on eigenvalues of M:
R = det M - k(trace M)2
where:
– det M =
1
– trace M =
1
2
+
2
– k = 0.04 - 0.06 (empirically defined)
Harris corner detection
Adaptive non-maximal suppression (ANMS)
•
Looking for local maxima of the interest function can lead to an uneven distribution of
feature points across the image. Particularly points can be denser in regions of higher
contrast.
•
A solution that results into a more uniform distribution of interest points is to only
detect features that are both local maxima and whose response value is significantly
(10%) greater than that of all of its neighbors within a radius r.
This is obtained by first sorting interest points by strength and then creating a list
sorted by decreasing suppression radius
•
Szeliski 2010