Computational Vision
 Edge Detection
 Canny Detector
 Line Detection
 Hough Transform
 Trucco: Chapter 4, pp. 76 – 80
Chapter 5, pp. 95 - 100
Computational Vision / Ioannis Stamos
Finding Corners
CSc 83020 3-D Computer Vision – Ioannis Stamos
What Is a Corner?
Large gradients in more than one direction.
CSc 83020 3-D Computer Vision – Ioannis Stamos
Edges vs. Corners
 Edges = maxima in intensity gradient
Edges vs. Corners
 Corners = lots of variation in direction of
gradient in a small neighborhood
Detecting Corners
 How to detect this variation?
Detecting Corners
 How to detect this variation?
 Not enough to check average
f
x
and
f
y
Detecting Corners
 Claim: the following covariance matrix
summarizes the statistics of the gradient
  fx2
C
 f x f y
f f
f
x
y
2
y



f
f
fx  , f y 
x
y
Summations over local neighborhoods
Detecting Corners
 Examine behavior of C by testing its effect
in simple cases
 Case #1: Single edge in local
neighborhood
Case#1: Single Edge
 Let (a,b) be gradient along edge
 Compute C (a,b):
a 
C  
b 

 fx 
f    
 fy 
  fx2

 f x f y
f f
f
x
y
 f f 

  f  f

y
2
T
a 
b 
 
a  
   
b  
 a 
 
 b 
Case #1: Single Edge
 However, in this simple case, the only
nonzero terms are those where f = (a,b)
 So, C (a,b) is just some multiple of (a,b)
Case #2: Corner
 Assume there is a corner, with
perpendicular gradients (a,b) and (c,d)
Case #2: Corner
 What is C (a,b)?
 Since (a,b)  (c,d) = 0, the only nonzero terms
are those where f = (a,b)
 So, C (a,b) is again just a multiple of (a,b)
 What is C (c,d)?
 Since (a,b)  (c,d) = 0, the only nonzero terms
are those where f = (c,d)
 So, C (c,d) is a multiple of (c,d)
Corner Detection
 Matrix times vector = multiple of vector
 Eigenvectors and eigenvalues!
 In particular, if C has one large
eigenvalue, there’s an edge
 If C has two large eigenvalues, have
corner
 Tomasi-Kanade corner detector
Corner Detection
Implementation
1. Compute image gradient
2. For each mm neighborhood,
compute matrix C
3. If smaller eigenvalue 2 is larger than
threshold , record a corner
4. Nonmaximum suppression: only keep
strongest corner in each mm window
Corner Detection Results
 Checkerboard
with noise
Trucco & Verri
Corner Detection Results
Corner Detection Results
Histogram of 2 (smaller eigenvalue)
A Simple Corner Detector
  I x2
C
 I x I y
I I
I
Find eigenvalues of C
x y
2
y
1



2 
If the smaller eigenvalue is above a threshold then we have a corner.
CSc 83020 3-D Computer Vision – Ioannis Stamos
A Simple Corner Detector
  I x2
C
 I x I y
I I
I
Iy  0
 a 0
C

0
0


a 0 
C

0
a


Find eigenvalues of C
x y
2
y
1



2 
If the smaller eigenvalue is above a threshold then we have a corner.
CSc 83020 3-D Computer Vision – Ioannis Stamos
A Simple Corner Detector
CSc 83020 3-D Computer Vision – Ioannis Stamos
Canny Edge Detection




Can we derive an optimal detector?
Good Detection: minimize false positives and false negatives.
Good Localization: close as possible to the true edges.
Single Response Constraint: one edge should be detected
for each true edge
Comparison
Sobel
CSc 83020 3-D Computer Vision – Ioannis Stamos
Canny
Canny Edge Detection
 3 STEPS:
 CANNY_ENHANCER
 NONMAX_SUPPRESION
 HYSTERESIS_THRESH
Localization-Detection Tradeoff
 Filter’s spatial scale
 Location &
 Detection criteria.
 Good tradeoff 1-D step edge detector:
(optimal)
 Approximated by the 1st derivative of the Gaussian.
Optimal 1-D step edge detectors…
Pixel (i,j)
Keep the one that gives you maximum response.
Expensive.
Solution…
y
x
Pixel (i,j)
Compute derivatives with respect to x & y directions.
Compute edge normal.
CANNY_ENHANCER
 Compute Ix and Iy the gradient of the image using a derivative
of Gaussian filter.
 Compute the edge strength from the magnitude of the gradient:
Es
 Compute the orientation of the edge from arctan(Iy / Ix ): Eo
[Canny ’86]
n
Problem with detector
original image
gradient magnitude (Es)
• Compute image derivatives
• if gradient magnitude >  and the value is a local max. along gradient
direction – pixel is an edge candidate
• how to detect one pixel thin edges ?
NONMAX_SUPPRESSION
Locate local maxima from Es.
•Find direction d that best approximates Eo(i,j)
•If Es(i,j) is smaller than at least one neighbor along d In(i,j)=0
else In(i,j)=Es(i,j).
135
90
45
0
Non-Maximum Supression
Non-maximum suppression:
Select the single maximum point across the width
of an edge.
CSc 83020 3-D Computer Vision – Ioannis Stamos
Linking to the Next Edge Point
Assume the marked
point q is an edge
point.
Take the normal to
the gradient at that
point and use this to
predict continuation
points (either r or p).
CSc 83020 3-D Computer Vision – Ioannis Stamos
Thresholding
 Edges are found by thresholding the output of
NONMAX_SUPRESSION
 If the threshold is too high:
 Very few (none) edges
 High MISDETECTIONS, many gaps
 If the threshold is too low:
 Too many (all pixels) edges
 High FALSE POSITIVES, many extra edges
Edge Detection With Hysteresis
Low threshold
Hysteresis (high and low threshold)
High threshold
Edge Hysteresis



Hysteresis: A lag or momentum factor
Idea: Maintain two thresholds khigh and klow
 Use khigh to find strong edges to start edge
chain
 Use klow to find weak edges which continue
edge chain
Typical ratio of thresholds is roughly
khigh / klow = 2
CSc 83020 3-D Computer Vision – Ioannis Stamos
Edge Tracking
Hysteresis thresholding [Canny ’86]
Weak edges
Strong edge
Strong edges reinforce weak edges.
edges
Weak edge removed
We call a pixel an edge if it is strong.
We also call a pixel an edge if it is weak but is connected to an edge.
A pixel is connected to an edge if it is in a direction perpendicular to the edge normal
Canny Edge Detection (Example)
gap is gone
Strong +
connected
weak edges
Original
image
Strong
edges
only
Weak
edges
courtesy of G. Loy
CSc 83020 3-D Computer Vision – Ioannis Stamos
HYSTERESIS_THRESH
 Input: In, Eo, high threshold, low threshold.
 Output: Lists of connected edges
(contours).
Edge Relaxation
Parallel – iterative method to adjust edge values on the basis of neighboring edges
Crack edges:
Edge Relaxation
Parallel – iterative method to adjust edge values on the basis of neighboring edges
Crack edges:
Notation:
Edge to be updated
Edge
No edge
Vertex types:
(0)
(1)
(2)
(3)
Edge Relaxation
Parallel – iterative method to adjust edge values on the basis of neighboring edges
Crack edges:
Notation:
Edge to be updated
Edge
No edge
Vertex types:
(0)
(1)
(2)
(3)
Action table for edge type:
Decrease Increase Unchanged
0-0
1-1
0-1
0-2
1-2
2-2
0-3
1-3
2-3
3-3
Edge Relaxation
Parallel – iterative method to adjust edge values on the basis of neighboring edges
Crack edges:
Notation:
Algorithm:
Edge to be updated
Edge
No edge
0. Compute initial confidence C0(e) of
each edge e
C0(e) =
Vertex types:
(0)
(1)
(2)
(3)
Action table for edge type:
Decrease Increase Unchanged
0-0
1-1
0-1
0-2
1-2
2-2
0-3
1-3
2-3
3-3
e
max( I )
1.
k=1
2.
Compute edge type for all e
3.
Modify Ck(e) based on Ck-1(e) and
edge type.
4.
If all Ck(e) have converged to 1 or 0
else go to step 1
Canny Edge Detector
Original: Lena
Computational Vision / Ioannis Stamos
Edges
Towards Global Features
Local versus global
CSc 83020 3-D Computer Vision – Ioannis Stamos
From Edges to Lines
CSc 83020 3-D Computer Vision – Ioannis Stamos
Detecting Lines
 What is the difference between line
detection and edge detection?
 Edges = local
 Lines = nonlocal
 Line detection usually performed on the
output of an edge detector
From Szymon Rusinkiewicz, Princeton
Detecting Lines
 Possible approaches ?
From Szymon Rusinkiewicz, Princeton
Detecting Lines
 Possible approaches:
 Brute force: enumerate all lines, check if present
 Hough transform: vote for lines to which detected
edges might belong
 Fitting: given guess for approximate location, refine it
 Second method efficient for finding
unknown lines, but not always accurate
From Szymon Rusinkiewicz, Princeton
Hough Transform
 General idea: transform from image
coordinates to parameter space of feature
 Need parameterized model of features
 For each pixel, determine all parameter values that
might have given rise to that pixel; vote
 At end, look for peaks in parameter space
From Szymon Rusinkiewicz, Princeton
Hough Transform for Lines
 Generic line: y = ax+b
 Parameters: a and b
From Szymon Rusinkiewicz, Princeton
Hough Transform for Lines
1. Initialize table of buckets, indexed by
a and b, to zero
2. For each detected edge pixel (x,y):
a. Determine all (a,b) such that y = ax+b
b. Increment bucket (a,b)
3. Buckets with many votes indicate probable lines
From Szymon Rusinkiewicz, Princeton
Hough Transform for Lines
a
b
From Szymon Rusinkiewicz, Princeton
Hough Transform for Lines
a
b
From Szymon Rusinkiewicz, Princeton
Difficulties with
Hough Transform for Lines
 Slope / intercept parameterization not ideal
 Non-uniform sampling of directions
 Can’t represent vertical lines
 Angle / distance parameterization
 Line represented as (r,q) where
x cos q + y sin q = r
r
q
From Szymon Rusinkiewicz, Princeton
Finding Lines Using the Hough
Transform
xi , yi 
  xi cosq  yi sin q
CSc 83020 3-D Computer Vision – Ioannis Stamos
Algorithm
 Discretize the parameter spaces ρ and θ.
 Create Accumulator array A(1..R,1..T).
 Set A(k,h)=0 for all k and h.
 For each image edge E(i,j)=1
 For h=1…T
 ρ =i cosθd(h)+j sinθd (h)
 Find index k: ρd is closest to ρ
 Increment A(h,k) by one.
 Find all local maxima (kp, hp) such that A (kp, hp)>τ
CSc 83020 3-D Computer Vision – Ioannis Stamos
Hough Transform Results
Forsyth & Ponce
Hough Transform Results
Forsyth & Ponce
Finding Lines Using the Hough
Transform
Strong local peaks
correspond to lines.
CSc 83020 3-D Computer Vision – Ioannis Stamos
Finding Lines Using the Hough
Transform
Resolution Issues
…
CSc 83020 3-D Computer Vision – Ioannis Stamos
Bucket Selection
 How to select bucket size?
From Szymon Rusinkiewicz, Princeton
Bucket Selection
 How to select bucket size?
 Too small: poor performance on noisy data, long
running times
 Too large: poor accuracy, possibility of false positives
 Large buckets + verification and refinement
 Problems distinguishing nearby lines
 Be smarter at selecting buckets
 Use gradient information to select subset of buckets
 More sensitive to noise
From Szymon Rusinkiewicz, Princeton
Hough Transform: Results
Image
Edge detection
CSc 83020 3-D Computer Vision – Ioannis Stamos
Hough Transform
Summary Hough Transform
 Smart counting
 Local evidence for global features
 Organized in a table
 Careful with parameterization!
 Problem: Curse of dimensionality
 Works great for simple features with 3 unknowns
 Will fail for complex objects
 Problem: Not a local algorithm
CSc 83020 3-D Computer Vision – Ioannis Stamos
Hough Transform
 What else can be detected using
Hough transform?
Hough Transform
 What else can be detected using
Hough transform?
 Anything, but dimensionality is key
Finding Circles by Hough Transform
y
b0
r
a0
Equation of Circle:
(xi,yi)
x
( xi  a0 ) 2  ( yi  b0 )2  r 2
CSc 83020 3-D Computer Vision – Ioannis Stamos
Finding Circles by Hough Transform
y
b0
r
a0
(xi,yi)
x
( xi  a0 ) 2  ( yi  b0 )2  r 2
Equation of Circle:
If radius r is known:
Circles!
b
y
(xi,yi)
x
Accumulator array A(a,b)
a
Finding Circles by Hough Transform
y
b0
r
(xi,yi)
a0
x
If r is not known
Use accumulator array A(a,b,r)
For each (xi,yi) increment A(a,b,r) such that
( xi  a )2  ( yi  b )2  r 2
CSc 83020 3-D Computer Vision – Ioannis Stamos
Using Gradient Information
Can save lot of computations!
Given: location (xi,yi)
y
Edge direction φi
x
CSc 83020 3-D Computer Vision – Ioannis Stamos
Using Gradient Information
Can save lot of computations!
Given: location (xi,yi)
y
Edge direction φi
x
Assume r is known:
(xi,yi)
y
φi
(a,b)
a=x-rcosφ
b=y-rsinφ
Need to increment
only one point
x in Accumulator Array.
Hough Transform for Curves
 Curve y=f(x,a)
 a=[a1, … , ap] the parameters of the curve.
 Limitation: size of parameter space wrt # of
parameters.
 Solution: variable-resolution parameter space.
CSc 83020 3-D Computer Vision – Ioannis Stamos
Hough Transform
 Pattern Matching.
 More efficient than template matching.
 Handles occlusion.
 Finds all instances of pattern.
 Handling inaccurate edge locations?
 Drawbacks?
Fitting
 Output of Hough transform often not
accurate enough
 Use as initial guess for fitting
Fitting Lines
Initial guess
Fitting Lines
Least-squares
minimization
Fitting Lines
Finding Lines
 Assume edge detection
 Each pixel is either edge or not
 How do we find the line?
CSc 83020 3-D Computer Vision – Ioannis Stamos
Finding Lines
 Assume edge detection
 Each pixel is either edge or not
 How do we find the line?
CSc 83020 3-D Computer Vision – Ioannis Stamos
Least Squares Fitting of Lines
y  mx  c
E
 yi  mxi  c 2
i
N
xi , yi 
Minimize E
E
0
m
E
0
c
yi  mxi  c
CSc 83020 3-D Computer Vision – Ioannis Stamos
Least Squares Fitting of Lines
E
 yi  mxi  c 2
i
N
Minimize E
E
0
m
E
0
c
Problem: E must be formulated carefully!
Least Squares Fitting of Lines
  x sin q  y cosq  0
E
  xi sin q  yi cosq 2
N
i
xi , yi 
Minimize E
E
0

E
0
q
  xi sin q  yi cosq