Robert Collins
CSE486, Penn State
Lecture 16:
Planar Homographies
Robert Collins
CSE486, Penn State
Motivation:
Points on Planar Surface
y
x
Robert Collins
CSE486, Penn State
Review : Forward Projection
World
Coords
Camera
Coords
Film
Coords
U
V
W
X
Y
Z
x
y
U
V
W
U
V
W
Mext
Mproj
X
Y
Z
Mint
M
m11 m12
m21 m22
m31 m31
m13
m23
m33
m14
m24
m34
Pixel
Coords
Maff
u
v
u
v
u
v
Robert Collins
CSE486, Penn State
World to Camera Transformation
PC PW
W
Y
X
R
Rotate to
align axes
C
Z
U
V
Translate by - C
(align origins)
P C = R ( PW - C )
= R PW + T
Robert Collins
CSE486, Penn State
X
x= f
Z
Y
y= f
Z
Perspective Matrix Equation
(Camera Coordinates)
 x'  f
 y ' =  0
  
 z '   0
0
f
0
X 
0 0  
Y

0 0  
Z 
1 0  
1
p = M int ⋅ PC
Robert Collins
CSE486, Penn State
Film to Pixel Coords
2D affine transformation from film
coords (x,y) to pixel coordinates (u,v):
u’
v’
w’
a11 a12 xa' 13
f



a21 a22 ya' 23 = 0
  
0 0  z 1'   0
Maff
0
f
0
X 
0 0  
Y

0 0  
Z 
1 0  
1
Mproj
u = Mint PC = Maff Mproj PC
Robert Collins
CSE486, Penn State
Projection of Points on Planar Surface
Perspective
projection
y
Film coordinates
x
Point
on plane
Rotation +
Translation
Robert Collins
CSE486, Penn State
Projection of Planar Points
Robert Collins
CSE486, Penn State
Projection of Planar Points (cont)
Homography H
(planar projective
transformation)
Robert Collins
CSE486, Penn State
Projection of Planar Points (cont)
Homography H
(planar projective
transformation)
Punchline: For planar surfaces, 3D to 2D perspective
projection reduces to a 2D to 2D transformation.
Punchline2: This transformation is INVERTIBLE!
Robert Collins
CSE486, Penn State
Special Case : Frontal Plane
What if the planar surface is perpendicular to
the optic axis (Z axis of camera coord system)?
Then world rotation matrix simplies:
Robert Collins
CSE486, Penn State
Frontal Plane
So the homography for a frontal plane simplies:
Similarity Transformation!
Robert Collins
CSE486, Penn State
Convert to Pixel Coords
Internal camera
parameters
a11 a12 a13
a21 a22 a23
0 0 1
v
pixels
u
Robert Collins
CSE486, Penn State
Planar Projection Diagram
Here’s where
transformation
groups get useful!
Robert Collins
CSE486, Penn State
General Planar Projection
H1
H
A1
Robert Collins
CSE486, Penn State
General Planar Projection
H1
H-1
A1
Robert Collins
CSE486, Penn State
Frontal Plane Projection
S1
A
A1
Robert Collins
CSE486, Penn State
Frontal Plane Projection
S1
A-1
A1
Robert Collins
CSE486, Penn State
General Planar Projection
H1
H2
H
Robert Collins
CSE486, Penn State
Summary: Planar Projection
Internal
params
Perspective
projection
u
Pixel coords
v
y
Homography
x
Rotation + Translation
Point on plane
Applying Homographies to Remove
Perspective Distortion
Robert Collins
CSE486, Penn State
from Hartley & Zisserman
4 point correspondences suffice for
the planar building facade
Robert Collins
CSE486, Penn State
Homographies for
Bird’s-eye Views
from Hartley & Zisserman
Robert Collins
CSE486, Penn State
Homographies for Mosaicing
from Hartley & Zisserman
Robert Collins
CSE486, Penn State
Two Practical Issues
How to estimate the homography given
four or more point correspondences
(will derive L.S. solution now)
How to (un)warp image pixel values to
produce a new picture (last class)
Robert Collins
CSE486, Penn State
Estimating a Homography
Matrix Form:
Equations:
Robert Collins
CSE486, Penn State
Degrees of Freedom?
There are 9 numbers h11,…,h33 , so are there 9 DOF?
No. Note that we can multiply all hij by nonzero k
without changing the equations:
Robert Collins
CSE486, Penn State
Enforcing 8 DOF
One approach: Set h33 = 1.
Second approach: Impose unit vector constraint
Subject to the
constraint:
Robert Collins
CSE486, Penn State
L.S. using Algebraic Distance
Setting h33 = 1
Multiplying through by denominator
Rearrange
Robert Collins
CSE486, Penn State
Algebraic Distance, h33=1 (cont)
2N x 8
Point 1
Point 2
Point 3
Point 4
additional
points
8x1
2N x 1
Robert Collins
CSE486, Penn State
Algebraic Distance, h33=1 (cont)
Linear
equations
Solve:
2Nx8
8x1
A h
8x2N
2Nx8
8x1
AT A h
8x8
Matlab:
=
(AT
h = A\b
b
8x2N
2Nx1
= AT
b
8x1
8x1
(AT A) h
h
=
2Nx1
= (AT b)
-1
A) (AT b)
Robert Collins
CSE486, Penn State
Caution: Numeric Conditioning
R.Hartley: “In Defense of the Eight Point Algorithm”
Observation: Linear estimation of projective transformation
parameters from point correspondences often suffer from poor
“conditioning” of the matrices involves. This means the solution
is sensitive to noise in the points (even if there are no outliers).
To get better answers, precondition the matrices by performing
a normalization of each point set by:
• translating center of mass to the origin
• scaling so that average distance of points from origin is sqrt(2).
• do this normalization to each point set independently
Robert Collins
CSE486, Penn State
PointSet1
Hartley’s PreConditioning
-1 H
H = T2-1 S2H?
norm S1 T1
PointSet2
T1
Translate
center of mass
to origin
T2
S1
Scale so average
point dist is sqrt(2)
S2
Estimate Hnorm
Robert Collins
CSE486, Penn State
A More General Approach
What might be wrong with setting h33 = 1?
If h33 actually = 0, we can’t get the right answer.
Robert Collins
CSE486, Penn State
Algebraic Distance, ||h||=1
||h|| = 1
Multiplying through by denominator
Rearrange
=0
=0
Robert Collins
CSE486, Penn State
Algebraic Distance, ||h||=1 (cont)
2N x 9
4
P
O
I
N
T
S
additional
points
9x1
2N x 1
Robert Collins
CSE486, Penn State
Algebraic Distance, ||h||=1 (cont)
Homogeneous
equations
2Nx9
9x2N
9x1
Solve:
2Nx9
A h
AT A h
9x9
9x1
=
9x2N
= AT
9x1
9x1
(AT A) h
= 0
SVD of ATA = U
D
2Nx1
0
2Nx1
0
UT
Let h be the column of U (unit eigenvector)
associated with the smallest eigenvalue in D.
(if only 4 points, that eigenvalue will be 0)