Projective 3D geometry
class 4
Multiple View Geometry
Comp 290-089
Marc Pollefeys
Content
• Background: Projective geometry (2D, 3D),
Parameter estimation, Algorithm evaluation.
• Single View: Camera model, Calibration, Single
View Geometry.
• Two Views: Epipolar Geometry, 3D
reconstruction, Computing F, Computing
structure, Plane and homographies.
• Three Views: Trifocal Tensor, Computing T.
• More Views: N-Linearities, Multiple view
reconstruction, Bundle adjustment, autocalibration, Dynamic SfM, Cheirality, Duality
Multiple View Geometry course schedule
(subject to change)
Jan. 7, 9
Intro & motivation
Projective 2D Geometry
Jan. 14, 16
(no course)
Projective 2D Geometry
Jan. 21, 23
Projective 3D Geometry
Parameter Estimation
Jan. 28, 30
Parameter Estimation
Algorithm Evaluation
Camera Models
Camera Calibration
Feb. 11, 13
Single View Geometry
Epipolar Geometry
Feb. 18, 20
3D reconstruction
Fund. Matrix Comp.
Feb. 25, 27
Structure Comp.
Planes & Homographies
Trifocal Tensor
Three View Reconstruction
Mar. 18, 20
Multiple View Geometry
MultipleView Reconstruction
Mar. 25, 27
Bundle adjustment
Papers
Apr. 1, 3
Auto-Calibration
Papers
Apr. 8, 10
Dynamic SfM
Papers
Apr. 15, 17
Cheirality
Papers
Apr. 22, 24
Duality
Project Demos
Feb. 4, 6
Mar. 4, 6
Last week …
line at infinity (affinities)
l   0,0,1
T
1 0 0 
C*  0 1 0
0 0 0
circular points (similarities)
C*  IJ T  JI T
lT C* m  0
(orthogonality)
Last week …
pole-polar relation
cross-ratio
l  Cx
x  C* l
conjugate points & lines
y Cx  0
T
m T C* l  0
Chasles’ theorem
A
projective conic classification
x 2  y 2  w2  0
affine conic classification
B
C
X
D
Fixed points and lines
H e  λe
T
H l  λl
(eigenvectors H =fixed points)
(1=2  pointwise fixed line)
(eigenvectors H-T =fixed lines)
Singular Value Decomposition
A mn  U mm Σ mn VnTn
 1 0  0 
0   0 
2



  
Σ

0
0


n






0
0

0


mn
1   2     n  0
UT U  I
VT V  I
A  U1  1 V1T  U 2  2 V2T    U n  n VnT
UΣ
Σ VT X
Singular Value Decomposition
• Homogeneous least-squares
min AX subject to X  1
• Span and null-space
S L  U1 U 2 ; N L  U3 U 4 
S R  V1V2 ; N R  V3V4 
A  UΣ V
T
solution X  Vn
 1 0
0 
2
Σ
0 0

0 0
0 0
0 0
0 0

0 0
• Closest rank r approximation
~
~ T
A  UΣ
UΣ V
~
  diag  1 ,  2 ,,  r , 0r 1,,
,, 0n
• Pseudo inverse
A   VΣ  U T   diag  11 ,  21 ,,  r1 , 0 ,, 0 
Projective 3D Geometry
• Points, lines, planes and quadrics
• Transformations
• П∞, ω∞ and Ω
∞
3D points
3D point
 X , Y , Z T in R3
T
X   X1 , X 2 , X 3 , X 4 
in P3
T
 X1 X 2 X 3 
T
X  
,
,
,1   X , Y , Z , 1
 X4 X4 X4 
projective transformation
X'  H X (4x4-1=15 dof)
X 4  0
Planes
3D plane
Transformation
X'  H X
π'  H -T π
π1 X  π 2Y  π3 Z  π 4  0
π1 X 1  π 2 X 2  π3 X 3  π 4 X 4  0
πTX  0
Euclidean representation
~
n . X d  0
n  π1 , π 2 , π3 
π4  d
T
~
T
X  X ,Y , Z 
X4 1
d/ n
Dual: points ↔ planes, lines ↔ lines
Planes from points
Solve π from X1T π  0, X T2 π  0 and X 3T π  0
X1T 
 T
X 2  π  0
X 3T 
 
 X1T 
 
(solve π as right nullspace of  X T2  )
 X 3T 
 
Or implicitly from coplanarity condition
 X 1  X 1 1  X 2 1  X 3 1 
 X X  X  X  
2
1
2
3 2
det X X1X 2 X32  0 2
0
 X 3  X 1 3  X 2 3  X 3 3 


 X 4  X 1 4  X 2 4  X 3 4 
X 1D234  X 2 D134  X 3 D124  X 4 D123  0
T
π  D234 ,D134 , D124 ,D123 
Points from planes
Solve X from π1T X  0, π T2 X  0 and π 3T X  0
 π1T 
 T
π 2  X  0
 π 3T 
 
(solve Xas right nullspace of
 π1T 
 T )
π 2 
 π 3T 
 
Representing a plane by its span
X Mx
M  X1X 2 X3 
πT M  0
p 
M 
I
π  a, b, c, d 
T
 b c d
p    , , 
 a a a
T
Lines
(4dof)
A T 
W  T
B 
λA μB
T


P
*
W   T
Q 
λP μQ
W W  WW
*
T
*T
 022
Example: X-axis
0 0 0 1 
W

1
0
0
0


0 0 1 0
W 

0
1
0
0


*
Points, lines and planes
W
M   T
X 
W* 
M T
π 
W
Mπ  0
X
W*
MX  0
π
Plücker matrices
Plücker matrix (4x4 skew-symmetric homogeneous matrix)
lij  Ai B j  Bi A j
L  ABT  BA T
1. L has rank 2
LW
*T
 0 42
2. 4dof
3. generalization of l  x  y
4. L independent of choice A and B
5. Transformation L'  HLH T
Example: x-axis
0 
1
0
0 
0 
0
T
L   1 0 0 0   0 0 0 1  
0 
0 
0
 
 

1
0 
1
0 0  1
0 0 0 
0 0 0

0 0 0
Plücker matrices
Dual Plücker matrix L*
L*  PQT  QPT
*'
L  H-T LH -1
Correspondence
*
*
*
l12 : l13 : l14 : l23 : l42 : l34  l34
: l42
: l23
: l14* : l13* : l12*
Join and incidence
π  L*X (plane through point and line)
L*X  0
(point on line)
X  Lπ
(intersection point of plane and line)
Lπ  0
(line in plane)
L1, L2 ,π  0
(coplanar lines)
Plücker line coordinates
  l12 , l13 , l14 , l23 , l42 , l34 
T
P
l12l34  l13l42  l14l23  0
5
on Klein quadric
 
T K  0
ˆ  A, B, Â,0B̂ 0 0 0 0 1 l12 
, 
0 0 0 0 1 0   l 
13 



ˆ
ˆ
ˆ
ˆ
ˆ  l lˆ
det A, B, Â, B̂  l12
l

l
l

l
l

l
l

l
l
34
13
42
14
23
23
14
42
0 0 0 1 0 0 l14  13 34 12
l12l13l14l23l42l34 ˆ T K   | ˆ
   0
0 0 1 0 0 0 l23 
0 1 0 0 0 0 l42 

 
1 0 0 0 0 0 l34 




Plücker line coordinates
 |   0
(Plücker internal constraint)
 | ˆ   detA, B, Â, B̂  0
(two lines intersect)
 | ˆ   detP, Q, P̂, Q̂  0
(two lines intersect)
 | ˆ   P AQ B Q AP B  0
T
T
T
T
(two lines intersect)
Quadrics and dual quadrics
X T QX  0
1.
2.
3.
4.
5.
6.
(Q : 4x4 symmetric matrix)


Q



9 d.o.f.
in general 9 points define quadric
det Q=0 ↔ degenerate quadric
pole – polar π  QX
(plane ∩ quadric)=conic C  M T QM
transformation Q' H -T QH-1
  
  
  

  
π : X  Mx
π T Q* π  0
1. relation to quadric Q  Q
(non-degenerate)
*
* T
2. transformation Q'  HQ H
*
-1
Quadric classification
Rank
Sign.
Diagonal
Equation
4
4
(1,1,1,1)
X2+ Y2+ Z2+1=0
2
(1,1,1,-1)
X2+ Y2+ Z2=1
Sphere
0
(1,1,-1,-1)
X2+ Y2= Z2+1
Hyperboloid (1S)
3
(1,1,1,0)
X2+ Y2+ Z2=0
Single point
1
(1,1,-1,0)
X 2 + Y 2 = Z2
Cone
2
(1,1,0,0)
X2 + Y2 = 0
Single line
0
(1,-1,0,0)
X 2 = Y2
Two planes
1
(1,0,0,0)
X2=0
Single plane
3
2
1
Realization
No real points
Quadric classification
Projectively equivalent to sphere:
sphere
ellipsoid
hyperboloid paraboloid
of two sheets
Ruled quadrics:
hyperboloids
of one sheet
Degenerate ruled quadrics:
cone
two planes
Twisted cubic
twisted
conic cubic
  a13
 22  a14 3 
 x1 
 1   a11  a12
12
13


 
  
2
3
 x2   A     a21  a22  a23  a24 
 x   A 2    a  a   a  2  a  3 
33
 3 
   31 32
 34 
 3   a  a   a  2  a  3 
x 
   41 42
 4
43
44

1.
2.
3.
4.
5.
3 intersection with plane (in general)
12 dof (15 for A – 3 for reparametrisation (1 θ θ2θ3)
2 constraints per point on cubic, defined by 6 points
projectively equivalent to (1 θ θ2θ3)
Horopter & degenerate case for reconstruction
Hierarchy of transformations
Projective
15dof
A
vT

t
v 
Affine
12dof
 A t
0 T 1


Similarity
7dof
s R t 
 0T 1


The absolute conic Ω∞
Euclidean
6dof
 R t
0 T 1


Volume
Intersection and tangency
Parallellism of planes,
Volume ratios, centroids,
The plane at infinity π∞
Screw decomposition
Any particular translation and rotation is
equivalent to a rotation about a screw axis
and a translation along the screw axis.
2D Euclidean Motion and the screw
decomposition
screw axis // rotation axis
t  t //  t 
The plane at infinity
0
 
T
A
0  0 
T

π  H A π  
   π 
 A t 1 0 
1
 
The plane at infinity π is a fixed plane under a
projective transformation H iff H is an affinity
1.
2.
3.
4.
canical position π   0,0,0,1
contains directions D   X1 , X 2 , X 3 ,0T
two planes are parallel  line of intersection in π∞
line // line (or plane)  point of intersection in π∞
T
The absolute conic
The absolute conic Ω∞ is a (point) conic on π.
In a metric frame:
2
2
2
X1  X 2  X 3 
0
X4

or conic for directions:  X , X , X I X , X , X T
1
2
3
1
2
3
(with no real points)
The absolute conic Ω∞ is a fixed conic under the
projective transformation H iff H is a similarity
1. Ω∞ is only fixed as a set
2. Circle intersect Ω∞ in two points
3. Spheres intersect π∞ in Ω∞
The absolute conic

d d 
Euclidean:
cos  
d d d d 

d  d 
cos


Projective:
d  d d  d 
T
1
2
T
1 1
T
1
T
1
d1T  d 2  0
 1
T
2
2

2
T
2

2
(orthogonality=conjugacy)
normal
plane
The absolute dual quadric
I
  T
0
*

0
0
The absolute conic Ω*∞ is a fixed conic under the
projective transformation H iff H is a similarity
1. 8 dof
2. plane at infinity π∞ is the nullvector of
3. Angles:
π T * π
cos  
1

*
2
π  π π  π 
T
1
*
 1
T
2
*

2
Next classes:
Parameter estimation
Direct Linear Transform
Iterative Estimation
Maximum Likelihood Est.
Robust Estimation