One view geometry---camera
modeling
X’
P3
u
X
u’
u
O
v
P2
1
Modeling
‘abstract’ camera: projection from P3 to P2
C:
P3  P2
Math: central proj.
Physics: pin-hole
As lines are preserved so that it is a linear transformation
and can be represented by a 3*4 matrix
 c11

C   c21
c
 31
c12
c13
c22
c23
c32
c33
c14 

c24   C33
c34 

This is the most general camera model without
considering optical distortion
c
2
X’
P3
u
X
u’
u
O
v
P2
3
Properties of the 3*4 matrix P
• d.o.f.?
• Rank(P) = ?
• ker(P)?
• row vectors, planes
• column vectors, directions
• principal plane: w=0
• calibration, how many pts?
• decomposition by QR,
• K intrinsic (5). R, t, extrinsic (6)
• geometric interpretation of K, R, t (backward from u/x=v/y=f/z to P)
• internal parameters and absolute conic
4
Geometric description
From concrete phy. paramters to algebraic param.
Central projection in Cartesian coordinates
• Camera coordinate frame
• image coordinate frame
• world coordinate frame
5
Camera coordinate frame
Z
u
Y
x
X
f
O
y
X
x
v
x
X y
Y

, 
f
Z f
Z
6
 x X
   
 y    Y 
 f  Z
   
In more familiar matrix form:
 x
1



 y    0
 f 
0



0
0
1
0
0
1

0 

0 
0



X 

Y 
Z 

1 

7
Image coordinate frame
Z
u
Y
x
X
f
O
y
X
u
x
v
y
o
x
v
8
Image coordiante frame: intrinsic parameters
u  u0
v  v0
pixels
pixels
ku 

, kv 

x
mm
y
mm
 u

K  0
 0

0
v
0
u0 

v0 
1 
Camera calibration matrix
• Focal length in horizontal/vertical pixels (2) (aspect ratio)
• the principal point (2)
• the skew (1) ?
9
• one rough example: 135 film
World (object) coordinate frame
Zw
Z
x
f
O
Yw
u
Y
X
Xw
y
X
Xw
x
v
10
World coordinate frame: extrinsic parameters
 Xc 


 Yc   R
0
Z 
c

 
 1 


 Xw 


t  Yw 



1
 Z w 
 1 


Finally, we should count properly ...
C 34  K I 33
R
0 
0
t

1
Relation between the abstract algebraic and
geometric models is in the intrinsic/extrinsic
parameters!
11
Summary of camera modelling
• 3 coordinate frame
• projection matrix
• decomposition
• intrinsic/extrinsic param
12
What is the calibration matrix K?
It is the image of the absolute conic, prove it first!
Point conic:
The dual conic:
KK 
T 1
KK T
13
u
T
KK 
T 1
u
(u  u 0 ) 2
 u2

( v  v0 ) 2
 v2
2
(u  u 0 )( v  v0 ) cos 
1  0
au v
(u  u 0 ) 2 ( v  v 0 ) 2
(u  u0 )(v  v0 ) cos 
2


2


1

i
 u2
 v2
au v
v
u

(u 0 , v 0 )
14
It turns the camera into an spherical one, or angular/direction sensor!
Direction vector:
d  K -1u
Angle between two rays ...
15
Don’t forget: when the world is
planar …
 u   c11
  
  v    c21
 w  c
   31
c12
c13
c22
c23
c32
c33
 u   c11
  
  v    c21
 w  c
   31
c12
c22
c32

c14 

c24 
z

c34 




y 
 0

1 

x
 x
c14  
 y 
c24  

c34 

1
 
A general plane homography!
16
Camera calibration
Given u i  X i from image processing or by hand 
• Estimate C
• decompose C into intrinsic/extrinsic
17
Calibration set-up:
3D calibration object
18
u i  ( u i , vi )  X i  ( xi , y i , z i )
c11 xi  c12 yi

ui 

c31 xi  c32 yi


c21 xi  c22 yi
vi 

c31 xi  c32 yi

 c13 zi
 c33 zi
 c23 zi
 c33 zi
 c14
 c34
 c24
 c34
The remaining pb is how to solve this ‘trivial’ system of equations!
19
Review of some basic numerical
algorithms
• linear algebra
• non-linear optimisation
• statistics
Go to see the slides ‘calibration.ppt’ for math review
20
Linear algebra in 5 mins
• Gaussian elimination
• LU decomposition
• Choleski (sym positive) LL^T
• orthogonal decomposition
• QR (Gram-Schmidt)
• SVD (the high(est)light of linear algebra!
A mn  UV T
• row space: first Vs
• null space: last Vs
• col space: first Us
• null space of the trans : last Us
21
Linear methods of computing P
• p34=1
• ||p||=1
• ||p3||=1
Geometric interpretation of these constraints
22
Non-linear …
• ||p3||=1 and
• no skew (p1 X p3) . (p2 X p3) = 0
It’s constrained optimizaion, but non linear …
23
Decomposition
• analytical by equating K(R,t)=P
• QR (more exactly it is RQ)
24
Linear, but non-optimal,
but we want optima, but non-linear,
methods of computing P
Each observation has a (0,sigma) independent Gaussian distribution:
1 ( x  x ) 2 /(2 2 ) 
Pr( x) 
e
e
2
2
d 2 ( x,x )
2 2
The probability of obtaining all measurements x given that the
camera matrix is P:
d 2 ( x, x )
1 
Pr( x | P )  
e
2
2
2 2
MLE (maximum likelihood estimate) is:
min  d 2 ( x, x )  min  d 2 ( x  x (cij, ...))
25
Even linear models, but end up with
non-linear optimization …
Non-linear models for optimization,
always end-up iterative linear
computing!
Yesterday, closed-form, algebraic
methods (gradients), for small scale,
Today, everything numerical in big
scale.
26
min 
c11xi  c12 yi  c13 zi  c14 2
c21xi  c22 yi  c23 zi  c24 2
((ui 
)  ( vi 
))
c31xi  c32 yi  c33 zi  c34
c31xi  c32 yi  c33 zi  c34
How to solve this non-linear system of equations?
27
Non-linear iterative optimisation
• J d = r from vector F(x+d)=F(x)+J d
• minimize the square of y-F(x+d)=y-F(x)-J d = r – J d
• normal equation is J^T J d = J^T r (Gauss-Newton)
• (H+lambda I) d = J^T r (LM)
Note: F is a vector of functions, i.e. min f=(y-F)^T(y-F)
28
General non-linear optimisation
• 1-order , d gradient descent d= g and H =I
• 2-order,
• Newton step: H d = -g
• Gauss-Newton for LS: f=(y-F)^T(y-F), H=J^TJ, g=-J^T r
• ‘restricted step’, trust-region, LM: (H+lambda W) d = -g
R. Fletcher: practical method of optimisation
f(x+d) = f(x)+g^T d + ½ d^T H d
Note: f is a scalar valued function here.
29
statistics
• ‘small errors’ --- classical estimation theory
• analytical based on first order appoxi.
• Monte Carlo
• ‘big errors’ --- robust stat.
• RANSAC
• LMS
• M-estimators
Talk about it later
30
(Non-linear) Optical distorsion
Radial distorsion could be modeled by
x  x (1   2 r 2   4 r 4  ...)
• alpha_i and eventually (xc,yc) are distorsion
parameters
• you may have any degree for r, but probably the
correction is not sufficient for first degree
• x is normalised coordinates, x’ is actual distorted
measures (people are confusing x and x’ 
• normalised as calibrated for ‘photogrammetrists’
(more difficult to write down the cost function), while
normalised just to [-1,1] to compute r (easy to integrate
into the cost function, still pixels)
31
Two approaches:
• add these terms into the non-linear projection equations
using a calibration object
• estimate them independently using the ‘colinearity’
constraint (see Devernay, photogrammetry) to fit lines
32
Solution uniqueness?
Rank of the matrix A?
33
Using a planar pattern
Why? it is more convenient to have a planar calibration pattern
than a 3D calibration object, so it’s very popular now for amateurs.
Cf. the paper by Zhenyou Zhang (ICCV99),
Sturm and Maybank (CVPR99)
Homework: read these papers.
34
 u   c11
  
  v    c21
 w  c
   31
c12
c13
c22
c23
c32
c33
 u   c11
  
  v    c21
 w  c
   31
c12
c22
c32

c14 

c24 
z


c34 

x 

y 
 0

1 
c14  x 
 
c24  y 
1
c34 
 
• first estimate the plane homogrphies Hi from u and x,
1. How to estimate H?
2. Why one may not be sufficient?
• extract parameters from the plane homographies
35
How to extract intrinsic parameters?
KK 
T 1
The absolute conic in image

The (transformed) absolute conic in the plane:
T
x (H
T
KK 
T 1
H )x  0
The circular points of the Euclidean plane (i,1,0) and (-i,1,0) go
thru this conic: two equations on K!
36
Summary of calibration
1.
2.
3.
4.
Get image-space points
Solve the linear system
Optimal sol. by non-linear method
Decomposition by RQ
37
Where is the camera?
Given u i  X i rom image processing or by hand 
• Camera pose (given K)
• where is the camera?
38
Camera pose: 3-point algorithm
• fundamental Euclidean constraint
• 3 point algorithm
• quarternion for rotation?
39
Where is the camera? A
projective setting
• on a plane, 1D camera,
•Circle by 3 pts and a constant angle
•Chasles conics
• 2D camera
• calibration singularities
40
41