Lecture 8
Camera Models
Professor Silvio Savarese
Computational Vision and Geometry Lab
Silvio Savarese
Lecture 8 -
15-Oct-14
Lecture 8
Camera Models
• Pinhole cameras
• Cameras & lenses
• The geometry of pinhole cameras
• Other camera models
Reading:
[FP] Chapter 1 “Cameras”
[FP] Chapter 2 “Geometric Camera Models”
[HZ] Chapter 6 “Camera Models”
Some slides in this lecture are courtesy to Profs. J. Ponce, S. Seitz, F-F Li
Silvio Savarese
Lecture 8 -
15-Oct-14
How do we see the world?
• Let’s design a camera
– Idea 1: put a piece of film in front of an object
– Do we get a reasonable image?
Pinhole camera
• Add a barrier to block off most of the rays
– This reduces blurring
– The opening known as the aperture
Some history…
Milestones:
• Leonardo da Vinci (1452-1519):
first record of camera obscura
Some history…
Milestones:
• Leonardo da Vinci (1452-1519):
first record of camera obscura
• Johann Zahn (1685): first
portable camera
Some history…
Milestones:
• Leonardo da Vinci (1452-1519):
first record of camera obscura
• Johann Zahn (1685): first
portable camera
• Joseph Nicephore Niepce (1822):
first photo - birth of photography
Photography (Niepce, “La
Table Servie,” 1822)
Some history…
Milestones:
• Leonardo da Vinci (1452-1519):
first record of camera obscura
• Johann Zahn (1685): first
portable camera
• Joseph Nicephore Niepce (1822):
first photo - birth of photography
• Daguerréotypes (1839)
• Photographic Film (Eastman, 1889)
• Cinema (Lumière Brothers, 1895)
• Color Photography (Lumière Brothers,
1908)
Photography (Niepce, “La
Table Servie,” 1822)
Let’s also not forget…
Motzu
(468-376 BC)
Oldest existent
book on geometry
in China
Aristotle
Al-Kindi (c. 801–873)
(384-322 BC)
Ibn al-Haitham
Also: Plato, Euclid
(965-1040)
Pinholeprojection
camera
Pinhole perspective
f
o
f = focal length
o = aperture = pinhole = center of the camera
Pinhole camera
x 
 x 


P   y   P   
y 

 z 

x'  f



 y'  f


x
z
y
z
Derived using similar triangles
Pinhole camera
i
f
k
O
P’=[x’, f ]
P = [x, z]
x x

f
z
Pinhole camera
f
f
Common to draw image plane in front of
the focal point.
What’s the transformation between these 2 planes?

x'  f



 y'  f


x
z
y
z
Pinhole camera
Is the size of the aperture important?
Kate lazuka ©
Shrinking
aperture
size
- Rays are mixed up
-Why the aperture cannot be too small?
-Less light passes through
-Diffraction effect
Adding lenses!
Cameras & Lenses
• A lens focuses light onto the film
Cameras & Lenses
“circle of
confusion”
• A lens focuses light onto the film
– There is a specific distance at which objects are “in
focus”
– Related to the concept of depth of field
Cameras & Lenses
• A lens focuses light onto the film
– There is a specific distance at which objects are “in
focus”
– Related to the concept of depth of field
Cameras & Lenses
focal point
f
• A lens focuses light onto the film
– All parallel rays converge to one point on a plane
located at the focal length f
– Rays passing through the center are not deviated
Cameras & Lenses
Z’
-z
From Snell’s law:
x

x '  z' z

 y'  z ' y

z
f
zo
z'  f  z o
R
f
2( n  1)
Thin Lenses
zo
z'  f  z o
R
f
2( n  1)
Focal length
Snell’s law:
n1 sin 1 = n2 sin 2
Small angles:
n1 1  n2 2
n1 = n (lens)
n1 = 1 (air)
x

x
'

z
'

z

 y'  z ' y

z
Issues with lenses: Radial Distortion
– Deviations are most noticeable for rays that pass through
the edge of the lens
No distortion
Pin cushion
Barrel (fisheye lens)
Image magnification
decreases with distance
from the optical axis
Lecture 2
Camera Models
• Pinhole cameras
• Cameras & lenses
• The geometry of pinhole cameras
• Intrinsic
• Extrinsic
• Other camera models
Silvio Savarese
Lecture 8 -
15-Oct-14
Pinholeprojection
camera
Pinhole perspective
f
o
f = focal length
o = center of the camera
x y
( x , y, z )  ( f , f )
z z
E
 
3
2
From retina plane to images
Pixels, bottom-left coordinate systems
Coordinate systems
yc
xc
Converting to pixels
1. Off set
y
yc
xc
C=[cx, cy]
x
x
y
( x , y, z )  ( f  c x , f  c y )
z
z
Converting to pixels
y
1. Off set
2. From metric to pixels
yc
xc
C=[cx, cy]
x
y
( x , y, z )  ( f k  c x , f l  c y )
z
z


Units: k,l : pixel/m
x
f :m
Non-square pixels
, 
: pixel
Converting to pixels
y
x
y
( x , y, z )  (  c x ,   c y )
z
z
yc
C=[cx, cy]
xc
• Matrix form?
x
A related question:
• Is this a linear transformation?
x y
( x , y, z )  ( f , f )
z z
Is this a linear transformation?
No — division by z is nonlinear
How to make it linear?
Homogeneous coordinates
homogeneous image
coordinates
homogeneous scene
coordinates
• Converting from homogeneous
coordinates
Camera Matrix
y
x
y
( x , y, z )  (  c x ,   c y )
z
z
yc
xc
C=[cx, cy]
x
 x  c x z  
X    y  c y z    0

  0
z
0
cx

cy
0
1
x 
0  
y


0
z
0  
1 
Perspective Projection
Transformation
x 
f x   f 0 0 0   
y





X    f y   0 f 0 0 
z
 z  0 0 1 0  
1 
M

f
X i  
f

x
z
y

z
X  M X
H
 
4
3
Camera Matrix
Camera
matrix K
X  M X
 KI 0 X

X   0
 0
0
cx

cy
0
1
x 
0  
y


0
z
0  
1 
Finite projective cameras
Skew parameter
y
yc
xc

C=[cx, cy]
x

X   0
 0
s
cx

cy
0
1
x 
0  
y


0
z
0  
1 
K has 5 degrees of freedom!
Lecture 2
Camera Models
• Pinhole cameras
• Cameras & lenses
• The geometry of pinhole cameras
• Intrinsic
• Extrinsic
• Other camera models
Silvio Savarese
Lecture 8 -
15-Oct-14
World reference system
R,T
jw
kw
Ow
iw
•The mapping so far is defined within the camera
reference system
• What if an object is represented in the world
reference system
3D Rotation of Points
Rotation around the coordinate axes, counter-clockwise:
0
1
Rx ( )  0 cos 
0 sin 
p’
Y’

p
y
x’
z
x

 sin  
cos  
 cos  0 sin  
R y (  )   0
1
0 
 sin  0 cos  
cos   sin  0
Rz ( )   sin  cos  0
 0
0
1
0
World reference system
R,T
jw
X
kw
Ow
iw
X’
R T 
In 4D homogeneous coordinates: X  
Xw

 0 1  44
Internal parameters
X '  K I
0 X  K I
R T 
0 
Xw

 0 1  44
External parameters
 K R T  X w
M
Projective cameras
R,T
jw
X
kw
Ow
iw
X’
X 31  M 3 x 4 X w K 33 R
T  34 X w 41
How many degrees of freedom?
5 + 3 + 3 =11!

K   0
 0
s

0
cx 
c y 
1 
Camera calibration
More details in CS231A
Estimate intrinsic and extrinsic parameters
from 1 or multiple images
X 31  M 3 x 4 X w K 33 R
T  34 X w 41
How many degrees of freedom?
5 + 3 + 3 =11!

K   0
 0
s

0
cx 
c y 
1 
Projective cameras
R,T
jw
X
kw
Ow
iw
X’
X31  M X w K 33 R
 m1 
 m1 X W 
 m 2  X W  m 2 X W 
m 3 
m 3 X W 
 m1 
m 
M

T 34 X w 41
 2
m 3 
E
m1 X w m 2 X w
(
,
)
m3 X w m3 X w

Properties of Projection
• Points project to points
• Lines project to lines
• Distant objects look smaller
Properties of Projection
•Angles are not preserved
•Parallel lines meet!
Parallel lines in the world
intersect in the image at a
“vanishing point”
Horizon line (vanishing line)
l
horizon
Horizon line (vanishing line)
One-point perspective
• Masaccio, Trinity,
Santa Maria
Novella, Florence,
1425-28
Credit slide S. Lazebnik
Lecture 2
Camera Models
• Pinhole cameras
• Cameras & lenses
• The geometry of pinhole cameras
• Intrinsic
• Extrinsic
• Other camera models
Silvio Savarese
Lecture 8 -
15-Oct-14
Projective camera
f’
R’
p’
q’
r’
Q’
O
P’
Weak perspective projection
When the relative scene depth is small compared to its distance from the camera
zo
f’
R’
p’
q’
r’
R
Q’
O
P’
Q
P
Weak perspective projection
When the relative scene depth is small compared to its distance from the camera
zo
f’
R’
p’
q’
r’
R
Q’
O
P’

 x'  



 y'  

Q
P
f'
x
z
f'

 x'   z x
0
 
f'

y'   y
f'
y

z0
z
Magnification m
Weak perspective projection
zo
f’
R’
p’
q’
r’
R
Q’
Q
O
P  M Pw
A b
M 

0
1


P
P’
Instead of
M  K R T 
A b


v
1


P  M Pw
 m1 
 m1 PW 
 m 2  PW  m 2 PW 
m 3 
m 3 PW 
A b
M 

v
1


 m1 
 m 2 
m 3 
E
m1 Pw m 2 Pw
(
,
)
m 3 Pw m 3 Pw
P  M Pw
 m1 
 m1 PW 
 m 2  PW  m 2 PW 
m 3 
 1 
E
 (m1 Pw , m 2 Pw )
magnification
Perspective
A b
M 

0
1


m1
 m1  


 m 2   
m2

m 3  0 0 0 1
Weak perspective
Orthographic (affine) projection
Distance from center of projection to image plane is infinite

 x'  



 y'  

f'
x
z
f'
y
z
 x'   x
 
 y'   y
Pros and Cons of These Models
• Weak perspective much simpler math.
– Accurate when object is small and distant.
– Most useful for recognition.
• Pinhole perspective much more accurate for
scenes.
– Used in structure from motion.
Weak perspective projection
The Kangxi Emperor's Southern Inspection Tour (1691-1698) By Wang Hui
Weak perspective projection
The Kangxi Emperor's Southern Inspection Tour (1691-1698) By Wang Hui
Things to remember