Correspondence-Free Determination
of the Affine Fundamental Matrix
2007. 2. 6 (Tue)
Young Ki Baik, Computer Vision Lab.
Correspondence-Free Determination of the Affine
fundamental Matrix
• References
• Correspondence-Free Determination of the
Affine Fundamental Matrix
•
Stefan Lehmann et. al. PAMI 2007
• Radon-based Structure from Motion
Without Correspondences
•
Ameesh Makadia et. al. CVPR 2005
• Robust Fundamental Matrix Determination
without Correspondences
•
Stefan Lehmann et. al. APRS 2005
2
Correspondence-Free Determination of the Affine
fundamental Matrix
• Contents
• The conventional method of SfM
• Features of the proposed method
• Theory of the proposed algorithm
• Experimental results
• Discussion
3
Correspondence-Free Determination of the
Affine fundamental Matrix
• Conventional SfM
Image
Sequence
Feature
Extraction/
Matching
Relating
Image
Projective
Reconstruction
AutoCalibration
Dense
Matching
3D Model
Building
4
Correspondence-Free Determination of the
Affine fundamental Matrix
• The Problem of conventional SfM
• The high sensitivity of fundamental matrix
•
Noise and outlier correspondences in feature data
severely affect the precision of the fundamental
matrix
•
Incomplete 3D reconstruction
5
Correspondence-Free Determination of the
Affine fundamental Matrix
• The Key Feature
• Correspondence-free
Finding Correspondence (X)
• Illumination changes-free (?)
• Intensity value (X)
• Position of features (O)
•
• Limitation
• Occlusion ? (X)
• Affine camera only!!
6
Correspondence-Free Determination of the
Affine fundamental Matrix
• Parallel projection
• Orthographic projection
7
Correspondence-Free Determination of the
Affine fundamental Matrix
• Mathematical Model
• Assumption
•
•
We have 3-dimensional N features.
The 3D feature space is represented by,
f 3 x, y, z  
N
  x  x , y  y , z  z
n
n
n

n 1
xn , yn , zn  : Individual feature locations
   : Dirac delta function
8
Correspondence-Free Determination of the
Affine fundamental Matrix
• Mathematical Model
• Assumption
•
•
•
Parallel projection model determines the 2D
feature projections along the lines that are
running parallel to the view axis (z-axis) of the
camera.
The model considers a continuous projection
plane with infinite extent.
The corresponding 2D projection data is…
f 2 x, y  

R
f 3 x, y, z dz 
N
  x  x , y  y 
n
n
n 1
9
Correspondence-Free Determination of the
Affine fundamental Matrix
• Mathematical Model
• Fourier spectra
•
The Fourier spectra of
be denoted as
F2  , 


e  j xn yn 


f 3 x, y, z e  j x y z dxdydz
R3
N

N
f 2 x, y e  j x y dxdy 
R2
F3  , ,   
f 2 x, y  and f 3 x, y, z  can

n 1
e  j xn yn zn 
n 1
F2  ,   F3  , ,   0
 ,,   : 3D frequency components
The projection - slice theorem
10
Correspondence-Free Determination of the
Affine fundamental Matrix
• Mathematical Model
• 2-view case
•
The 3D correspondence feature point
T




P  x, y, z  , P  x , y , z 
T
•
Relation between images
P  RP  t
R,t : nonhomogeneous 3D rotation and translation matrix
•
The 3D frequency vector
Δ   , ,  
T
11
Correspondence-Free Determination of the
Affine fundamental Matrix
• Mathematical Model
• 2-view case
•
Relation between 3D spectrums
F3Δ   e
 F R T Δ
 j tT Δ
F3Δ   e
3

 j PT Δ

P  RP  t
The equation shows that rotation R also establishes
the transformation between corresponding frequency
indices in the 3D Fourier spaces of the original and
the transformed spectrum or scene.
12
Correspondence-Free Determination of the
Affine fundamental Matrix
• Mathematical Model
• Matching line
F2  ,   F3  , ,   0
•
The magnitudes of two spectra along these lines
will be identical, while the phases will show a
linear offset dependent upon the translational
component of transformation.
•
The proposed method is
to detect these matching
lines.
13
Correspondence-Free Determination of the
Affine fundamental Matrix
• Mathematical Model
• Matching line angle pair  ,  
•
Angle pair of the matching lines with respect to
the  ,   axes of the frequency spectra F and F’,
respectively.


14
Correspondence-Free Determination of the
Affine fundamental Matrix
• Mathematical Model
• Analysis of the transformation parameters
•
 , ,  ,  as the corresponding frequency
locations along the matching lines of the
spectrum F of the first and the spectrum F’ of the
second set of 2D features, respectively.
•
It follows that,
F2  ,   e  j x0 y0  F2  ,  t  x0 , y0 , z0 
15
Correspondence-Free Determination of the
Affine fundamental Matrix
• Mathematical Model
• Analysis of the transformation parameters
F2  ,   e  j x0 y0  F2  ,  t  x0 , y0 , z0 
   cos ,   sin 
    cos ,    sin 
F2 v cos , v sin   e  j v F2 v cos , v sin 
  x0 cos   y0 sin 
F1 v   F2 v cos , v sin 
16
Correspondence-Free Determination of the
Affine fundamental Matrix
• Mathematical Model
• Derivation of a 3D rotation matrix
R  Rz  Rx R z 
R z 
 cos   sin   0 




  sin 
cos
0
 0
0
1 

0
1

Rx   0 cos
 0 sin 

R z 
 : unknownangle
 cos

   sin 
 0



 sin  
cos 
0
sin 
cos
0
0

0
1 
17
Correspondence-Free Determination of the
Affine fundamental Matrix
• Estimation of the fundamental matrix
• By using 3D rotation matrix, we can obtain the
relation between 2D projection point (x’,y’) of a 3D
feature (x,y,z) with translation.
x  cos  cos  sin   cos sin  x
 cos  sin   sin   cos cos y
 sin   sin  z  x0
y   sin   cos  cos  cos sin  x
 sin   sin   cos  cos cos  y
 cos  sin  z  y0
18
Correspondence-Free Determination of the
Affine fundamental Matrix
• Estimation of the fundamental matrix
• In the orthographic projection case, all epipolar
lines are parallel.
• Then we can denote the epipolar line of 2D feature
point (x,y) as
px  qy  c
c depends on (x, v)
c  [ pcos  cos  sin   cos sin  
 qsin   cos  cos  cos sin  ]x
 [ pcos  sin   sin   cos cos 
 qsin   sin   cos  cos cos ] y
 [ psin   sin    qcos  sin  ]z  px0  qy0
19
Correspondence-Free Determination of the
Affine fundamental Matrix
• Estimation of the fundamental matrix
px  qy  c
p  cos , q  sin  
c  [ pcos  cos  sin   cos sin  
 qsin   cos  cos  cos sin  ]x
 [ pcos  sin   sin   cos cos 
 qsin   sin   cos  cos cos ] y
 [ psin   sin    qcos  sin  ]z  px0  qy0
cos x  sin y  cos x  sin y  
20
Correspondence-Free Determination of the
Affine fundamental Matrix
• Estimation of the fundamental matrix
cos x  sin y  cos x  sin y  
0 0 a


F  0 0 b
c d e


ax  by  cx  dy  e  0
0
cos  
 0


F  0
0
 sin  
  cos   sin  





 , ,  ?
21
Correspondence-Free Determination of the
Affine fundamental Matrix
• Estimation of matching line angle
• For the practical purpose, corresponding discrete
spectra should be defined as follows.
bk  F1 2kf 
ck  F12kf 
f : frequency resolution
   2f : circular frequency resolution
 exp  jk  x
N
bk 
n

cos  y n sin  
n 1
N
ck 
 exp  jk  x cos   y sin  
n
n 1
n
22
Correspondence-Free Determination of the
Affine fundamental Matrix
• Estimation of matching line angle
• The final object function
 
 
d 2  arg max  
 , , 



  jk  
bk  ck e
k 1
N
2
b c
2
 

 



2
• Discrete Fourier-Mellin transformation method
•
To find out the matching line
(According to the well known shift theorem of
the FT, a shift in the space domain corresponds
to a phase shift in the frequency domain.)
23
Correspondence-Free Determination of the
Affine fundamental Matrix
• Overall flow
24
Correspondence-Free Determination of the
Affine fundamental Matrix
• Experimental result
•
•
•
•
test images : telephoto lens
Feature points : Harris corner detection method
Ideal epipolar lines are the horizontal lines.
The proposed method shows us good result relative
to conventional methods.
25
Camera Calibration Methods for Wide Angle view
• Discussion
• Key feature
•
Correspondence-free method for obtaining the
fundamental matrix is presented.

Matching line exists between the Fourier
transformed data.
• Limitation
•
Proposed method


Considers only affine projection model
Does not treat occlusion problem
• Future work
•
Applying projective projection model
26