FINITE ELEMENT BASED SEQUENTIAL BAYESIAN
NON-RIGID STRUCTURE FROM MOTION
Agudo A., Calvo B., Montiel J. M. M. - Universidad de Zaragoza
{aagudo, bcalvo, josemari}@unizar.es
Abstract
Navier’s equations modelling linear
elastic solid deformations are embedded within an Extended Kalman
Filter (EKF) to compute a sequential Bayesian estimate for Non-Rigid
Structure from Motion (NRSfM). The
scene is coded as a Finite Element
Method (FEM) elastic thin-plate solid,
where the discretization nodes are the
sparse set of scene points salient in the
image. It is assumed a set of Gaussian
forces acting on solid nodes to cause
incremental scene deformation
Contributions
Estimating structure and motion for a
non-rigidity deforming scene:
•
•
•
•
30 Hz. Real-time performance
No prior data association
Full-perspective camera
Dealing with isometric and nonisometric scene deformations
Future Work
• Comparison with respect to
other NRSfM methods
• Extend to extreme deformations
• Extend to process medical image
sequences
Our Approach
cam k+m
cam k
trajectory
cam k+n
∆SW
35
∆SW
30
25
20
35
30
25
25
20
0
20
20
10
0
20
0
10
10
15
non-rigid
g
structure30 k
10
20
5
0
-5
-10
40
15
10
non-rigid
5
0
structure
-5
5
k+n
20
boundary points 40
non-rigid points
30
• Normalized forces to factor out
material properties
• O n3 computational complexity:
– EKF O n3
3
– MP pseudoinverse O 12n
non-rigid
40
structure
k+m
50
10
0
30
-10
-20
-10
-15
50
-15
50
CVPR‐12

• Cameracentric EKF-based
• EKF sequential estimation processing every single frame
xv
W
• Deforming scene
thin-plate FEM
model depending on Young’s
modulus E and Poisson’s ratio ν
xv
xv
• Data association is embedded
within EKF prediction-update
loop
ar
xr
a
a
a = anr
a = anr
W
• FEM K matrix Moore-Penrose
(MP) pseudoinverse
W
xr = xv + ar
• Most of the state vector is dynamic: qualities Kalman Filter
Experimental Results
Multiply Deformed Silicone Sequence:
Paper Sequences:
• Accurate wrt ground truth
• Measurements natural landmarks
• Consistent estimates
• Static or waving camera
• Waving hand-held camera
• Irregular triangles
• Non-Isometric deformation
• Isometric deformation
Acknowledgments
Frame #274
0.7
0.6
0.1
Z [m]
0.65
0.65
0
0
-0.05
0
-0.05
Y [m]
X [m]
01
0.1
0
0.05
-0.05
-0.1
-0.1
Y [m]
X [m]
0
0.15
0.05
01
0.1
0
-0.05
-0.05
0.6
0.1
0.05
-0.1
-0.1
0.65
0.15
0.05
01
0.1
0
0.05
Y [m]
0.15
0.05
01
0.1
0.7
0.65
0.6
0.1
0.6
0.1
0.15
0.05
Frame #495
0.7
0.7
Z [m]
Z [m]
Frame #443
Frame #413
Z [m]
This work was supported by the
Spanish MICINN DIP2009-07130 and
DPI2011-27939-C02-01 grants
-0.05
-0.05
-0.1
0.05
0
-0.1
Y [m]
X [m]
-0.05
-0.1
-0.1
X [m]
References
Additional material:
Frame #412
Frame #256
0.75
0.7
0.7
0.7
Silicone
1.5 · 10−3
0.499
1.5 · 10−5
Paper
1.0 · 10−4
0.499
2.0 · 10−9
Z [m]
Z [m]
0.75
0.65
0.65
0.65
Parameter
h (m)
ν
∆f
Eh (m) (std.)
Frame #596
0.75
Z [m]
[1] Agudo A., Calvo B. and Montiel J. M. M.
FEM Models to Code Non-Rigid EKF
Monocular SLAM. In Workshop on Dynamic
Shape Capture and Analysis (ICCV) 2011
[2] Agudo A., Calvo B. and Montiel J. M. M.
Finite Element based Sequential Bayesian
Non-Rigid Structure from Motion.
In
Conference on Computer Vision and Pattern
Recognition (CVPR) 2012
0.1
0.1
0.1
0 05
0.05
0 15
0.15
0.1
0
-0.05
0
-0.1
Y [m]
0.05
0 05
0.05
0 15
0.15
0.1
0
-0.05
-0.1
-0.05
-0.1
X [m]
0
Y [m]
0.05
0 05
0.05
0 15
0.15
0.1
0
-0.05
-0.1
-0.05
-0.1
X [m]
0
Y [m]
-0.05
-0.1
X [m]
• Initialization: rigid EKF-SLAM
0.05