Single View Modeling
• Guillaume Dugas-Phocion
• Li Zhang
• Jean-Sebastien Samson
• Steve M. Seitz
Problem
Extract a 3D model from a single image
Approach
Free-form surface modeling without strong assumptions

No assumptions about
– Shading
– Geometry

Creation of a surface model
– User-added constraints
– Fast and simple user interface
– Real-time adjustement of the model
Related work

Single View Modeling
– A. Criminisi, I. Reid, and A. Zisserman
“Single View Metrology”, 1999.
– J. J. Koenderink, A. J. van Doorn, A. M. L. Kappers
and J. T. Todd
“Pictorial Relief”, 1998.
– S.B. Kang,
“Depth Painting for Image-Based Rendering Applications”, 1998.
– V. Blanz and T. Vetter,
“A Morphable Model For The Synthesis Of 3D Faces”,
1999.
Related work (2)

Surface Modeling
– R. Szeliski,
“Fast Surface Interpolation Using Hierarchical Basis
Functions”, 1990.
– S. Gortler and M. Cohen,
“Variational Modeling with Wavelets”, 1994.
Organisation
Organisation
I.
II.
Constraints & Discontinuities
Hierarchical structure
a. Quad-Tree
b. The new wavelet-based transformation
III. Results and examples
IV. Future work
I Constraints & Discontinuities
Framework
Build a surface model by :

Setting some constraints
– Fixed points
– Surface normals

Adding a smoothness to the surface
 Modifying this smoothness locally
– Position and normal discontinuities
– Fairing curves
Demo
I Constraints & Discontinuities
Surface representation
Height field
•
•
•
Logical solution for a single view
Advantages : easy math, fast convergence
Drawbacks : holes, no layer structure
I Constraints & Discontinuities
Position constraints
Fix the height of a surface point

Linear constraint
 2 possibilities :
– The point is a grid point :
 Expression : f(xi) = h
– The point is NOT a grid point
 Interpolation to get f(x)
I Constraints & Discontinuities
Normal constraints
Fix the local orientation of the surface

a


• Dy f(P) =

• Dx f(P) =
P : point where we set the
normal constraint
f() – f()
a
f() – f()
a
I Constraints & Discontinuities
Smoothness function
Give a smooth-like behavior to the surface

Solution : minimize a smoothness function
σ=
 D
surface

xx
² f(x,y)
+ Dyy² f(x,y) +
2 · Dxy² f(x,y)
dx dy
Discrete version :
σd =
 w · (D
i
xx
²
+ Dyy² +
grid points
wi =
2 · Dxy² ) · f(pi)
 Φ (x,y)
i
dx dy
I Constraints & Discontinuities
Solving the problem
How to obtain a sparse linear system ?



min σd = vT H v
Smoothness function :
2nd order polynomial in height values :
 Hv=0
Constraints :
1st order polynomial in height values :
Av=b
Final sparse system :
H
AT
v
0
=
A
0

b
I Constraints & Discontinuities
Discontinuities
ATT
H
v
0
=
A
0

b
Adding discontinuities =
modifying the smoothness of the surface =
deleting some terms in the smoothness function
σd =
 w · (D
i
xx
² + Dyy² + 2 · Dxy² ) · f(xi)
I Constraints & Discontinuities
Position discontinuities
Adding a position discontinuity =
In the H matrix, deleting the terms
involved in corresponding edges
I Constraints & Discontinuities
Normal discontinuities
Adding a normal discontinuity =
In the H matrix, deleting the terms
involved in the corresponding points
II Hierarchical structure
Hierarchical structure
Quad-tree structure

Increase the level of details
 Without slowing the system
 Recursive structure (wavelets)
Demo
II Hierarchical structure
Hierarchical structure
Ways of changing the resolution.

User specified

Along a curve

High curvature
II Hierarchical structure
Speed up the solver
no convergence
wavelets. ?
How1D
to hypothesis,
accelerate the
f=
 f(x ) · Φ
Demo
i
i
II Hierarchical structure
Speed up the solver
1D hypothesis, basic wavelets.
v : height field
c : wavelet coefficients
W : wavelet transformation
v=Wc
WT H W
WT AT
c 0
=
AW
0
’
b
II Hierarchical structure
Speed up the solver
Wavelet transformation (from coefficient to height)
1D Convolution
hI = (hA + hB) / 2 + cI
2D Convolution
II Hierarchical structure
Speed up the solver
What happens with discontinuities ?
1D Convolution
Instead
of :
hI = (hA + hB) / 2 + cI
We
get :
hI = hB + cI
Demo
II Hierarchical structure
Speed up the solver
2D Convolution
Instead
of :
hO = (hI + hJ + hK + hL) / 4
+ cO
We
get :
hO = (hI + hL) / 2 + cO
III Results
Results
Demo
IV Future work
Future work

Handle large images
 Handle perspective
 Fill the holes
 Relight the scene
 Constraint detector ?
Questions ?
Further details
Position constraints

Easiest constraint : fixing the height of the ith grid point
– Expression :
f(xi) = h
– Corresponding row of the A matrix :
[0…0 1 0…0]
i-1 i i+1
– Corresponding value in the b vector :
the user-set height h
Position constraints (2)

If the point we want to fix is not a grid point :
we compute its height with the direct neighbors (interpolation)
– f(P) = i · f(I) + j · f(J) + k · f(K) + l · f(L) = h
L
K
P
I
J
– Corresponding row in the A matrix :
[ 0 … cI … cJ … cK … cL … 0 ]
(index)
iI … iJ … iK … iL
– Corresponding value in the b vector : the same h
Linear problem

Each term as a 2nd order polynom in height values :
f(xi,i) + f(xi,i) - 2·f(xi,i)
________________________
Dxx² f(xi,i) = [
]² valid for a regular grid
δx²
=
∑∑x
k,l , m, n f(xk,l)·f(xm,n)
k,l
m,n
= vT·Hxx i,i ·v
v:
vector of the height map.
Hxx i,i : specific operator for Dxx² f(xi,i)
dim = n²
4
dim = n
1D wavelet basis functions
Speed up the solver
2D regular basis functions

the Impulse Response of a Linear System

Concrete Paving Equipment Market to receive overwhelming hike in Revenues by 2025

Powerpoint