Partial functional
correspondence
Michael Bronstein
University of Lugano
Intel Corporation
Lyon, 7 July 2016
1/59
Microsoft Kinect 2010
(Acquired by Intel in 2012)
4/59
Different form factor computers featuring Intel RealSense 3D camera
7/59
Deluge of geometric data
3D sensors
Repositories
3D printers
8/59
Applications
Deformable fusion
Motion capture
Motion transfer
Texture mapping
Dou et al. 2015; Sumner, Popović 2004; Faceshift; Cow image: Moore 2014
9/59
Shape correspondence problem
Isometric
10/59
Shape correspondence problem
Isometric
Partial
10/59
Shape correspondence problem
Isometric
Partial
Different representation
10/59
Shape correspondence problem
Isometric
Different representation
Partial
Non-isometric
10/59
Computer Graphics Forum
SGP 2016
Computer Graphics Forum
SGP 2016 Best paper award
11/59
Outline
Background: Spectral analysis on manifolds
Functional correspondence
Partial functional correspondence
Non-rigid puzzles
12/59
Riemannian geometry in one minute
Tangent plane Tm M = local
Euclidean representation of
manifold (surface) M around m
Tm M m
M
13/59
Riemannian geometry in one minute
Tangent plane Tm M = local
Euclidean representation of
manifold (surface) M around m
Tm M m
m0
Riemannian metric
h·, ·iTm M : Tm M × Tm M → R
Tm0 M
depending smoothly on m
13/59
Riemannian geometry in one minute
Tangent plane Tm M = local
Euclidean representation of
manifold (surface) M around m
Tm M m
m0
Riemannian metric
h·, ·iTm M : Tm M × Tm M → R
Tm0 M
depending smoothly on m
Isometry = metric-preserving shape
deformation
13/59
Riemannian geometry in one minute
Tangent plane Tm M = local
Euclidean representation of
manifold (surface) M around m
Tm M m
v
expm (v)
Riemannian metric
h·, ·iTm M : Tm M × Tm M → R
depending smoothly on m
Isometry = metric-preserving shape
deformation
Exponential map
expm : Tm M → M
‘unit step along geodesic’
13/59
Laplace-Beltrami operator
m
f
Smooth field f : M → R
14/59
Laplace-Beltrami operator
m
f ◦ expm
f
Smooth field f ◦ expm : Tm M → R
14/59
Laplace-Beltrami operator
Intrinsic gradient
m
f ◦ expm
∇M f (m) = ∇(f ◦ expm )(0)
Taylor expansion
f
(f ◦ expm )(v) ≈
f (m) + h∇M f (m), viTm M
14/59
Laplace-Beltrami operator
Intrinsic gradient
m
f ◦ expm
∇M f (m) = ∇(f ◦ expm )(0)
Taylor expansion
f
(f ◦ expm )(v) ≈
f (m) + h∇M f (m), viTm M
Laplace-Beltrami operator
∆M f (m) = ∆(f ◦ expm )(0)
14/59
Laplace-Beltrami operator
Intrinsic gradient
m
f ◦ expm
∇M f (m) = ∇(f ◦ expm )(0)
Taylor expansion
f
(f ◦ expm )(v) ≈
f (m) + h∇M f (m), viTm M
Laplace-Beltrami operator
∆M f (m) = ∆(f ◦ expm )(0)
Intrinsic (expressed solely in terms of the Riemannian metric)
14/59
Laplace-Beltrami operator
Intrinsic gradient
m
f ◦ expm
∇M f (m) = ∇(f ◦ expm )(0)
Taylor expansion
f
(f ◦ expm )(v) ≈
f (m) + h∇M f (m), viTm M
Laplace-Beltrami operator
∆M f (m) = ∆(f ◦ expm )(0)
Intrinsic (expressed solely in terms of the Riemannian metric)
Isometry-invariant
14/59
Laplace-Beltrami operator
Intrinsic gradient
m
f ◦ expm
∇M f (m) = ∇(f ◦ expm )(0)
Taylor expansion
f
(f ◦ expm )(v) ≈
f (m) + h∇M f (m), viTm M
Laplace-Beltrami operator
∆M f (m) = ∆(f ◦ expm )(0)
Intrinsic (expressed solely in terms of the Riemannian metric)
Isometry-invariant
Self-adjoint h∆M f, giL2 (M)=hf, ∆M giL2 (M)
14/59
Laplace-Beltrami operator
Intrinsic gradient
m
f ◦ expm
∇M f (m) = ∇(f ◦ expm )(0)
Taylor expansion
f
(f ◦ expm )(v) ≈
f (m) + h∇M f (m), viTm M
Laplace-Beltrami operator
∆M f (m) = ∆(f ◦ expm )(0)
Intrinsic (expressed solely in terms of the Riemannian metric)
Isometry-invariant
Self-adjoint h∆M f, giL2 (M)=hf, ∆M giL2 (M) ⇒ orthogonal eigenfunctions
14/59
Laplace-Beltrami operator
Intrinsic gradient
m
f ◦ expm
∇M f (m) = ∇(f ◦ expm )(0)
Taylor expansion
f
(f ◦ expm )(v) ≈
f (m) + h∇M f (m), viTm M
Laplace-Beltrami operator
∆M f (m) = ∆(f ◦ expm )(0)
Intrinsic (expressed solely in terms of the Riemannian metric)
Isometry-invariant
Self-adjoint h∆M f, giL2 (M)=hf, ∆M giL2 (M) ⇒ orthogonal eigenfunctions
Positive semidefinite ⇒ non-negative eigenvalues
14/59
Discrete Laplacian
j
wij
αij
βij
i ai
i
Undirected graph (V, E)
X
(∆f )i ≈
wij (fi − fj )
αij
Triangular mesh (V, E, F )
(∆f )i ≈
(i,j)∈E
wij =
1 X
wij (fi − fj )
ai
(i,j)∈E
 cot αij +cot βij
(i, j) ∈ Ei

2
 1
2

 −
cot
P αij
k6=i wik
0
(i, j) ∈ Eb
i=j
else
ai = local area element
Tutte 1963; MacNeal 1949; Duffin 1959; Pinkall, Polthier 1993
15/59
Fourier analysis (Euclidean spaces)
A function f : [−π, π] → R can be written as Fourier series
X 1 Z π
f (x) =
f (ξ)eiωξ dξ e−iωx
2π −π
ω
= α1
+ α2
+ α3
+...
16/59
Fourier analysis (Euclidean spaces)
A function f : [−π, π] → R can be written as Fourier series
X 1 Z π
f (x) =
f (ξ)eiωξ dξ e−iωx
2π −π
ω
|
{z
}
fˆ(ω)=hf,e−iωx iL2 ([−π,π])
= α1
+ α2
+ α3
+...
16/59
Fourier analysis (Euclidean spaces)
A function f : [−π, π] → R can be written as Fourier series
X 1 Z π
f (x) =
f (ξ)eiωξ dξ e−iωx
2π −π
ω
|
{z
}
fˆ(ω)=hf,e−iωx iL2 ([−π,π])
= α1
+ α2
+ α3
+...
Fourier basis = Laplacian eigenfunctions: ∆e−iωx = ω 2 e−iωx
16/59
Fourier analysis (non-Euclidean spaces)
A function f : M → R can be written as Fourier series
XZ
f (m) =
f (ξ)φk (ξ)dξ φk (m)
k≥1 | M
{z
}
fˆk =hf,φk iL2 (M)
=
f
α1
+
φ1
α2
+
φ2
α3
+ ...
φ3
Fourier basis = Laplacian eigenfunctions: ∆M φk = λk φk
17/59
Outline
Background: Spectral analysis on manifolds
Functional correspondence
Partial functional correspondence
Non-rigid puzzles
18/59
Point-wise correspondence
t
n
m
M
N
Point-wise maps t : M → N
19/59
Functional correspondence
g
f
F (M)
F (N )
T
Functional maps T : F(M) → F(N )
Ovsjanikov et al. 2012
19/59
Functional correspondence
f
↓
T
↓
g
Ovsjanikov et al. 2012
20/59
Functional correspondence
f
≈ a1
+ a2
+ · · · + ak
≈ b1
+ b2
+ · · · + bk
↓
T
↓
g
Ovsjanikov et al. 2012
20/59
Functional correspondence
≈ a1
f
g
↓
↓
T
↓
C>
↓
≈ b1
+ a2
+ · · · + ak
Translates Fourier coefficients from Φ to Ψ
+ b2
+ · · · + bk
Ovsjanikov et al. 2012
20/59
Functional correspondence
≈ a1
f
↓
T
↓
g
+ a2
+ · · · + ak
↓
≈
Ψk C>Φ>
k
↓
≈ b1
Translates Fourier coefficients from Φ to Ψ
+ b2
+ · · · + bk
g> Ψk = f > Φk C
where Φk = (φ1 , . . . , φk ), Ψk = (ψ 1 , . . . , ψ k ) are truncated
Laplace-Beltrami eigenbases
Ovsjanikov et al. 2012
20/59
Functional correspondence in Laplacian eigenbases
For isometric simple spectrum shapes C is diagonal since ψ i = ±Tφi
21/59
Computing functional correspondence
Ovsjanikov et al. 2012
22/59
Computing functional correspondence
f1
f2
···
fq
g1
g2
···
gq
Given ordered set of functions f 1 , . . . , f q on M and corresponding
functions g1 , . . . , gq on N (gi ≈ Tf i )
Ovsjanikov et al. 2012
22/59
Computing functional correspondence
f1
f2
···
fq
g1
g2
···
gq
Given ordered set of functions f 1 , . . . , f q on M and corresponding
functions g1 , . . . , gq on N (gi ≈ Tf i )
C found by solving a system of qk equations with k 2 variables
G> Ψk = F> Φk C
where F = (f 1 , . . . , f q ) and G = (g1 , . . . , gq ) are n × q and m × q
matrices
Ovsjanikov et al. 2012
22/59
Key issues
How to recover point-wise correspondence with some guarantees
(e.g. bijectivity)?
How to automatically find corresponding functions F, G?
Near isometric shapes: easy (a lot of structure!)
Non-isometric shapes: hard
Does not work well in case of missing parts and topological noise
23/59
Partial Laplacian eigenvectors
ζ2
ζ3
ζ4
ζ5
ζ6
ζ7
ζ8
ζ9
ψ2
ψ3
ψ4
ψ5
ψ6
ψ7
ψ8
ψ9
φ2
φ3
φ4
φ5
φ6
φ7
φ8
φ9
Laplacian eigenvectors of a shape with missing parts
(Neumann boundary conditions)
Rodolà, Cosmo, B, Torsello, Cremers 2016
24/59
Partial Laplacian eigenvectors
Functional correspondence matrix C
Rodolà, Cosmo, B, Torsello, Cremers 2016
25/59
Perturbation analysis: intuition
∆M
φ1
∆M
φ1
φ2
φ3
M
M̄
∆M̄
φ2
φ̄1
φ3
φ̄2
φ̄3
Ignoring boundary interaction: disjoint parts (block-diagonal matrix)
Eigenvectors = Mixture of eigenvectors of the parts
Rodolà, Cosmo, B, Torsello, Cremers 2016
26/59
Perturbation analysis: eigenvalues
8.00
·10−2
M
6.00
4.00
r
k
2.00
0.00
10
20
30
40
N
50
eigenvalue number
Slope
r
k
≈
|M|
|N |
(depends on the area of the cut)
Consistent with Weil’s law
Rodolà, Cosmo, B, Torsello, Cremers 2016
27/59
Perturbation analysis: details
∆M
M
M̄
∆M+tDM
tE>
tE
∆M̄+tDM̄
∆M̄
Rodolà, Cosmo, B, Torsello, Cremers 2016
28/59
Perturbation analysis: details
∆M
M
M̄
∆M+tDM
tE>
tE
∆M̄+tDM̄
∆M̄
“How would the Laplacian eigenvalues and eigenvectors of the red
part change if we attached a blue part to it?”
Rodolà, Cosmo, B, Torsello, Cremers 2016
28/59
Perturbation analysis: details
M
M̄
PM
n×n
P
n × n̄
DM
E
“How would the Laplacian eigenvalues and eigenvectors of the red
part change if we attached a blue part to it?”
Rodolà, Cosmo, B, Torsello, Cremers 2016
28/59
Perturbation analysis: details
Denote ∆M + tPM = Φ(t)Λ(t)Φ(t)> , ∆M̄ = Φ̄Λ̄Φ̄> , Φ = Φ(0), and
Λ = Λ(0).
Theorem 1 (eigenvalues) The derivative of the non-trivial eigenvalues
is given by
d
0
0
>
λi = φi PM φi
PM =
0 DM
dt
Rodolà, Cosmo, B, Torsello, Cremers 2016
29/59
Perturbation analysis: details
Denote ∆M + tPM = Φ(t)Λ(t)Φ(t)> , ∆M̄ = Φ̄Λ̄Φ̄> , Φ = Φ(0), and
Λ = Λ(0).
Theorem 1 (eigenvalues) The derivative of the non-trivial eigenvalues
is given by
d
0
0
>
λi = φi PM φi
PM =
0 DM
dt
Theorem 2 (eigenvectors) Assuming λi 6= λj for i 6= j and λi 6= λ̄j for
all i, j, the derivative of the non-trivial eigenvectors is given by
n
n̄
X
X
φ>
φ>
d
i P φ̄j
i PM φj
φi =
φj +
φ̄
dt
λi − λj
λi − λ̄j j
j=1
j=1
P=
0 0
E 0
j6=i
Rodolà, Cosmo, B, Torsello, Cremers 2016
29/59
Perturbation analysis: boundary interaction strength
20
10
Value of f
Eigenvector perturbation depends on length and position of the boundary
R
d
Perturbation strength k dt
ΦkF ≤ c ∂M f (m)dm, where
f (m) =
2
n X
φi (m)φj (m)
λi − λj
i,j=1
j6=i
Rodolà, Cosmo, B, Torsello, Cremers 2016
30/59
Laplacian perturbation: typical picture
Plate
Punctured plate
Figure: Filoche, Mayboroda 2009
31/59
Partial functional maps
Model shape M
Query shape N
Part M ⊆ M ≈ isometric to N
Data F, G
Partial functional map
T
M
N
TG ≈ F(M )
M
Rodolà, Cosmo, B, Torsello, Cremers 2016
32/59
Partial functional maps
Model shape M
Query shape N
Part M ⊆ M ≈ isometric to N
Data F, G
Partial functional map
C
M
N
G> ΨC ≈ F(M )> Φ
M
Rodolà, Cosmo, B, Torsello, Cremers 2016
32/59
Partial functional maps
Model shape M
Query shape N
Part M ⊆ M ≈ isometric to N
Data F, G
Partial functional map
C
v
N
G> ΨC ≈ F> diag(v)Φ
v ∈ F(M) indicator function of M
M
Rodolà, Cosmo, B, Torsello, Cremers 2016
32/59
Partial functional maps
Model shape M
Query shape N
Part M ⊆ M ≈ isometric to N
Data F, G
Partial functional map
C
v
N
G> ΨC ≈ F> diag(η(v))Φ
v ∈ F(M) indicator function of M
M
η(t) = 21 (tanh(2t − 1) + 1)
Optimization problem w.r.t. correspondence C and part v
min kG> ΨC − F> diag(η(v))Φk2,1 + ρcorr (C) + ρpart (v)
C,v
Rodolà, Cosmo, B, Torsello, Cremers 2016
32/59
Partial functional maps
min kG> ΨC − F> diag(η(v))Φk2,1 + ρcorr (C) + ρpart (v)
C,v
Rodolà, Cosmo, B, Torsello, Cremers 2016
33/59
Partial functional maps
min kG> ΨC − F> diag(η(v))Φk2,1 + ρcorr (C) + ρpart (v)
C,v
Part regularization
Z
ρpart (v) = µ1 |N | −
2
Z
η(v)dm
+ µ2
M
where ξ(t) ≈ δ η(t) −
1
2
ξ(v)k∇M vkdm
M
Rodolà, Cosmo, B, Torsello, Cremers 2016; BB 2008
33/59
Partial functional maps
min kG> ΨC − F> diag(η(v))Φk2,1 + ρcorr (C) + ρpart (v)
C,v
Part regularization
2
Z
Z
ρpart (v) = µ1 |N | −
η(v)dm
+ µ2
ξ(v)k∇M vkdm
M
| M
{z
}
|
{z
}
area preservation
where ξ(t) ≈ δ η(t) −
1
2
Mumford−Shah
Rodolà, Cosmo, B, Torsello, Cremers 2016; BB 2008
33/59
Partial functional maps
min kG> ΨC − F> diag(η(v))Φk2,1 + ρcorr (C) + ρpart (v)
C,v
Part regularization
2
Z
Z
ρpart (v) = µ1 |N | −
η(v)dm
+ µ2
ξ(v)k∇M vkdm
M
| M
{z
}
|
{z
}
area preservation
where ξ(t) ≈ δ η(t) −
1
2
Mumford−Shah
Correspondence regularization
X
X
((C> C)ii − di )2
ρcorr (C) = µ3 kC ◦ Wk2F + µ4
(C> C)2ij + µ5
i6=j
i
Rodolà, Cosmo, B, Torsello, Cremers 2016; BB 2008
33/59
Partial functional maps
min kG> ΨC − F> diag(η(v))Φk2,1 + ρcorr (C) + ρpart (v)
C,v
Part regularization
2
Z
Z
ρpart (v) = µ1 |N | −
η(v)dm
+ µ2
ξ(v)k∇M vkdm
M
| M
{z
}
|
{z
}
area preservation
where ξ(t) ≈ δ η(t) −
1
2
Mumford−Shah
Correspondence regularization
X
X
ρcorr (C) = µ3 kC ◦ Wk2F + µ4
(C> C)2ij + µ5
((C> C)ii − di )2
|
{z
}
i
i6=j
slant
{z
}
|
{z
} |
≈ orthogonality
rank≈r
Rodolà, Cosmo, B, Torsello, Cremers 2016; BB 2008
33/59
Structure of partial functional correspondence
4
2
0
C
W
C> C
0
20
40
60
80 100
singular values
Rodolà, Cosmo, B, Torsello, Cremers 2016
34/59
Alternating minimization
C-step: fix v∗ , solve for correspondence C
min kG> ΨC − F> diag(η(v∗ ))Φk2,1 + ρcorr (C)
C
v-step: fix C∗ , solve for part v
min kG> ΨC∗ − F> diag(η(v))Φk2,1 + ρpart (v)
v
Rodolà, Cosmo, B, Torsello, Cremers 2016
35/59
Alternating minimization
C-step: fix v∗ , solve for correspondence C
min kG> ΨC − F> diag(η(v∗ ))Φk2,1 + ρcorr (C)
C
v-step: fix C∗ , solve for part v
min kG> ΨC∗ − F> diag(η(v))Φk2,1 + ρpart (v)
v
Iteration 1
2
3
4
Rodolà, Cosmo, B, Torsello, Cremers 2016
35/59
Example of convergence
Time (sec.)
1010
0
5
10
15
20
25
C-step
v-step
109
Energy
108
107
106
105
104
0
20
40
60
80
100
Iteration
Rodolà, Cosmo, B, Torsello, Cremers 2016
36/59
Examples of partial functional maps
Rodolà, Cosmo, B, Torsello, Cremers 2016
37/59
Examples of partial functional maps
Rodolà, Cosmo, B, Torsello, Cremers 2016
37/59
Examples of partial functional maps
Rodolà, Cosmo, B, Torsello, Cremers 2016
37/59
Examples of partial functional maps
Rodolà, Cosmo, B, Torsello, Cremers 2016
37/59
Partial functional maps vs Functional maps
100
100
% Correspondences
80
150
50
PFM
Func. maps
60
40
50
20
0
100
150
0
0.05
0.1
0.15
0.2
0.25
Geodesic error
Correspondence performance for different rank values k
Rodolà, Cosmo, B, Torsello, Cremers 2016
38/59
Partial correspondence performance
Cuts
Holes
% Correspondences
100
80
60
40
20
0
0
0.05
0.1
0.15
0.2
0.25
0
Geodesic Error
PFM
0.05
0.1
0.15
0.2
0.25
Geodesic Error
RF
IM
EN
GT
SHREC’16 Partial Matching benchmark Rodolà et al. 2016; Methods: Rodolà,
Cosmo, B, Torsello, Cremers 2016 (PFM); Sahillioğlu, Yemez 2012 (IM); Rodolà,
Bronstein, Albarelli, Bergamasco, Torsello 2012 (GT); Rodolà et al. 2013 (EN);
Rodolà et al. 2014 (RF)
39/59
Partial correspondence performance
Cuts
Holes
Mean geodesic error
1
0.8
0.6
0.4
0.2
0
20
40
60
80
20
Partiality (%)
PFM
40
60
80
Partiality (%)
RF
IM
EN
GT
SHREC’16 Partial Matching benchmark Rodolà et al. 2016; Methods: Rodolà,
Cosmo, B, Torsello, Cremers 2016 (PFM); Sahillioğlu, Yemez 2012 (IM); Rodolà,
Bronstein, Albarelli, Bergamasco, Torsello 2012 (GT); Rodolà et al. 2013 (EN);
Rodolà et al. 2014 (RF)
40/59
Deep learning + Partial functional maps
Correspondence
0.1
0.0
Correspondence error
Boscaini, Masci, Rodolà, B 2016
41/59
Deep learning + Partial functional maps
Correspondence
0.1
0.0
Correspondence error
Boscaini, Masci, Rodolà, B 2016
42/59
Outline
Background: Spectral analysis on manifolds
Functional correspondence
Partial functional correspondence
Non-rigid puzzles
43/59
Litani, BB 2012
Partial correspondence
Rodolà, Cosmo, B, Torsello, Cremers 2016
45/59
Non-rigid puzzle
Litani, Rodolà, BB, Cremers 2016
45/59
Partial Laplacian eigenvectors
Functional correspondence matrix C
Rodolà, Cosmo, B, Torsello, Cremers 2016
46/59
Key observation
M
N
N
M
CN N
slant ∝
|N |
|N |
CM M
slant ∝
|M |
|M|
Litani, Rodolà, BB, Cremers 2016
47/59
Key observation
M
N
N
M
CN M = CN N CN M CMM
slant ∝
|N | |M|
|N | |M |
Litani, Rodolà, BB, Cremers 2016
47/59
Key observation
M
N
N
M
CN M = CN N CN M CMM
slant ∝
|N |
|N | |M|
=
|N | |M |
|M|
Litani, Rodolà, BB, Cremers 2016
47/59
Non-rigid puzzles problem formulation
Model
Input
Model M
Parts N1 , . . . , Np
Output
Segmentation Mi ⊆ M
Located parts Ni ⊆ Ni
Correspondences Ci
Clutter Nic
Missing parts M0
Parts
M1
M2
C1
C2
N2c
N2
N1
N1c
N1
M0
M
N2
Litani, Rodolà, BB, Cremers 2016
48/59
Non-rigid puzzles problem formulation
Model
Data Fi , Gi
Model basis Φ, Φ(Mi )
Part bases Ψi , Ψi (Ni )
Data term
Parts
M1
M2
C1
C2
N2c
N2
N1
N1c
N1
>
F>
i Φ(Mi ) ≈ Gi Ψi (Ni )Ci
M0
M
N2
Litani, Rodolà, BB, Cremers 2016
48/59
Non-rigid puzzles problem formulation
min
p
X
Ci
Mi ⊆M,Ni ⊆Ni
>
kG>
i Ψ i (Ni )Ci − Fi Φ (Mi )k2,1
i=1
+ λM
p
X
ρpart (Mi ) + λN
i=0
p
X
+ λcorr
p
X
ρpart (Ni )
i=1
ρcorr (Ci )
i=1
s.t. Mi ∩ Mj = ∅ ∀i 6= j
M0 ∪ M1 ∪ · · · ∪ Mp = M
|Mi | = |Ni | ≥ α|Ni |,
Litani, Rodolà, BB, Cremers 2016
49/59
Non-rigid puzzles problem formulation
min
Ci
ui ,vi
p
X
Ψi Ci − F>
Φk2,1
kG>
i diag(η(ui ))Ψ
i diag(η(vi ))Φ
i=1
+ λM
p
X
ρpart (η(vi )) + λN
i=0
p
X
+ λcorr
p
X
ρpart (η(ui ))
i=1
ρcorr (Ci )
i=1
s.t.
p
X
η(ui ) = 1
i=1
a>
M ui
>
= a>
N vi ≥ αaNi 1
Litani, Rodolà, BB, Cremers 2016
49/59
Convergence example
Outer iteration 1
Litani, Rodolà, BB, Cremers 2016
50/59
Convergence example
Outer iteration 2
Litani, Rodolà, BB, Cremers 2016
50/59
Convergence example
Outer iteration 3
Litani, Rodolà, BB, Cremers 2016
50/59
Convergence example
30
80
32
90
34
100
36
110
Time (sec)
38
120
40
42
130
Iteration number
44
140
46
150
48
160
Litani, Rodolà, BB, Cremers 2016
51/59
“Perfect puzzle” example
Model/Part
Transformation
Clutter
Missing part
Data term
Synthetic (TOSCA)
Isometric
No
No
Dense (SHOT)
Litani, Rodolà, BB, Cremers 2016
52/59
“Perfect puzzle” example
Model/Part
Transformation
Clutter
Missing part
Data term
Synthetic (TOSCA)
Isometric
No
No
Dense (SHOT)
Segmentation
Litani, Rodolà, BB, Cremers 2016
52/59
“Perfect puzzle” example
Model/Part
Transformation
Clutter
Missing part
Data term
Synthetic (TOSCA)
Isometric
No
No
Dense (SHOT)
Correspondence
Litani, Rodolà, BB, Cremers 2016
52/59
Overlapping parts example
Model/Part
Transformation
Clutter
Missing part
Data term
Synthetic (FAUST)
Near-isometric
Yes (overlap)
No
Dense (SHOT)
Segmentation
Litani, Rodolà, BB, Cremers 2016
53/59
Overlapping parts example
Model/Part
Transformation
Clutter
Missing part
Data term
Synthetic (FAUST)
Near-isometric
Yes (overlap)
No
Dense (SHOT)
Correspondence
Litani, Rodolà, BB, Cremers 2016
53/59
Overlapping parts example
Model/Part
Transformation
Clutter
Missing part
Data term
Synthetic (FAUST)
Near-isometric
Yes (overlap)
No
Dense (SHOT)
0.1
0.0
Correspondence error
Litani, Rodolà, BB, Cremers 2016
53/59
Missing parts example
Model/Part
Transformation
Clutter
Missing part
Data term
Synthetic (TOSCA)
Isometric
Yes (extra part)
Yes
Dense (SHOT)
Litani, Rodolà, BB, Cremers 2016
54/59
Missing parts example
Model/Part
Transformation
Clutter
Missing part
Data term
Synthetic (TOSCA)
Isometric
Yes (extra part)
Yes
Dense (SHOT)
Segmentation
Litani, Rodolà, BB, Cremers 2016
54/59
Missing parts example
Model/Part
Transformation
Clutter
Missing part
Data term
Synthetic (TOSCA)
Isometric
Yes (extra part)
Yes
Dense (SHOT)
Correspondence
Litani, Rodolà, BB, Cremers 2016
54/59
Scanned data example
Model/Part
Transformation
Clutter
Missing part
Data term
Synthetic (TOSCA) / Scan
Non-Isometric
No
No
Sparse deltas
Litani, Rodolà, BB, Cremers 2016
55/59
Scanned data example
Model/Part
Transformation
Clutter
Missing part
Data term
Synthetic (TOSCA) / Scan
Non-Isometric
No
No
Sparse deltas
Segmentation
Litani, Rodolà, BB, Cremers 2016
55/59
Non-rigid puzzle vs Partial functional map
Non-rigid puzzle
Partial functional map (pair-wise)
Rodolà, Cosmo, B, Torsello, Cremers 2016; Litani, Rodolà, BB, Cremers 2016
56/59
Summary
New insights about spectral properties of Laplacians
Extension of functional correspondence framework to the partial
setting
Practically working methods for challenging shape correspondence
settings
Code available (SGP Reproducibility Stamp)
Some over-engineering - can be done simpler! (stay tuned...)
57/59
E. Rodolà
A. Bronstein
O. Litany
L. Cosmo
A. Torsello
D. Cremers
Supported by
58/59
Thank you!
59/59