Aligning Surfaces Without Aligning Surfaces
Geoffrey Oxholm and Ko Nishino
Department of Computer Science, Drexel University
Finding Matching Boundary Regions
Non-overlapping Alignment
Surface alignment is the process of finding
matching views, and aligning them.
Past work focus on overlapping cases
I Separate views overlap each other
I Surfaces attach to each other
Non-overlapping problems have no such information
I Time critical systems: Medical imaging, robotics
I Thin or fragile objects: Forensics, paleontology & archaeology
Our approach
Contributions
I Makes no assumptions about the
1. Novel scale-space boundary contour
representation
shape of the object, or its painted
texture
2. Image registration based boundary
I Leverages geometric and
contour matching
photometric continuity along and
3. Two part least squares alignment
across matching boundary regions
Results
Basic user interface
I We augment our method with a basic user
interface.
I Likely matches are presnted to the user who
confirms or rejects them.
Now that we’ve represented the boundaries with images, we find matching boundary
contours as matching image-regions.
1.
2.
3.
Boundary Contour Representation
Extracting the boundary contour
First, we apply a longest common
substring search on the coarsest
scale, i.e., the top row, to estimate
matches.
6 View Alignment
Real-world data
I The object to the right is a store bought vase,
broken for experimentation.
I It is assembeled fully automatically.
I The other two datasets are colonial era
historical artifacts.
I Reassembly takes 40% less time with our
method than by hand.
Next, we refine the matches using all
scales at once. Using one region as
the template, we refine the location of
the other region using normalized
cross-correlation image registration.
-
Finally we account for “fan-out” error,
which is caused by non-matching
values at the end of each match by
iteratively expand the base of the
matching regions.
Matches found
Close points are then added
Our result
Hand assembeled ground-truth
16 View Alignment
Aligning Views
Color image
I
I
Depth
Mask
Raw 2D
Raw 3D
Processed
3D boundary
image
boundary
boundary
Inputs: color image and a range image for each view
Using these, we generate a mask, which we then use to extract a the raw 2D
boundary of the view. This allows us to extract the raw 3D boundary, which we
then smooth and subsample.
1.
R, t
...
kλ̈ × λ̇k
(λ̇ × λ̈) · λ
κ=
τ =
,
kλ̇k3
kλ̇ × λ̈k2
...
˙
¨
where , , and denote the first, second,
and third order derivatives of the contour λ(t).
Photometry : Red and green chromaticity
I Computed as the channel value divided by
sum of channel values
I Account for changes in illumination
rd
I Are compact (3 channel is redundant)
Encoding Scale
I
I
We compute these values and store them as
the base row S0 = f (t) of an image S.
Moving up the image to row r corresponds
to circular convolution with Gaussian
smoothing kernel
Sr = N (σr ) ~ S0 ,
where σr ∝ r.
2.
σr
A
B
i=1
where R and t are the rotation and translation
that align the n point boundary contours.
Encoding Shape and Color
Geometry : Curvature and torsion
Align the boundary contours so that the points
{ai | 1 ≤ i ≤ n} from piece A are aligned with
the points {bi | 1 ≤ i ≤ n} from piece B.
n
X
argmin
kbiR + t − aik2 ,
f (t)κ – Curvature
σr
Align the surfaces to maximize geometric
continuity. To do so, we minimize the gradient
of surface normals.


ai(−k)
for k < 0
b
qi(k) = 0
for k = 0

b
bi(k) R H(θ) for k > 0 ,
b i(k) are the series of surface
where b
ai(k) and b
normals from ai and bi, respectively (shown at
right), and H(θ) is a single parameter rotation
matrix about the boundary contour’s principal
axis, or “hinge line.”
We may now find the optimal hinge angle θ
2
n X
∂qi .
argmin
(0)
∂k
θ
f (t)τ – Torsion
σr
f (t)cr – Red chromaticity
σr
11 View Alignment
A
−k
k)
−
(
b
ai
bi (k
b
)R
H(θ)
+k
0
B
A
i=1
f (t)cg – Green chromaticity
σr
f (t) – Simplifed representation
3.
I
I
B
Once two views are aligned, they are merged into a new view.
The boundary is then encoded with its own scale-space image, and entered
back into the system.
Repeat until complete
Conclusion
I Our method allows for rapid alignment of multiple, non-overlapping views.
I This makes archaeology faster, and safer, since handling the objects is minimized.
Acknowledgements
This work was supported in part by National Science Foundation CAREER Award IIS-0746717
and IIS-0803670.