3D Laser Stripe Scanner
or “A Really Poor Man’s DeltaSphere”
Chad Hantak
December 6, 2004
Overview
•
•
•
•
•
•
•
Introduction
Acquisition Device
Calibration
Software Framework
Processing
Limitations
Future Work
Introduction
or “What’s the project about?”
• 3D Scanning System
• Similar to DeltaSphere
• Range Samples from fixed COP
• Acquire sparse depth with color
• Acquisition Device
• Cheap
• Simple to operate
DeltaSphere
• Dense Depth
• Time of Flight Acquisition
• Collects Range Samples over 360
degrees of Azimuth and 135 degrees of
elevation
• Cons
• Slow
• Scanning
• Processing
• Expensive
Acquisition Device Components
• Laser Level
• Available at hardware stores
• Used for leveling (pictures, shelves, …)
• Emits a plane of laser light (Vertical or
Horizontal)
• Camcorder
• Canon GL2 (3 CCD)
• Little overkill for this
Acquisition Device Construction
• Camcorder rotates about fixed point
• Laser Level is fixed next to
camcorder
Depth Range Acquisition
“The Theory”
• Calibrated Rig
• Know laser stripe’s plane equation in camera’s
coordinate system
• During Rotation
• Determine amount of rotation
• Find laser’s stripe points in image
• Determine where image rays intersect stripe
plane
• Yields depth samples
• At same time acquire color
Calibration
• Need to know laser stripe plane in
camera coordinate system
• Record calibration grid with laser
stripe activated
Bouget’s Matlab Toolkit
• Camera Intrinsics
• Standard Bouget’s Toolkit
• Extras from toolkit
• Calibration plane in camera’s
coordinate system for each Image
Laser Stripe’s Plane
• For each image
• Find stripe points (user clicks)
• Determine world rays
• Intersect with grid’s plane for world
points
• Take all the world points and fit
them to a plane (least squares)
• Result is laser stripe plane
Software Framework
• Implemented in C++
• Libraries
• OpenCV
• Image Processing
• http://www.intel.com/research/mrl/research/opencv/
• DirectShow
• Interface to Camcorder
• http://msdn.microsoft.com/library/default.asp?url=/librar
y/en-us/directshow/htm/introductiontodirectshow.asp
Two Executables
• depthPanorama
• Hook into camcorder
• Acquires / processes frames
• Saves into shared memory
• panoramaView
• Reads from shared memory
• Displays result to user
depthPanorama
• 3 Systems
• RotationTracker
• Estimates the amount of rotation
• ColorProcessor
• Creates the color panorama
• DepthProcessor
• Creates the depth panorama
RotationTracker
• Estimates the amount of rotation
between two frames
• Uses Lucas-Kanade Point Tracking
• Estimate rotation if “enough” points
moved
• Uses RANSAC to estimate the
amount of rotation
ColorProcessor
• Updates the Color Panorama
• Process
• Only update after “enough” rotation
• Extract and resize left half of frame into
panorama’s memory
• Right half of frame contains the laser
stripe
Depth Processor
• Updates the Depth Panorama
• Process
• Find the laser stripe points in the frame
(samples every x scan lines)
• Turn image points into world rays
• Intersect rays with laser stripe plane
• For valid intersections, place depth
value in depth panorama
• For “some” invalid intersections,
attempt to interpolate
Finding Laser Stripe Image Points
or “The Hard Part”
• Know in the frame where points are (right half)
• Extension to ModelCamera system
• Process scan line looking for candidate peaks with
symmetry
• Exploit fact camcorder is just rotating
• Determine homography from one frame to the next,
warp previous frame, subtract from current frame
• Homography is based on amount of rotation between
frames
• Since laser stripe moves differently it will stand out
better in resultant image
• There is still noise in the image
• Need some image processing
• Edge Detection, Erosion, Dilation
Turing Image Points into Depth
Samples
• Know the camcorder intrinsics
• Allows image points to be turned into rays into
world from camcorder’s COP
• Intersect these rays with plane
• Splat depth to neighboring depth samples
• Associate a confidence level with each depth
sample
• Perfect; confidence = 1.0
• Interpolated; confidence = 0.75
• Splatted; confidence = 0.6
• When updating if incoming confidence higher,
use it
No Intersection
• Due to noise, image points of stripe
may not be correct
• World ray may not intersect stripe
plane
• If neighboring samples are valid,
interpolate missing depth value
• Lower confidence
panoramaView
• Simply reads values sent by
depthPanorama
• Displays texture-mapped quads of
depth values
• Updates continually from shared
memory
Results
Current Problems
• Rotation tracking
• Off by a few degrees at end of 360 degree
pan (< 10 degrees)
• Tweaking LK Tracking parameters
• Smooth out the rotation (use LK & correlation
to determine when rotation complete)
• Stripe point determination
• Adjusting image processing parameters
• Depth discontinuities
• Take them into account when rendering
Future Work
• Fix current problems
• Better laser light / Point detection
• Stronger at distance
• Want longer range scans
• Combine different scans
• Introduces new slew of problems
References
•
•
•
R. Laganiere, “Programming computer vision applications: A step-by-step guide to
the use of the Intel OpenCV library and the Microsoft DirectShow technology”,
http://www.site.uottawa.ca/~laganier/tutorial/opencv+directshow/.
V. Popescu, E. Sacks and G. Bahmutov, “Interactive Modeling from Dense Color and
Sparse Depth” , 3DPTV, 2004.
S. Sinha and M. Pollefeys, “Towards Calibrating a Pan-Tilt-Zoom Camera Network”,
5th Workshop on Omnidirectional Vision, Camera Networks, and Non-classical
Cameras (OMNIVIS), May 16, 2004.