Occluded Imaging with Time of Flight Cameras
Anonymous Submission Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 1/31 Imaging in Time Provides Richer Data
Conven=onal Photography I (u, v)
Time of Flight Imaging I (u, v, t ) = A (u, v ) sin ( 2π fωt + ϕ (u, v )) + ζ (u, v, t )
Anonymous Submission Amplitude
Occluded Imaging with Time of Flight Cameras Phase Slide 2/31 Time of Flight 3D Cameras
fω
I (u, v, t ) = A (u, v ) sin ( 2π fωt + ϕ (u, v )) + ζ (u, v, t )
cΦ
z=
, c ≈ 3 ×108 m/s
2π fω
Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 3/31 Output: Phase and Amplitude
Reflec=on/Albedo (monochrome) Anonymous Submission Occluded Imaging with Time of Flight Cameras Phase/Range/Depth Slide 4/31 Virtual Sensor Array
Wall à Lensless Imaging Chip Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 5/31 Virtual Sensor Array
Map the measurements to complex domain Wall à Lensless Imaging Chip Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 6/31 Virtual Sensor Array
Map the measurements to complex domain Transport phasor from light to wall: Wall à Lensless Imaging Chip Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 7/31 Virtual Sensor Array
Map the measurements to complex domain Transport phasor from light to wall: Transport phasor from wall to camera Wall à Lensless Imaging Chip Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 8/31 Virtual Sensor Array
Measured Phasor is a Composite Mul=plica=on. Wall à Lensless Imaging Chip Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 9/31 Virtual Sensor Array
Measured Phasor is a Composite Mul=plica=on. Assuming Wall in Focus Where angle has been integrated out. Wall à Lensless Imaging Chip Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 10/31 Virtual Sensor Array
Measured Phasor is a Composite Mul=plica=on. Assuming Wall in Focus Where angle has been integrated out. Wall à Lensless Imaging Chip Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 11/31 Idea: Exploit Spatial Diversity
Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 12/31 Grid Localization
Without Loss of Generality, consider 2-­‐D localiza=on using a 1-­‐D slice of measurements M sensors Flexible for Far Field and Near Field Cases Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 13/31 Grid Localization
Without Loss of Generality, consider 2-­‐D localiza=on using a 1-­‐D slice of measurements Define the Set of Grid Points N elements
Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 14/31 Grid Localization
Without Loss of Generality, consider 2-­‐D localiza=on using a 1-­‐D slice of measurements Define the Set of Grid Points N elements
Define the Set of Targets K elements
Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 15/31 Grid Localization
Without Loss of Generality, consider 2-­‐D localiza=on using a 1-­‐D slice of measurements Define the Set of Grid Points N elements
Define the Set of Targets K elements
K << N Provides Intuition for Sparsity
Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 16/31 Grid Localization
Without Loss of Generality, consider 2-­‐D localiza=on using a 1-­‐D slice of measurements Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 17/31 Grid Localization
Without Loss of Generality, consider 2-­‐D localiza=on using a 1-­‐D slice of measurements Define weights of confidence Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 18/31 Grid Localization
Without Loss of Generality, consider 2-­‐D localiza=on using a 1-­‐D slice of measurements Define weights of confidence Define the Operator Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 19/31 Grid Localization
Without Loss of Generality, consider 2-­‐D localiza=on using a 1-­‐D slice of measurements Define weights of confidence Define the Operator Build a DIc=onary Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 20/31 Grid Localization
Without Loss of Generality, consider 2-­‐D localiza=on using a 1-­‐D slice of measurements Define weights of confidence Define the Operator Build a DIc=onary Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 21/31 The Next Question…
•  Do we have uniqueness in solu=on? •  How many sources can we recover? •  How do the physical constraints of the problem map to guarantees? Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 22/31 Coherence
Define the mutual coherence For robust recovery, seems “natural” that low coherence is beber. Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 23/31 Coherence
Define the mutual coherence For robust recovery, seems “natural” that low coherence is beber. Thm from Elad, “Sparse and Redundant Representa=ons”, 2010. Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 24/31 Beampattern and Coherence
Define the mutual coherence Beampabern is the rows/columns of G For underdetermined systems, G is rank-­‐deficient, hence high coherence. Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 25/31 Physical Parameters
Frequency and Gridding relates to coherence Suppose i’ and j’ are indices of the two most similar columns of D. Then we can explicitly write the coherence. Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 26/31 Physical Parameters
Frequency and Gridding relates to coherence Aperture Size Direc=onality Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 27/31 Superresolution through Sparse Priors
Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 28/31 Beampattern
Wideband to Eliminate Gra=ng Lobes Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 29/31 Selected Demonstrations
Real Time Localiza=on Imaging Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 30/31 Concluding Remarks
•  Key is to provide guarantees on problems that ordinarily do not come with guarantees. •  VSA model can be used for larger scale scenes -­‐-­‐-­‐ use the environment as your lens. •  Sparsity typically manifests in such problems. Anonymous Submission Occluded Imaging with Time of Flight Cameras Slide 31/31