Multi-view stereo methods frequently fail to properly reconstruct
3D scene geometry if visible texture is sparse or
the scene exhibits difficult self-occlusions. Time-of-Flight
(ToF) depth sensors can provide 3D information regardless
of texture but with only limited resolution and accuracy. To
find an optimal reconstruction, we propose an integrated
multi-view sensor fusion approach that combines information
from multiple color cameras and multiple ToF depth
sensors. First, multi-view ToF sensor measurements are
combined to obtain a coarse but complete model. Then, the
initial model is refined by means of a probabilistic multiview
fusion framework, optimizing over an energy function
that aggregates ToF depth sensor information with multiview
stereo and silhouette constraints. We obtain high quality
dense and detailed 3D models of scenes challenging for
stereo alone, while simultaneously reducing complex noise
of ToF sensors.
Images
Data set 1: room
input
multi-view stereo
our fusion method
Data set 2: girl
input
multi-view stereo
one ToF camera and a pair of stereo cameras
our fusion method
before refinement
after refinement
Data set 3: macbeth
input
multi-view stereo
one ToF camera and a pair of stereo cameras
our fusion method
Data set 4: whale
input
multi-view stereo
one ToF camera and a pair of stereo cameras
before refinement
our fusion method
Video
Publication
Y.M. Kim, C. Theobalt, J. Diebel, J. Kosecka, B. Micusik, S. Thrun,
Multi-view Image and ToF Sensor Fusion for Dense 3D Reconstruction,
2009 IEEE Workshop on 3-D Digital Imaging and Modeling (3DIM)
[PDF][poster]
