Abstract
The efficient fusion of depth maps is a key part of most state-of-the-art 3Dreconstruction methods. Besides requiring high accuracy, these depth fusionmethods need to be scalable and real-time capable. To this end, we present anovel real-time capable machine learning-based method for depth map fusion.Similar to the seminal depth map fusion approach by Curless and Levoy, we onlyupdate a local group of voxels to ensure real-time capability. Instead of asimple linear fusion of depth information, we propose a neural network thatpredicts non-linear updates to better account for typical fusion errors. Ournetwork is composed of a 2D depth routing network and a 3D depth fusion networkwhich efficiently handle sensor-specific noise and outliers. This is especiallyuseful for surface edges and thin objects for which the original approachsuffers from thickening artifacts. Our method outperforms the traditionalfusion approach and related learned approaches on both synthetic and real data.We demonstrate the performance of our method in reconstructing fine geometricdetails from noise and outlier contaminated data on various scenes