We present 3DVNet, a novel multi-view stereo (MVS) depth-prediction method
that combines the advantages of previous depth-based and volumetric MVS
approaches. Our key idea is the use of a 3D scene-modeling network that
iteratively updates a set of coarse depth predictions, resulting in highly
accurate predictions which agree on the underlying scene geometry. Unlike
existing depth-prediction techniques, our method uses a volumetric 3D
convolutional neural network (CNN) that operates in world space on all depth
maps jointly. The network can therefore learn meaningful scene-level priors.
Furthermore, unlike existing volumetric MVS techniques, our 3D CNN operates on
a feature-augmented point cloud, allowing for effective aggregation of
multi-view information and flexible iterative refinement of depth maps.
Experimental results show our method exceeds state-of-the-art accuracy in both
depth prediction and 3D reconstruction metrics on the ScanNet dataset, as well
as a selection of scenes from the TUM-RGBD and ICL-NUIM datasets. This shows
that our method is both effective and generalizes to new settings.

3DVNet 是一种结合了先前基于深度和体积的 MVS 方法优点的新型多视图立体 (multi-view stereo) 深度预测方法，使用基于体积的三维卷积神经网络来对所有深度图进行操作，并能够学习有意义的场景级先验知识，其在深度预测和 3D 重建度量上均超越了现有技术，证明该方法在新的环境下具有很好的效果和通用性。