We present 3DVNet, a novel multi-view stereo (MVS) depth-prediction method that combines the advantages of previous depth-based and volumetric MVS approaches. Our key idea is the use of a 3D scene-modeling network that iteratively updates a set of coarse depth predictions, resulting in highly accurate predictions which agree on the underlying scene geometry. Unlike existing depth-prediction techniques, our method uses a volumetric 3D convolutional neural network (CNN) that operates in world space on all depth maps jointly. The network can therefore learn meaningful scene-level priors. Furthermore, unlike existing volumetric MVS techniques, our 3D CNN operates on a feature-augmented point cloud, allowing for effective aggregation of multi-view information and flexible iterative refinement of depth maps. Experimental results show our method exceeds state-of-the-art accuracy in both depth prediction and 3D reconstruction metrics on the ScanNet dataset, as well as a selection of scenes from the TUM-RGBD and ICL-NUIM datasets. This shows that our method is both effective and generalizes to new settings.

3DVNet是一种结合了先前基于深度和体积的MVS方法优点的新型多视图立体(multi-view stereo)深度预测方法，使用基于体积的三维卷积神经网络来对所有深度图进行操作，并能够学习有意义的场景级先验知识，其在深度预测和3D重建度量上均超越了现有技术，证明该方法在新的环境下具有很好的效果和通用性。

3DVNet: 多视角深度预测和体积细化