Almost all previous deep learning-based multi-view stereo (MVS) approaches focus on improving reconstruction quality. Besides quality, efficiency is also a desirable feature for MVS in real scenarios. Towards this end, this paper presents a Fast-MVSNet, a novel sparse-to-dense coarse-to-fine framework, for fast and accurate depth estimation in MVS. Specifically, in our Fast-MVSNet, we first construct a sparse cost volume for learning a sparse and high-resolution depth map. Then we leverage a small-scale convolutional neural network to encode the depth dependencies for pixels within a local region to densify the sparse high-resolution depth map. At last, a simple but efficient Gauss-Newton layer is proposed to further optimize the depth map. On one hand, the high-resolution depth map, the data-adaptive propagation method and the Gauss-Newton layer jointly guarantee the effectiveness of our method. On the other hand, all modules in our Fast-MVSNet are lightweight and thus guarantee the efficiency of our approach. Besides, our approach is also memory-friendly because of the sparse depth representation. Extensive experimental results show that our method is 5$\times$ and 14$\times$ faster than Point-MVSNet and R-MVSNet, respectively, while achieving comparable or even better results on the challenging Tanks and Temples dataset as well as the DTU dataset. Code is available at https://github.com/svip-lab/FastMVSNet.

本研究旨在提高多视角立体成像(MVS)技术的效率，并提出了一种基于深度学习的Fast-MVSNet稀疏到稠密分层框架，其在局部区域内使用小型卷积神经网络对深度依赖关系进行编码，通过高分辨率的稀疏深度图以及高效的高斯牛顿层优化深度图，取得与Point-MVSNet和R-MVSNet相同或更好的结果，可在Tanks and Temples和DTU数据集中五到十四倍地提高速度。

Fast-MVSNet: 基于学习传播和高斯牛顿精细化的稀疏到密集多视角立体视觉