We present an end-to-end deep learning architecture for depth map inference
from multi-view images. In the network, we first extract deep visual image
features, and then build the 3D cost volume upon the reference camera frustum
via the differentiable homography warping. Next, we apply 3D convolutions to
regularize and regress the initial depth map, which is then refined with the
reference image to generate the final output. Our framework flexibly adapts
arbitrary N-view inputs using a variance-based cost metric that maps multiple
features into one cost feature. The proposed MVSNet is demonstrated on the
large-scale indoor DTU dataset. With simple post-processing, our method not
only significantly outperforms previous state-of-the-arts, but also is several
times faster in runtime. We also evaluate MVSNet on the complex outdoor Tanks
and Temples dataset, where our method ranks first before April 18, 2018 without
any fine-tuning, showing the strong generalization ability of MVSNet.

本文提出了一种用于计算多视图深度图的端到端深度学习框架 (MVSNet)，首先提取深度视觉图像特征，然后利用不同 iable homography 变形构建 3D 成本体积，并使用 3D 卷积对初始深度图进行规则化和回归，最终与参考图像相结合生成最终输出，其通过引入基于方差的成本度量来适应任意 N-view 输入， 其表现优于现有同类方法并能很好地泛化于室内室外数据。