We propose a cost volume-based neural network for depth inference from
multi-view images. We demonstrate that building a cost volume pyramid in a
coarse-to-fine manner instead of constructing a cost volume at a fixed
resolution leads to a compact, lightweight network and allows us inferring high
resolution depth maps to achieve better reconstruction results. To this end, we
first build a cost volume based on uniform sampling of fronto-parallel planes
across the entire depth range at the coarsest resolution of an image. Then,
given current depth estimate, we construct new cost volumes iteratively on the
pixelwise depth residual to perform depth map refinement. While sharing similar
insight with Point-MVSNet as predicting and refining depth iteratively, we show
that working on cost volume pyramid can lead to a more compact, yet efficient
network structure compared with the Point-MVSNet on 3D points. We further
provide detailed analyses of the relation between (residual) depth sampling and
image resolution, which serves as a principle for building compact cost volume
pyramid. Experimental results on benchmark datasets show that our model can
perform 6x faster and has similar performance as state-of-the-art methods. Code
is available at this https URL

本文提出了一种基于成本体积的神经网络，可以从多视图图像中进行深度推断，在构建成本体积金字塔的过程中，通过从粗到细的方式代替固定分辨率的方式，使得网络结构更紧凑轻量，并允许我们推断高分辨率深度图以获得更好的重建结果。