Recent cost volume pyramid based deep neural networks have unlocked the
potential of efficiently leveraging high-resolution images for depth inference
from multi-view stereo. In general, those approaches assume that the depth of
each pixel follows a unimodal distribution. Boundary pixels usually follow a
multi-modal distribution as they represent different depths; Therefore, the
assumption results in an erroneous depth prediction at the coarser level of the
cost volume pyramid and can not be corrected in the refinement levels leading
to wrong depth predictions. In contrast, we propose constructing the cost
volume by non-parametric depth distribution modeling to handle pixels with
unimodal and multi-modal distributions. Our approach outputs multiple depth
hypotheses at the coarser level to avoid errors in the early stage. As we
perform local search around these multiple hypotheses in subsequent levels, our
approach does not maintain the rigid depth spatial ordering and, therefore, we
introduce a sparse cost aggregation network to derive information within each
volume. We evaluate our approach extensively on two benchmark datasets: DTU and
Tanks & Temples. Our experimental results show that our model outperforms
existing methods by a large margin and achieves superior performance on
boundary regions. Code is available at this https URL

通过非参数深度分布建模构建成本体积，避免错误检测。同时，我们使用稀疏成本聚合网络推导每个体积中的信息。在 DTU 和坦克与神殿这两个基准数据集上广泛评估我们的方法，实验结果表明，我们的模型以显着优势胜过现有方法，并在边界区域实现卓越性能。

基于非参数深度分布建模的多视角立体匹配深度推断

Non-parametric Depth Distribution Modelling based Depth Inference for  Multi-view Stereo

We propose a cost volume-based neural network for depth inference from
multi-view images. We demonstrate that building a cost volume pyramid in a
coarse-to-fine manner instead of constructing a cost volume at a fixed
resolution leads to a compact, lightweight network and allows us inferring high
resolution depth maps to achieve better reconstruction results. To this end, we
first build a cost volume based on uniform sampling of fronto-parallel planes
across the entire depth range at the coarsest resolution of an image. Then,
given current depth estimate, we construct new cost volumes iteratively on the
pixelwise depth residual to perform depth map refinement. While sharing similar
insight with Point-MVSNet as predicting and refining depth iteratively, we show
that working on cost volume pyramid can lead to a more compact, yet efficient
network structure compared with the Point-MVSNet on 3D points. We further
provide detailed analyses of the relation between (residual) depth sampling and
image resolution, which serves as a principle for building compact cost volume
pyramid. Experimental results on benchmark datasets show that our model can
perform 6x faster and has similar performance as state-of-the-art methods. Code
is available at this https URL

本文提出了一种基于成本体积的神经网络，可以从多视图图像中进行深度推断，在构建成本体积金字塔的过程中，通过从粗到细的方式代替固定分辨率的方式，使得网络结构更紧凑轻量，并允许我们推断高分辨率深度图以获得更好的重建结果。