Multi-view stereopsis (MVS) tries to recover the 3D model from 2D images. As
the observations become sparser, the significant 3D information loss makes the
MVS problem more challenging. Instead of only focusing on densely sampled
conditions, we investigate sparse-MVS with large baseline angles since the
sparser sensation is more practical and more cost-efficient. By investigating
various observation sparsities, we show that the classical depth-fusion
pipeline becomes powerless for the case with a larger baseline angle that
worsens the photo-consistency check. As another line of the solution, we
present SurfaceNet+, a volumetric method to handle the 'incompleteness' and the
'inaccuracy' problems induced by a very sparse MVS setup. Specifically, the
former problem is handled by a novel volume-wise view selection approach. It
owns superiority in selecting valid views while discarding invalid occluded
views by considering the geometric prior. Furthermore, the latter problem is
handled via a multi-scale strategy that consequently refines the recovered
geometry around the region with the repeating pattern. The experiments
demonstrate the tremendous performance gap between SurfaceNet+ and
state-of-the-art methods in terms of precision and recall. Under the extreme
sparse-MVS settings in two datasets, where existing methods can only return
very few points, SurfaceNet+ still works as well as in the dense MVS setting.
The benchmark and the implementation are publicly available at
this https URL

本文针对少量稀疏数据下的 Multi-view stereopsis（MVS）问题，提出一种基于体积的方法 SurfaceNet+ 来解决 ' 不完整性 ' 和 ' 不准确性 ' 问题，并利用一种新颖的体量视图选择方法来选择有效视图并且丢弃无效被遮挡的视图，这一方法在精度和召回率方面具有比其他现有方法更大的性能优势。

SurfaceNet+: 一种用于极度稀疏的多视角立体视觉的端到端 3D 神经网络

SurfaceNet+: An End-to-end 3D Neural Network for Very Sparse Multi-view  Stereopsis

We present a learning based approach for multi-view stereopsis (MVS). While
current deep MVS methods achieve impressive results, they crucially rely on
ground-truth 3D training data, and acquisition of such precise 3D geometry for
supervision is a major hurdle. Our framework instead leverages photometric
consistency between multiple views as supervisory signal for learning depth
prediction in a wide baseline MVS setup. However, naively applying photo
consistency constraints is undesirable due to occlusion and lighting changes
across views. To overcome this, we propose a robust loss formulation that: a)
enforces first order consistency and b) for each point, selectively enforces
consistency with some views, thus implicitly handling occlusions. We
demonstrate our ability to learn MVS without 3D supervision using a real
dataset, and show that each component of our proposed robust loss results in a
significant improvement. We qualitatively observe that our reconstructions are
often more complete than the acquired ground truth, further showing the merits
of this approach. Lastly, our learned model generalizes to novel settings, and
our approach allows adaptation of existing CNNs to datasets without
ground-truth 3D by unsupervised finetuning. Project webpage:
this https URL

提出了一种基于学习的多视点立体感知方法，可以在宽基线 MVS 环境下学习深度预测并通过稳健的损失函数处理遮挡和光照等问题。该方法不需要使用三维监督数据进行训练，并且可以通过无监督的微调实现对新数据集的适应。

通过强健的光度一致性学习无监督的多视图立体成像

Learning Unsupervised Multi-View Stereopsis via Robust Photometric  Consistency

We present a learnt system for multi-view stereopsis. In contrast to recent
learning based methods for 3D reconstruction, we leverage the underlying 3D
geometry of the problem through feature projection and unprojection along
viewing rays. By formulating these operations in a differentiable manner, we
are able to learn the system end-to-end for the task of metric 3D
reconstruction. End-to-end learning allows us to jointly reason about shape
priors while conforming geometric constraints, enabling reconstruction from
much fewer images (even a single image) than required by classical approaches
as well as completion of unseen surfaces. We thoroughly evaluate our approach
on the ShapeNet dataset and demonstrate the benefits over classical approaches
as well as recent learning based methods.

通过特征投影和反投影沿视线的方式，我们提出了一种用于多视图立体成像的学习系统，通过可微化运算，我们能够对 3D 形状进行度量重构，从而实现在几张相对较少的图像（甚至一张图像）中重建 3D 图像，并且证明了我们的方法在 ShapeNet 数据集上优于传统方法和最新的基于学习的方法。