Combining the signed distance function (SDF) and differentiable volume
rendering has emerged as a powerful paradigm for surface reconstruction from
multi-view images without 3D supervision. However, current methods are impeded
by requiring long-time per-scene optimizations and cannot generalize to new
scenes. In this paper, we present GenS, an end-to-end generalizable neural
surface reconstruction model. Unlike coordinate-based methods that train a
separate network for each scene, we construct a generalized multi-scale volume
to directly encode all scenes. Compared with existing solutions, our
representation is more powerful, which can recover high-frequency details while
maintaining global smoothness. Meanwhile, we introduce a multi-scale
feature-metric consistency to impose the multi-view consistency in a more
discriminative multi-scale feature space, which is robust to the failures of
the photometric consistency. And the learnable feature can be self-enhanced to
continuously improve the matching accuracy and mitigate aggregation ambiguity.
Furthermore, we design a view contrast loss to force the model to be robust to
those regions covered by few viewpoints through distilling the geometric prior
from dense input to sparse input. Extensive experiments on popular benchmarks
show that our model can generalize well to new scenes and outperform existing
state-of-the-art methods even those employing ground-truth depth supervision.
Code is available at this https URL

我们提出了 GenS，一个端到端的通用神经表面重建模型，可以通过结合有符号距离函数（SDF）和可微体渲染，将多视图图像无需 3D 监督进行表面重建。与现有解决方案相比，我们的表示更强大，能够恢复高频细节并同时保持全局平滑性。同时，我们引入多尺度特征度量一致性，以在更具区分性的多尺度特征空间中施加多视图一致性，从而抵抗光度一致性的失效。我们还设计了一种视角对比损失，通过将密集输入中的几何先验精炼到稀疏输入中，强制模型对少视角覆盖的区域具有鲁棒性。在流行的基准测试上进行的大量实验证明，我们的模型能够很好地推广到新场景，并且胜过现有的最先进方法，即使那些使用地面真实深度监督的方法。

GenS：多视角图像的通用神经表面重建

GenS: Generalizable Neural Surface Reconstruction from Multi-View Images

In this paper, we study the problem of stereo matching from a pair of images
with different resolutions, e.g., those acquired with a tele-wide camera
system. Due to the difficulty of obtaining ground-truth disparity labels in
diverse real-world systems, we start from an unsupervised learning perspective.
However, resolution asymmetry caused by unknown degradations between two views
hinders the effectiveness of the generally assumed photometric consistency. To
overcome this challenge, we propose to impose the consistency between two views
in a feature space instead of the image space, named feature-metric
consistency. Interestingly, we find that, although a stereo matching network
trained with the photometric loss is not optimal, its feature extractor can
produce degradation-agnostic and matching-specific features. These features can
then be utilized to formulate a feature-metric loss to avoid the photometric
inconsistency. Moreover, we introduce a self-boosting strategy to optimize the
feature extractor progressively, which further strengthens the feature-metric
consistency. Experiments on both simulated datasets with various degradations
and a self-collected real-world dataset validate the superior performance of
the proposed method over existing solutions.

本研究提出了一种基于无监督学习的视差匹配方法，通过特征度量一致性而非图像度量一致性来解决不同分辨率图像之间的挑战，并引入了一种自我增强的策略来优化特征提取器，在模拟数据集和真实世界数据集上，取得了优于现有解决方案的表现。