This paper addresses the challenge of reconstructing surfaces from sparse
view inputs, where ambiguity and occlusions due to missing information pose
significant hurdles. We present a novel approach, named EpiS, that incorporates
Epipolar information into the reconstruction process. Existing methods in
sparse-view neural surface learning have mainly focused on mean and variance
considerations using cost volumes for feature extraction. In contrast, our
method aggregates coarse information from the cost volume into Epipolar
features extracted from multiple source views, enabling the generation of
fine-grained Signal Distance Function (SDF)-aware features. Additionally, we
employ an attention mechanism along the line dimension to facilitate feature
fusion based on the SDF feature. Furthermore, to address the information gaps
in sparse conditions, we integrate depth information from monocular depth
estimation using global and local regularization techniques. The global
regularization utilizes a triplet loss function, while the local regularization
employs a derivative loss function. Extensive experiments demonstrate that our
approach outperforms state-of-the-art methods, especially in cases with sparse
and generalizable conditions.

稀疏视图输入中重建表面的挑战一直存在着，本文提出了一种名为 EpiS 的新方法，它将对极线信息纳入重建过程，聚合了来自多个源视图的线级信息，生成精细的信号距离函数 (SDF) 感知特征，并通过注意力机制实现了特征融合。此外，本文还采用全局和局部正则化技术，整合了从单目深度估计中得到的深度信息，用于解决稀疏条件下的信息缺失问题。广泛的实验证明，我们的方法在稀疏和可推广条件下优于现有的方法。

使用极线几何从稀疏视角进行神经表面重建

Neural Surface Reconstruction from Sparse Views Using Epipolar Geometry

3D decomposition/segmentation still remains a challenge as large-scale 3D
annotated data is not readily available. Contemporary approaches typically
leverage 2D machine-generated segments, integrating them for 3D consistency.
While the majority of these methods are based on NeRFs, they face a potential
weakness that the instance/semantic embedding features derive from independent
MLPs, thus preventing the segmentation network from learning the geometric
details of the objects directly through radiance and density. In this paper, we
propose ClusteringSDF, a novel approach to achieve both segmentation and
reconstruction in 3D via the neural implicit surface representation,
specifically Signal Distance Function (SDF), where the segmentation rendering
is directly integrated with the volume rendering of neural implicit surfaces.
Although based on ObjectSDF++, ClusteringSDF no longer requires the
ground-truth segments for supervision while maintaining the capability of
reconstructing individual object surfaces, but purely with the noisy and
inconsistent labels from pre-trained models.As the core of ClusteringSDF, we
introduce a high-efficient clustering mechanism for lifting the 2D labels to 3D
and the experimental results on the challenging scenes from ScanNet and Replica
datasets show that ClusteringSDF can achieve competitive performance compared
against the state-of-the-art with significantly reduced training time.

通过神经隐式表面表示，即信号距离函数（SDF），提出了 ClusteringSDF 方法，实现了 3D 中的分割和重建。这项研究通过高效的聚类机制将 2D 标签提升到 3D，并在 ScanNet 和 Replica 数据集上的挑战性场景中取得了与现有技术相竞争的表现，而且训练时间大大缩短。