Existing neural implicit surface reconstruction methods have achieved
impressive performance in multi-view 3D reconstruction by leveraging explicit
geometry priors such as depth maps or point clouds as regularization. However,
the reconstruction results still lack fine details because of the over-smoothed
depth map or sparse point cloud. In this work, we propose a neural implicit
surface reconstruction pipeline with guidance from 3D Gaussian Splatting to
recover highly detailed surfaces. The advantage of 3D Gaussian Splatting is
that it can generate dense point clouds with detailed structure. Nonetheless, a
naive adoption of 3D Gaussian Splatting can fail since the generated points are
the centers of 3D Gaussians that do not necessarily lie on the surface. We thus
introduce a scale regularizer to pull the centers close to the surface by
enforcing the 3D Gaussians to be extremely thin. Moreover, we propose to refine
the point cloud from 3D Gaussians Splatting with the normal priors from the
surface predicted by neural implicit models instead of using a fixed set of
points as guidance. Consequently, the quality of surface reconstruction
improves from the guidance of the more accurate 3D Gaussian splatting. By
jointly optimizing the 3D Gaussian Splatting and the neural implicit model, our
approach benefits from both representations and generates complete surfaces
with intricate details. Experiments on Tanks and Temples verify the
effectiveness of our proposed method.

神经隐式表面重建方法通过引导来自三维高斯化簇粘附的点云，结合来自神经隐式模型的表面预测精炼来提高细节的重建效果。

NeuSG: 基于神经网络的隐式表面重建与 3D 高斯喷点引导

NeuSG: Neural Implicit Surface Reconstruction with 3D Gaussian Splatting  Guidance

This paper addresses the problem of generating dense point clouds from given
sparse point clouds to model the underlying geometric structures of
objects/scenes. To tackle this challenging issue, we propose a novel end-to-end
learning-based framework. Specifically, by taking advantage of the linear
approximation theorem, we first formulate the problem explicitly, which boils
down to determining the interpolation weights and high-order approximation
errors. Then, we design a lightweight neural network to adaptively learn
unified and sorted interpolation weights as well as the high-order refinements,
by analyzing the local geometry of the input point cloud. The proposed method
can be interpreted by the explicit formulation, and thus is more
memory-efficient than existing ones. In sharp contrast to the existing methods
that work only for a pre-defined and fixed upsampling factor, the proposed
framework only requires a single neural network with one-time training to
handle various upsampling factors within a typical range, which is highly
desired in real-world applications. In addition, we propose a simple yet
effective training strategy to drive such a flexible ability. In addition, our
method can handle non-uniformly distributed and noisy data well. Extensive
experiments on both synthetic and real-world data demonstrate the superiority
of the proposed method over state-of-the-art methods both quantitatively and
qualitatively.

本文提出了一种基于学习的框架，通过自适应地分析输入点云的局部几何结构，设计出轻量级神经网络来统一、排序插值权重以及高阶细化，从而生成具有密集点云的物体 / 场景，并处理非均匀分布和噪声数据，该方法于现实世界中的各种上采样因子都可运作，并在合成和实际数据上均获得了优于现有方法的表现。