With the emergence of Gaussian Splats, recent efforts have focused on
large-scale scene geometric reconstruction. However, most of these efforts
either concentrate on memory reduction or spatial space division, neglecting
information in the semantic space. In this paper, we propose a novel method,
named SA-GS, for fine-grained 3D geometry reconstruction using semantic-aware
3D Gaussian Splats. Specifically, we leverage prior information stored in large
vision models such as SAM and DINO to generate semantic masks. We then
introduce a geometric complexity measurement function to serve as soft
regularization, guiding the shape of each Gaussian Splat within specific
semantic areas. Additionally, we present a method that estimates the expected
number of Gaussian Splats in different semantic areas, effectively providing a
lower bound for Gaussian Splats in these areas. Subsequently, we extract the
point cloud using a novel probability density-based extraction method,
transforming Gaussian Splats into a point cloud crucial for downstream tasks.
Our method also offers the potential for detailed semantic inquiries while
maintaining high image-based reconstruction results. We provide extensive
experiments on publicly available large-scale scene reconstruction datasets
with highly accurate point clouds as ground truth and our novel dataset. Our
results demonstrate the superiority of our method over current state-of-the-art
Gaussian Splats reconstruction methods by a significant margin in terms of
geometric-based measurement metrics. Code and additional results will soon be
available on our project page.

采用语义感知的 3D 高斯斑点，提出了一种新的方法 SA-GS 用于细粒度的 3D 几何重建，并在大规模场景重建数据集上展示了其卓越性能。