We propose NEDS-SLAM, an Explicit Dense semantic SLAM system based on 3D
Gaussian representation, that enables robust 3D semantic mapping, accurate
camera tracking, and high-quality rendering in real-time. In the system, we
propose a Spatially Consistent Feature Fusion model to reduce the effect of
erroneous estimates from pre-trained segmentation head on semantic
reconstruction, achieving robust 3D semantic Gaussian mapping. Additionally, we
employ a lightweight encoder-decoder to compress the high-dimensional semantic
features into a compact 3D Gaussian representation, mitigating the burden of
excessive memory consumption. Furthermore, we leverage the advantage of 3D
Gaussian splatting, which enables efficient and differentiable novel view
rendering, and propose a Virtual Camera View Pruning method to eliminate
outlier GS points, thereby effectively enhancing the quality of scene
representations. Our NEDS-SLAM method demonstrates competitive performance over
existing dense semantic SLAM methods in terms of mapping and tracking accuracy
on Replica and ScanNet datasets, while also showing excellent capabilities in
3D dense semantic mapping.

NEDS-SLAM 是一种基于 3D 高斯表示的显式稠密语义 SLAM 系统，可以实时进行鲁棒的 3D 语义映射、准确的相机跟踪和高质量的渲染。

NEDS-SLAM：一种新颖的神经显式稠密语义 SLAM 框架，使用 3D 高斯光滑插值

NEDS-SLAM: A Novel Neural Explicit Dense Semantic SLAM Framework using  3D Gaussian Splatting

We propose SemGauss-SLAM, the first semantic SLAM system utilizing 3D
Gaussian representation, that enables accurate 3D semantic mapping, robust
camera tracking, and high-quality rendering in real-time. In this system, we
incorporate semantic feature embedding into 3D Gaussian representation, which
effectively encodes semantic information within the spatial layout of the
environment for precise semantic scene representation. Furthermore, we propose
feature-level loss for updating 3D Gaussian representation, enabling
higher-level guidance for 3D Gaussian optimization. In addition, to reduce
cumulative drift and improve reconstruction accuracy, we introduce
semantic-informed bundle adjustment leveraging semantic associations for joint
optimization of 3D Gaussian representation and camera poses, leading to more
robust tracking and consistent mapping. Our SemGauss-SLAM method demonstrates
superior performance over existing dense semantic SLAM methods in terms of
mapping and tracking accuracy on Replica and ScanNet datasets, while also
showing excellent capabilities in novel-view semantic synthesis and 3D semantic
mapping.

提出 SemGauss-SLAM，第一个利用 3D 高斯表示的语义 SLAM 系统，实现实时精确的 3D 语义地图构建、鲁棒的相机跟踪和高质量渲染。该系统将语义特征嵌入 3D 高斯表示，有效地在环境的空间布局中编码语义信息以获得精确的语义场景表示；同时，提出特征级损失来更新 3D 高斯表示，为 3D 高斯优化提供更高层次的指导；此外，通过引入语义感知束调整，利用语义关联进行 3D 高斯表示和相机位姿的联合优化，从而减小累积漂移并提高重建精度。在 Replica 和 ScanNet 数据集上，与现有的密集语义 SLAM 方法相比，我们的 SemGauss-SLAM 方法在地图构建和相机跟踪的准确性方面表现出更好的性能，并展示出新视角语义合成和 3D 语义映射的优秀能力。

SemGauss-SLAM：密集语义高斯分层 SLAM

SemGauss-SLAM: Dense Semantic Gaussian Splatting SLAM

In this paper, we present a complete pipeline for 3D semantic mapping solely
based on a stereo camera system. The pipeline comprises a direct sparse visual
odometry front-end as well as a back-end for global optimization including GNSS
integration, and semantic 3D point cloud labeling. We propose a simple but
effective temporal voting scheme which improves the quality and consistency of
the 3D point labels. Qualitative and quantitative evaluations of our pipeline
are performed on the KITTI-360 dataset. The results show the effectiveness of
our proposed voting scheme and the capability of our pipeline for efficient
large-scale 3D semantic mapping. The large-scale mapping capabilities of our
pipeline is furthermore demonstrated by presenting a very large-scale semantic
map covering 8000 km of roads generated from data collected by a fleet of
vehicles.

本文提出一种仅基于立体相机系统完成 3D 语义映射的完整流程，其中包括全局优化的直接稀疏视觉里程表前端和 GNSS 集成，以及语义 3D 点云标记的后端。作者提出了一种简单但有效的时间投票方案，提高了 3D 点标签的质量和一致性，并在 KITTI-360 数据集上进行了定性和定量评估，结果表明了提出的投票方案的有效性和流水线对于大规模高效 3D 语义映射的能力。另外，作者还展示了流水线的大规模映射能力，展示了由车队收集的数据生成的覆盖 8000 公里道路的大规模语义地图。