Humans naturally retain memories of permanent elements, while ephemeral
moments often slip through the cracks of memory. This selective retention is
crucial for robotic perception, localization, and mapping. To endow robots with
this capability, we introduce 3D Gaussian Mapping (3DGM), a self-supervised,
camera-only offline mapping framework grounded in 3D Gaussian Splatting. 3DGM
converts multitraverse RGB videos from the same region into a Gaussian-based
environmental map while concurrently performing 2D ephemeral object
segmentation. Our key observation is that the environment remains consistent
across traversals, while objects frequently change. This allows us to exploit
self-supervision from repeated traversals to achieve environment-object
decomposition. More specifically, 3DGM formulates multitraverse environmental
mapping as a robust differentiable rendering problem, treating pixels of the
environment and objects as inliers and outliers, respectively. Using robust
feature distillation, feature residuals mining, and robust optimization, 3DGM
jointly performs 3D mapping and 2D segmentation without human intervention. We
build the Mapverse benchmark, sourced from the Ithaca365 and nuPlan datasets,
to evaluate our method in unsupervised 2D segmentation, 3D reconstruction, and
neural rendering. Extensive results verify the effectiveness and potential of
our method for self-driving and robotics.

通过 3D 高斯建模，使用自监督离线地图构建方法实现了基于相机的环境建模和 2D 目标分割，利用多次遍历的自我监督学习对环境和目标进行分解，通过特征提取、特征残差挖掘和鲁棒优化实现 3D 建图和 2D 分割，验证了该方法在无人驾驶和机器人技术方面的有效性和潜力。