Embodied agents require robust navigation systems to operate in unstructured
environments, making the robustness of Simultaneous Localization and Mapping
(SLAM) models critical to embodied agent autonomy. While real-world datasets
are invaluable, simulation-based benchmarks offer a scalable approach for
robustness evaluations. However, the creation of a challenging and controllable
noisy world with diverse perturbations remains under-explored. To this end, we
propose a novel, customizable pipeline for noisy data synthesis, aimed at
assessing the resilience of multi-modal SLAM models against various
perturbations. The pipeline comprises a comprehensive taxonomy of sensor and
motion perturbations for embodied multi-modal (specifically RGB-D) sensing,
categorized by their sources and propagation order, allowing for procedural
composition. We also provide a toolbox for synthesizing these perturbations,
enabling the transformation of clean environments into challenging noisy
simulations. Utilizing the pipeline, we instantiate the large-scale
Noisy-Replica benchmark, which includes diverse perturbation types, to evaluate
the risk tolerance of existing advanced RGB-D SLAM models. Our extensive
analysis uncovers the susceptibilities of both neural (NeRF and Gaussian
Splatting -based) and non-neural SLAM models to disturbances, despite their
demonstrated accuracy in standard benchmarks. Our code is publicly available at
this https URL

我们提出了一种新颖的、可定制的噪声数据合成管道，旨在评估多模态 SLAM 模型对各种干扰的弹性，通过使用所提供的工具箱，将清洁环境转化为具有挑战性的噪声模拟。使用该管道，我们构建了大规模 Noisy-Replica 基准，并评估现有高级 RGB-D SLAM 模型对多种干扰类型的风险容忍度。

从完美到嘈杂的世界模拟：用于 SLAM 鲁棒性评估的可定制的多模态干扰

From Perfect to Noisy World Simulation: Customizable Embodied  Multi-modal Perturbations for SLAM Robustness Benchmarking

Robustness is a crucial factor for the successful deployment of robots in
unstructured environments, particularly in the domain of Simultaneous
Localization and Mapping (SLAM). Simulation-based benchmarks have emerged as a
highly scalable approach for robustness evaluation compared to real-world data
collection. However, crafting a challenging and controllable noisy world with
diverse perturbations remains relatively under-explored. To this end, we
propose a novel, customizable pipeline for noisy data synthesis, aimed at
assessing the resilience of multi-modal SLAM models against various
perturbations. This pipeline incorporates customizable hardware setups,
software components, and perturbed environments. In particular, we introduce
comprehensive perturbation taxonomy along with a perturbation composition
toolbox, allowing the transformation of clean simulations into challenging
noisy environments. Utilizing the pipeline, we instantiate the Robust-SLAM
benchmark, which includes diverse perturbation types, to evaluate the risk
tolerance of existing advanced multi-modal SLAM models. Our extensive analysis
uncovers the susceptibilities of existing SLAM models to real-world
disturbance, despite their demonstrated accuracy in standard benchmarks. Our
perturbation synthesis toolbox, SLAM robustness evaluation pipeline, and
Robust-SLAM benchmark will be made publicly available at
this https URL

我们提出了一个用于评估多模态 SLAM 模型对各种扰动的鲁棒性的新型、可定制的噪声数据合成流程，并利用此流程实例化了 Robust-SLAM 基准，揭示了现有 SLAM 模型在真实世界干扰下的易感性。