Recent progress in semantic scene understanding has primarily been enabled by
the availability of semantically annotated bi-modal (camera and lidar) datasets
in urban environments. However, such annotated datasets are also needed for
natural, unstructured environments to enable semantic perception for
applications, including conservation, search and rescue, environment
monitoring, and agricultural automation. Therefore, we introduce WildScenes, a
bi-modal benchmark dataset consisting of multiple large-scale traversals in
natural environments, including semantic annotations in high-resolution 2D
images and dense 3D lidar point clouds, and accurate 6-DoF pose information.
The data is (1) trajectory-centric with accurate localization and globally
aligned point clouds, (2) calibrated and synchronized to support bi-modal
inference, and (3) containing different natural environments over 6 months to
support research on domain adaptation. Our 3D semantic labels are obtained via
an efficient automated process that transfers the human-annotated 2D labels
from multiple views into 3D point clouds, thus circumventing the need for
expensive and time-consuming human annotation in 3D. We introduce benchmarks on
2D and 3D semantic segmentation and evaluate a variety of recent deep-learning
techniques to demonstrate the challenges in semantic segmentation in natural
environments. We propose train-val-test splits for standard benchmarks as well
as domain adaptation benchmarks and utilize an automated split generation
technique to ensure the balance of class label distributions. The data,
evaluation scripts and pretrained models will be released upon acceptance at
this https URL

近期在语义场景理解方面的进展主要得益于城市环境中具有语义注释的双模态（相机和激光雷达）数据集的可用性。然而，为了实现自然、非结构化环境下的语义感知应用，包括自然保护、搜救、环境监测和农业自动化，我们也需要具有语义注释的数据集。因此，我们介绍了 WildScenes，一个双模态基准数据集，其中包括自然环境下多个大规模遍历，包括高分辨率 2D 图像和密集 3D 激光雷达点云的语义注释，以及准确的 6 自由度姿态信息。我们的 3D 语义标签通过一种高效的自动化过程得到，该过程将多视角中的人工标注 2D 标签转移到 3D 点云中，从而避免了在 3D 中进行昂贵和耗时的人工注释。我们引入了关于 2D 和 3D 语义分割的基准，并评估了各种最新的深度学习技术，以展示在自然环境中语义分割中的挑战。我们提出了供标准基准和域自适应基准使用的训练 - 验证 - 测试划分，并利用一种自动划分生成技术来确保类别标签分布的平衡。该数据集、评估脚本和预训练模型将在接受之后发布于该 URL。