Due to the lack of real multi-agent data and time-consuming of labeling,
existing multi-agent cooperative perception algorithms usually select the
simulated sensor data for training and validating. However, the perception
performance is degraded when these simulation-trained models are deployed to
the real world, due to the significant domain gap between the simulated and
real data. In this paper, we propose the first Simulation-to-Reality transfer
learning framework for multi-agent cooperative perception using a novel Vision
Transformer, named as S2R-ViT, which considers both the Implementation Gap and
Feature Gap between simulated and real data. We investigate the effects of
these two types of domain gaps and propose a novel uncertainty-aware vision
transformer to effectively relief the Implementation Gap and an agent-based
feature adaptation module with inter-agent and ego-agent discriminators to
reduce the Feature Gap. Our intensive experiments on the public multi-agent
cooperative perception datasets OPV2V and V2V4Real demonstrate that the
proposed S2R-ViT can effectively bridge the gap from simulation to reality and
outperform other methods significantly for point cloud-based 3D object
detection.

本篇论文提出了一种名为 S2R-ViT 的新型视觉转换器的模拟到现实转换学习框架，以解决现有多智能体协作感知算法的模拟与现实数据之间的显著领域差距问题，并进行了公开数据集上的实验证明，其在点云 3D 目标检测方面的性能优于其他方法。