Vehicle-to-Everything (V2X) collaborative perception is crucial for
autonomous driving. However, achieving high-precision V2X perception requires a
significant amount of annotated real-world data, which can always be expensive
and hard to acquire. Simulated data have raised much attention since they can
be massively produced at an extremely low cost. Nevertheless, the significant
domain gap between simulated and real-world data, including differences in
sensor type, reflectance patterns, and road surroundings, often leads to poor
performance of models trained on simulated data when evaluated on real-world
data. In addition, there remains a domain gap between real-world collaborative
agents, e.g. different types of sensors may be installed on autonomous vehicles
and roadside infrastructures with different extrinsics, further increasing the
difficulty of sim2real generalization. To take full advantage of simulated
data, we present a new unsupervised sim2real domain adaptation method for V2X
collaborative detection named Decoupled Unsupervised Sim2Real Adaptation
(DUSA). Our new method decouples the V2X collaborative sim2real domain
adaptation problem into two sub-problems: sim2real adaptation and inter-agent
adaptation. For sim2real adaptation, we design a Location-adaptive Sim2Real
Adapter (LSA) module to adaptively aggregate features from critical locations
of the feature map and align the features between simulated data and real-world
data via a sim/real discriminator on the aggregated global feature. For
inter-agent adaptation, we further devise a Confidence-aware Inter-agent
Adapter (CIA) module to align the fine-grained features from heterogeneous
agents under the guidance of agent-wise confidence maps. Experiments
demonstrate the effectiveness of the proposed DUSA approach on unsupervised
sim2real adaptation from the simulated V2XSet dataset to the real-world
DAIR-V2X-C dataset.

车联网 (V2X) 的协作感知对于自动驾驶至关重要。然而，实现高精度的 V2X 感知需要大量的真实世界数据进行标注，这往往昂贵且难以获取。模拟数据因其可大规模低成本生成而备受关注。然而，模拟数据与真实世界数据之间存在显著的领域差异，包括传感器类型、反射模式和道路环境差异，导致在真实世界数据上评估的模型性能较差。为了充分利用模拟数据，我们提出了一种新的无监督 sim2real 领域适应方法，名为 Decoupled Unsupervised Sim2Real Adaptation (DUSA)。我们的新方法将 V2X 协作 sim2real 领域适应问题分解为两个子问题：sim2real 领域适应和协同代理适应。通过针对 sim2real 领域适应，设计了一种自适应聚合特征的位置适应型 Sim2Real Adapter (LSA) 模块，通过聚合全局特征上的模拟 / 真实判别器来调整来自特征图的关键位置的特征，并对模拟数据和真实世界数据之间的特征进行对齐。针对协同代理适应，我们进一步设计了一种自信度感知的协同代理 Adapter (CIA) 模块，通过代理自信度图的引导来调整异构代理的细粒度特征。实验证明了所提出的 DUSA 方法在从模拟 V2XSet 数据集到真实世界 DAIR-V2X-C 数据集的无监督 sim2real 领域适应任务中的有效性。

DUSA: 解耦无监督 Sim2Real 车辆协同感知适应

DUSA: Decoupled Unsupervised Sim2Real Adaptation for  Vehicle-to-Everything Collaborative Perception

Modern perception systems of autonomous vehicles are known to be sensitive to
occlusions and lack the capability of long perceiving range. It has been one of
the key bottlenecks that prevents Level 5 autonomy. Recent research has
demonstrated that the Vehicle-to-Vehicle (V2V) cooperative perception system
has great potential to revolutionize the autonomous driving industry. However,
the lack of a real-world dataset hinders the progress of this field. To
facilitate the development of cooperative perception, we present V2V4Real, the
first large-scale real-world multi-modal dataset for V2V perception. The data
is collected by two vehicles equipped with multi-modal sensors driving together
through diverse scenarios. Our V2V4Real dataset covers a driving area of 410
km, comprising 20K LiDAR frames, 40K RGB frames, 240K annotated 3D bounding
boxes for 5 classes, and HDMaps that cover all the driving routes. V2V4Real
introduces three perception tasks, including cooperative 3D object detection,
cooperative 3D object tracking, and Sim2Real domain adaptation for cooperative
perception. We provide comprehensive benchmarks of recent cooperative
perception algorithms on three tasks. The V2V4Real dataset can be found at
this https URL

本文介绍了 V2V4Real 多模态数据集和近期合作感知算法在三项任务上的全面基准测试。通过两辆车行驶，由多模态传感器收集数据，该数据集包括 20K LiDAR 帧、40K RGB 帧、240K 5 类三维边界框的标注和覆盖所有行驶路线的 HDMaps。该数据集对协作 3D 物体检测、协作 3D 物体跟踪和 Sim2Real 领域自适应合作感知等三个任务提出了需要解决的问题。