Autonomous Vehicles (AVs) use multiple sensors to gather information about
their surroundings. By sharing sensor data between Connected Autonomous
Vehicles (CAVs), the safety and reliability of these vehicles can be improved
through a concept known as cooperative perception. However, recent approaches
in cooperative perception only share single sensor information such as cameras
or LiDAR. In this research, we explore the fusion of multiple sensor data
sources and present a framework, called CoBEVFusion, that fuses LiDAR and
camera data to create a Bird's-Eye View (BEV) representation. The CAVs process
the multi-modal data locally and utilize a Dual Window-based Cross-Attention
(DWCA) module to fuse the LiDAR and camera features into a unified BEV
representation. The fused BEV feature maps are shared among the CAVs, and a 3D
Convolutional Neural Network is applied to aggregate the features from the
CAVs. Our CoBEVFusion framework was evaluated on the cooperative perception
dataset OPV2V for two perception tasks: BEV semantic segmentation and 3D object
detection. The results show that our DWCA LiDAR-camera fusion model outperforms
perception models with single-modal data and state-of-the-art BEV fusion
models. Our overall cooperative perception architecture, CoBEVFusion, also
achieves comparable performance with other cooperative perception models.

通过利用多传感器融合的方法，我们提出了一种名为 CoBEVFusion 的框架，将 LiDAR 和相机数据融合为鸟瞰图（BEV）表示，以改善协同感知在自动驾驶车辆中的安全性和可靠性。我们的 DWCA LiDAR-camera fusion 模型在 BEV 语义分割和 3D 物体检测任务中表现优于单模态数据和最先进的 BEV 融合模型。同时，CoBEVFusion 整体协同感知架构的性能与其他协同感知模型相当。