Despite radar's popularity in the automotive industry, for fusion-based 3D
object detection, most existing works focus on LiDAR and camera fusion. In this
paper, we propose TransCAR, a Transformer-based Camera-And-Radar fusion
solution for 3D object detection. Our TransCAR consists of two modules. The
first module learns 2D features from surround-view camera images and then uses
a sparse set of 3D object queries to index into these 2D features. The
vision-updated queries then interact with each other via transformer
self-attention layer. The second module learns radar features from multiple
radar scans and then applies transformer decoder to learn the interactions
between radar features and vision-updated queries. The cross-attention layer
within the transformer decoder can adaptively learn the soft-association
between the radar features and vision-updated queries instead of
hard-association based on sensor calibration only. Finally, our model estimates
a bounding box per query using set-to-set Hungarian loss, which enables the
method to avoid non-maximum suppression. TransCAR improves the velocity
estimation using the radar scans without temporal information. The superior
experimental results of our TransCAR on the challenging nuScenes datasets
illustrate that our TransCAR outperforms state-of-the-art Camera-Radar
fusion-based 3D object detection approaches.

该研究提出一种基于 Transformer 的摄像头和雷达融合方案 TransCAR，用于 3D 物体检测，并通过 set-to-set 的匈牙利损失估计每个查询的边界框，最终在挑战的 nuScenes 数据集中得到了优越的实验结果，超越了现有的基于相机 - 雷达融合的 3D 物体检测方法。