Three-dimensional object detection is one of the key tasks in autonomous
driving. To reduce costs in practice, low-cost multi-view cameras for 3D object
detection are proposed to replace the expansive LiDAR sensors. However, relying
solely on cameras is difficult to achieve highly accurate and robust 3D object
detection. An effective solution to this issue is combining multi-view cameras
with the economical millimeter-wave radar sensor to achieve more reliable
multi-modal 3D object detection. In this paper, we introduce RCBEVDet, a
radar-camera fusion 3D object detection method in the bird's eye view (BEV).
Specifically, we first design RadarBEVNet for radar BEV feature extraction.
RadarBEVNet consists of a dual-stream radar backbone and a Radar Cross-Section
(RCS) aware BEV encoder. In the dual-stream radar backbone, a point-based
encoder and a transformer-based encoder are proposed to extract radar features,
with an injection and extraction module to facilitate communication between the
two encoders. The RCS-aware BEV encoder takes RCS as the object size prior to
scattering the point feature in BEV. Besides, we present the Cross-Attention
Multi-layer Fusion module to automatically align the multi-modal BEV feature
from radar and camera with the deformable attention mechanism, and then fuse
the feature with channel and spatial fusion layers. Experimental results show
that RCBEVDet achieves new state-of-the-art radar-camera fusion results on
nuScenes and view-of-delft (VoD) 3D object detection benchmarks. Furthermore,
RCBEVDet achieves better 3D detection results than all real-time camera-only
and radar-camera 3D object detectors with a faster inference speed at 21~28
FPS. The source code will be released at this https URL

RCBEVDet 是一种在鸟瞰视角下将雷达和相机融合的三维物体检测方法，通过设计 RadarBEVNet 网络进行雷达鸟瞰特征提取，并结合交叉注意力多层融合模块实现自动对齐和融合多模态特征，取得了领先于现有方法的检测结果。

RCBEVDet: 用于三维物体检测的鸟瞰雷达 - 摄像头融合

RCBEVDet: Radar-camera Fusion in Bird's Eye View for 3D Object Detection

In multimodal traffic monitoring, we gather traffic statistics for distinct
transportation modes, such as pedestrians, cars and bicycles, in order to
analyze and improve people's daily mobility in terms of safety and convenience.
On account of its robustness to bad light and adverse weather conditions, and
inherent speed measurement ability, the radar sensor is a suitable option for
this application. However, the sparse radar data from conventional commercial
radars make it extremely challenging for transportation mode classification.
Thus, we propose to use a high-resolution millimeter-wave(mmWave) radar sensor
to obtain a relatively richer radar point cloud representation for a traffic
monitoring scenario. Based on a new feature vector, we use the multivariate
Gaussian mixture model (GMM) to do the radar point cloud segmentation, i.e.
`point-wise' classification, in an unsupervised learning environment. In our
experiment, we collected radar point clouds for pedestrians and cars, which
also contained the inevitable clutter from the surroundings. The experimental
results using GMM on the new feature vector demonstrated a good segmentation
performance in terms of the intersection-over-union (IoU) metrics. The detailed
methodology and validation metrics are presented and discussed.

以毫米波雷达为传感器，提出基于高分辨率雷达点云的混合高斯模型无监督学习算法，在多模态交通监测中取得了良好的分割效果。