Sensor fusion is crucial for an accurate and robust perception system on
autonomous vehicles. Most existing datasets and perception solutions focus on
fusing cameras and LiDAR. However, the collaboration between camera and radar
is significantly under-exploited. The incorporation of rich semantic
information from the camera, and reliable 3D information from the radar can
potentially achieve an efficient, cheap, and portable solution for 3D object
perception tasks. It can also be robust to different lighting or all-weather
driving scenarios due to the capability of mmWave radars. In this paper, we
introduce the CRUW3D dataset, including 66K synchronized and well-calibrated
camera, radar, and LiDAR frames in various driving scenarios. Unlike other
large-scale autonomous driving datasets, our radar data is in the format of
radio frequency (RF) tensors that contain not only 3D location information but
also spatio-temporal semantic information. This kind of radar format can enable
machine learning models to generate more reliable object perception results
after interacting and fusing the information or features between the camera and
radar.

传感器融合在自主车辆的准确和健壮的感知系统中至关重要。大多数现有的数据集和感知解决方案都专注于相机和激光雷达的融合，然而相机和雷达之间的协作被显著地低估。本论文介绍了 CRUW3D 数据集，包括 66K 个在不同行驶情景中同步并进行了校准的相机、雷达和激光雷达帧。与其他大规模自动驾驶数据集不同，我们的雷达数据以射频（RF）张量的格式呈现，包含了 3D 位置信息和时空语义信息。这种雷达格式可以使机器学习模型在相机和雷达之间交互和融合信息或特征后生成更可靠的物体感知结果。