In the field of 3D object detection for autonomous driving, LiDAR-Camera (LC)
fusion is the top-performing sensor configuration. Still, LiDAR is relatively
high cost, which hinders adoption of this technology for consumer automobiles.
Alternatively, camera and radar are commonly deployed on vehicles already on
the road today, but performance of Camera-Radar (CR) fusion falls behind LC
fusion. In this work, we propose Camera-Radar Knowledge Distillation (CRKD) to
bridge the performance gap between LC and CR detectors with a novel
cross-modality KD framework. We use the Bird's-Eye-View (BEV) representation as
the shared feature space to enable effective knowledge distillation. To
accommodate the unique cross-modality KD path, we propose four distillation
losses to help the student learn crucial features from the teacher model. We
present extensive evaluations on the nuScenes dataset to demonstrate the
effectiveness of the proposed CRKD framework. The project page for CRKD is
this https URL

在自动驾驶的三维物体检测领域，LiDAR - 相机融合是性能最佳的传感器配置，但是由于成本较高，阻碍了该技术在消费级汽车中的采用。本研究提出了一种相机 - 雷达知识蒸馏（CRKD）框架，旨在弥合相机 - 雷达融合和 LiDAR - 相机融合之间的性能差距，并通过引入鸟瞰图表示作为共享特征空间，实现有效的知识蒸馏。通过四个蒸馏损失函数帮助学生模型从教师模型中学习关键特征，并在 nuScenes 数据集上进行了广泛的评估，展示了 CRKD 框架的有效性。

CRKD：拥有跨模态知识蒸馏的增强型相机 - 雷达目标检测

CRKD: Enhanced Camera-Radar Object Detection with Cross-modality  Knowledge Distillation

A popular approach for constructing bird's-eye-view (BEV) representation in
3D detection is to lift 2D image features onto the viewing frustum space based
on explicitly predicted depth distribution. However, depth distribution can
only characterize the 3D geometry of visible object surfaces but fails to
capture their internal space and overall geometric structure, leading to sparse
and unsatisfactory 3D representations. To mitigate this issue, we present
BEV-IO, a new 3D detection paradigm to enhance BEV representation with instance
occupancy information. At the core of our method is the newly-designed instance
occupancy prediction (IOP) module, which aims to infer point-level occupancy
status for each instance in the frustum space. To ensure training efficiency
while maintaining representational flexibility, it is trained using the
combination of both explicit and implicit supervision. With the predicted
occupancy, we further design a geometry-aware feature propagation mechanism
(GFP), which performs self-attention based on occupancy distribution along each
ray in frustum and is able to enforce instance-level feature consistency. By
integrating the IOP module with GFP mechanism, our BEV-IO detector is able to
render highly informative 3D scene structures with more comprehensive BEV
representations. Experimental results demonstrate that BEV-IO can outperform
state-of-the-art methods while only adding a negligible increase in parameters
(0.2%) and computational overhead (0.24%in GFLOPs).

提出了一种新的 3D 检测模型 BEV-IO，利用实例占用信息和几何感知特征传播机制增强了鸟瞰图表示，实验结果表明 BEV-IO 检测器能够提供更全面的 BEV 表示并且优于现有的方法。

BEV-IO: 用实例占用量增强鸟瞰图三维检测

BEV-IO: Enhancing Bird's-Eye-View 3D Detection with Instance Occupancy

The goal of perception for autonomous vehicles is to extract semantic
representations from multiple sensors and fuse these representations into a
single "bird's-eye-view" coordinate frame for consumption by motion planning.
We propose a new end-to-end architecture that directly extracts a
bird's-eye-view representation of a scene given image data from an arbitrary
number of cameras. The core idea behind our approach is to "lift" each image
individually into a frustum of features for each camera, then "splat" all
frustums into a rasterized bird's-eye-view grid. By training on the entire
camera rig, we provide evidence that our model is able to learn not only how to
represent images but how to fuse predictions from all cameras into a single
cohesive representation of the scene while being robust to calibration error.
On standard bird's-eye-view tasks such as object segmentation and map
segmentation, our model outperforms all baselines and prior work. In pursuit of
the goal of learning dense representations for motion planning, we show that
the representations inferred by our model enable interpretable end-to-end
motion planning by "shooting" template trajectories into a bird's-eye-view cost
map output by our network. We benchmark our approach against models that use
oracle depth from lidar. Project page with code:
this https URL .

本文提出了一种新的端到端架构，能够直接从任意数量的相机图像数据中提取场景的鸟瞰图表示，并将其整合成单一的场景表示，该模型能够学习如何融合所有相机的预测结果，并针对密集表示进行运动规划。