3D object detection models that exploit both LiDAR and camera sensor features
are top performers in large-scale autonomous driving benchmarks. A transformer
is a popular network architecture used for this task, in which so-called object
queries act as candidate objects. Initializing these object queries based on
current sensor inputs is a common practice. For this, existing methods strongly
rely on LiDAR data however, and do not fully exploit image features. Besides,
they introduce significant latency. To overcome these limitations we propose
EfficientQ3M, an efficient, modular, and multimodal solution for object query
initialization for transformer-based 3D object detection models. The proposed
initialization method is combined with a "modality-balanced" transformer
decoder where the queries can access all sensor modalities throughout the
decoder. In experiments, we outperform the state of the art in
transformer-based LiDAR object detection on the competitive nuScenes benchmark
and showcase the benefits of input-dependent multimodal query initialization,
while being more efficient than the available alternatives for LiDAR-camera
initialization. The proposed method can be applied with any combination of
sensor modalities as input, demonstrating its modularity.

提出了 EfficientQ3M，一种高效、模块化、多模态的解决方案，用于基于 transformer 的 3D 目标检测模型中的对象查询初始化。通过与 “模态平衡” 的 transformer 解码器相结合，该方法能够在解码器过程中使查询能够访问所有传感器模态，并在竞争激烈的 nuScenes 基准测试中超越现有方法，展示了基于输入的多模态查询初始化的优势，并且比 LiDAR - 摄像机初始化的可用替代方法更高效。该方法可以应用于任何组合的传感器模态作为输入，展示了它的模块化特性。