Existing top-performance 3D object detectors typically rely on the
multi-modal fusion strategy. This design is however fundamentally restricted
due to overlooking the modality-specific useful information and finally
hampering the model performance. To address this limitation, in this work we
introduce a novel modality interaction strategy where individual per-modality
representations are learned and maintained throughout for enabling their unique
characteristics to be exploited during object detection. To realize this
proposed strategy, we design a DeepInteraction architecture characterized by a
multi-modal representational interaction encoder and a multi-modal predictive
interaction decoder. Experiments on the large-scale nuScenes dataset show that
our proposed method surpasses all prior arts often by a large margin.
Crucially, our method is ranked at the first position at the highly competitive
nuScenes object detection leaderboard.

本文提出一种新的模态交互策略，采用 DeepInteraction 结构对多模态特征进行交互和预测，此方法在大规模 nuScenes 数据集上取得比之前所有技术更好的结果，并跻身 nuScenes 目标检测榜首。