In this work, we present a unified framework for multi-modality 3D object
detection, named UVTR. The proposed method aims to unify multi-modality
representations in the voxel space for accurate and robust single- or
cross-modality 3D detection. To this end, the modality-specific space is first
designed to represent different inputs in the voxel feature space. Different
from previous work, our approach preserves the voxel space without height
compression to alleviate semantic ambiguity and enable spatial connections. To
make full use of the inputs from different sensors, the cross-modality
interaction is then proposed, including knowledge transfer and modality fusion.
In this way, geometry-aware expressions in point clouds and context-rich
features in images are well utilized for better performance and robustness. The
transformer decoder is applied to efficiently sample features from the unified
space with learnable positions, which facilitates object-level interactions. In
general, UVTR presents an early attempt to represent different modalities in a
unified framework. It surpasses previous work in single- or multi-modality
entries. The proposed method achieves leading performance in the nuScenes test
set for both object detection and the following object tracking task. Code is
made publicly available at this https URL

UVTR 提出了一种统一的框架，旨在提高单模或跨模 3D 检测的准确性和鲁棒性，通过设计模态特定空间，保留体素空间并开发不同传感器的输入，实现几何感知点云和图像的上下文特征的交互，从而在 nuScenes 测试集中获得了领先的性能。