3D single object tracking plays a crucial role in computer vision. Mainstream
methods mainly rely on point clouds to achieve geometry matching between target
template and search area. However, textureless and incomplete point clouds make
it difficult for single-modal trackers to distinguish objects with similar
structures. To overcome the limitations of geometry matching, we propose a
Multi-modal Multi-level Fusion Tracker (MMF-Track), which exploits the image
texture and geometry characteristic of point clouds to track 3D target.
Specifically, we first propose a Space Alignment Module (SAM) to align RGB
images with point clouds in 3D space, which is the prerequisite for
constructing inter-modal associations. Then, in feature interaction level, we
design a Feature Interaction Module (FIM) based on dual-stream structure, which
enhances intra-modal features in parallel and constructs inter-modal semantic
associations. Meanwhile, in order to refine each modal feature, we introduce a
Coarse-to-Fine Interaction Module (CFIM) to realize the hierarchical feature
interaction at different scales. Finally, in similarity fusion level, we
propose a Similarity Fusion Module (SFM) to aggregate geometry and texture
clues from the target. Experiments show that our method achieves
state-of-the-art performance on KITTI (39% Success and 42% Precision gains
against previous multi-modal method) and is also competitive on NuScenes.

提出 MMF-Track，一种多模态多级融合跟踪器，利用点云的几何和图像纹理特征跟踪 3D 目标，在 KITTI 上实现了最先进的性能表现，并在 NuScenes 上具有竞争力。