We propose a unified object-aware temporal learning framework for multi-view 3D detection and tracking tasks. Having observed that the efficacy of the temporal fusion strategy in recent multi-view perception methods may be weakened by distractors and background clutters in historical frames, we propose a cyclic learning mechanism to improve the robustness of multi-view representation learning. The essence is constructing a backward bridge to propagate information from model predictions (e.g., object locations and sizes) to image and BEV features, which forms a circle with regular inference. After backward refinement, the responses of target-irrelevant regions in historical frames would be suppressed, decreasing the risk of polluting future frames and improving the object awareness ability of temporal fusion. We further tailor an object-aware association strategy for tracking based on the cyclic learning model. The cyclic learning model not only provides refined features, but also delivers finer clues (e.g., scale level) for tracklet association. The proposed cycle learning method and association module together contribute a novel and unified multi-task framework. Experiments on nuScenes show that the proposed model achieves consistent performance gains over baselines of different designs (i.e., dense query-based BEVFormer, sparse query-based SparseBEV and LSS-based BEVDet4D) on both detection and tracking evaluation.

我们提出了一个统一的面向对象的时间学习框架，用于多视角三维检测和跟踪任务。通过构建一个循环学习机制以提高多视角表示学习的鲁棒性，使模型预测的信息能够向后传播，从而减小历史帧中与目标无关的区域的响应，降低污染未来帧的风险并提高时间融合的对象感知能力。我们进一步根据循环学习模型，针对跟踪问题定制了一个面向对象的关联策略。所提出的循环学习方法和关联模块共同构成了一个新颖而统一的多任务框架。在nuScenes上的实验证明，所提出的模型在检测和跟踪评估上相对于不同设计（如基于密集查询的BEVFormer，基于稀疏查询的SparseBEV和基于LSS的BEVDet4D）的基线模型都取得了一致的性能提升。

循环就业者：面向多视角3D检测和跟踪的对象感知时序表示学习