Efficiently modeling spatio-temporal relations of objects is a key challenge
in visual object tracking (VOT). Existing methods track by appearance-based
similarity or long-term relation modeling, resulting in rich temporal contexts
between consecutive frames being easily overlooked. Moreover, training trackers
from scratch or fine-tuning large pre-trained models needs more time and memory
consumption. In this paper, we present ACTrack, a new tracking framework with
additive spatio-temporal conditions. It preserves the quality and capabilities
of the pre-trained Transformer backbone by freezing its parameters, and makes a
trainable lightweight additive net to model spatio-temporal relations in
tracking. We design an additive siamese convolutional network to ensure the
integrity of spatial features and perform temporal sequence modeling to
simplify the tracking pipeline. Experimental results on several benchmarks
prove that ACTrack could balance training efficiency and tracking performance.

我们提出了 ACTrack，一个新的跟踪框架，通过冻结参数来保留预训练的 Transformer 骨干的质量和能力，并使用可训练的轻量级加法网络来建模跟踪中的时空关系，实验结果证明 ACTrack 能够在训练效率和跟踪性能之间取得平衡。

ACTrack: 为视觉目标跟踪添加时空条件

ACTrack: Adding Spatio-Temporal Condition for Visual Object Tracking

Following the tracking-by-attention paradigm, this paper introduces an
object-centric, transformer-based framework for tracking in 3D. Traditional
model-based tracking approaches incorporate the geometric effect of object- and
ego motion between frames with a geometric motion model. Inspired by this, we
propose S.T.A.R.-Track, which uses a novel latent motion model (LMM) to
additionally adjust object queries to account for changes in viewing direction
and lighting conditions directly in the latent space, while still modeling the
geometric motion explicitly. Combined with a novel learnable track embedding
that aids in modeling the existence probability of tracks, this results in a
generic tracking framework that can be integrated with any query-based
detector. Extensive experiments on the nuScenes benchmark demonstrate the
benefits of our approach, showing state-of-the-art performance for DETR3D-based
trackers while drastically reducing the number of identity switches of tracks
at the same time.

本文介绍了一种基于注意力跟踪范式的物体中心、基于变压器的三维跟踪框架。我们提出了 S.T.A.R.-Track，该框架使用新颖的隐式运动模型（LMM）将目标查询进行调整，以直接考虑视角和照明条件的变化，同时显式建立几何运动模型。此外，我们还提出了一种可学习的跟踪嵌入，有助于建立跟踪存在概率的模型。在 nuScenes 基准测试中进行的广泛实验显示出我们方法的优势，不仅在 DETR3D 跟踪器方面表现出最先进的性能，同时还显着减少了轨迹的身份切换次数。

S.T.A.R.-Track：适应空时外观表示的端到端 3D 物体跟踪潜在运动模型

S.T.A.R.-Track: Latent Motion Models for End-to-End 3D Object Tracking  with Adaptive Spatio-Temporal Appearance Representations

Transformer-based trackers have achieved strong accuracy on the standard
benchmarks. However, their efficiency remains an obstacle to practical
deployment on both GPU and CPU platforms. In this paper, to overcome this
issue, we propose a fully transformer tracking framework, coined as
\emph{MixFormerV2}, without any dense convolutional operation and complex score
prediction module. Our key design is to introduce four special prediction
tokens and concatenate them with the tokens from target template and search
areas. Then, we apply the unified transformer backbone on these mixed token
sequence. These prediction tokens are able to capture the complex correlation
between target template and search area via mixed attentions. Based on them, we
can easily predict the tracking box and estimate its confidence score through
simple MLP heads. To further improve the efficiency of MixFormerV2, we present
a new distillation-based model reduction paradigm, including dense-to-sparse
distillation and deep-to-shallow distillation. The former one aims to transfer
knowledge from the dense-head based MixViT to our fully transformer tracker,
while the latter one is used to prune some layers of the backbone. We
instantiate two types of MixForemrV2, where the MixFormerV2-B achieves an AUC
of 70.6\% on LaSOT and an AUC of 57.4\% on TNL2k with a high GPU speed of 165
FPS, and the MixFormerV2-S surpasses FEAR-L by 2.7\% AUC on LaSOT with a
real-time CPU speed.

本文提出了一种基于全 Transformer 结构的跟踪框架 MixFormerV2，通过引入四个特殊的预测 tokens，结合目标模板和搜索区域的 tokens，采用 Transformer 骨干网络进行跟踪预测，进而通过简单的 MLP heads 预测跟踪框和评估置信度得分，并提出了一种新的蒸馏模型的减少算法，包括密集到稀疏和深层到浅层的蒸馏，以进一步提高跟踪器性能。