Recent Transformer-based visual tracking models have showcased superior
performance. Nevertheless, prior works have been resource-intensive, requiring
prolonged GPU training hours and incurring high GFLOPs during inference due to
inefficient training methods and convolution-based target heads. This intensive
resource use renders them unsuitable for real-world applications. In this
paper, we present DETRack, a streamlined end-to-end visual object tracking
framework. Our framework utilizes an efficient encoder-decoder structure where
the deformable transformer decoder acting as a target head, achieves higher
sparsity than traditional convolution heads, resulting in decreased GFLOPs. For
training, we introduce a novel one-to-many label assignment and an auxiliary
denoising technique, significantly accelerating model's convergence.
Comprehensive experiments affirm the effectiveness and efficiency of our
proposed method. For instance, DETRack achieves 72.9% AO on challenging GOT-10k
benchmarks using only 20% of the training epochs required by the baseline, and
runs with lower GFLOPs than all the transformer-based trackers.

DETRack 是一种高效的端到端视觉目标跟踪框架，利用编码器 - 解码器结构和可变形 Transformer 解码器作为目标头，显著减少了 GFLOPs 并通过新颖的一对多标签分配和辅助去噪技术加快了模型的收敛速度。实验证实了该方法的有效性和高效性。

形变变换器用于视觉跟踪的高效训练

Efficient Training for Visual Tracking with Deformable Transformer

Video question answering is a challenging task that requires understanding
jointly the language input, the visual information in individual video frames,
as well as the temporal information about the events occurring in the video. In
this paper, we propose a novel multi-stream video encoder for video question
answering that uses multiple video inputs and a new video-text iterative
co-tokenization approach to answer a variety of questions related to videos. We
experimentally evaluate the model on several datasets, such as MSRVTT-QA,
MSVD-QA, IVQA, outperforming the previous state-of-the-art by large margins.
Simultaneously, our model reduces the required GFLOPs from 150-360 to only 67,
producing a highly efficient video question answering model.

本文提出了一种新型的多流视频编码器，使用多个视频输入和新的视频文本迭代共同标记方法来回答与视频相关的各种问题，同时将所需的 GFLOPs 从 150-360 减少到只有 67，实现了高效的视频问答模型，并在 MSRVTT-QA，MSVD-QA，IVQA 等多个数据集上实验评估，优于以往的最先进技术。