Robustness and discrimination power are two fundamental requirements in
visual object tracking. In most tracking paradigms, we find that the features
extracted by the popular Siamese-like networks cannot fully discriminatively
model the tracked targets and distractor objects, hindering them from
simultaneously meeting these two requirements. While most methods focus on
designing robust correlation operations, we propose a novel target-dependent
feature network inspired by the self-/cross-attention scheme. In contrast to
the Siamese-like feature extraction, our network deeply embeds cross-image
feature correlation in multiple layers of the feature network. By extensively
matching the features of the two images through multiple layers, it is able to
suppress non-target features, resulting in instance-varying feature extraction.
The output features of the search image can be directly used for predicting
target locations without extra correlation step. Moreover, our model can be
flexibly pre-trained on abundant unpaired images, leading to notably faster
convergence than the existing methods. Extensive experiments show our method
achieves the state-of-the-art results while running at real-time. Our feature
networks also can be applied to existing tracking pipelines seamlessly to raise
the tracking performance. Code will be available.

我们提出了一种基于自注意力 / 交叉注意力的目标依赖特征网络，通过深度嵌入多层特征的交叉图像特征相关性，抑制了非目标特征，从而实现了实例变化的特征提取，在不需要额外的相关步骤的情况下可以直接用于预测目标位置，且可以灵活地在丰富的无配对图像上进行预训练，实现了较快的收敛速度和最先进的实时跟踪性能。

Correlation-Aware Deep Tracking

We introduce a novel backbone architecture to improve target-perception
ability of feature representation for tracking. Specifically, having observed
that de facto frameworks perform feature matching simply using the outputs from
backbone for target localization, there is no direct feedback from the matching
module to the backbone network, especially the shallow layers. More concretely,
only the matching module can directly access the target information (in the
reference frame), while the representation learning of candidate frame is blind
to the reference target. As a consequence, the accumulation effect of
target-irrelevant interference in the shallow stages may degrade the feature
quality of deeper layers. In this paper, we approach the problem from a
different angle by conducting multiple branch-wise interactions inside the
Siamese-like backbone networks (InBN). At the core of InBN is a general
interaction modeler (GIM) that injects the prior knowledge of reference image
to different stages of the backbone network, leading to better
target-perception and robust distractor-resistance of candidate feature
representation with negligible computation cost. The proposed GIM module and
InBN mechanism are general and applicable to different backbone types including
CNN and Transformer for improvements, as evidenced by our extensive experiments
on multiple benchmarks. In particular, the CNN version (based on SiamCAR)
improves the baseline with 3.2/6.9 absolute gains of SUC on LaSOT/TNL2K,
respectively. The Transformer version obtains SUC scores of 65.7/52.0 on
LaSOT/TNL2K, which are on par with recent state of the arts. Code and models
will be released.

本文介绍了一种新的骨干架构，使用交互模型器对 Siamese-like 骨干网络中不同阶段注入参考图像的先验知识，从而提高了特征表征的目标感知能力和鲁棒性，适用于不同的骨干类型的改进。