Multimodal vision-language (VL) learning has noticeably pushed the tendency
toward generic intelligence owing to emerging large foundation models. However,
tracking, as a fundamental vision problem, surprisingly enjoys less bonus from
recent flourishing VL learning. We argue that the reasons are two-fold: the
lack of large-scale vision-language annotated videos and ineffective
vision-language interaction learning of current works. These nuisances motivate
us to design more effective vision-language representation for tracking,
meanwhile constructing a large database with language annotation for model
learning. Particularly, in this paper, we first propose a general attribute
annotation strategy to decorate videos in six popular tracking benchmarks,
which contributes a large-scale vision-language tracking database with more
than 23,000 videos. We then introduce a novel framework to improve tracking by
learning a unified-adaptive VL representation, where the cores are the proposed
asymmetric architecture search and modality mixer (ModaMixer). To further
improve VL representation, we introduce a contrastive loss to align different
modalities. To thoroughly evidence the effectiveness of our method, we
integrate the proposed framework on three tracking methods with different
designs, i.e., the CNN-based SiamCAR, the Transformer-based OSTrack, and the
hybrid structure TransT. The experiments demonstrate that our framework can
significantly improve all baselines on six benchmarks. Besides empirical
results, we theoretically analyze our approach to show its rationality. By
revealing the potential of VL representation, we expect the community to divert
more attention to VL tracking and hope to open more possibilities for future
tracking with diversified multimodal messages.

我们设计了一种有效的视觉 - 语言表示方法，同时为追踪问题构建了一个大型带有语言注释的数据库。通过引入异构架构搜索和模态混合器等核心技术，以及对不同模态之间进行对比损失的引入，我们能够显著改善追踪问题的解决方案，并期望将更多注意力转向视觉 - 语言追踪，为未来的多样化多模态消息追踪开辟更多可能性。

将更多关注转移到视觉语言对象追踪上

Divert More Attention to Vision-Language Object Tracking

It is still challenging to build an AI system that can perform tasks that
involve vision and language at human level. So far, researchers have singled
out individual tasks separately, for each of which they have designed networks
and trained them on its dedicated datasets. Although this approach has seen a
certain degree of success, it comes with difficulties of understanding
relations among different tasks and transferring the knowledge learned for a
task to others. We propose a multi-task learning approach that enables to learn
vision-language representation that is shared by many tasks from their diverse
datasets. The representation is hierarchical, and prediction for each task is
computed from the representation at its corresponding level of the hierarchy.
We show through experiments that our method consistently outperforms previous
single-task-learning methods on image caption retrieval, visual question
answering, and visual grounding. We also analyze the learned hierarchical
representation by visualizing attention maps generated in our network.

该研究提出了一种多任务学习的方法，将来自各种不同数据集的任务共享视觉语言表示。结果表明该方法在图像字幕检索、视觉问答和视觉定位方面比先前的单任务学习方法表现更好，同时通过可视化注意力图分析了学习到的分层表示。