Multimodal vision-language (VL) learning has noticeably pushed the tendency
toward generic intelligence owing to emerging large foundation models. However,
tracking, as a fundamental vision problem, surprisingly enjoys less bonus from
recent flourishing VL learning. We argue that the reasons are two-fold: the
lack of large-scale vision-language annotated videos and ineffective
vision-language interaction learning of current works. These nuisances motivate
us to design more effective vision-language representation for tracking,
meanwhile constructing a large database with language annotation for model
learning. Particularly, in this paper, we first propose a general attribute
annotation strategy to decorate videos in six popular tracking benchmarks,
which contributes a large-scale vision-language tracking database with more
than 23,000 videos. We then introduce a novel framework to improve tracking by
learning a unified-adaptive VL representation, where the cores are the proposed
asymmetric architecture search and modality mixer (ModaMixer). To further
improve VL representation, we introduce a contrastive loss to align different
modalities. To thoroughly evidence the effectiveness of our method, we
integrate the proposed framework on three tracking methods with different
designs, i.e., the CNN-based SiamCAR, the Transformer-based OSTrack, and the
hybrid structure TransT. The experiments demonstrate that our framework can
significantly improve all baselines on six benchmarks. Besides empirical
results, we theoretically analyze our approach to show its rationality. By
revealing the potential of VL representation, we expect the community to divert
more attention to VL tracking and hope to open more possibilities for future
tracking with diversified multimodal messages.

我们设计了一种有效的视觉 - 语言表示方法，同时为追踪问题构建了一个大型带有语言注释的数据库。通过引入异构架构搜索和模态混合器等核心技术，以及对不同模态之间进行对比损失的引入，我们能够显著改善追踪问题的解决方案，并期望将更多注意力转向视觉 - 语言追踪，为未来的多样化多模态消息追踪开辟更多可能性。