Many RGBT tracking researches primarily focus on modal fusion design, while
overlooking the effective handling of target appearance changes. While some
approaches have introduced historical frames or fuse and replace initial
templates to incorporate temporal information, they have the risk of disrupting
the original target appearance and accumulating errors over time. To alleviate
these limitations, we propose a novel Transformer RGBT tracking approach, which
mixes spatio-temporal multimodal tokens from the static multimodal templates
and multimodal search regions in Transformer to handle target appearance
changes, for robust RGBT tracking. We introduce independent dynamic template
tokens to interact with the search region, embedding temporal information to
address appearance changes, while also retaining the involvement of the initial
static template tokens in the joint feature extraction process to ensure the
preservation of the original reliable target appearance information that
prevent deviations from the target appearance caused by traditional temporal
updates. We also use attention mechanisms to enhance the target features of
multimodal template tokens by incorporating supplementary modal cues, and make
the multimodal search region tokens interact with multimodal dynamic template
tokens via attention mechanisms, which facilitates the conveyance of
multimodal-enhanced target change information. Our module is inserted into the
transformer backbone network and inherits joint feature extraction,
search-template matching, and cross-modal interaction. Extensive experiments on
three RGBT benchmark datasets show that the proposed approach maintains
competitive performance compared to other state-of-the-art tracking algorithms
while running at 39.1 FPS.

提出了一种新颖的 Transformer RGBT 跟踪方法，通过融合静态多模态模板和多模态搜索区域中的时空多模态令牌来处理目标外观变化，以实现鲁棒的 RGBT 跟踪，并通过注意机制增强多模态模板令牌的目标特征，从而促进多模态增强的目标变化信息的传递。该方法在三个 RGBT 基准数据集上的实验证明了与其他最先进的跟踪算法相比具有竞争力的性能，帧率可达 39.1。