Many RGB-T trackers attempt to attain robust feature representation by
utilizing an adaptive weighting scheme (or attention mechanism). Different from
these works, we propose a new dynamic modality-aware filter generation module
(named MFGNet) to boost the message communication between visible and thermal
data by adaptively adjusting the convolutional kernels for various input images
in practical tracking. Given the image pairs as input, we first encode their
features with the backbone network. Then, we concatenate these feature maps and
generate dynamic modality-aware filters with two independent networks. The
visible and thermal filters will be used to conduct a dynamic convolutional
operation on their corresponding input feature maps respectively. Inspired by
residual connection, both the generated visible and thermal feature maps will
be summarized with input feature maps. The augmented feature maps will be fed
into the RoI align module to generate instance-level features for subsequent
classification. To address issues caused by heavy occlusion, fast motion and
out-of-view, we propose to conduct a joint local and global search by
exploiting a new direction-aware target driven attention mechanism. The spatial
and temporal recurrent neural network is used to capture the direction-aware
context for accurate global attention prediction. Extensive experiments on
three large-scale RGB-T tracking benchmark datasets validated the effectiveness
of our proposed algorithm. The source code of this paper is available at
\textcolor{magenta}{https://github.com/wangxiao5791509/MFG_RGBT_Tracking_PyTorch}.

提出一种名为 MFGNet 的动态模态感知滤波器生成模块，其能够通过自适应调整卷积核来增强可见和热数据之间的信息交流，以解决 RGB-T 跟踪中的遮挡、快速运动和视野范围限制等问题，并且实验表明该算法的有效性。