Unsupervised domain adaptation (UDA) in videos is a challenging task that
remains not well explored compared to image-based UDA techniques. Although
vision transformers (ViT) achieve state-of-the-art performance in many computer
vision tasks, their use in video domain adaptation has still been little
explored. Our key idea is to use the transformer layers as a feature encoder
and incorporate spatial and temporal transferability relationships into the
attention mechanism. A Transferable-guided Attention (TransferAttn) framework
is then developed to exploit the capacity of the transformer to adapt
cross-domain knowledge from different backbones. To improve the transferability
of ViT, we introduce a novel and effective module named Domain
Transferable-guided Attention Block~(DTAB). DTAB compels ViT to focus on the
spatio-temporal transferability relationship among video frames by changing the
self-attention mechanism to a transferability attention mechanism. Extensive
experiments on UCF-HMDB, Kinetics-Gameplay, and Kinetics-NEC Drone datasets
with different backbones, like ResNet101, I3D, and STAM, verify the
effectiveness of TransferAttn compared with state-of-the-art approaches. Also,
we demonstrate that DTAB yields performance gains when applied to other
state-of-the-art transformer-based UDA methods from both video and image
domains. The code will be made freely available.

使用转换器图层作为特征编码器，将空间和时间的可传递性关系融入到注意机制中，提出了 TransferAttn 框架以适应不同骨干的跨领域知识，通过改变自注意机制为可传递性注意机制，引入 DTAB 模块来增加 ViT 的可传递性，验证了 TransferAttn 和 DTAB 在多个数据集和不同骨干下的有效性。