In this paper, we efficiently transfer the surpassing representation power of
the vision foundation models, such as ViT and Swin, for video understanding
with only a few trainable parameters. Previous adaptation methods have
simultaneously considered spatial and temporal modeling with a unified
learnable module but still suffered from fully leveraging the representative
capabilities of image transformers. We argue that the popular dual-path
(two-stream) architecture in video models can mitigate this problem. We propose
a novel DualPath adaptation separated into spatial and temporal adaptation
paths, where a lightweight bottleneck adapter is employed in each transformer
block. Especially for temporal dynamic modeling, we incorporate consecutive
frames into a grid-like frameset to precisely imitate vision transformers'
capability that extrapolates relationships between tokens. In addition, we
extensively investigate the multiple baselines from a unified perspective in
video understanding and compare them with DualPath. Experimental results on
four action recognition benchmarks prove that pretrained image transformers
with DualPath can be effectively generalized beyond the data domain.

本文提出了一种新的 DualPath 适应性算法，将图像 transformers 的表示能力有效地转移至视频理解中，包括空间和时间建模，极大地拓展了预训练表示的泛化，同时在四个动作识别基准测试上进行了广泛实验比较。

从图像到视频的双路径变换器自适应

Dual-path Adaptation from Image to Video Transformers

Graph Convolutional Networks (GCNs), which model skeleton data as graphs,
have obtained remarkable performance for skeleton-based action recognition.
Particularly, the temporal dynamic of skeleton sequence conveys significant
information in the recognition task. For temporal dynamic modeling, GCN-based
methods only stack multi-layer 1D local convolutions to extract temporal
relations between adjacent time steps. With the repeat of a lot of local
convolutions, the key temporal information with non-adjacent temporal distance
may be ignored due to the information dilution. Therefore, these methods still
remain unclear how to fully explore temporal dynamic of skeleton sequence. In
this paper, we propose a Temporal Enhanced Graph Convolutional Network (TE-GCN)
to tackle this limitation. The proposed TE-GCN constructs temporal relation
graph to capture complex temporal dynamic. Specifically, the constructed
temporal relation graph explicitly builds connections between semantically
related temporal features to model temporal relations between both adjacent and
non-adjacent time steps. Meanwhile, to further explore the sufficient temporal
dynamic, multi-head mechanism is designed to investigate multi-kinds of
temporal relations. Extensive experiments are performed on two widely used
large-scale datasets, NTU-60 RGB+D and NTU-120 RGB+D. And experimental results
show that the proposed model achieves the state-of-the-art performance by
making contribution to temporal modeling for action recognition.

本篇文章介绍了一种称之为 Temporal Enhanced Graph Convolutional Network (TE-GCN) 的新模型，用于构建骨骼动作识别应用中的时间关系图，以提取骨架序列的时间动态。文章对两个广泛使用的大型数据集进行了广泛的实验，在动作识别中取得了最先进的性能，同时通过多头机制进一步探索时间关系图。