Inspired by the success of transformer-based pre-training methods on natural
language tasks and further computer vision tasks, researchers have begun to
apply transformer to video processing. This survey aims to give a comprehensive
overview on transformer-based pre-training methods for Video-Language learning.
We first briefly introduce the transformer tructure as the background
knowledge, including attention mechanism, position encoding etc. We then
describe the typical paradigm of pre-training & fine-tuning on Video-Language
processing in terms of proxy tasks, downstream tasks and commonly used video
datasets. Next, we categorize transformer models into Single-Stream and
Multi-Stream structures, highlight their innovations and compare their
performances. Finally, we analyze and discuss the current challenges and
possible future research directions for Video-Language pre-training.

本文综述了基于 Transformer 的预训练方法在视频语言学习方面的应用，包括代理任务、下游任务和常用视频数据集，将 Transformer 模型分为单流和多流结构，并比较它们的性能。最后，我们分析和讨论了当前挑战和了可能的未来研究方向。