Pre-training video transformers generally requires a large amount of data, presenting significant challenges in terms of data collection costs and concerns related to privacy, licensing, and inherent biases. Synthesizing data is one of the promising ways to solve these issues, yet pre-training solely on synthetic data has its own challenges. In this paper, we introduce an effective self-supervised learning framework for videos that leverages readily available and less costly static images. Specifically, we define the Pseudo Motion Generator (PMG) module that recursively applies image transformations to generate pseudo-motion videos from images. These pseudo-motion videos are then leveraged in masked video modeling. Our approach is applicable to synthetic images as well, thus entirely freeing video pre-training from data collection costs and other concerns in real data. Through experiments in action recognition tasks, we demonstrate that this framework allows effective learning of spatio-temporal features through pseudo-motion videos, significantly improving over existing methods which also use static images and partially outperforming those using both real and synthetic videos. These results uncover fragments of what video transformers learn through masked video modeling.

本研究解决了视频变换器预训练中面临的数据收集成本、隐私及偏见问题。通过引入伪运动生成器（PMG）模块，该框架利用静态图像生成伪运动视频，进而进行遮蔽视频建模。实验结果显示，该方法有效增强了时空特征的学习，超越了现有静态图像的方法，并在一定程度上超越了使用真实和合成视频的方法。

无数据收集的遮蔽视频建模