The goal of this paper is to self-train a 3D convolutional neural network on
an unlabeled video collection for deployment on small-scale video collections.
As smaller video datasets benefit more from motion than appearance, we strive
to train our network using optical flow, but avoid its computation during
inference. We propose the first motion-augmented self-training regime, we call
MotionFit. We start with supervised training of a motion model on a small, and
labeled, video collection. With the motion model we generate pseudo-labels for
a large unlabeled video collection, which enables us to transfer knowledge by
learning to predict these pseudo-labels with an appearance model. Moreover, we
introduce a multi-clip loss as a simple yet efficient way to improve the
quality of the pseudo-labeling, even without additional auxiliary tasks. We
also take into consideration the temporal granularity of videos during
self-training of the appearance model, which was missed in previous works. As a
result we obtain a strong motion-augmented representation model suited for
video downstream tasks like action recognition and clip retrieval. On
small-scale video datasets, MotionFit outperforms alternatives for knowledge
transfer by 5%-8%, video-only self-supervision by 1%-7% and semi-supervised
learning by 9%-18% using the same amount of class labels.

本文介绍了一种称为 MotionFit 的自训练方法，采用 3D 卷积神经网络、运动模型和假标签来提高视频下游任务的效果。该方法在小规模视频数据集中表现优异，远超其他知识迁移算法、半监督学习和仅使用视频的自我监督学习。