With the advent of large-scale multimodal video datasets, especially
sequences with audio or transcribed speech, there has been a growing interest
in self-supervised learning of video representations. Most prior work
formulates the objective as a contrastive metric learning problem between the
modalities. To enable effective learning, however, these strategies require a
careful selection of positive and negative samples often combined with
hand-designed curriculum policies. In this work we remove the need for negative
sampling by taking a generative modeling approach that poses the objective as a
translation problem between modalities. Such a formulation allows us to tackle
a wide variety of downstream video understanding tasks by means of a single
unified framework, without the need for large batches of negative samples
common in contrastive metric learning. We experiment with the large-scale
HowTo100M dataset for training, and report performance gains over the
state-of-the-art on several downstream tasks including video classification
(EPIC-Kitchens), question answering (TVQA), captioning (TVC, YouCook2, and
MSR-VTT), and text-based clip retrieval (YouCook2 and MSR-VTT).

本文介绍了在大规模多模式视频数据集上的自我监督学习的发展；提出了一种基于生成模型的方法，以翻译问题的形式解决了这一问题，并将其应用于多种下游视频理解任务中。结果表明，本方法在性能上优于基于对比度度量学习的方法。