We propose a general framework for self-supervised learning of transferable
visual representations based on Video-Induced Visual Invariances (VIVI). We
consider the implicit hierarchy present in the videos and make use of (i)
frame-level invariances (e.g. stability to color and contrast perturbations),
(ii) shot/clip-level invariances (e.g. robustness to changes in object
orientation and lighting conditions), and (iii) video-level invariances
(semantic relationships of scenes across shots/clips), to define a holistic
self-supervised loss. Training models using different variants of the proposed
framework on videos from the YouTube-8M (YT8M) data set, we obtain
state-of-the-art self-supervised transfer learning results on the 19 diverse
downstream tasks of the Visual Task Adaptation Benchmark (VTAB), using only
1000 labels per task. We then show how to co-train our models jointly with
labeled images, outperforming an ImageNet-pretrained ResNet-50 by 0.8 points
with 10x fewer labeled images, as well as the previous best supervised model by
3.7 points using the full ImageNet data set.

通过使用 Video-Induced Visual Invariances (VIVI) 的自监督学习框架，本文提出了一种可以在 19 种视觉任务中实现超过最佳监督模型的自监督转移学习方法。