Recently, pretext-task based methods are proposed one after another in
self-supervised video feature learning. Meanwhile, contrastive learning methods
also yield good performance. Usually, new methods can beat previous ones as
claimed that they could capture "better" temporal information. However, there
exist setting differences among them and it is hard to conclude which is
better. It would be much more convincing in comparison if these methods have
reached as closer to their performance limits as possible. In this paper, we
start from one pretext-task baseline, exploring how far it can go by combining
it with contrastive learning, data pre-processing, and data augmentation. A
proper setting has been found from extensive experiments, with which huge
improvements over the baselines can be achieved, indicating a joint
optimization framework can boost both pretext task and contrastive learning. We
denote the joint optimization framework as Pretext-Contrastive Learning (PCL).
The other two pretext task baselines are used to validate the effectiveness of
PCL. And we can easily outperform current state-of-the-art methods in the same
training manner, showing the effectiveness and the generality of our proposal.
It is convenient to treat PCL as a standard training strategy and apply it to
many other works in self-supervised video feature learning.

本文提出了一种联合优化框架 Pretext-Contrastive Learning (PCL) 可以显著提高预训练任务和对比学习的表现，在自监督视频特征学习任务中可以超过当前的 SOTA 方法。