We propose a new self-supervised CNN pre-training technique based on a novel
auxiliary task called "odd-one-out learning". In this task, the machine is
asked to identify the unrelated or odd element from a set of otherwise related
elements. We apply this technique to self-supervised video representation
learning where we sample subsequences from videos and ask the network to learn
to predict the odd video subsequence. The odd video subsequence is sampled such
that it has wrong temporal order of frames while the even ones have the correct
temporal order. Therefore, to generate a odd-one-out question no manual
annotation is required. Our learning machine is implemented as multi-stream
convolutional neural network, which is learned end-to-end. Using odd-one-out
networks, we learn temporal representations for videos that generalizes to
other related tasks such as action recognition.
On action classification, our method obtains 60.3\% on the UCF101 dataset
using only UCF101 data for training which is approximately 10% better than
current state-of-the-art self-supervised learning methods. Similarly, on HMDB51
dataset we outperform self-supervised state-of-the art methods by 12.7% on
action classification task.

本文提出了一种基于奇偶 - 例外学习的自监督卷积神经网络预训练技术，其可以应用在视频表示学习以及动作识别上，且在 UCF101 和 HMDB51 dataset 上优于当前自监督学习方法约 10% 和 12.7%。

使用 Odd-One-Out 网络进行自监督视频表示学习

Self-Supervised Video Representation Learning With Odd-One-Out Networks

We present an unsupervised visual feature learning algorithm driven by
context-based pixel prediction. By analogy with auto-encoders, we propose
Context Encoders -- a convolutional neural network trained to generate the
contents of an arbitrary image region conditioned on its surroundings. In order
to succeed at this task, context encoders need to both understand the content
of the entire image, as well as produce a plausible hypothesis for the missing
part(s). When training context encoders, we have experimented with both a
standard pixel-wise reconstruction loss, as well as a reconstruction plus an
adversarial loss. The latter produces much sharper results because it can
better handle multiple modes in the output. We found that a context encoder
learns a representation that captures not just appearance but also the
semantics of visual structures. We quantitatively demonstrate the effectiveness
of our learned features for CNN pre-training on classification, detection, and
segmentation tasks. Furthermore, context encoders can be used for semantic
inpainting tasks, either stand-alone or as initialization for non-parametric
methods.

本研究提出了一种无监督的视觉特征学习方法，称为 Context Encoders，利用卷积神经网络通过上下文预测像素来对图像进行重构，该方法通过重构加对抗性损失双重学习获得更加清晰的结果，这种学习到的特征在分类、检测和分割等任务中表现出了很好的效果。