Deep supervised learning has achieved great success in the last decade.
However, its deficiencies of dependence on manual labels and vulnerability to
attacks have driven people to explore a better solution. As an alternative,
self-supervised learning attracts many researchers for its soaring performance
on representation learning in the last several years. Self-supervised
representation learning leverages input data itself as supervision and benefits
almost all types of downstream tasks. In this survey, we take a look into new
self-supervised learning methods for representation in computer vision, natural
language processing, and graph learning. We comprehensively review the existing
empirical methods and summarize them into three main categories according to
their objectives: generative, contrastive, and generative-contrastive
(adversarial). We further investigate related theoretical analysis work to
provide deeper thoughts on how self-supervised learning works. Finally, we
briefly discuss open problems and future directions for self-supervised
learning. An outline slide for the survey is provided.

本文介绍自监督学习作为深度学习替代方案的优势及其在计算机视觉、自然语言处理和图学习中的应用，分类总结了现有的自监督学习方法，并进一步探讨其相关的理论分析工作，最后简要讨论了自监督学习的未来方向和开放性问题.

自监督学习：生成或对比

Self-supervised Learning: Generative or Contrastive

In recent years, deep artificial neural networks (including recurrent ones)
have won numerous contests in pattern recognition and machine learning. This
historical survey compactly summarises relevant work, much of it from the
previous millennium. Shallow and deep learners are distinguished by the depth
of their credit assignment paths, which are chains of possibly learnable,
causal links between actions and effects. I review deep supervised learning
(also recapitulating the history of backpropagation), unsupervised learning,
reinforcement learning & evolutionary computation, and indirect search for
short programs encoding deep and large networks.

这篇文章总结了深度神经网络在模式识别和机器学习中的应用，包括深度监督学习、无监督学习和强化学习，在搜索深层网络的过程中进行了间接搜索。