Recently, learning from vast unlabeled data, especially self-supervised
learning, has been emerging and attracted widespread attention. Self-supervised
learning followed by the supervised fine-tuning on a few labeled examples can
significantly improve label efficiency and outperform standard supervised
training using fully annotated data. In this work, we present a novel
self-supervised deep learning paradigm based on online hard negative pair
mining. Specifically, we design a student-teacher network to generate
multi-view of the data for self-supervised learning and integrate hard negative
pair mining into the training. Then we derive a new triplet-like loss
considering both positive sample pairs and mined hard negative sample pairs.
Extensive experiments demonstrate the effectiveness of the proposed method and
its components on ILSVRC-2012.

本文介绍了一种基于在线负样本对挖掘的自我监督深度学习范式，包括学生教师网络、正负样本对挖掘和三元组式的损失函数等，实验表明该方法可以有效地提高标签效率并在 ILSVRC-2012 上表现优异。

对抗对比自监督学习

Adversarial Contrastive Self-Supervised Learning

Deep metric learning aims to learn embeddings that contain semantic
similarity information among data points. To learn better embeddings, methods
to generate synthetic hard samples have been proposed. Existing methods of
synthetic hard sample generation are adopting autoencoders or generative
adversarial networks, but this leads to more hyper-parameters, harder
optimization, and slower training speed. In this paper, we address these
problems by proposing a novel method of synthetic hard sample generation called
symmetrical synthesis. Given two original feature points from the same class,
the proposed method firstly generates synthetic points with each other as an
axis of symmetry. Secondly, it performs hard negative pair mining within the
original and synthetic points to select a more informative negative pair for
computing the metric learning loss. Our proposed method is hyper-parameter free
and plug-and-play for existing metric learning losses without network
modification. We demonstrate the superiority of our proposed method over
existing methods for a variety of loss functions on clustering and image
retrieval tasks. Our implementations is publicly available.

本文提出了一种新的通过对称性生成合成难样本的方法 Symmetrical Synthesis，解决了既有的自编码器和生成对抗网络生成合成难样本的高超参数、难度大和训练速度慢的问题，无需修改网络并适用于现有的度量学习损失函数，在聚类和图像检索任务中比现有方法表现更优越。