In this paper we revisit the idea of pseudo-labeling in the context of
semi-supervised learning where a learning algorithm has access to a small set
of labeled samples and a large set of unlabeled samples. Pseudo-labeling works
by applying pseudo-labels to samples in the unlabeled set by using a model
trained on the combination of the labeled samples and any previously
pseudo-labeled samples, and iteratively repeating this process in a
self-training cycle. Current methods seem to have abandoned this approach in
favor of consistency regularization methods that train models under a
combination of different styles of self-supervised losses on the unlabeled
samples and standard supervised losses on the labeled samples. We empirically
demonstrate that pseudo-labeling can in fact be competitive with the
state-of-the-art, while being more resilient to out-of-distribution samples in
the unlabeled set. We identify two key factors that allow pseudo-labeling to
achieve such remarkable results (1) applying curriculum learning principles and
(2) avoiding concept drift by restarting model parameters before each
self-training cycle. We obtain 94.91% accuracy on CIFAR-10 using only 4,000
labeled samples, and 68.87% top-1 accuracy on Imagenet-ILSVRC using only 10% of
the labeled samples. The code is available at
this https URL

本文重新审视了伪标记的概念，提出了一种基于半监督学习的方法，通过将伪标记应用于无标签集中的样本，并利用已训练好的模型标记这些样本，然后迭代重复此过程来训练模型。本文通过实验证明，伪标记方法可以取得与现有最先进方法相媲美甚至更好的结果，并且更能抵御未知分布样本。作者指出采用学习课程原理以及在每个自我训练周期前重启模型参数是实现这一点的两个关键因素。在 CIFAR-10 数据集上，本文仅使用了 4,000 个标记样本，达到了 94.91% 的准确率，在 Imagenet-ILSVRC 数据集上，本文仅使用了 10％的标记样本，达到了 68.87％的 top-1 的准确率。