Active learning is the set of techniques for intelligently labeling large
unlabeled datasets to reduce the labeling effort. In parallel, recent
developments in self-supervised and semi-supervised learning (S4L) provide
powerful techniques, based on data-augmentation, contrastive learning, and
self-training, that enable superior utilization of unlabeled data which led to
a significant reduction in required labeling in the standard machine learning
benchmarks. A natural question is whether these paradigms can be unified to
obtain superior results. To this aim, this paper provides a novel algorithmic
framework integrating self-supervised pretraining, active learning, and
consistency-regularized self-training. We conduct extensive experiments with
our framework on CIFAR10 and CIFAR100 datasets. These experiments enable us to
isolate and assess the benefits of individual components which are evaluated
using state-of-the-art methods (e.g.~Core-Set, VAAL, simCLR, FixMatch). Our
experiments reveal two key insights: (i) Self-supervised pre-training
significantly improves semi-supervised learning, especially in the few-label
regime, (ii) The benefit of active learning is undermined and subsumed by S4L
techniques. Specifically, we fail to observe any additional benefit of
state-of-the-art active learning algorithms when combined with state-of-the-art
S4L techniques.

本研究提供了一个将自我监督预训练、主动学习和一致性正则化自我训练整合的新算法框架，并在 CIFAR10 和 CIFAR100 数据集上进行了实验，揭示了自我监督预训练在半监督学习中的重要性，被 S4L 技术所替代的主动学习的价值。