Semi-supervised learning (SSL) has seen great strides when labeled data is
scarce but unlabeled data is abundant. Critically, most recent work assume that
such unlabeled data is drawn from the same distribution as the labeled data. In
this work, we show that state-of-the-art SSL algorithms suffer a degradation in
performance in the presence of unlabeled auxiliary data that does not
necessarily possess the same class distribution as the labeled set. We term
this problem as Auxiliary-SSL and propose AuxMix, an algorithm that leverages
self-supervised learning tasks to learn generic features in order to mask
auxiliary data that are not semantically similar to the labeled set. We also
propose to regularize learning by maximizing the predicted entropy for
dissimilar auxiliary samples. We show an improvement of 5% over existing
baselines on a ResNet-50 model when trained on CIFAR10 dataset with 4k labeled
samples and all unlabeled data is drawn from the Tiny-ImageNet dataset. We
report competitive results on several datasets and conduct ablation studies.

本研究提出了一种叫做 AuxMix 的算法，该算法利用自监督学习任务来学习通用特征，以掩盖那些不同于标记集的语义上不相似的辅助数据，并正则化学习通过预测熵最大化来处理不同数据的不确定性，结果显示该算法相比现有的基线在 ResNet-50 模型上训练 CIFAR10 数据集时提高了 5％。