Semi-Supervised Learning (SSL) under class distribution mismatch aims to
tackle a challenging problem wherein unlabeled data contain lots of unknown
categories unseen in the labeled ones. In such mismatch scenarios, traditional
SSL suffers severe performance damage due to the harmful invasion of the
instances with unknown categories into the target classifier. In this study, by
strict mathematical reasoning, we reveal that the SSL error under class
distribution mismatch is composed of pseudo-labeling error and invasion error,
both of which jointly bound the SSL population risk. To alleviate the SSL
error, we propose a robust SSL framework called Weight-Aware Distillation (WAD)
that, by weights, selectively transfers knowledge beneficial to the target task
from unsupervised contrastive representation to the target classifier.
Specifically, WAD captures adaptive weights and high-quality pseudo labels to
target instances by exploring point mutual information (PMI) in representation
space to maximize the role of unlabeled data and filter unknown categories.
Theoretically, we prove that WAD has a tight upper bound of population risk
under class distribution mismatch. Experimentally, extensive results
demonstrate that WAD outperforms five state-of-the-art SSL approaches and one
standard baseline on two benchmark datasets, CIFAR10 and CIFAR100, and an
artificial cross-dataset. The code is available at
this https URL

通过严格的数学推理，本研究揭示了半监督学习中类分布不匹配所导致的错误由伪标签错误和入侵错误共同限制了半监督学习的群体风险，并提出了一种稳健的半监督学习框架 Weight-Aware Distillation（WAD）以通过权重从无监督对比表示中选择性地将对目标任务有益的知识转移到目标分类器，通过在表示空间中探索点互信息（PMI）捕捉自适应权重和高质量伪标签以最大限度地发挥无标签数据的作用并过滤未知类别，并理论上证明 WAD 在类分布不匹配下具有群体风险的紧密上界；实验证明 WAD 在两个基准数据集（CIFAR10 和 CIFAR100）和一个人工交叉数据集上优于五种最先进的半监督学习方法和一个标准基准线