Self-training algorithms, which train a model to fit pseudolabels predicted
by another previously-learned model, have been very successful for learning
with unlabeled data using neural networks. However, the current theoretical
understanding of self-training only applies to linear models. This work
provides a unified theoretical analysis of self-training with deep networks for
semi-supervised learning, unsupervised domain adaptation, and unsupervised
learning. At the core of our analysis is a simple but realistic "expansion"
assumption, which states that a low probability subset of the data must expand
to a neighborhood with large probability relative to the subset. We also assume
that neighborhoods of examples in different classes have minimal overlap. We
prove that under these assumptions, the minimizers of population objectives
based on self-training and input-consistency regularization will achieve high
accuracy with respect to ground-truth labels. By using off-the-shelf
generalization bounds, we immediately convert this result to sample complexity
guarantees for neural nets that are polynomial in the margin and Lipschitzness.
Our results help explain the empirical successes of recently proposed
self-training algorithms which use input consistency regularization.

本研究提出了一种统一的理论分析方法，以解释使用深度神经网络进行自训练的半监督学习、无监督领域适应和无监督学习。结果表明，在使用输入一致性正则化的自训练和输入准确性正则化的人口目标的最小化器下，假定数据的一个低概率子集相对于该子集的大概率邻域具有扩张性，而不同类别的样本邻域具有最小的重叠，可以达到与基准真实标签高精度的结果。