Self-training has shown great potential in semi-supervised learning. Its core
idea is to use the model learned on labeled data to generate pseudo-labels for
unlabeled samples, and in turn teach itself. To obtain valid supervision,
active attempts typically employ a momentum teacher for pseudo-label prediction
yet observe the confirmation bias issue, where the incorrect predictions may
provide wrong supervision signals and get accumulated in the training process.
The primary cause of such a drawback is that the prevailing self-training
framework acts as guiding the current state with previous knowledge, because
the teacher is updated with the past student only. To alleviate this problem,
we propose a novel self-training strategy, which allows the model to learn from
the future. Concretely, at each training step, we first virtually optimize the
student (i.e., caching the gradients without applying them to the model
weights), then update the teacher with the virtual future student, and finally
ask the teacher to produce pseudo-labels for the current student as the
guidance. In this way, we manage to improve the quality of pseudo-labels and
thus boost the performance. We also develop two variants of our
future-self-training (FST) framework through peeping at the future both deeply
(FST-D) and widely (FST-W). Taking the tasks of unsupervised domain adaptive
semantic segmentation and semi-supervised semantic segmentation as the
instances, we experimentally demonstrate the effectiveness and superiority of
our approach under a wide range of settings. Code will be made publicly
available.

提出了一种新的自我训练策略，允许模型从未来中学习，可以提高合成标签的质量，从而提高半监督学习任务的性能表现。