We present a neural semi-supervised learning model termed Self-Pretraining.
Our model is inspired by the classic self-training algorithm. However, as
opposed to self-training, Self-Pretraining is threshold-free, it can
potentially update its belief about previously labeled documents, and can cope
with the semantic drift problem. Self-Pretraining is iterative and consists of
two classifiers. In each iteration, one classifier draws a random set of
unlabeled documents and labels them. This set is used to initialize the second
classifier, to be further trained by the set of labeled documents. The
algorithm proceeds to the next iteration and the classifiers' roles are
reversed. To improve the flow of information across the iterations and also to
cope with the semantic drift problem, Self-Pretraining employs an iterative
distillation process, transfers hypotheses across the iterations, utilizes a
two-stage training model, uses an efficient learning rate schedule, and employs
a pseudo-label transformation heuristic. We have evaluated our model in three
publicly available social media datasets. Our experiments show that
Self-Pretraining outperforms the existing state-of-the-art semi-supervised
classifiers across multiple settings. Our code is available at
this https URL

该研究提出了一种名为 Self-Pretraining 的神经半监督学习模型，该模型可以无阈值地更新先前标记的文档的信念，并且可以处理语义漂移问题，使用迭代蒸馏过程，跨迭代传输假设，利用两阶段训练模型，使用高效的学习率调度和使用伪标签转换启发式方法。

利用自我预训练的半监督文本分类

Semi-Supervised Text Classification via Self-Pretraining

Weakly-supervised instance segmentation (WSIS) has been considered as a more
challenging task than weakly-supervised semantic segmentation (WSSS). Compared
to WSSS, WSIS requires instance-wise localization, which is difficult to
extract from image-level labels. To tackle the problem, most WSIS approaches
use off-the-shelf proposal techniques that require pre-training with instance
or object level labels, deviating the fundamental definition of the
fully-image-level supervised setting. In this paper, we propose a novel
approach including two innovative components. First, we propose a semantic
knowledge transfer to obtain pseudo instance labels by transferring the
knowledge of WSSS to WSIS while eliminating the need for the off-the-shelf
proposals. Second, we propose a self-refinement method to refine the pseudo
instance labels in a self-supervised scheme and to use the refined labels for
training in an online manner. Here, we discover an erroneous phenomenon,
semantic drift, that occurred by the missing instances in pseudo instance
labels categorized as background class. This semantic drift occurs confusion
between background and instance in training and consequently degrades the
segmentation performance. We term this problem as semantic drift problem and
show that our proposed self-refinement method eliminates the semantic drift
problem. The extensive experiments on PASCAL VOC 2012 and MS COCO demonstrate
the effectiveness of our approach, and we achieve a considerable performance
without off-the-shelf proposal techniques. The code is available at
this https URL

本文提出一种创新的方法来解决弱监督实例分割任务中的实例定位难题，并使用自我训练方案来改进伪标注，消除语义漂移问题。通过在 PASCAL VOC 2012 和 MS COCO 上广泛实验，证明了该方法的有效性，无需使用与实例或对象有关的预先训练技术。