Weakly-supervised instance segmentation (WSIS) has been considered as a more
challenging task than weakly-supervised semantic segmentation (WSSS). Compared
to WSSS, WSIS requires instance-wise localization, which is difficult to
extract from image-level labels. To tackle the problem, most WSIS approaches
use off-the-shelf proposal techniques that require pre-training with instance
or object level labels, deviating the fundamental definition of the
fully-image-level supervised setting. In this paper, we propose a novel
approach including two innovative components. First, we propose a semantic
knowledge transfer to obtain pseudo instance labels by transferring the
knowledge of WSSS to WSIS while eliminating the need for the off-the-shelf
proposals. Second, we propose a self-refinement method to refine the pseudo
instance labels in a self-supervised scheme and to use the refined labels for
training in an online manner. Here, we discover an erroneous phenomenon,
semantic drift, that occurred by the missing instances in pseudo instance
labels categorized as background class. This semantic drift occurs confusion
between background and instance in training and consequently degrades the
segmentation performance. We term this problem as semantic drift problem and
show that our proposed self-refinement method eliminates the semantic drift
problem. The extensive experiments on PASCAL VOC 2012 and MS COCO demonstrate
the effectiveness of our approach, and we achieve a considerable performance
without off-the-shelf proposal techniques. The code is available at
this https URL

本文提出一种创新的方法来解决弱监督实例分割任务中的实例定位难题，并使用自我训练方案来改进伪标注，消除语义漂移问题。通过在 PASCAL VOC 2012 和 MS COCO 上广泛实验，证明了该方法的有效性，无需使用与实例或对象有关的预先训练技术。

从语义到实例分割：通过语义知识转移和自我优化进行弱监督实例分割

Beyond Semantic to Instance Segmentation: Weakly-Supervised Instance  Segmentation via Semantic Knowledge Transfer and Self-Refinement

Weakly-supervised instance segmentation aims to detect and segment object
instances precisely, given imagelevel labels only. Unlike previous methods
which are composed of multiple offline stages, we propose Sequential Label
Propagation and Enhancement Networks (referred as Label-PEnet) that
progressively transform image-level labels to pixel-wise labels in a
coarse-to-fine manner. We design four cascaded modules including multi-label
classification, object detection, instance refinement and instance
segmentation, which are implemented sequentially by sharing the same backbone.
The cascaded pipeline is trained alternatively with a curriculum learning
strategy that generalizes labels from high-level images to low-level pixels
gradually with increasing accuracy. In addition, we design a proposal
calibration module to explore the ability of classification networks to find
key pixels that identify object parts, which serves as a post validation
strategy running in the inverse order. We evaluate the efficiency of our
Label-PEnet in mining instance masks on standard benchmarks: PASCAL VOC 2007
and 2012. Experimental results show that Label-PEnet outperforms the
state-of-the-art algorithms by a clear margin, and obtains comparable
performance even with the fully-supervised approaches.

本文介绍了一种基于 Weakly-supervised instance segmentation 方法、称为 Label-PEnet 的分割网络，能够从图像级标签逐步转化到像素级标签，主要包含四个模块：多标签分类、目标检测、实例细化和实例分割，并通过 Curriculum Learning 策略将标签从高层次图像逐渐泛化到低层次像素。我们同时设计了一种 Proposal Calibration 模块来验证分类网络找到识别对象部件的关键像素的能力。实验结果表明，相比现有方法，Label-PEnet 性能更好，甚至可以达到全监督方法的水平。