We introduce a new setting of Novel Class Discovery in Semantic Segmentation
(NCDSS), which aims at segmenting unlabeled images containing new classes given
prior knowledge from a labeled set of disjoint classes. In contrast to existing
approaches that look at novel class discovery in image classification, we focus
on the more challenging semantic segmentation. In NCDSS, we need to distinguish
the objects and background, and to handle the existence of multiple classes
within an image, which increases the difficulty in using the unlabeled data. To
tackle this new setting, we leverage the labeled base data and a saliency model
to coarsely cluster novel classes for model training in our basic framework.
Additionally, we propose the Entropy-based Uncertainty Modeling and
Self-training (EUMS) framework to overcome noisy pseudo-labels, further
improving the model performance on the novel classes. Our EUMS utilizes an
entropy ranking technique and a dynamic reassignment to distill clean labels,
thereby making full use of the noisy data via self-supervised learning. We
build the NCDSS benchmark on the PASCAL-5$^i$ dataset and COCO-20$^i$ dataset.
Extensive experiments demonstrate the feasibility of the basic framework
(achieving an average mIoU of 49.81% on PASCAL-5$^i$) and the effectiveness of
EUMS framework (outperforming the basic framework by 9.28% mIoU on
PASCAL-5$^i$).

这篇论文介绍了一种新的新类发现语义分割方法，使用基于熵的不确定性建模和自训练技术来提高模型性能，实验结果展示了这种方法对于 PASCAL-5i 数据集上的平均 IoU 提高了 9.28%。