Incremental semantic segmentation aims to continually learn the segmentation of new coming classes without accessing the training data of previously learned classes. However, most current methods fail to address catastrophic forgetting and background shift since they 1) treat all previous classes equally without considering different forgetting paces caused by imbalanced gradient back-propagation; 2) lack strong semantic guidance between classes. To tackle the above challenges, in this paper, we propose a Gradient-Semantic Compensation (GSC) model, which surmounts incremental semantic segmentation from both gradient and semantic perspectives. Specifically, to address catastrophic forgetting from the gradient aspect, we develop a step-aware gradient compensation that can balance forgetting paces of previously seen classes via re-weighting gradient backpropagation. Meanwhile, we propose a soft-sharp semantic relation distillation to distill consistent inter-class semantic relations via soft labels for alleviating catastrophic forgetting from the semantic aspect. In addition, we develop a prototypical pseudo re-labeling that provides strong semantic guidance to mitigate background shift. It produces high-quality pseudo labels for old classes in the background by measuring distances between pixels and class-wise prototypes. Extensive experiments on three public datasets, i.e., Pascal VOC 2012, ADE20K, and Cityscapes, demonstrate the effectiveness of our proposed GSC model.

本研究提出了一种Gradient-Semantic Compensation (GSC)模型，从梯度和语义的角度克服了增量语义分割中的灾难性遗忘和背景转移问题。通过重新加权梯度反向传播来平衡先前训练类别的遗忘速度，同时提出了软锐语义关系蒸馏来减轻语义方面的灾难性遗忘，并使用原型伪重新标签方法提供强大的语义引导以减轻背景转移问题。在Pascal VOC 2012、ADE20K和Cityscapes三个公共数据集上的大量实验证明了提出的GSC模型的有效性。

渐变语义补偿对于增量语义分割的应用