Semantic segmentation plays a crucial role in enabling comprehensive scene understanding for robotic systems. However, generating annotations is challenging, requiring labels for every pixel in an image. In scenarios like autonomous driving, there's a need to progressively incorporate new classes as the operating environment of the deployed agent becomes more complex. For enhanced annotation efficiency, ideally, only pixels belonging to new classes would be annotated. This approach is known as Continual Semantic Segmentation (CSS). Besides the common problem of classical catastrophic forgetting in the continual learning setting, CSS suffers from the inherent ambiguity of the background, a phenomenon we refer to as the "background shift'', since pixels labeled as background could correspond to future classes (forward background shift) or previous classes (backward background shift). As a result, continual learning approaches tend to fail. This paper proposes a Backward Background Shift Detector (BACS) to detect previously observed classes based on their distance in the latent space from the foreground centroids of previous steps. Moreover, we propose a modified version of the cross-entropy loss function, incorporating the BACS detector to down-weight background pixels associated with formerly observed classes. To combat catastrophic forgetting, we employ masked feature distillation alongside dark experience replay. Additionally, our approach includes a transformer decoder capable of adjusting to new classes without necessitating an additional classification head. We validate BACS's superior performance over existing state-of-the-art methods on standard CSS benchmarks.

该论文提出了一种名为BACS的向后背景位移检测器，用于检测以前观察到的类，并建议一种修改后的交叉熵损失函数，结合BACS检测器，降低与以前观察到的类相关联的背景像素的权重，以解决连续语义分割面临的问题。同时，该方法通过掩蔽特征蒸馏和暗体验重播来应对灾难性遗忘，并使用能够适应新类别的变压器解码器，而不需要额外的分类头。该方法在标准连续语义分割基准测试上验证了BACS相对于现有最先进方法的卓越表现。

BACS: 背景感知持续语义分割