Image segmentation based on continual learning exhibits a critical drop of
performance, mainly due to catastrophic forgetting and background shift, as
they are required to incorporate new classes continually. In this paper, we
propose a simple, yet effective Continual Image Segmentation method with
incremental Dynamic Query (CISDQ), which decouples the representation learning
of both old and new knowledge with lightweight query embedding. CISDQ mainly
includes three contributions: 1) We define dynamic queries with adaptive
background class to exploit past knowledge and learn future classes naturally.
2) CISDQ proposes a class/instance-aware Query Guided Knowledge Distillation
strategy to overcome catastrophic forgetting by capturing the inter-class
diversity and intra-class identity. 3) Apart from semantic segmentation, CISDQ
introduce the continual learning for instance segmentation in which
instance-wise labeling and supervision are considered. Extensive experiments on
three datasets for two tasks (i.e., continual semantic and instance
segmentation are conducted to demonstrate that CISDQ achieves the
state-of-the-art performance, specifically, obtaining 4.4% and 2.9% mIoU
improvements for the ADE 100-10 (6 steps) setting and ADE 100-5 (11 steps)
setting.

基于持续学习的图像分割方法，通过增量动态查询实现了概念解耦，克服了灾难性遗忘和背景漂移问题，同时在语义分割和实例分割任务上取得了最先进的性能。