Deep Neural Networks (DNNs) based semantic segmentation of the robotic
instruments and tissues can enhance the precision of surgical activities in
robot-assisted surgery. However, in biological learning, DNNs cannot learn
incremental tasks over time and exhibit catastrophic forgetting, which refers
to the sharp decline in performance on previously learned tasks after learning
a new one. Specifically, when data scarcity is the issue, the model shows a
rapid drop in performance on previously learned instruments after learning new
data with new instruments. The problem becomes worse when it limits releasing
the dataset of the old instruments for the old model due to privacy concerns
and the unavailability of the data for the new or updated version of the
instruments for the continual learning model. For this purpose, we develop a
privacy-preserving synthetic continual semantic segmentation framework by
blending and harmonizing (i) open-source old instruments foreground to the
synthesized background without revealing real patient data in public and (ii)
new instruments foreground to extensively augmented real background. To boost
the balanced logit distillation from the old model to the continual learning
model, we design overlapping class-aware temperature normalization (CAT) by
controlling model learning utility. We also introduce multi-scale
shifted-feature distillation (SD) to maintain long and short-range spatial
relationships among the semantic objects where conventional short-range spatial
features with limited information reduce the power of feature distillation. We
demonstrate the effectiveness of our framework on the EndoVis 2017 and 2018
instrument segmentation dataset with a generalized continual learning setting.
Code is available at~https://github.com/XuMengyaAmy/Synthetic_CAT_SD.

基于深度神经网络的语义分割可提高机器人辅助手术中手术器械和组织的精确性。然而，在生物学学习中，深度神经网络无法随着时间的推移学习增量任务，表现出灾难性遗忘现象。为了解决这个问题，本研究开发了一种隐私保护的合成持续语义分割框架，通过合成背景和新器械来均衡地融合旧模型和持续学习模型，同时引入了多尺度特征蒸馏以保持语义对象之间的长短距离空间关系。