Recent advances have been made in applying convolutional neural networks to
achieve more precise prediction results for medical image segmentation
problems. However, the success of existing methods has highly relied on huge
computational complexity and massive storage, which is impractical in the
real-world scenario. To deal with this problem, we propose an efficient
architecture by distilling knowledge from well-trained medical image
segmentation networks to train another lightweight network. This architecture
empowers the lightweight network to get a significant improvement on
segmentation capability while retaining its runtime efficiency. We further
devise a novel distillation module tailored for medical image segmentation to
transfer semantic region information from teacher to student network. It forces
the student network to mimic the extent of difference of representations
calculated from different tissue regions. This module avoids the ambiguous
boundary problem encountered when dealing with medical imaging but instead
encodes the internal information of each semantic region for transferring.
Benefited from our module, the lightweight network could receive an improvement
of up to 32.6% in our experiment while maintaining its portability in the
inference phase. The entire structure has been verified on two widely accepted
public CT datasets LiTS17 and KiTS19. We demonstrate that a lightweight network
distilled by our method has non-negligible value in the scenario which requires
relatively high operating speed and low storage usage.

我们提出了一种高效的网络结构，通过从训练充分的医学图像分割网络中提炼知识来训练另一个轻量级网络，从而使得轻量级网络在保留其运行效率的同时，显著提高了其分割能力。通过使用我们为医学图像分割量身定制的新颖蒸馏模式，从教师网络向学生网络传递语义区域信息，避免了处理医学图像时遇到的模糊边界问题。在我们的实验中，轻量级网络的性能提高了 32.6％，同时在推理阶段保持了可移植性，并在 LiTS17 和 KiTS19 两个公认的公共 CT 数据集上进行了验证。