This paper proposes a new knowledge distillation method tailored for image
semantic segmentation, termed Intra- and Inter-Class Knowledge Distillation
(I2CKD). The focus of this method is on capturing and transferring knowledge
between the intermediate layers of teacher (cumbersome model) and student
(compact model). For knowledge extraction, we exploit class prototypes derived
from feature maps. To facilitate knowledge transfer, we employ a triplet loss
in order to minimize intra-class variances and maximize inter-class variances
between teacher and student prototypes. Consequently, I2CKD enables the student
to better mimic the feature representation of the teacher for each class,
thereby enhancing the segmentation performance of the compact network.
Extensive experiments on three segmentation datasets, i.e., Cityscapes, Pascal
VOC and CamVid, using various teacher-student network pairs demonstrate the
effectiveness of the proposed method.

这篇论文提出了一种针对图像语义分割的新型知识蒸馏方法，称为 Intra- and Inter-Class Knowledge Distillation（I2CKD）。该方法的重点是捕捉和转移教师（笨重模型）和学生（紧凑模型）的中间层之间的知识。为了进行知识提取，我们利用特征图生成了类原型。为了促进知识传递，我们采用三元损失来最小化教师和学生原型之间的类内方差，最大化类间方差。因此，I2CKD 能够使学生更好地模拟教师每个类的特征表示，从而提高紧凑网络的分割性能。使用各种教师 - 学生网络对 Cityscapes、Pascal VOC 和 CamVid 三个分割数据集进行广泛实验，证明了所提方法的有效性。