Generative adversarial networks (GANs) have shown significant potential in modeling high dimensional distributions of image data, especially on image-to-image translation tasks. However, due to the complexity of these tasks, state-of-the-art models often contain a tremendous amount of parameters, which results in large model size and long inference time. In this work, we propose a novel method to address this problem by applying knowledge distillation together with distillation of a semantic relation preserving matrix. This matrix, derived from the teacher's feature encoding, helps the student model learn better semantic relations. In contrast to existing compression methods designed for classification tasks, our proposed method adapts well to the image-to-image translation task on GANs. Experiments conducted on 5 different datasets and 3 different pairs of teacher and student models provide strong evidence that our methods achieve impressive results both qualitatively and quantitatively.

本文提出了使用知识蒸馏和语义保留矩阵以压缩学生模型大小的方法，该方法在5个数据集和3个教师-学生模型对的实验中，证明可以在生成对抗网络的图像到图像翻译任务中取得令人印象深刻的质量和数量上的优秀结果。

图像到图像翻译的语义关系保持知识蒸馏