Many researchers have sought ways of model compression to reduce the size of
a deep neural network (DNN) with minimal performance degradation in order to
use DNNs in embedded systems. Among the model compression methods, a method
called knowledge transfer is to train a student network with a stronger teacher
network. In this paper, we propose a novel knowledge transfer method which uses
convolutional operations to paraphrase teacher's knowledge and to translate it
for the student. This is done by two convolutional modules, which are called a
paraphraser and a translator. The paraphraser is trained in an unsupervised
manner to extract the teacher factors which are defined as paraphrased
information of the teacher network. The translator located at the student
network extracts the student factors and helps to translate the teacher factors
by mimicking them. We observed that our student network trained with the
proposed factor transfer method outperforms the ones trained with conventional
knowledge transfer methods.

本文提出了一种基于卷积操作的知识转移方法，通过使用两个模块，即叙述模块和翻译模块，将教师网络的知识转化为学生网络的知识，从而实现模型压缩和知识转移，实验结果表明这种方法优于传统的知识转移方法。