Knowledge distillation (KD) is a well-known method to reduce inference
latency by compressing a cumbersome teacher model to a small student model.
Despite the success of KD in the classification task, applying KD to
recommender models is challenging due to the sparsity of positive feedback, the
ambiguity of missing feedback, and the ranking problem associated with the
top-N recommendation. To address the issues, we propose a new KD model for the
collaborative filtering approach, namely collaborative distillation (CD).
Specifically, (1) we reformulate a loss function to deal with the ambiguity of
missing feedback. (2) We exploit probabilistic rank-aware sampling for the
top-N recommendation. (3) To train the proposed model effectively, we develop
two training strategies for the student model, called the teacher- and the
student-guided training methods, selecting the most useful feedback from the
teacher model. Via experimental results, we demonstrate that the proposed model
outperforms the state-of-the-art method by 2.7-33.2% and 2.7-29.1% in hit rate
(HR) and normalized discounted cumulative gain (NDCG), respectively. Moreover,
the proposed model achieves the performance comparable to the teacher model.

本文提出了一种新的知识蒸馏模型 —— 协作蒸馏模型 (CD)，该模型采用概率排序感知采样、转换误差函数以处理反馈不足以及提供两个训练方法 (即教师指导和学生指导)，在推荐领域中取得了 2.7-33.2% 和 2.7-29.1% 的命中率 (HR) 和标准化折现累计增益 (NDCG) 方面的优异表现。