Knowledge Distillation (KD), which transfers the knowledge of a well-trained
large model (teacher) to a small model (student), has become an important area
of research for practical deployment of recommender systems. Recently, Relaxed
Ranking Distillation (RRD) has shown that distilling the ranking information in
the recommendation list significantly improves the performance. However, the
method still has limitations in that 1) it does not fully utilize the
prediction errors of the student model, which makes the training not fully
efficient, and 2) it only distills the user-side ranking information, which
provides an insufficient view under the sparse implicit feedback. This paper
presents Dual Correction strategy for Distillation (DCD), which transfers the
ranking information from the teacher model to the student model in a more
efficient manner. Most importantly, DCD uses the discrepancy between the
teacher model and the student model predictions to decide which knowledge to be
distilled. By doing so, DCD essentially provides the learning guidance tailored
to "correcting" what the student model has failed to accurately predict. This
process is applied for transferring the ranking information from the user-side
as well as the item-side to address sparse implicit user feedback. Our
experiments show that the proposed method outperforms the state-of-the-art
baselines, and ablation studies validate the effectiveness of each component.

本文提出了 Dual Correction strategy for Distillation (DCD)，该方法通过梯度对齐实现知识转移，将评分信息从教师模型传递到学生模型，以更高效地训练推荐系统，同时解决了每个模型预测错误时所出现的问题，并通过用户和物品两个方面来处理稀疏隐式反馈，实验结果表明，该方法优于现有的基线模型。