As a promising solution for model compression, knowledge distillation (KD)
has been applied in recommender systems (RS) to reduce inference latency.
Traditional solutions first train a full teacher model from the training data,
and then transfer its knowledge (\ie \textit{soft labels}) to supervise the
learning of a compact student model. However, we find such a standard
distillation paradigm would incur serious bias issue -- popular items are more
heavily recommended after the distillation. This effect prevents the student
model from making accurate and fair recommendations, decreasing the
effectiveness of RS.
In this work, we identify the origin of the bias in KD -- it roots in the
biased soft labels from the teacher, and is further propagated and intensified
during the distillation. To rectify this, we propose a new KD method with a
stratified distillation strategy. It first partitions items into multiple
groups according to their popularity, and then extracts the ranking knowledge
within each group to supervise the learning of the student. Our method is
simple and teacher-agnostic -- it works on distillation stage without affecting
the training of the teacher model. We conduct extensive theoretical and
empirical studies to validate the effectiveness of our proposal. We release our
code at: this https URL.

本文提出了一种新的知识蒸馏方法，即分层蒸馏策略，用于解决推荐系统中已有的知识蒸馏方法中存在的推荐偏差问题。该方法通过对热门物品进行分层处理，提取每个组中的排名知识，用于监督学生模型的学习，避免了推荐结果的不准确和不公平。