Data-free knowledge distillation (DFKD) aims to obtain a lightweight student
model without original training data. Existing works generally synthesize data
from the pre-trained teacher model to replace the original training data for
student learning. To more effectively train the student model, the synthetic
data shall be customized to the current student learning ability. However, this
is ignored in the existing DFKD methods and thus negatively affects the student
training. To address this issue, we propose Customizing Synthetic Data for
Data-Free Student Learning (CSD) in this paper, which achieves adaptive data
synthesis using a self-supervised augmented auxiliary task to estimate the
student learning ability. Specifically, data synthesis is dynamically adjusted
to enlarge the cross entropy between the labels and the predictions from the
self-supervised augmented task, thus generating hard samples for the student
model. The experiments on various datasets and teacher-student models show the
effectiveness of our proposed method. Code is available at:
$\href{https://github.com/luoshiya/CSD}{this https URL}$

自适应数据合成（CSD）通过自监督增强辅助任务来估计学生的学习能力，从而实现自适应数据合成，扩大标签与预测之间的交叉熵，为学生模型生成困难样本，证实了该方法的有效性。