Data-free knowledge distillation (KD) helps transfer knowledge from a
pre-trained model (known as the teacher model) to a smaller model (known as the
student model) without access to the original training data used for training
the teacher model. However, the security of the synthetic or
out-of-distribution (OOD) data required in data-free KD is largely unknown and
under-explored. In this work, we make the first effort to uncover the security
risk of data-free KD w.r.t. untrusted pre-trained models. We then propose
Anti-Backdoor Data-Free KD (ABD), the first plug-in defensive method for
data-free KD methods to mitigate the chance of potential backdoors being
transferred. We empirically evaluate the effectiveness of our proposed ABD in
diminishing transferred backdoor knowledge while maintaining compatible
downstream performances as the vanilla KD. We envision this work as a milestone
for alarming and mitigating the potential backdoors in data-free KD. Codes are
released at this https URL

本文旨在探讨数据无关知识蒸馏的安全问题并提出了 Anti-Backdoor Data-Free KD，这是第一个用于缓解潜在后门被转移的数据无关 KD 方法的防御性插件。