Adversarial Training (AT) has been widely proved to be an effective method to
improve the adversarial robustness against adversarial examples for Deep Neural
Networks (DNNs). As a variant of AT, Adversarial Robustness Distillation (ARD)
has demonstrated its superior performance in improving the robustness of small
student models with the guidance of large teacher models. However, both AT and
ARD encounter the robust fairness problem: these models exhibit strong
robustness when facing part of classes (easy class), but weak robustness when
facing others (hard class). In this paper, we give an in-depth analysis of the
potential factors and argue that the smoothness degree of samples' soft labels
for different classes (i.e., hard class or easy class) will affect the robust
fairness of DNN models from both empirical observation and theoretical
analysis. Based on the above finding, we propose an Anti-Bias Soft Label
Distillation (ABSLD) method to mitigate the adversarial robust fairness problem
within the framework of Knowledge Distillation (KD). Specifically, ABSLD
adaptively reduces the student's error risk gap between different classes to
achieve fairness by adjusting the class-wise smoothness degree of samples' soft
labels during the training process, and the smoothness degree of soft labels is
controlled by assigning different temperatures in KD to different classes.
Extensive experiments demonstrate that ABSLD outperforms state-of-the-art AT,
ARD, and robust fairness methods in terms of overall performance of robustness
and fairness.

通过对软标签的平滑程度进行深入分析和理论分析，我们提出了一种反偏差软标签蒸馏（ABSLD）方法，以在知识蒸馏（KD）框架中减轻对手性鲁棒性公平问题，通过调整训练过程中样本的软标签的类别平滑程度来减少学生模型在不同类别之间的错误风险差距，软标签的平滑程度由在 KD 中为不同类别分配不同的温度来控制。大量实验表明，ABSLD 在鲁棒性和公平性的整体性能方面优于最先进的对手训练、对手鲁棒性蒸馏和公平性方法。