In standard adversarial training, models are optimized to fit one-hot labels
within allowable adversarial perturbation budgets. However, the ignorance of
underlying distribution shifts brought by perturbations causes the problem of
robust overfitting. To address this issue and enhance adversarial robustness,
we analyze the characteristics of robust models and identify that robust models
tend to produce smoother and well-calibrated outputs. Based on the observation,
we propose a simple yet effective method, Annealing Self-Distillation
Rectification (ADR), which generates soft labels as a better guidance mechanism
that accurately reflects the distribution shift under attack during adversarial
training. By utilizing ADR, we can obtain rectified distributions that
significantly improve model robustness without the need for pre-trained models
or extensive extra computation. Moreover, our method facilitates seamless
plug-and-play integration with other adversarial training techniques by
replacing the hard labels in their objectives. We demonstrate the efficacy of
ADR through extensive experiments and strong performances across datasets.

提出一种通过生成软标签辅助来增强对抗训练模型的方法 Annealing Self-Distillation Rectification (ADR)，该方法显著改善了模型的鲁棒性并且易于与其他对抗训练技术进行无缝集成。