Adversarial training (AT) has become a popular choice for training robust networks. However, by virtue of its formulation, AT tends to sacrifice clean accuracy heavily in favor of robustness. Furthermore, AT with a large perturbation budget can cause models to get stuck at poor local minima and behave like a constant function, always predicting the same class. To address the above concerns we propose Adversarial Training with Early Stopping (ATES). The design of ATES is guided by principles from curriculum learning that emphasizes on starting "easy" and gradually ramping up on the "difficulty" of training. We do so by early stopping the adversarial example generation step in AT, progressively increasing difficulty of the samples the network trains on. This stabilizes network training even for large perturbation budgets and allows the network to operate at a better clean accuracy versus robustness trade-off curve compared to AT. Functionally, this leads to a significant improvement in both clean accuracy and robustness for ATES models.

本文提出了一种平滑对抗训练方法(SAT)，它利用课程学习的思想对对抗训练过程进行平滑处理，提高了对抗训练的稳定性并在干净精度与鲁棒性之间实现了更好的权衡，具体方法基于海森矩阵的特征值以及softmax概率，实验结果显示该方法在CIFAR-100和Imagenette数据集上比AT方法显著提升了干净度与鲁棒度。

SAT:基于课程的损失平滑改进对抗训练