Fast adversarial training (FAT) effectively improves the efficiency of
standard adversarial training (SAT). However, initial FAT encounters
catastrophic overfitting, i.e.,the robust accuracy against adversarial attacks
suddenly and dramatically decreases. Though several FAT variants spare no
effort to prevent overfitting, they sacrifice much calculation cost. In this
paper, we explore the difference between the training processes of SAT and FAT
and observe that the attack success rate of adversarial examples (AEs) of FAT
gets worse gradually in the late training stage, resulting in overfitting. The
AEs are generated by the fast gradient sign method (FGSM) with a zero or random
initialization. Based on the observation, we propose a prior-guided FGSM
initialization method to avoid overfitting after investigating several
initialization strategies, improving the quality of the AEs during the whole
training process. The initialization is formed by leveraging historically
generated AEs without additional calculation cost. We further provide a
theoretical analysis for the proposed initialization method. We also propose a
simple yet effective regularizer based on the prior-guided initialization,i.e.,
the currently generated perturbation should not deviate too much from the
prior-guided initialization. The regularizer adopts both historical and current
adversarial perturbations to guide the model learning. Evaluations on four
datasets demonstrate that the proposed method can prevent catastrophic
overfitting and outperform state-of-the-art FAT methods. The code is released
at this https URL

本文针对快速对抗训练 (FAT) 出现的过拟合现象进行了研究，提出了一种基于先验引导的快速梯度符号方法 (FGSM) 初始化策略和正则化方法，有效预防了过拟合，提高了对抗训练的效率。实验结果表明该方法优于现有的同类方法。