Fast adversarial training (FAT) is an efficient method to improve robustness.
However, the original FAT suffers from catastrophic overfitting, which
dramatically and suddenly reduces robustness after a few training epochs.
Although various FAT variants have been proposed to prevent overfitting, they
require high training costs. In this paper, we investigate the relationship
between adversarial example quality and catastrophic overfitting by comparing
the training processes of standard adversarial training and FAT. We find that
catastrophic overfitting occurs when the attack success rate of adversarial
examples becomes worse. Based on this observation, we propose a positive
prior-guided adversarial initialization to prevent overfitting by improving
adversarial example quality without extra training costs. This initialization
is generated by using high-quality adversarial perturbations from the
historical training process. We provide theoretical analysis for the proposed
initialization and propose a prior-guided regularization method that boosts the
smoothness of the loss function. Additionally, we design a prior-guided
ensemble FAT method that averages the different model weights of historical
models using different decay rates. Our proposed method, called FGSM-PGK,
assembles the prior-guided knowledge, i.e., the prior-guided initialization and
model weights, acquired during the historical training process. Evaluations of
four datasets demonstrate the superiority of the proposed method.

本文提出了一种名为 FGSM-PGK 的方法，它使用先前的训练过程中高质量的对抗扰动来生成正样本引导的对抗初始化以及使用不同的衰减率平均不同模型权重的先验引导的集成快速对抗训练方法以提高对抗性能，从而解决了 catastrophic overfitting 问题。