PGD-based and FGSM-based are two popular adversarial training (AT) approaches
for obtaining adversarially robust models. Compared with PGD-based AT,
FGSM-based one is significantly faster but fails with catastrophic overfitting
(CO). For mitigating CO in such Fast AT, there are two popular existing
strategies: random start (RandStart) and Gradient Alignment (GradAlign). The
former works only for a relatively small perturbation 8/255 with the l_\infty
constraint, and GradAlign improves it by extending the perturbation size to
16/255 (with the l_\infty constraint) but at the cost of being 3 to 4 times
slower. How to avoid CO in Fast AT for a large perturbation size but without
increasing the computation overhead remains as an unsolved issue, for which our
work provides a frustratingly simple (yet effective) solution. Specifically,
our solution lies in just noise augmentation (NoiseAug) which is a non-trivial
byproduct of simplifying GradAlign. By simplifying GradAlign we have two
findings: (i) aligning logit instead of gradient in GradAlign requires half the
training time but achieves higher performance than GradAlign; (ii) the
alignment operation can also be removed by only keeping noise augmentation
(NoiseAug). Simplified from GradAlign, our NoiseAug has a surprising
resemblance with RandStart except that we inject noise on the image instead of
perturbation. To understand why injecting noise to input prevents CO, we verify
that this is caused not by data augmentation effect (inject noise on image) but
by improved local linearity. We provide an intuitive explanation for why
NoiseAug improves local linearity without explicit regularization. Extensive
results demonstrate that our NoiseAug achieves SOTA results in FGSM AT. The
code will be released after accepted.

文章提出了一个基于噪声增强（NoiseAug）的简单而有效的快速对抗训练方法，相较于现有的随机起点和梯度对齐策略，该方法避免了灾难性过拟合，能够在大扰动条件下提高模型的可靠性和鲁棒性。

噪声增强的快速对抗训练：RandStart 和 GradAlign 的统一视角

Fast Adversarial Training with Noise Augmentation: A Unified Perspective on RandStart and GradAlign

Adversarial training (AT) has been demonstrated to be effective in improving
model robustness by leveraging adversarial examples for training. However, most
AT methods are in face of expensive time and computational cost for calculating
gradients at multiple steps in generating adversarial examples. To boost
training efficiency, fast gradient sign method (FGSM) is adopted in fast AT
methods by calculating gradient only once. Unfortunately, the robustness is far
from satisfactory. One reason may arise from the initialization fashion.
Existing fast AT generally uses a random sample-agnostic initialization, which
facilitates the efficiency yet hinders a further robustness improvement. Up to
now, the initialization in fast AT is still not extensively explored. In this
paper, we boost fast AT with a sample-dependent adversarial initialization,
i.e., an output from a generative network conditioned on a benign image and its
gradient information from the target network. As the generative network and the
target network are optimized jointly in the training phase, the former can
adaptively generate an effective initialization with respect to the latter,
which motivates gradually improved robustness. Experimental evaluations on four
benchmark databases demonstrate the superiority of our proposed method over
state-of-the-art fast AT methods, as well as comparable robustness to advanced
multi-step AT methods. The code is released at
this https URL

本研究针对 Adversarial training 应用 Fast Gradient Sign Method (FGSM) 进行改进，提出了一种基于 generative network 和 target network 联合优化的 sample-dependent adversarial initialization 方案来提升模型的鲁棒性，并在四个基准数据库上进行了实验验证，结果表明该方法比现有的 Fast AT 方法更加优越。