Catastrophic overfitting is a phenomenon observed during Adversarial Training
(AT) with the Fast Gradient Sign Method (FGSM) where the test robustness
steeply declines over just one epoch in the training stage. Prior work has
attributed this loss in robustness to a sharp decrease in $\textit{local
linearity}$ of the neural network with respect to the input space, and has
demonstrated that introducing a local linearity measure as a regularization
term prevents catastrophic overfitting. Using a simple neural network
architecture, we experimentally demonstrate that maintaining high local
linearity might be $\textit{sufficient}$ to prevent catastrophic overfitting
but is not $\textit{necessary.}$ Further, inspired by Parseval networks, we
introduce a regularization term to AT with FGSM to make the weight matrices of
the network orthogonal and study the connection between orthogonality of the
network weights and local linearity. Lastly, we identify the $\textit{double
descent}$ phenomenon during the adversarial training process.

本研究通过实验证明保持高的局部线性度可能足以防止灾难性过拟合，但这并非必要条件。此外，引入一种规范化术语，使网络权重矩阵正交，并研究了网络权重正交和局部线性之间的联系，并在对抗训练过程中识别了双重下降现象。