Adversarial training (AT) is currently one of the most successful methods to
obtain the adversarial robustness of deep neural networks. However, the
phenomenon of robust overfitting, i.e., the robustness starts to decrease
significantly during AT, has been problematic, not only making practitioners
consider a bag of tricks for a successful training, e.g., early stopping, but
also incurring a significant generalization gap in the robustness. In this
paper, we propose an effective regularization technique that prevents robust
overfitting by optimizing an auxiliary `consistency' regularization loss during
AT. Specifically, we discover that data augmentation is a quite effective tool
to mitigate the overfitting in AT, and develop a regularization that forces the
predictive distributions after attacking from two different augmentations of
the same instance to be similar with each other. Our experimental results
demonstrate that such a simple regularization technique brings significant
improvements in the test robust accuracy of a wide range of AT methods. More
remarkably, we also show that our method could significantly help the model to
generalize its robustness against unseen adversaries, e.g., other types or
larger perturbations compared to those used during training. Code is available
at this https URL

本文提出了一种通过优化辅助一致性规则损失来避免鲁棒过度拟合的有效正则化技术，在 Adversarial training 过程中使用数据扩增来强制攻击后的预测分布相似。实验结果表明，这种简单的方法可以显著提高各种 AT 方法的测试准确性，并对模型作出更具实际意义的泛化。