Adversarial training (AT) is a canonical method for enhancing the robustness
of deep neural networks (DNNs). However, recent studies empirically
demonstrated that it suffers from robust overfitting, i.e., a long time AT can
be detrimental to the robustness of DNNs. This paper presents a theoretical
explanation of robust overfitting for DNNs. Specifically, we non-trivially
extend the neural tangent kernel (NTK) theory to AT and prove that an
adversarially trained wide DNN can be well approximated by a linearized DNN.
Moreover, for squared loss, closed-form AT dynamics for the linearized DNN can
be derived, which reveals a new AT degeneration phenomenon: a long-term AT will
result in a wide DNN degenerates to that obtained without AT and thus cause
robust overfitting. Based on our theoretical results, we further design a
method namely Adv-NTK, the first AT algorithm for infinite-width DNNs.
Experiments on real-world datasets show that Adv-NTK can help infinite-width
DNNs enhance comparable robustness to that of their finite-width counterparts,
which in turn justifies our theoretical findings. The code is available at
this https URL

深度神经网络的对抗训练存在鲁棒过拟合问题，本文通过理论分析提出了神经切线核理论在对抗训练中的扩展，并证明了经过对抗训练的宽深度神经网络可以很好近似为一个线性化的神经网络；进一步，设计了用于无限宽度深度神经网络的对抗训练算法 Adv-NTK，实验证明 Adv-NTK 能够使无限宽度深度神经网络提高鲁棒性。