Despite the effectiveness in improving the robustness of neural networks,
adversarial training has suffered from the natural accuracy degradation
problem, i.e., accuracy on natural samples has reduced significantly. In this
study, we reveal that natural accuracy degradation is highly related to the
disruption of the natural sample topology in the representation space by
quantitative and qualitative experiments. Based on this observation, we propose
Topology-pReserving Adversarial traINing (TRAIN) to alleviate the problem by
preserving the topology structure of natural samples from a standard model
trained only on natural samples during adversarial training. As an additional
regularization, our method can easily be combined with various popular
adversarial training algorithms in a plug-and-play manner, taking advantage of
both sides. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny ImageNet
show that our proposed method achieves consistent and significant improvements
over various strong baselines in most cases. Specifically, without additional
data, our proposed method achieves up to 8.78% improvement in natural accuracy
and 4.50% improvement in robust accuracy.

通过保留自然样本的拓扑结构，我们的研究提出了一种名为 Topology-pReserving Adversarial traINing (TRAIN) 的对抗训练方法，以缓解自然准确率降低的问题，并在多个测试数据集上实验证明在自然准确率和鲁棒准确率上都取得了显著的改善。