In practice, deep neural networks have been found to be vulnerable to various
types of noise, such as adversarial examples and corruption. Various
adversarial defense methods have accordingly been developed to improve
adversarial robustness for deep models. However, simply training on data mixed
with adversarial examples, most of these models still fail to defend against
the generalized types of noise. Motivated by the fact that hidden layers play a
highly important role in maintaining a robust model, this paper proposes a
simple yet powerful training algorithm, named \emph{Adversarial Noise
Propagation} (ANP), which injects noise into the hidden layers in a layer-wise
manner. ANP can be implemented efficiently by exploiting the nature of the
backward-forward training style. Through thorough investigations, we determine
that different hidden layers make different contributions to model robustness
and clean accuracy, while shallow layers are comparatively more critical than
deep layers. Moreover, our framework can be easily combined with other
adversarial training methods to further improve model robustness by exploiting
the potential of hidden layers. Extensive experiments on MNIST, CIFAR-10,
CIFAR-10-C, CIFAR-10-P, and ImageNet demonstrate that ANP enables the strong
robustness for deep models against both adversarial and corrupted ones, and
also significantly outperforms various adversarial defense methods.

本篇论文提出了一种名为 Adversarial Noise Propagation 的训练算法，将噪声注入隐藏层以提高深度模型的鲁棒性，并证明不同的隐藏层对模型鲁棒性和整洁的准确性有不同的贡献。该算法可有效地与其他对抗性训练方法相结合，进一步提高模型的鲁棒性。在 MNIST、CIFAR-10、CIFAR-10-C、CIFAR-10-P 和 ImageNet 上的广泛实验表明，ANP 实现了深度模型对对抗性和受损数据均具有强大的鲁棒性，并明显优于各种对抗性防御方法。