Since the Lipschitz properties of convolutional neural networks (CNNs) are
widely considered to be related to adversarial robustness, we theoretically
characterize the $\ell_1$ norm and $\ell_\infty$ norm of 2D multi-channel
convolutional layers and provide efficient methods to compute the exact
$\ell_1$ norm and $\ell_\infty$ norm. Based on our theorem, we propose a novel
regularization method termed norm decay, which can effectively reduce the norms
of convolutional layers and fully-connected layers. Experiments show that
norm-regularization methods, including norm decay, weight decay, and singular
value clipping, can improve generalization of CNNs. However, they can slightly
hurt adversarial robustness. Observing this unexpected phenomenon, we compute
the norms of layers in the CNNs trained with three different adversarial
training frameworks and surprisingly find that adversarially robust CNNs have
comparable or even larger layer norms than their non-adversarially robust
counterparts. Furthermore, we prove that under a mild assumption, adversarially
robust classifiers can be achieved using neural networks, and an adversarially
robust neural network can have an arbitrarily large Lipschitz constant. For
this reason, enforcing small norms on CNN layers may be neither necessary nor
effective in achieving adversarial robustness. The code is available at
this https URL

通过对 2D 多通道卷积层的理论分析，提出了规范化方法 norm decay，它可以有效地降低卷积层和全连接层的范数，并证明了在一定条件下，可以使用神经网络实现对抗性强度分类器，并且在提高通用性方面具有改进效果，但可能对攻击鲁棒性有轻微影响。