We investigate the HSIC (Hilbert-Schmidt independence criterion) bottleneck
as a regularizer for learning an adversarially robust deep neural network
classifier. In addition to the usual cross-entropy loss, we add regularization
terms for every intermediate layer to ensure that the latent representations
retain useful information for output prediction while reducing redundant
information. We show that the HSIC bottleneck enhances robustness to
adversarial attacks both theoretically and experimentally. In particular, we
prove that the HSIC bottleneck regularizer reduces the sensitivity of the
classifier to adversarial examples. Our experiments on multiple benchmark
datasets and architectures demonstrate that incorporating an HSIC bottleneck
regularizer attains competitive natural accuracy and improves adversarial
robustness, both with and without adversarial examples during training. Our
code and adversarially robust models are publicly available.

通过加入 HSIC 瓶颈作为规范化器，我们通过增加每个中间层的正则化项，使潜在的表示保留输出预测的有用信息，同时减少冗余信息，并在理论和实验上证明了 HSIC 瓶颈增强了对抗性攻击的鲁棒性。

重新审视希尔伯特 - 施密特信息瓶颈在对抗鲁棒性中的应用

Revisiting Hilbert-Schmidt Information Bottleneck for Adversarial  Robustness

We introduce the HSIC (Hilbert-Schmidt independence criterion) bottleneck for
training deep neural networks. The HSIC bottleneck is an alternative to the
conventional cross-entropy loss and backpropagation that has a number of
distinct advantages. It mitigates exploding and vanishing gradients, resulting
in the ability to learn very deep networks without skip connections. There is
no requirement for symmetric feedback or update locking. We find that the HSIC
bottleneck provides performance on MNIST/FashionMNIST/CIFAR10 classification
comparable to backpropagation with a cross-entropy target, even when the system
is not encouraged to make the output resemble the classification labels.
Appending a single layer trained with SGD (without backpropagation) to reformat
the information further improves performance.

介绍了使用 HSIC 瓶颈训练深度神经网络，它可以缓解爆炸和消失梯度的问题，实现了在没有跳跃连接的情况下学习非常深的网络，而且在不需要对称反馈或更新锁定的情况下，可以获得 MNIST / FashionMNIST / CIFAR10 分类等优异的性能。同时，使用 SGD 训练单独的一层可以进一步提高性能。