Adversarial pruning compresses models while preserving robustness. Current
methods require access to adversarial examples during pruning. This
significantly hampers training efficiency. Moreover, as new adversarial attacks
and training methods develop at a rapid rate, adversarial pruning methods need
to be modified accordingly to keep up. In this work, we propose a novel
framework to prune a previously trained robust neural network while maintaining
adversarial robustness, without further generating adversarial examples. We
leverage concurrent self-distillation and pruning to preserve knowledge in the
original model as well as regularizing the pruned model via the Hilbert-Schmidt
Information Bottleneck. We comprehensively evaluate our proposed framework and
show its superior performance in terms of both adversarial robustness and
efficiency when pruning architectures trained on the MNIST, CIFAR-10, and
CIFAR-100 datasets against five state-of-the-art attacks. Code is available at
this https URL.

该研究通过运用自蒸馏和信息瓶颈方案，提出新的神经网络剪枝框架，使其可以在保持鲁棒性的情况下压缩其大小并提高训练效率。

无需对抗样本修剪对抗鲁棒神经网络

Pruning Adversarially Robust Neural Networks without Adversarial Examples

Adversarial examples provoke weak reliability and potential security issues
in deep neural networks. Although adversarial training has been widely studied
to improve adversarial robustness, it works in an over-parameterized regime and
requires high computations and large memory budgets. To bridge adversarial
robustness and model compression, we propose a novel adversarial pruning
method, Masking Adversarial Damage (MAD) that employs second-order information
of adversarial loss. By using it, we can accurately estimate adversarial
saliency for model parameters and determine which parameters can be pruned
without weakening adversarial robustness. Furthermore, we reveal that model
parameters of initial layer are highly sensitive to the adversarial examples
and show that compressed feature representation retains semantic information
for the target objects. Through extensive experiments on three public datasets,
we demonstrate that MAD effectively prunes adversarially trained networks
without loosing adversarial robustness and shows better performance than
previous adversarial pruning methods.

深度神经网络中的对抗样本会引起弱可靠性和潜在安全问题。为了解决对抗鲁棒性和模型压缩之间的问题，我们提出了一种基于对抗损失的二阶信息的新对抗剪枝方法，称为 MAD。通过在三个公共数据集上进行的广泛实验，我们证明了 MAD 有效地剪枝了对抗训练的网络，而不会失去其对抗鲁棒性，并显示出比以前的对抗剪枝方法更好的性能。

遮盖对抗损伤：寻找稳健、稀疏网络的对抗显著性

Masking Adversarial Damage: Finding Adversarial Saliency for Robust and  Sparse Network

Adversarially robust machine learning has received much recent attention.
However, prior attacks and defenses for non-parametric classifiers have been
developed in an ad-hoc or classifier-specific basis. In this work, we take a
holistic look at adversarial examples for non-parametric classifiers, including
nearest neighbors, decision trees, and random forests. We provide a general
defense method, adversarial pruning, that works by preprocessing the dataset to
become well-separated. To test our defense, we provide a novel attack that
applies to a wide range of non-parametric classifiers. Theoretically, we derive
an optimally robust classifier, which is analogous to the Bayes Optimal. We
show that adversarial pruning can be viewed as a finite sample approximation to
this optimal classifier. We empirically show that our defense and attack are
either better than or competitive with prior work on non-parametric
classifiers. Overall, our results provide a strong and broadly-applicable
baseline for future work on robust non-parametrics. Code available at
this https URL .

文章系统研究了非参数分类器的对抗性问题，提出了全新的数据预处理方案 adverasrial pruning，并且得到了理论最优的鲁棒分类器，实验结果显示该方法更好或竞争性更好。