Adversarial training serves as one of the most popular and effective methods
to defend against adversarial perturbations. However, most defense mechanisms
only consider a single type of perturbation while various attack methods might
be adopted to perform stronger adversarial attacks against the deployed model
in real-world scenarios, e.g., $\ell_2$ or $\ell_\infty$. Defending against
various attacks can be a challenging problem since multi-perturbation
adversarial training and its variants only achieve suboptimal robustness
trade-offs, due to the theoretical limit to multi-perturbation robustness for a
single model. Besides, it is impractical to deploy large models in some
storage-efficient scenarios. To settle down these drawbacks, in this paper we
propose a novel multi-perturbation adversarial training framework,
parameter-saving adversarial training (PSAT), to reinforce multi-perturbation
robustness with an advantageous side effect of saving parameters, which
leverages hypernetworks to train specialized models against a single
perturbation and aggregate these specialized models to defend against multiple
perturbations. Eventually, we extensively evaluate and compare our proposed
method with state-of-the-art single/multi-perturbation robust methods against
various latest attack methods on different datasets, showing the robustness
superiority and parameter efficiency of our proposed method, e.g., for the
CIFAR-10 dataset with ResNet-50 as the backbone, PSAT saves approximately 80\%
of parameters with achieving the state-of-the-art robustness trade-off
accuracy.

本文提出了一个新颖的多扰动对抗训练框架，即参数节省的对抗训练（PSAT），强化多扰动鲁棒性的同时具有参数节省的优势，并通过使用超网络训练专门针对单一扰动的模型并集成这些专门模型以抵御多种扰动。最终，我们在不同数据集上对我们的方法与最新攻击方法进行了广泛评估和比较，显示了我们提出的方法在鲁棒性和参数效率方面的优势，例如，对于 CIFAR-10 数据集，以 ResNet-50 为骨干网络，PSAT 在保持最先进的鲁棒性的基础上节省了大约 80% 的参数。