This paper focuses on an important type of black-box attacks, i.e.,
transfer-based adversarial attacks, where the adversary generates adversarial
examples by a substitute (source) model and utilize them to attack an unseen
target model, without knowing its information. Existing methods tend to give
unsatisfactory adversarial transferability when the source and target models
are from different types of DNN architectures (e.g. ResNet-18 and Swin
Transformer). In this paper, we observe that the above phenomenon is induced by
the output inconsistency problem. To alleviate this problem while effectively
utilizing the existing DNN models, we propose a common knowledge learning (CKL)
framework to learn better network weights to generate adversarial examples with
better transferability, under fixed network architectures. Specifically, to
reduce the model-specific features and obtain better output distributions, we
construct a multi-teacher framework, where the knowledge is distilled from
different teacher architectures into one student network. By considering that
the gradient of input is usually utilized to generated adversarial examples, we
impose constraints on the gradients between the student and teacher models, to
further alleviate the output inconsistency problem and enhance the adversarial
transferability. Extensive experiments demonstrate that our proposed work can
significantly improve the adversarial transferability.

本文研究黑盒攻击的一种重要类型 —— 基于转移的对抗攻击，并提出了一种通用知识学习框架，通过构建多教师框架来减少模型特定特征，获得更好的输出分布，加强对抗转移性能，同时通过对学生和教师模型之间的梯度施加约束来进一步缓解输出不一致问题和增强对抗转移性能。实验表明，所提出的方法可以显著提高对抗转移性能。