Adversarial Propagation (AdvProp) is an effective way to improve recognition
models, leveraging adversarial examples. Nonetheless, AdvProp suffers from the
extremely slow training speed, mainly because: a) extra forward and backward
passes are required for generating adversarial examples; b) both original
samples and their adversarial counterparts are used for training (i.e.,
2$\times$ data). In this paper, we introduce Fast AdvProp, which aggressively
revamps AdvProp's costly training components, rendering the method nearly as
cheap as the vanilla training. Specifically, our modifications in Fast AdvProp
are guided by the hypothesis that disentangled learning with adversarial
examples is the key for performance improvements, while other training recipes
(e.g., paired clean and adversarial training samples, multi-step adversarial
attackers) could be largely simplified.
Our empirical results show that, compared to the vanilla training baseline,
Fast AdvProp is able to further model performance on a spectrum of visual
benchmarks, without incurring extra training cost. Additionally, our ablations
find Fast AdvProp scales better if larger models are used, is compatible with
existing data augmentation methods (i.e., Mixup and CutMix), and can be easily
adapted to other recognition tasks like object detection. The code is available
here: this https URL

本文介绍了一种名为 Fast AdvProp 的改进 Adversarial Propagation（AdvProp）的方法，通过去除不必要的训练组件，提高了训练速度，并采用不同于其他训练配方的具有纠缠性的学习方法，进一步提高了模型性能，在不增加训练成本的情况下，成功地在多个视觉基准上取得了优异表现。