Adversarial training has shown promise in building robust models against
adversarial examples. A major drawback of adversarial training is the
computational overhead introduced by the generation of adversarial examples. To
overcome this limitation, adversarial training based on single-step attacks has
been explored. Previous work improves the single-step adversarial training from
different perspectives, e.g., sample initialization, loss regularization, and
training strategy. Almost all of them treat the underlying model as a black
box. In this work, we propose to exploit the interior building blocks of the
model to improve efficiency. Specifically, we propose to dynamically sample
lightweight subnetworks as a surrogate model during training. By doing this,
both the forward and backward passes can be accelerated for efficient
adversarial training. Besides, we provide theoretical analysis to show the
model robustness can be improved by the single-step adversarial training with
sampled subnetworks. Furthermore, we propose a novel sampling strategy where
the sampling varies from layer to layer and from iteration to iteration.
Compared with previous methods, our method not only reduces the training cost
but also achieves better model robustness. Evaluations on a series of popular
datasets demonstrate the effectiveness of the proposed FB-Better. Our code has
been released at this https URL

在这项工作中，我们提出了一种利用模型的内部构建块来提高效率的方法，通过在训练过程中动态地采样轻量级子网络作为替代模型，加速前向和后向传递以实现高效的对抗训练，并通过理论分析证明了单步对抗训练和采样子网络可以提高模型的鲁棒性。相比之前的方法，我们的方法不仅减少了训练成本，而且实现了更好的模型鲁棒性。