Despite being a heavily researched topic, Adversarial Training (AT) is
rarely, if ever, deployed in practical AI systems for two primary reasons: (i)
the gained robustness is frequently accompanied by a drop in generalization and
(ii) generating adversarial examples (AEs) is computationally prohibitively
expensive. To address these limitations, we propose SMAAT, a new AT algorithm
that leverages the manifold conjecture, stating that off-manifold AEs lead to
better robustness while on-manifold AEs result in better generalization.
Specifically, SMAAT aims at generating a higher proportion of off-manifold AEs
by perturbing the intermediate deepnet layer with the lowest intrinsic
dimension. This systematically results in better scalability compared to
classical AT as it reduces the PGD chains length required for generating the
AEs. Additionally, our study provides, to the best of our knowledge, the first
explanation for the difference in the generalization and robustness trends
between vision and language models, ie., AT results in a drop in generalization
in vision models whereas, in encoder-based language models, generalization
either improves or remains unchanged. We show that vision transformers and
decoder-based models tend to have low intrinsic dimensionality in the earlier
layers of the network (more off-manifold AEs), while encoder-based models have
low intrinsic dimensionality in the later layers. We demonstrate the efficacy
of SMAAT; on several tasks, including robustifying (i) sentiment classifiers,
(ii) safety filters in decoder-based models, and (iii) retrievers in RAG
setups. SMAAT requires only 25-33% of the GPU time compared to standard AT,
while significantly improving robustness across all applications and
maintaining comparable generalization.

通过改进的 Adversarial Training 算法 SMAAT，本研究提供了 vision 和 language 模型在 generalization 和 robustness 趋势差异的首个解释，同时展示了 SMAAT 在多个任务中显著提高了鲁棒性，且相比于标准的 AT 仅需约 25-33% 的 GPU 时间。