The major part of the vanilla vision transformer (ViT) is the attention block
that brings the power of mimicking the global context of the input image. For
better performance, ViT needs large-scale training data. To overcome this data
hunger limitation, many ViT-based networks, or hybrid-ViT, have been proposed
to include local context during the training. The robustness of ViTs and its
variants against adversarial attacks has not been widely investigated in the
literature like CNNs. This work studies the robustness of ViT variants 1)
against different Lp-based adversarial attacks in comparison with CNNs, 2)
under adversarial examples (AEs) after applying preprocessing defense methods
and 3) under the adaptive attacks using expectation over transformation (EOT)
framework. To that end, we run a set of experiments on 1000 images from
ImageNet-1k and then provide an analysis that reveals that vanilla ViT or
hybrid-ViT are more robust than CNNs. For instance, we found that 1) Vanilla
ViTs or hybrid-ViTs are more robust than CNNs under Lp-based attacks and under
adaptive attacks. 2) Unlike hybrid-ViTs, Vanilla ViTs are not responding to
preprocessing defenses that mainly reduce the high frequency components.
Furthermore, feature maps, attention maps, and Grad-CAM visualization jointly
with image quality measures, and perturbations' energy spectrum are provided
for an insight understanding of attention-based models.

本文针对 Vision Transformer 在对抗攻击下的稳健性问题进行了探究，实验证明 Vanilla ViTs 或 Hybrid-ViTs 的对抗攻击鲁棒性比 CNNs 更强。通过提供特征图、注意力图等分析，对注意力模型进行了深入理解。