Previous studies have shown the vulnerability of vision transformers to
adversarial patches, but these studies all rely on a critical assumption: the
attack patches must be perfectly aligned with the patches used for linear
projection in vision transformers. Due to this stringent requirement, deploying
adversarial patches for vision transformers in the physical world becomes
impractical, unlike their effectiveness on CNNs. This paper proposes a novel
method for generating an adversarial patch (G-Patch) that overcomes the
alignment constraint, allowing the patch to launch a targeted attack at any
position within the field of view. Specifically, instead of directly optimizing
the patch using gradients, we employ a GAN-like structure to generate the
adversarial patch. Our experiments show the effectiveness of the adversarial
patch in achieving universal attacks on vision transformers, both in digital
and physical-world scenarios. Additionally, further analysis reveals that the
generated adversarial patch exhibits robustness to brightness restriction,
color transfer, and random noise. Real-world attack experiments validate the
effectiveness of the G-Patch to launch robust attacks even under some very
challenging conditions.

通过使用类似 GAN 的结构，本文提出了一种新的方法生成对抗补丁，该补丁克服了与视觉 Transformer 进行线性投影的贴片完美对齐的约束，使得该补丁能够在视野内的任何位置发动定向攻击，并且无论是在数字环境还是在真实世界的情况下都表现出普遍攻击的有效性。