Vision transformers (ViTs) have been successfully deployed in a variety of
computer vision tasks, but they are still vulnerable to adversarial samples.
Transfer-based attacks use a local model to generate adversarial samples and
directly transfer them to attack a target black-box model. The high efficiency
of transfer-based attacks makes it a severe security threat to ViT-based
applications. Therefore, it is vital to design effective transfer-based attacks
to identify the deficiencies of ViTs beforehand in security-sensitive
scenarios. Existing efforts generally focus on regularizing the input gradients
to stabilize the updated direction of adversarial samples. However, the
variance of the back-propagated gradients in intermediate blocks of ViTs may
still be large, which may make the generated adversarial samples focus on some
model-specific features and get stuck in poor local optima. To overcome the
shortcomings of existing approaches, we propose the Token Gradient
Regularization (TGR) method. According to the structural characteristics of
ViTs, TGR reduces the variance of the back-propagated gradient in each internal
block of ViTs in a token-wise manner and utilizes the regularized gradient to
generate adversarial samples. Extensive experiments on attacking both ViTs and
CNNs confirm the superiority of our approach. Notably, compared to the
state-of-the-art transfer-based attacks, our TGR offers a performance
improvement of 8.8% on average.

本篇论文提出了一种名为 Token Gradient Regularization（TGR）的方法，通过在每个内部块中以 token 方式减少 ViTs 反向传播梯度的方差来生成对抗样本，从而攻击 Vision transformers 和 CNNs，并对比现有的迁移攻击方式，在性能上提供了平均 8.8% 的改善。