In this work we demonstrate the vulnerability of vision transformers (ViTs)
to gradient-based inversion attacks. During this attack, the original data
batch is reconstructed given model weights and the corresponding gradients. We
introduce a method, named GradViT, that optimizes random noise into naturally
looking images via an iterative process. The optimization objective consists of
(i) a loss on matching the gradients, (ii) image prior in the form of distance
to batch-normalization statistics of a pretrained CNN model, and (iii) a total
variation regularization on patches to guide correct recovery locations. We
propose a unique loss scheduling function to overcome local minima during
optimization. We evaluate GadViT on ImageNet1K and MS-Celeb-1M datasets, and
observe unprecedentedly high fidelity and closeness to the original (hidden)
data. During the analysis we find that vision transformers are significantly
more vulnerable than previously studied CNNs due to the presence of the
attention mechanism. Our method demonstrates new state-of-the-art results for
gradient inversion in both qualitative and quantitative metrics. Project page
at this https URL.

该论文展示了视觉转换器（ViTs）对基于渐变的反演攻击的易受攻击性。作者提供了一种名为 GradViT 的方法，可以将随机噪声优化为自然图像，以通过迭代过程重建原始数据批次。作者发现 Vision Transformers 由于注意机制的存在，比之前研究过的 CNNs 容易受到攻击。作者的方法具有卓越的定量与定性表现。