Diffusion models have been widely deployed in various image generation tasks,
demonstrating an extraordinary connection between image and text modalities.
However, they face challenges of being maliciously exploited to generate
harmful or sensitive images by appending a specific suffix to the original
prompt. Existing works mainly focus on using single-modal information to
conduct attacks, which fails to utilize multi-modal features and results in
less than satisfactory performance. Integrating multi-modal priors (MMP), i.e.
both text and image features, we propose a targeted attack method named
MMP-Attack in this work. Specifically, the goal of MMP-Attack is to add a
target object into the image content while simultaneously removing the original
object. The MMP-Attack shows a notable advantage over existing works with
superior universality and transferability, which can effectively attack
commercial text-to-image (T2I) models such as DALL-E 3. To the best of our
knowledge, this marks the first successful attempt of transfer-based attack to
commercial T2I models. Our code is publicly available at
https://github.com/ydc123/MMP-Attack.

采用多模态先验信息，我们提出了一种名为 MMP-Attack 的目标攻击方法，它可以在图像内容中添加目标对象并同时删除原始对象。MMP-Attack 相对于现有方法具有更好的普适性和可迁移性，可以有效地攻击商业文本到图像模型（如 DALL-E 3），这是对商业文本到图像模型进行迁移式攻击的首次成功尝试。

作弊的后缀：基于多模态先验的针对文本到图像扩散模型的定向攻击

Cheating Suffix: Targeted Attack to Text-To-Image Diffusion Models with  Multi-Modal Priors

Deep neural networks are vulnerable to adversarial examples, posing a threat
to the models' applications and raising security concerns. An intriguing
property of adversarial examples is their strong transferability. Several
methods have been proposed to enhance transferability, including ensemble
attacks which have demonstrated their efficacy. However, prior approaches
simply average logits, probabilities, or losses for model ensembling, lacking a
comprehensive analysis of how and why model ensembling significantly improves
transferability. In this paper, we propose a similar targeted attack method
named Similar Target~(ST). By promoting cosine similarity between the gradients
of each model, our method regularizes the optimization direction to
simultaneously attack all surrogate models. This strategy has been proven to
enhance generalization ability. Experimental results on ImageNet validate the
effectiveness of our approach in improving adversarial transferability. Our
method outperforms state-of-the-art attackers on 18 discriminative classifiers
and adversarially trained models.

在这篇论文中，我们提出了一种名为 Similar Target（ST）的类似目标攻击方法，通过促进每个模型梯度的余弦相似性，我们的方法规范化了优化方向，从而同时攻击所有替代模型，验证了我们的方法在提高对抗迁移性方面的有效性。在 ImageNet 上的实验结果证实了我们的方法在提高对抗迁移性方面的有效性，优于 18 个判别分类器和对抗训练模型的最先进攻击方法。