Advanced text-to-image models such as DALL-E 2 and Midjourney possess the
capacity to generate highly realistic images, raising significant concerns
regarding the potential proliferation of unsafe content. This includes adult,
violent, or deceptive imagery of political figures. Despite claims of rigorous
safety mechanisms implemented in these models to restrict the generation of
not-safe-for-work (NSFW) content, we successfully devise and exhibit the first
prompt attacks on Midjourney, resulting in the production of abundant
photorealistic NSFW images. We reveal the fundamental principles of such prompt
attacks and suggest strategically substituting high-risk sections within a
suspect prompt to evade closed-source safety measures. Our novel framework,
SurrogatePrompt, systematically generates attack prompts, utilizing large
language models, image-to-text, and image-to-image modules to automate attack
prompt creation at scale. Evaluation results disclose an 88% success rate in
bypassing Midjourney's proprietary safety filter with our attack prompts,
leading to the generation of counterfeit images depicting political figures in
violent scenarios. Both subjective and objective assessments validate that the
images generated from our attack prompts present considerable safety hazards.

我们成功地设计并展示了对 Midjourney 的首个 prompt 攻击，从而生成大量逼真的不适宜工作场所的图像，揭示了此类攻击的基本原理，并建议通过策略性地替换可疑提示中的高风险部分来规避闭源安全措施。我们的新框架 SurrogatePrompt 可以系统地利用大型语言模型、图像到文本和图像到图像模块自动扩展攻击提示的创建。评估结果显示，在我们的攻击提示下，绕过 Midjourney 的专有安全过滤器的成功率达到 88％，导致生成描绘政治人物处于暴力情境中的冒牌图像。主观和客观评估均验证我们的攻击提示生成的图像存在相当大的安全风险。

文本图像模型的安全过滤器绕过方法

SurrogatePrompt: Bypassing the Safety Filter of Text-To-Image Models via  Substitution

AI-based text-to-image generation has undergone a significant leap in the
production of visually comprehensive and aesthetic imagery over the past year,
to the point where differentiating between a man-made piece of art and an
AI-generated image is becoming more difficult. Generative Models such as Stable
Diffusion, Midjourney and others are expected to affect several major
industries in technological and ethical aspects. Striking the balance between
raising human standard of life and work vs exploiting one group of people to
enrich another is a complex and crucial part of the discussion. Due to the
rapid growth of this technology, the way in which its models operate, and gray
area legalities, visual and artistic domains - including the video game
industry, are at risk of being taken over from creators by AI infrastructure
owners. This paper is a literature review examining the concerns facing both AI
developers and users today, including identity theft, data laundering and more.
It discusses legalization challenges and ethical concerns, and concludes with
how AI generative models can be tremendously useful in streamlining the process
of visual creativity in both static and interactive media given proper
regulation.
Keywords: AI text-to-image generation, Midjourney, Stable Diffusion, AI
Ethics, Game Design, Digital Art, Data Laundering

本文是一篇文献综述，探讨了 AI 生成模型在静态和交互媒体中简化视觉创造流程的巨大潜力及其在技术和伦理方面对多个重要产业的影响，涵盖了 AI 文本到图像生成、Midjourney、Stable Diffusion、AI 伦理、游戏设计、数字艺术和数据洗白等方面的关注点和挑战，以及如何通过适当的监管解决相关的法律、伦理等问题。