Text-to-image (T2I) models have been shown to be vulnerable to misuse, particularly in generating not-safe-for-work (NSFW) content, raising serious ethical concerns. In this work, we present PromptGuard, a novel content moderation technique that draws inspiration from the system prompt mechanism in large language models (LLMs) for safety alignment. Unlike LLMs, T2I models lack a direct interface for enforcing behavioral guidelines. Our key idea is to optimize a safety soft prompt that functions as an implicit system prompt within the T2I model's textual embedding space. This universal soft prompt (P*) directly moderates NSFW inputs, enabling safe yet realistic image generation without altering the inference efficiency or requiring proxy models. Extensive experiments across three datasets demonstrate that PromptGuard effectively mitigates NSFW content generation while preserving high-quality benign outputs. PromptGuard achieves 7.8 times faster than prior content moderation methods, surpassing eight state-of-the-art defenses with an optimal unsafe ratio down to 5.84%.

本研究解决了文本到图像模型生成不安全内容的问题，提出了一种新颖的内容管理技术PromptGuard，灵感来自大型语言模型的系统提示机制。研究表明，PromptGuard能够有效减少不安全内容的生成，同时保持高质量的安全输出，其速度比以前的方法快7.8倍，对比八种最先进的防御方法，其最佳不安全率降低至5.84%。

PromptGuard：软提示引导的文本到图像模型不安全内容的软化管理