Recent advancements in Text-to-Image (T2I) models have raised significant
safety concerns about their potential misuse for generating inappropriate or
Not-Safe-For-Work (NSFW) contents, despite existing countermeasures such as
NSFW classifiers or model fine-tuning for inappropriate concept removal.
Addressing this challenge, our study unveils GuardT2I, a novel moderation
framework that adopts a generative approach to enhance T2I models' robustness
against adversarial prompts. Instead of making a binary classification,
GuardT2I utilizes a Large Language Model (LLM) to conditionally transform text
guidance embeddings within the T2I models into natural language for effective
adversarial prompt detection, without compromising the models' inherent
performance. Our extensive experiments reveal that GuardT2I outperforms leading
commercial solutions like OpenAI-Moderation and Microsoft Azure Moderator by a
significant margin across diverse adversarial scenarios.

通过使用生成方法增强文本到图像模型的鲁棒性，GuardT2I 框架在对抗性场景下显著优于 OpenAI-Moderation 和 Microsoft Azure Moderator 等商业解决方案。