Text-conditioned image generation models have recently achieved astonishing
image quality and alignment results. Consequently, they are employed in a
fast-growing number of applications. Since they are highly data-driven, relying
on billion-sized datasets randomly scraped from the web, they also produce
unsafe content. As a contribution to the Adversarial Nibbler challenge, we
distill a large set of over 1,000 potential adversarial inputs from existing
safety benchmarks. Our analysis of the gathered prompts and corresponding
images demonstrates the fragility of input filters and provides further
insights into systematic safety issues in current generative image models.

文本条件的图像生成模型在图像质量和对齐性方面取得了惊人的结果，然而它们依赖于从网络随机获取的数量庞大的数据集，因此也会生成不安全的内容。作为对 Adversarial Nibbler 挑战的贡献，我们从现有的安全基准中提取了超过 1,000 个潜在对抗性输入，通过对收集到的提示和相应的图像进行分析，揭示了输入过滤器的脆弱性并进一步深入研究了当前生成图像模型中的系统安全问题。