Owing to the unrestricted nature of the content in the training data, large
text-to-image diffusion models, such as Stable Diffusion (SD), are capable of
generating images with potentially copyrighted or dangerous content based on
corresponding textual concepts information. This includes specific intellectual
property (IP), human faces, and various artistic styles. However, Negative
Prompt, a widely used method for content removal, frequently fails to conceal
this content due to inherent limitations in its inference logic. In this work,
we propose a novel strategy named \textbf{Degeneration-Tuning (DT)} to shield
contents of unwanted concepts from SD weights. By utilizing Scrambled Grid to
reconstruct the correlation between undesired concepts and their corresponding
image domain, we guide SD to generate meaningless content when such textual
concepts are provided as input. As this adaptation occurs at the level of the
model's weights, the SD, after DT, can be grafted onto other conditional
diffusion frameworks like ControlNet to shield unwanted concepts. In addition
to qualitatively showcasing the effectiveness of our DT method in protecting
various types of concepts, a quantitative comparison of the SD before and after
DT indicates that the DT method does not significantly impact the generative
quality of other contents. The FID and IS scores of the model on COCO-30K
exhibit only minor changes after DT, shifting from 12.61 and 39.20 to 13.04 and
38.25, respectively, which clearly outperforms the previous methods.

通过使用一种名为 Degeneration-Tuning (DT) 的新策略，本研究提出了一种方法来保护文本到图像扩散模型中不需要的概念内容，同时对其他内容的生成质量影响较小。