Text-to-image diffusion models particularly Stable Diffusion, have revolutionized the field of computer vision. However, the synthesis quality often deteriorates when asked to generate images that faithfully represent complex prompts involving multiple attributes and objects. While previous studies suggest that blended text embeddings lead to improper attribute binding, few have explored this in depth. In this work, we critically examine the limitations of the CLIP text encoder in understanding attributes and investigate how this affects diffusion models. We discern a phenomenon of attribute bias in the text space and highlight a contextual issue in padding embeddings that entangle different concepts. We propose \textbf{Magnet}, a novel training-free approach to tackle the attribute binding problem. We introduce positive and negative binding vectors to enhance disentanglement, further with a neighbor strategy to increase accuracy. Extensive experiments show that Magnet significantly improves synthesis quality and binding accuracy with negligible computational cost, enabling the generation of unconventional and unnatural concepts.

本研究解决了文本到图像扩散模型在生成复杂提示时的合成质量下降问题，尤其是在属性绑定方面。通过深入分析CLIP文本编码器的局限性，提出了无训练的“磁铁”方法，有效改善属性的解缠问题，从而显著提高了合成质量和绑定准确性，推动了非常规概念的生成。

磁铁：了解视觉-语言模型的运作后，我们才能明白文本到图像扩散模型的工作机制