Text-to-image diffusion models have advanced towards more controllable
generation via supporting various image conditions (e.g., depth map) beyond
text. However, these models are learned based on the premise of perfect
alignment between the text and image conditions. If this alignment is not
satisfied, the final output could be either dominated by one condition, or
ambiguity may arise, failing to meet user expectations. To address this issue,
we present a training-free approach called "Decompose and Realign'' to further
improve the controllability of existing models when provided with partially
aligned conditions. The ``Decompose'' phase separates conditions based on pair
relationships, computing scores individually for each pair. This ensures that
each pair no longer has conflicting conditions. The "Realign'' phase aligns
these independently calculated scores via a cross-attention mechanism to avoid
new conflicts when combing them back. Both qualitative and quantitative results
demonstrate the effectiveness of our approach in handling unaligned conditions,
which performs favorably against recent methods and more importantly adds
flexibility to the controllable image generation process.

本文提出了一种名为 “分解和重新对齐” 的无需训练的方法，通过分离存在于图像和文本之间的对应关系来改善现有模型的可控性，从而实现对部分对齐条件的控制。我们采用交叉注意机制来避免在重新组合分别计算的分数时产生新的冲突，在定性和定量结果上都证明了我们方法的有效性。