Customization generation techniques have significantly advanced the synthesis
of specific concepts across varied contexts. Multi-concept customization
emerges as the challenging task within this domain. Existing approaches often
rely on training a Low-Rank Adaptations (LoRA) fusion matrix of multiple LoRA
to merge various concepts into a single image. However, we identify this
straightforward method faces two major challenges: 1) concept confusion, which
occurs when the model cannot preserve distinct individual characteristics, and
2) concept vanishing, where the model fails to generate the intended subjects.
To address these issues, we introduce LoRA-Composer, a training-free framework
designed for seamlessly integrating multiple LoRAs, thereby enhancing the
harmony among different concepts within generated images. LoRA-Composer
addresses concept vanishing through Concept Injection Constraints, enhancing
concept visibility via an expanded cross-attention mechanism. To combat concept
confusion, Concept Isolation Constraints are introduced, refining the
self-attention computation. Furthermore, Latent Re-initialization is proposed
to effectively stimulate concept-specific latent within designated regions. Our
extensive testing showcases a notable enhancement in LoRA-Composer's
performance compared to standard baselines, especially when eliminating the
image-based conditions like canny edge or pose estimations. Code is released at
this https URL

自定义生成技术在不同背景下合成特定概念的技术已经取得了显著进展。多概念自定义是该领域内的一项具有挑战性的任务。现有方法往往依赖于训练多个低秩适应矩阵（LoRA）的融合矩阵，将各种概念合并到单个图像中。然而，我们发现这种直接方法面临两个主要挑战：1）概念混淆，即模型无法保留独特的个体特征，2）概念消失，即模型无法生成预期的主题。为了解决这些问题，我们引入了一种名为 LoRA-Composer 的无需训练的框架，用于无缝集成多个 LoRA，从而增强生成图像中不同概念之间的和谐。LoRA-Composer 通过概念注入约束来解决概念消失问题，通过扩展交叉注意机制增强概念可见性。为了对抗概念混淆，引入了概念隔离约束，对自注意力计算进行了改进。此外，提出了潜在重新初始化方法来有效激发指定区域内的特定概念潜在性。我们的广泛测试显示，与标准基线方法相比，LoRA-Composer 的性能有明显的提升，特别是在消除基于图像的条件（如边缘检测或姿势估计）时。代码发布在此 https URL