Large-scale text-to-image models that can generate high-quality and diverse
images based on textual prompts have shown remarkable success. These models aim
ultimately to create complex scenes, and addressing the challenge of
multi-subject generation is a critical step towards this goal. However, the
existing state-of-the-art diffusion models face difficulty when generating
images that involve multiple subjects. When presented with a prompt containing
more than one subject, these models may omit some subjects or merge them
together. To address this challenge, we propose a novel approach based on a
guiding principle. We allow the diffusion model to initially propose a layout,
and then we rearrange the layout grid. This is achieved by enforcing
cross-attention maps (XAMs) to adhere to proposed masks and by migrating pixels
from latent maps to new locations determined by us. We introduce new loss terms
aimed at reducing XAM entropy for clearer spatial definition of subjects,
reduce the overlap between XAMs, and ensure that XAMs align with their
respective masks. We contrast our approach with several alternative methods and
show that it more faithfully captures the desired concepts across a variety of
text prompts.

基于大规模文本到图像模型，我们提出了一种新颖的方法，通过引入交叉注意力图，来解决多主题生成的挑战并提高图像生成的质量和多样性。我们的方法通过重新排列布局网格和引入新的损失项，实现了对生成图像中主题的更清晰定义和避免主题重叠。与其他方法相比，我们的方法在各种文本提示下更准确地捕捉了所需的概念。

多目标生成中获得优秀布局

Obtaining Favorable Layouts for Multiple Object Generation

Text-to-image diffusion models have an unprecedented ability to generate
diverse and high-quality images. However, they often struggle to faithfully
capture the intended semantics of complex input prompts that include multiple
subjects. Recently, numerous layout-to-image extensions have been introduced to
improve user control, aiming to localize subjects represented by specific
tokens. Yet, these methods often produce semantically inaccurate images,
especially when dealing with multiple semantically or visually similar
subjects. In this work, we study and analyze the causes of these limitations.
Our exploration reveals that the primary issue stems from inadvertent semantic
leakage between subjects in the denoising process. This leakage is attributed
to the diffusion model's attention layers, which tend to blend the visual
features of different subjects. To address these issues, we introduce Bounded
Attention, a training-free method for bounding the information flow in the
sampling process. Bounded Attention prevents detrimental leakage among subjects
and enables guiding the generation to promote each subject's individuality,
even with complex multi-subject conditioning. Through extensive
experimentation, we demonstrate that our method empowers the generation of
multiple subjects that better align with given prompts and layouts.

我们研究了文本到图像扩散模型中的语义泄漏问题，并提出了一种名为有界注意力的方法，通过限定信息流的路径来解决泄漏问题，从而提高多主题生成的质量。

做回自己：多主体文本到图像生成的有限关注

Be Yourself: Bounded Attention for Multi-Subject Text-to-Image  Generation

Diffusion models excel at text-to-image generation, especially in
subject-driven generation for personalized images. However, existing methods
are inefficient due to the subject-specific fine-tuning, which is
computationally intensive and hampers efficient deployment. Moreover, existing
methods struggle with multi-subject generation as they often blend features
among subjects. We present FastComposer which enables efficient, personalized,
multi-subject text-to-image generation without fine-tuning. FastComposer uses
subject embeddings extracted by an image encoder to augment the generic text
conditioning in diffusion models, enabling personalized image generation based
on subject images and textual instructions with only forward passes. To address
the identity blending problem in the multi-subject generation, FastComposer
proposes cross-attention localization supervision during training, enforcing
the attention of reference subjects localized to the correct regions in the
target images. Naively conditioning on subject embeddings results in subject
overfitting. FastComposer proposes delayed subject conditioning in the
denoising step to maintain both identity and editability in subject-driven
image generation. FastComposer generates images of multiple unseen individuals
with different styles, actions, and contexts. It achieves
300$\times$-2500$\times$ speedup compared to fine-tuning-based methods and
requires zero extra storage for new subjects. FastComposer paves the way for
efficient, personalized, and high-quality multi-subject image creation. Code,
model, and dataset are available at
this https URL

FastComposer 使用图像编码器提取的主题嵌入来增强扩散模型中的通用文本调节，从而实现了高效的个性化、多主题文本到图像的生成。 FastComposer 通过跨注意力本地化监督和延迟立体雕塑来处理多主题生成中的身份混合问题，并且相对于基于微调的方法获得了数百到数千倍的加速。