Recent advancements in text-to-image generation models have dramatically
enhanced the generation of photorealistic images from textual prompts, leading
to an increased interest in personalized text-to-image applications,
particularly in multi-subject scenarios. However, these advances are hindered
by two main challenges: firstly, the need to accurately maintain the details of
each referenced subject in accordance with the textual descriptions; and
secondly, the difficulty in achieving a cohesive representation of multiple
subjects in a single image without introducing inconsistencies. To address
these concerns, our research introduces the MS-Diffusion framework for
layout-guided zero-shot image personalization with multi-subjects. This
innovative approach integrates grounding tokens with the feature resampler to
maintain detail fidelity among subjects. With the layout guidance, MS-Diffusion
further improves the cross-attention to adapt to the multi-subject inputs,
ensuring that each subject condition acts on specific areas. The proposed
multi-subject cross-attention orchestrates harmonious inter-subject
compositions while preserving the control of texts. Comprehensive quantitative
and qualitative experiments affirm that this method surpasses existing models
in both image and text fidelity, promoting the development of personalized
text-to-image generation.

我们的研究引入了 MS-Diffusion 框架，用于基于布局指导的零样本图像个性化处理多主题，融合了引用令牌和特征重采样以保持细节的准确性，并通过跨主题注意力来实现各自主题条件对特定区域的影响，通过全面的定量和定性实验，证明该方法在图像和文本的准确性方面优于现有模型，推动个性化的文本到图像生成的发展。