With the unprecedented performance being achieved by text-to-image (T2I)
diffusion models, T2I customization further empowers users to tailor the
diffusion model to new concepts absent in the pre-training dataset, termed
subject-driven generation. Moreover, extracting several new concepts from a
single image enables the model to learn multiple concepts, and simultaneously
decreases the difficulties of training data preparation, urging the
disentanglement of multiple concepts to be a new challenge. However, existing
models for disentanglement commonly require pre-determined masks or retain
background elements. To this end, we propose an attention-guided method,
AttenCraft, for multiple concept disentanglement. In particular, our method
leverages self-attention and cross-attention maps to create accurate masks for
each concept within a single initialization step, omitting any required mask
preparation by humans or other models. The created masks are then applied to
guide the cross-attention activation of each target concept during training and
achieve concept disentanglement. Additionally, we introduce Uniform sampling
and Reweighted sampling schemes to alleviate the non-synchronicity of feature
acquisition from different concepts, and improve generation quality. Our method
outperforms baseline models in terms of image-alignment, and behaves comparably
on text-alignment. Finally, we showcase the applicability of AttenCraft to more
complicated settings, such as an input image containing three concepts. The
project is available at this https URL

使用 AttenCraft 方法，我们提出了一种基于注意力引导的多概念解缠方法，它使用了自注意力和交叉注意力图来生成精确的概念掩模，并在训练过程中应用于每个目标概念的交叉注意力激活，从而在图像对齐方面优于基线模型，并在文本对齐方面表现相当。