Recent advancements in personalizing text-to-image (T2I) diffusion models
have shown the capability to generate images based on personalized visual
concepts using a limited number of user-provided examples. However, these
models often struggle with maintaining high visual fidelity, particularly in
manipulating scenes as defined by textual inputs. Addressing this, we introduce
ComFusion, a novel approach that leverages pretrained models generating
composition of a few user-provided subject images and predefined-text scenes,
effectively fusing visual-subject instances with textual-specific scenes,
resulting in the generation of high-fidelity instances within diverse scenes.
ComFusion integrates a class-scene prior preservation regularization, which
leverages composites the subject class and scene-specific knowledge from
pretrained models to enhance generation fidelity. Additionally, ComFusion uses
coarse generated images, ensuring they align effectively with both the instance
image and scene texts. Consequently, ComFusion maintains a delicate balance
between capturing the essence of the subject and maintaining scene
fidelity.Extensive evaluations of ComFusion against various baselines in T2I
personalization have demonstrated its qualitative and quantitative superiority.

最近在个性化文本到图像（T2I）扩散模型方面的进展表明，使用有限数量的用户提供的示例生成基于个性化视觉概念的图像的能力已被证明。然而，这些模型在维护高视觉保真度方面经常遇到困难，特别是在根据文本输入进行场景操作时。为了解决这个问题，我们引入了 ComFusion，这是一种新颖的方法，利用预训练模型生成几个用户提供的主题图像和预定义文本场景的组合，有效地将视觉主题实例与文本特定场景融合，从而在多样的场景中生成高保真度的实例。ComFusion 整合了一种类别场景先验保留正则化方法，利用预训练模型的组合主题和场景特定知识，增强了生成的保真度。此外，ComFusion 使用粗糙生成的图像，确保它们与实例图像和场景文本有效对齐。因此，ComFusion 在捕捉主题的本质和保持场景保真度之间保持了一个微妙的平衡。对 T2I 个性化中的各种基线进行了广泛评估，证明了 ComFusion 在定性和定量上的优越性。