Prior studies have made significant progress in image inpainting guided by
either text or subject image. However, the research on editing with their
combined guidance is still in the early stages. To tackle this challenge, we
present LAR-Gen, a novel approach for image inpainting that enables seamless
inpainting of masked scene images, incorporating both the textual prompts and
specified subjects. Our approach adopts a coarse-to-fine manner to ensure
subject identity preservation and local semantic coherence. The process
involves (i) Locate: concatenating the noise with masked scene image to achieve
precise regional editing, (ii) Assign: employing decoupled cross-attention
mechanism to accommodate multi-modal guidance, and (iii) Refine: using a novel
RefineNet to supplement subject details. Additionally, to address the issue of
scarce training data, we introduce a novel data construction pipeline. This
pipeline extracts substantial pairs of data consisting of local text prompts
and corresponding visual instances from a vast image dataset, leveraging
publicly available large models. Extensive experiments and varied application
scenarios demonstrate the superiority of LAR-Gen in terms of both identity
preservation and text semantic consistency. Project page can be found at
https://ali-vilab.github.io/largen-page/.

LAR-Gen 是一种图像修复方法，可以无缝地修复遮挡的场景图像，结合了文本提示和指定主题，通过粗到细的方式保证主题身份保持和本地语义一致性，并引入了一种新的数据构建流程来解决稀缺训练数据的问题，广泛的实验和各种应用场景展示了 LAR-Gen 在保持身份和文本语义一致性方面的优越性。