This study introduces Text-Guided Subject-Driven Image Inpainting, a novel
task that combines text and exemplar images for image inpainting. While both
text and exemplar images have been used independently in previous efforts,
their combined utilization remains unexplored. Simultaneously accommodating
both conditions poses a significant challenge due to the inherent balance
required between editability and subject fidelity. To tackle this challenge, we
propose a two-step approach DreamInpainter. First, we compute dense subject
features to ensure accurate subject replication. Then, we employ a
discriminative token selection module to eliminate redundant subject details,
preserving the subject's identity while allowing changes according to other
conditions such as mask shape and text prompts. Additionally, we introduce a
decoupling regularization technique to enhance text control in the presence of
exemplar images. Our extensive experiments demonstrate the superior performance
of our method in terms of visual quality, identity preservation, and text
control, showcasing its effectiveness in the context of text-guided
subject-driven image inpainting.

本研究介绍了一种新颖的任务 —— 基于文本导向的主题驱动图像修复，它将文本和示例图像结合起来进行图像修复。通过同时适应这两个条件，我们提出了一种两步法的方法 ——DreamInpainter，该方法通过计算密集的主题特征来确保准确的主题复制，并利用一个有区分性的令牌选择模块来消除多余的主题细节，保留主题的身份，同时允许根据其他条件（如遮罩形状和文本提示）进行更改。此外，我们引入了一种解耦正则化技术，以增强在存在示例图像的情况下对文本的控制。我们广泛的实验证明了我们的方法在视觉质量、身份保护和文本控制方面的卓越性能，展示了它在文本导向的主题驱动图像修复的情境中的有效性。