Image editing has advanced significantly with the introduction of
text-conditioned diffusion models. Despite this progress, seamlessly adding
objects to images based on textual instructions without requiring user-provided
input masks remains a challenge. We address this by leveraging the insight that
removing objects (Inpaint) is significantly simpler than its inverse process of
adding them (Paint), attributed to the utilization of segmentation mask
datasets alongside inpainting models that inpaint within these masks.
Capitalizing on this realization, by implementing an automated and extensive
pipeline, we curate a filtered large-scale image dataset containing pairs of
images and their corresponding object-removed versions. Using these pairs, we
train a diffusion model to inverse the inpainting process, effectively adding
objects into images. Unlike other editing datasets, ours features natural
target images instead of synthetic ones; moreover, it maintains consistency
between source and target by construction. Additionally, we utilize a large
Vision-Language Model to provide detailed descriptions of the removed objects
and a Large Language Model to convert these descriptions into diverse,
natural-language instructions. We show that the trained model surpasses
existing ones both qualitatively and quantitatively, and release the
large-scale dataset alongside the trained models for the community.

通过利用分割蒙版数据集和在蒙版内修复的修复模型，我们可以实现自动逆向修复过程，将对象添加到图像中，在没有用户提供的输入蒙版的情况下，提供了一种无缝地根据文本指令添加对象的方法。