Have you ever imagined how it would look if we placed new objects into paintings? For example, what would it look like if we placed a basketball into Claude Monet's ``Water Lilies, Evening Effect''? We propose Reference-based Painterly Inpainting, a novel task that crosses the wild reference domain gap and implants novel objects into artworks. Although previous works have examined reference-based inpainting, they are not designed for large domain discrepancies between the target and the reference, such as inpainting an artistic image using a photorealistic reference. This paper proposes a novel diffusion framework, dubbed RefPaint, to ``inpaint more wildly'' by taking such references with large domain gaps. Built with an image-conditioned diffusion model, we introduce a ladder-side branch and a masked fusion mechanism to work with the inpainting mask. By decomposing the CLIP image embeddings at inference time, one can manipulate the strength of semantic and style information with ease. Experiments demonstrate that our proposed RefPaint framework produces significantly better results than existing methods. Our method enables creative painterly image inpainting with reference objects that would otherwise be difficult to achieve. Project page: https://vita-group.github.io/RefPaint/

提出了一种新颖的基于参考的画家绘画修复任务，使用扩散框架RefPaint，通过引入图像条件的扩散模型、阶梯分支和掩模融合机制，以及分解CLIP图像嵌入来操作语义和风格信息的强度，实现了对艺术作品中大领域差异的参考对象进行修复，优于现有方法，实现了具有创造性的绘画图像修复。

基于参考画作的喷绘修复方法：通过扩展参考画领域填补差距