Image completion is a task that aims to fill in the missing region of a
masked image with plausible contents.However, existing image completion methods
tend to fill in the missing region with the surrounding texture instead of
hallucinating a visual instance that is suitable in accordance with the context
of the scene. In this work, we propose a novel image completion model, dubbed
Refill, that hallucinates the missing instance that harmonizes well with - and
thus preserves - the original context. Refill first adopts a transformer
architecture that considers the types, locations of the visible instances, and
the location of the missing region. Then, Refill completes the missing
foreground and background semantic segmentation masks within the missing
region, providing pixel-level semantic and structural guidance to generate
missing contents with seamless boundaries. Finally, we condition the image
synthesis blocks of Refill using the completed segmentation mask to generate
photo-realistic contents to fill out the missing region. Experimental results
show the superiority of Refill over state-of-the-art image completion
approaches on various natural images.

本文提出了一种名为 Refill 的基于 Transformer 网络和前景背景语义分割的图像修复模型，通过对原始场景上下文的考虑，生成高质量的缺失部分内容。实验证明该模型优于现有的 state-of-the-art 方法。