This paper presents a new adversarial training framework for image inpainting
with segmentation confusion adversarial training (SCAT) and contrastive
learning. SCAT plays an adversarial game between an inpainting generator and a
segmentation network, which provides pixel-level local training signals and can
adapt to images with free-form holes. By combining SCAT with standard global
adversarial training, the new adversarial training framework exhibits the
following three advantages simultaneously: (1) the global consistency of the
repaired image, (2) the local fine texture details of the repaired image, and
(3) the flexibility of handling images with free-form holes. Moreover, we
propose the textural and semantic contrastive learning losses to stabilize and
improve our inpainting model's training by exploiting the feature
representation space of the discriminator, in which the inpainting images are
pulled closer to the ground truth images but pushed farther from the corrupted
images. The proposed contrastive losses better guide the repaired images to
move from the corrupted image data points to the real image data points in the
feature representation space, resulting in more realistic completed images. We
conduct extensive experiments on two benchmark datasets, demonstrating our
model's effectiveness and superiority both qualitatively and quantitatively.

本文提出了一种新的对抗训练框架来完成带有分割混淆对抗训练（SCAT）和对比学习的图像修复任务。通过将 SCAT 与标准全局对抗训练相结合，该框架同时具有以下三个优点：修复图像的全局一致性、局部细节特征以及处理任意形状的缺陷。此外，我们还提出了纹理和语义对比学习的损失函数来稳定并改进我们的修复模型的训练，更好地指导修复图像从受损的图像数据点移到真实图像数据点。在两个基准数据集上的广泛实验表明，我们的模型在定性和定量上都具有良好的效果和优越性。