We present a generative image inpainting system to complete images with
free-form mask and guidance. The system is based on gated convolutions learned
from millions of images without additional labelling efforts. The proposed
gated convolution solves the issue of vanilla convolution that treats all input
pixels as valid ones, generalizes partial convolution by providing a learnable
dynamic feature selection mechanism for each channel at each spatial location
across all layers. Moreover, as free-form masks may appear anywhere in images
with any shape, global and local GANs designed for a single rectangular mask
are not applicable. Thus, we also present a patch-based GAN loss, named
SN-PatchGAN, by applying spectral-normalized discriminator on dense image
patches. SN-PatchGAN is simple in formulation, fast and stable in training.
Results on automatic image inpainting and user-guided extension demonstrate
that our system generates higher-quality and more flexible results than
previous methods. Our system helps user quickly remove distracting objects,
modify image layouts, clear watermarks and edit faces. Code, demo and models
are available at: this https URL

本文提出了一种建基于门控卷积的生成式图像修复系统，用于完成自由形状掩码和引导的图片修复任务。所提出的门控卷积解决了通用卷积中将所有输入像素视为有效像素的问题，并提供了一种可学习的动态特征选择机制，以适应于每个通道和空间位置上的输入。作者还提出了一种名为 SN-PatchGAN 的基于图像块的 GAN 损失，可以应用于自由形状掩码和根据用户指导进行图像扩展，并证明所提出的方法比以往任何方法都更加高质量和灵活性。