We propose PD-GAN, a probabilistic diverse GAN for image inpainting. Given an
input image with arbitrary hole regions, PD-GAN produces multiple inpainting
results with diverse and visually realistic content. Our PD-GAN is built upon a
vanilla GAN which generates images based on random noise. During image
generation, we modulate deep features of input random noise from coarse-to-fine
by injecting an initially restored image and the hole regions in multiple
scales. We argue that during hole filling, the pixels near the hole boundary
should be more deterministic (i.e., with higher probability trusting the
context and initially restored image to create natural inpainting boundary),
while those pixels lie in the center of the hole should enjoy more degrees of
freedom (i.e., more likely to depend on the random noise for enhancing
diversity). To this end, we propose spatially probabilistic diversity
normalization (SPDNorm) inside the modulation to model the probability of
generating a pixel conditioned on the context information. SPDNorm dynamically
balances the realism and diversity inside the hole region, making the generated
content more diverse towards the hole center and resemble neighboring image
content more towards the hole boundary. Meanwhile, we propose a perceptual
diversity loss to further empower PD-GAN for diverse content generation.
Experiments on benchmark datasets including CelebA-HQ, Places2 and Paris Street
View indicate that PD-GAN is effective for diverse and visually realistic image
restoration.

PD-GAN 是一种基于 GAN 的图像修复方法，其利用 SPDNorm 和感知多样性损失来平衡图像修复的真实性和多样性，实验证明其可以生成更多样性和真实性更高的修复图像。