Semantic Image Synthesis (SIS) is a subclass of image-to-image translation
where a semantic layout is used to generate a photorealistic image.
State-of-the-art conditional Generative Adversarial Networks (GANs) need a huge
amount of paired data to accomplish this task while generic unpaired
image-to-image translation frameworks underperform in comparison, because they
color-code semantic layouts and learn correspondences in appearance instead of
semantic content. Starting from the assumption that a high quality generated
image should be segmented back to its semantic layout, we propose a new
Unsupervised paradigm for SIS (USIS) that makes use of a self-supervised
segmentation loss and whole image wavelet based discrimination. Furthermore, in
order to match the high-frequency distribution of real images, a novel
generator architecture in the wavelet domain is proposed. We test our
methodology on 3 challenging datasets and demonstrate its ability to bridge the
performance gap between paired and unpaired models.

本文提出了一种新的无监督学习范例 (Unsupervised Paradigm for SIS, USIS)，通过使用自监督分割损失和基于整个图像小波的鉴别方法，结合在小波域中的生成器结构，使得语义图像合成 (SIS) 在不需要大量的配对数据的情况下实现，从而弥补了配对和非配对模型之间的性能差距。在三个具有挑战性的数据集上进行测试，证明了新方法的有效性。