Semantic image synthesis (SIS) aims to generate realistic images that match
given semantic masks. Despite recent advances allowing high-quality results and
precise spatial control, they require a massive semantic segmentation dataset
for training the models. Instead, we propose to employ a pre-trained
unconditional generator and rearrange its feature maps according to proxy
masks. The proxy masks are prepared from the feature maps of random samples in
the generator by simple clustering. The feature rearranger learns to rearrange
original feature maps to match the shape of the proxy masks that are either
from the original sample itself or from random samples. Then we introduce a
semantic mapper that produces the proxy masks from various input conditions
including semantic masks. Our method is versatile across various applications
such as free-form spatial editing of real images, sketch-to-photo, and even
scribble-to-photo. Experiments validate advantages of our method on a range of
datasets: human faces, animal faces, and buildings.

通过使用预训练的无条件生成器和重新排列特征图来生成与给定语义掩模相匹配的逼真图像。该方法在各种应用中表现出优势，包括真实图像的自由空间编辑，素描转照片，甚至涂鸦转照片。