Semantic image synthesis is a process for generating photorealistic images
from a single semantic mask. To enrich the diversity of multimodal image
synthesis, previous methods have controlled the global appearance of an output
image by learning a single latent space. However, a single latent code is often
insufficient for capturing various object styles because object appearance
depends on multiple factors. To handle individual factors that determine object
styles, we propose a class- and layer-wise extension to the variational
autoencoder (VAE) framework that allows flexible control over each object class
at the local to global levels by learning multiple latent spaces. Furthermore,
we demonstrate that our method generates images that are both plausible and
more diverse compared to state-of-the-art methods via extensive experiments
with real and synthetic datasets inthree different domains. We also show that
our method enables a wide range of applications in image synthesis and editing
tasks.

本文提出了一种基于变分自编码器的类别和层次的扩展方法，能够通过学习多个潜在空间在本地和全局水平上灵活地控制每个对象类别以处理确定对象样式的各种因素，生成的图像在真实数据集、合成数据集三个不同领域的广泛实验中都表现出更加合理、更具多样性的特点，可用于图像合成和编辑任务。