With the remarkable recent progress on learning deep generative models, it
becomes increasingly interesting to develop models for controllable image
synthesis from reconfigurable inputs. This paper focuses on a recent emerged
task, layout-to-image, to learn generative models that are capable of
synthesizing photo-realistic images from spatial layout (i.e., object bounding
boxes configured in an image lattice) and style (i.e., structural and
appearance variations encoded by latent vectors). This paper first proposes an
intuitive paradigm for the task, layout-to-mask-to-image, to learn to unfold
object masks of given bounding boxes in an input layout to bridge the gap
between the input layout and synthesized images. Then, this paper presents a
method built on Generative Adversarial Networks for the proposed
layout-to-mask-to-image with style control at both image and mask levels.
Object masks are learned from the input layout and iteratively refined along
stages in the generator network. Style control at the image level is the same
as in vanilla GANs, while style control at the object mask level is realized by
a proposed novel feature normalization scheme, Instance-Sensitive and
Layout-Aware Normalization. In experiments, the proposed method is tested in
the COCO-Stuff dataset and the Visual Genome dataset with state-of-the-art
performance obtained.

该论文提出了一种新的范例 (layout-to-mask-to-image) 来实现从给定布局生成物体掩膜的任务，给出了一种通过实例感知和布局感知归一化来实现物体掩膜水平的风格控制，使用生成对抗网络 (GANs) 在两个控制水平上进行风格控制的方法，并在 COCO-Stuff 数据集和 Visual Genome 数据集上获得了最新成果。