Despite significant recent progress on generative models, controlled
generation of images depicting multiple and complex object layouts is still a
difficult problem. Among the core challenges are the diversity of appearance a
given object may possess and, as a result, exponential set of images consistent
with a specified layout. To address these challenges, we propose a novel
approach for layout-based image generation; we call it Layout2Im. Given the
coarse spatial layout (bounding boxes + object categories), our model can
generate a set of realistic images which have the correct objects in the
desired locations. The representation of each object is disentangled into a
specified/certain part (category) and an unspecified/uncertain part
(appearance). The category is encoded using a word embedding and the appearance
is distilled into a low-dimensional vector sampled from a normal distribution.
Individual object representations are composed together using convolutional
LSTM, to obtain an encoding of the complete layout, and then decoded to an
image. Several loss terms are introduced to encourage accurate and diverse
generation. The proposed Layout2Im model significantly outperforms the previous
state of the art, boosting the best reported inception score by 24.66% and
28.57% on the very challenging COCO-Stuff and Visual Genome datasets,
respectively. Extensive experiments also demonstrate our method's ability to
generate complex and diverse images with multiple objects.

Layout2Im 是一种基于布局的图像生成方法，将物体表示分解为指定和不确定部分，并使用卷积 LSTM 编码并解码整个布局，以生成准确且多样化的图像，提高了最佳 Inception 分数 24.66％至 28.57％。