Despite some exciting progress on high-quality image generation from
structured(scene graphs) or free-form(sentences) descriptions, most of them
only guarantee the image-level semantical consistency, i.e. the generated image
matching the semantic meaning of the description. They still lack the
investigations on synthesizing the images in a more controllable way, like
finely manipulating the visual appearance of every object. Therefore, to
generate the images with preferred objects and rich interactions, we propose a
semi-parametric method, PasteGAN, for generating the image from the scene graph
and the image crops, where spatial arrangements of the objects and their
pair-wise relationships are defined by the scene graph and the object
appearances are determined by the given object crops. To enhance the
interactions of the objects in the output, we design a Crop Refining Network
and an Object-Image Fuser to embed the objects as well as their relationships
into one map. Multiple losses work collaboratively to guarantee the generated
images highly respecting the crops and complying with the scene graphs while
maintaining excellent image quality. A crop selector is also proposed to pick
the most-compatible crops from our external object tank by encoding the
interactions around the objects in the scene graph if the crops are not
provided. Evaluated on Visual Genome and COCO-Stuff dataset, our proposed
method significantly outperforms the SOTA methods on Inception Score, Diversity
Score and Fr\'echet Inception Distance. Extensive experiments also demonstrate
our method's ability to generate complex and diverse images with given objects.

提出了一种名为 PasteGAN 的半参数方法，结合场景图和图像裁剪来生成具有所需对象和丰富交互的图像，使用 Crop Refining Network 和 Object-Image Fuser 设计算法来增强对象间的交互，并设计了 Crop Selector 来选择最兼容的对象进行生成。在 Visual Genome 和 COCO-Stuff 数据集上通过评估表明，该方法显著优于 SOTA 方法。