In this paper, we address the task of semantics-guided image outpainting,
which is to complete an image by generating semantically practical content.
Different from most existing image outpainting works, we approach the above
task by understanding and completing image semantics at the scene graph level.
In particular, we propose a novel network of Scene Graph Transformer (SGT),
which is designed to take node and edge features as inputs for modeling the
associated structural information. To better understand and process graph-based
inputs, our SGT uniquely performs feature attention at both node and edge
levels. While the former views edges as relationship regularization, the latter
observes the co-occurrence of nodes for guiding the attention process. We
demonstrate that, given a partial input image with its layout and scene graph,
our SGT can be applied for scene graph expansion and its conversion to a
complete layout. Following state-of-the-art layout-to-image conversions works,
the task of image outpainting can be completed with sufficient and practical
semantics introduced. Extensive experiments are conducted on the datasets of
MS-COCO and Visual Genome, which quantitatively and qualitatively confirm the
effectiveness of our proposed SGT and outpainting frameworks.

本文提出一种基于场景图转换器的理解和完成图像语义的方法，使用图上的节点和边缘特征进行建模，并采用节点和边缘级别的特征注意力机制，通过 MS-COCO 和 Visual Genome 数据集的实验证明这一模型的有效性及场景图扩展和完感布局转换的实际应用。