Understanding, reasoning, and manipulating semantic concepts of images have
been a fundamental research problem for decades. Previous work mainly focused
on direct manipulation on natural image manifold through color strokes,
key-points, textures, and holes-to-fill. In this work, we present a novel
hierarchical framework for semantic image manipulation. Key to our hierarchical
framework is that we employ a structured semantic layout as our intermediate
representation for manipulation. Initialized with coarse-level bounding boxes,
our structure generator first creates pixel-wise semantic layout capturing the
object shape, object-object interactions, and object-scene relations. Then our
image generator fills in the pixel-level textures guided by the semantic
layout. Such framework allows a user to manipulate images at object-level by
adding, removing, and moving one bounding box at a time. Experimental
evaluations demonstrate the advantages of the hierarchical manipulation
framework over existing image generation and context hole-filing models, both
qualitatively and quantitatively. Benefits of the hierarchical framework are
further demonstrated in applications such as semantic object manipulation,
interactive image editing, and data-driven image manipulation.

本论文提出了一种基于结构化语义布局的图像操作分层框架，使用户可以通过添加、删除和移动一个边界框来对图像进行对象级别的操作。实验证明，该分层框架可以在语义对象操作、交互式图像编辑和数据驱动的图像操作等方面带来优势。