Learning to insert an object instance into an image in a semantically
coherent manner is a challenging and interesting problem. Solving it requires
(a) determining a location to place an object in the scene and (b) determining
its appearance at the location. Such an object insertion model can potentially
facilitate numerous image editing and scene parsing applications. In this
paper, we propose an end-to-end trainable neural network for the task of
inserting an object instance mask of a specified class into the semantic label
map of an image. Our network consists of two generative modules where one
determines where the inserted object mask should be (i.e., location and scale)
and the other determines what the object mask shape (and pose) should look
like. The two modules are connected together via a spatial transformation
network and jointly trained. We devise a learning procedure that leverage both
supervised and unsupervised data and show our model can insert an object at
diverse locations with various appearances. We conduct extensive experimental
validations with comparisons to strong baselines to verify the effectiveness of
the proposed network.

本文提出了一种基于端到端可训练神经网络的方法，以在图像的语义标签地图中插入特定类别的对象实例。该神经网络由两个生成模块组成，一个用于确定插入的对象掩码位置和比例，另一个用于确定对象掩码形状和姿态。两个模块由一个空间变换网络连接并联合训练，该学习过程利用了监督和无监督数据集，实验证明该模型能够在图像中的多个位置插入不同外观的物体，并具有较好的准确性。