Conditional image synthesis for generating photorealistic images serves
various applications for content editing to content generation. Previous
conditional image synthesis algorithms mostly rely on semantic maps, and often
fail in complex environments where multiple instances occlude each other. We
propose a panoptic aware image synthesis network to generate high fidelity and
photorealistic images conditioned on panoptic maps which unify semantic and
instance information. To achieve this, we efficiently use panoptic maps in
convolution and upsampling layers. We show that with the proposed changes to
the generator, we can improve on the previous state-of-the-art methods by
generating images in complex instance interaction environments in higher
fidelity and tiny objects in more details. Furthermore, our proposed method
also outperforms the previous state-of-the-art methods in metrics of mean IoU
(Intersection over Union), and detAP (Detection Average Precision).

提出一种基于全景图的图像合成网络，通过有效地利用全景图在卷积和上采样层来生成高保真的图像，解决了在复杂环境下多个实例互相遮挡的问题，并且在均值 IoU 和 DetAP 等指标上优于之前的最新技术方法。