Despite the progress in semantic image synthesis, it remains a challenging
problem to generate photo-realistic parts from input semantic map. Integrating
part segmentation map can undoubtedly benefit image synthesis, but is
bothersome and inconvenient to be provided by users. To improve part synthesis,
this paper presents to infer Parts from Object ShapE (iPOSE) and leverage it
for improving semantic image synthesis. However, albeit several part
segmentation datasets are available, part annotations are still not provided
for many object categories in semantic image synthesis. To circumvent it, we
resort to few-shot regime to learn a PartNet for predicting the object part map
with the guidance of pre-defined support part maps. PartNet can be readily
generalized to handle a new object category when a small number (e.g., 3) of
support part maps for this category are provided. Furthermore, part semantic
modulation is presented to incorporate both inferred part map and semantic map
for image synthesis. Experiments show that our iPOSE not only generates objects
with rich part details, but also enables to control the image synthesis
flexibly. And our iPOSE performs favorably against the state-of-the-art methods
in terms of quantitative and qualitative evaluation. Our code will be publicly
available at this https URL

本文提出了 iPOSE 方法，利用预先定义少量的部分支持地图来学习一个 PartNet，从而推断部分并将其应用于语义图像合成，其中还引入了部分语义调制，通过实验结果表明该方法在 qualitativer 和 quantitativer 方面都优于其他方法。