Despite the progress in semantic image synthesis, it remains a challenging
problem to generate photo-realistic parts from input semantic map. Integrating
part segmentation map can undoubtedly benefit image synthesis, but is
bothersome and inconvenient to be provided by users. To improve part synthesis,
this paper presents to infer Parts from Object ShapE (iPOSE) and leverage it
for improving semantic image synthesis. However, albeit several part
segmentation datasets are available, part annotations are still not provided
for many object categories in semantic image synthesis. To circumvent it, we
resort to few-shot regime to learn a PartNet for predicting the object part map
with the guidance of pre-defined support part maps. PartNet can be readily
generalized to handle a new object category when a small number (e.g., 3) of
support part maps for this category are provided. Furthermore, part semantic
modulation is presented to incorporate both inferred part map and semantic map
for image synthesis. Experiments show that our iPOSE not only generates objects
with rich part details, but also enables to control the image synthesis
flexibly. And our iPOSE performs favorably against the state-of-the-art methods
in terms of quantitative and qualitative evaluation. Our code will be publicly
available at this https URL

本文提出了 iPOSE 方法，利用预先定义少量的部分支持地图来学习一个 PartNet，从而推断部分并将其应用于语义图像合成，其中还引入了部分语义调制，通过实验结果表明该方法在 qualitativer 和 quantitativer 方面都优于其他方法。

从物体形状推断和利用部件以提高语义图像合成

Inferring and Leveraging Parts from Object Shape for Improving Semantic  Image Synthesis

Single-image 3D shape reconstruction is an important and long-standing
problem in computer vision. A plethora of existing works is constantly pushing
the state-of-the-art performance in the deep learning era. However, there
remains a much more difficult and under-explored issue on how to generalize the
learned skills over unseen object categories that have very different shape
geometry distributions. In this paper, we bring in the concept of compositional
generalizability and propose a novel framework that could better generalize to
these unseen categories. We factorize the 3D shape reconstruction problem into
proper sub-problems, each of which is tackled by a carefully designed neural
sub-module with generalizability concerns. The intuition behind our formulation
is that object parts (slates and cylindrical parts), their relationships
(adjacency and translation symmetry), and shape substructures (T-junctions and
a symmetric group of parts) are mostly shared across object categories, even
though object geometries may look very different (e.g. chairs and cabinets).
Experiments on PartNet show that we achieve superior performance than
state-of-the-art. This validates our problem factorization and network designs.

本文在单目图像 3D 形状重建领域引入了组合通用性的概念，并提出了一个新的框架来 better generalize 到那些有着非常不同的形状几何分布的看不见的目标类别。实验证明，我们的方法在 PartNet 中达到了比最先进的方法更高的性能，验证了我们的问题分解和网络设计的有效性。