Recognizing 3D part instances from a 3D point cloud is crucial for 3D
structure and scene understanding. Several learning-based approaches use
semantic segmentation and instance center prediction as training tasks and fail
to further exploit the inherent relationship between shape semantics and part
instances. In this paper, we present a new method for 3D part instance
segmentation. Our method exploits semantic segmentation to fuse nonlocal
instance features, such as center prediction, and further enhances the fusion
scheme in a multi- and cross-level way. We also propose a semantic region
center prediction task to train and leverage the prediction results to improve
the clustering of instance points. Our method outperforms existing methods with
a large-margin improvement in the PartNet benchmark. We also demonstrate that
our feature fusion scheme can be applied to other existing methods to improve
their performance in indoor scene instance segmentation tasks.

本文提出了一种基于语义分割和实例中心预测的方法，进一步利用形状语义学和部件实例之间的内在关系，同时利用多个层次的特征融合以及语义区域中心预测任务来提高实例点的聚类性能，该方法在 PartNet 基准测试中表现优异。

多级三维零件实例分割的语义分割辅助实例特征融合

Semantic Segmentation-Assisted Instance Feature Fusion for Multi-Level  3D Part Instance Segmentation

We present a novel end-to-end single-shot method that segments countable
object instances (things) as well as background regions (stuff) into a
non-overlapping panoptic segmentation at almost video frame rate. Current
state-of-the-art methods are far from reaching video frame rate and mostly rely
on merging instance segmentation with semantic background segmentation, making
them impractical to use in many applications such as robotics. Our approach
relaxes this requirement by using an object detector but is still able to
resolve inter- and intra-class overlaps to achieve a non-overlapping
segmentation. On top of a shared encoder-decoder backbone, we utilize multiple
branches for semantic segmentation, object detection, and instance center
prediction. Finally, our panoptic head combines all outputs into a panoptic
segmentation and can even handle conflicting predictions between branches as
well as certain false predictions. Our network achieves 32.6% PQ on MS-COCO at
23.5 FPS, opening up panoptic segmentation to a broader field of applications.

本文提出了一种全新的端到端单次方法，可以将可计数的物体实例 (事物) 以及背景区域 (物料) 分段成为一个几乎达到视频帧率的非重叠全景分割。