Existing point cloud semantic segmentation networks cannot identify unknown
classes and update their knowledge, due to a closed-set and static perspective
of the real world, which would induce the intelligent agent to make bad
decisions. To address this problem, we propose a Probability-Driven Framework
(PDF) for open world semantic segmentation that includes (i) a lightweight
U-decoder branch to identify unknown classes by estimating the uncertainties,
(ii) a flexible pseudo-labeling scheme to supply geometry features along with
probability distribution features of unknown classes by generating pseudo
labels, and (iii) an incremental knowledge distillation strategy to incorporate
novel classes into the existing knowledge base gradually. Our framework enables
the model to behave like human beings, which could recognize unknown objects
and incrementally learn them with the corresponding knowledge. Experimental
results on the S3DIS and ScanNetv2 datasets demonstrate that the proposed PDF
outperforms other methods by a large margin in both important tasks of open
world semantic segmentation.

提出了一种概率驱动框架（PDF）用于开放世界语义分割，在点云语义分割网络中引入轻量级的 U-decoder 分支来识别未知类别，并通过生成伪标签提供未知类别的几何特征和概率分布特征，再通过增量式知识蒸馏策略逐渐将新类别纳入现有知识库，使模型能够像人一样识别未知对象并逐步学习相关知识，实验结果表明，该 PDF 方法在开放世界语义分割的重要任务中大幅优于其他方法。

基于概率驱动的开放世界三维点云语义分割框架

PDF: A Probability-Driven Framework for Open World 3D Point Cloud  Semantic Segmentation

This work addresses the task of open world semantic segmentation using RGBD
sensing to discover new semantic classes over time. Although there are many
types of objects in the real-word, current semantic segmentation methods make a
closed world assumption and are trained only to segment a limited number of
object classes. Towards a more open world approach, we propose a novel method
that incrementally learns new classes for image segmentation. The proposed
system first segments each RGBD frame using both color and geometric
information, and then aggregates that information to build a single segmented
dense 3D map of the environment. The segmented 3D map representation is a key
component of our approach as it is used to discover new object classes by
identifying coherent regions in the 3D map that have no semantic label. The use
of coherent region in the 3D map as a primitive element, rather than
traditional elements such as surfels or voxels, also significantly reduces the
computational complexity and memory use of our method. It thus leads to
semi-real-time performance at {10.7}Hz when incrementally updating the dense 3D
map at every frame. Through experiments on the NYUDv2 dataset, we demonstrate
that the proposed method is able to correctly cluster objects of both known and
unseen classes. We also show the quantitative comparison with the
state-of-the-art supervised methods, the processing time of each step, and the
influences of each component.

本文提出了一种使用 RGBD sensing 的增量学习方法来进行开放式世界的语义分割，使用三维地图表示方法来标记没有语义标签的区域，实现对新的物体类别的识别及聚类。通过实验验证，该方法能够正确聚类已知和未知类别的对象，并且比现有的监督式方法具有更快的处理时间及更高的性能。