Existing 3D instance segmentation methods typically assume that all semantic
classes to be segmented would be available during training and only seen
categories are segmented at inference. We argue that such a closed-world
assumption is restrictive and explore for the first time 3D indoor instance
segmentation in an open-world setting, where the model is allowed to
distinguish a set of known classes as well as identify an unknown object as
unknown and then later incrementally learning the semantic category of the
unknown when the corresponding category labels are available. To this end, we
introduce an open-world 3D indoor instance segmentation method, where an
auto-labeling scheme is employed to produce pseudo-labels during training and
induce separation to separate known and unknown category labels. We further
improve the pseudo-labels quality at inference by adjusting the unknown class
probability based on the objectness score distribution. We also introduce
carefully curated open-world splits leveraging realistic scenarios based on
inherent object distribution, region-based indoor scene exploration and
randomness aspect of open-world classes. Extensive experiments reveal the
efficacy of the proposed contributions leading to promising open-world 3D
instance segmentation performance.

已有的 3D 实例分割方法通常假设在训练过程中可获取所有待分割的语义类别，并在推理时仅分割已见类别。我们认为这种封闭世界的假设过于限制，首次探索了在开放世界环境中进行 3D 室内实例分割，使模型能够区分已知类别，并将未知对象标注为未知，并在相应的类别标签可用时逐步学习未知对象的语义类别。为此，我们引入了一种开放世界的 3D 室内实例分割方法，其中采用自动标注方案在训练过程中生成伪标签，并引入分离机制来区分已知和未知类别标签。我们还通过基于目标性得分分布调整未知类别概率的方法，在推理过程中改进了伪标签的质量。我们还引入了基于真实场景的谨慎策划的开放世界数据集，根据物体分布、基于区域的室内场景探索和开放世界类别的随机性方面。广泛的实验表明了所提出的方法有效性，为开放世界 3D 实例分割性能带来了希望。

开放世界中的三维室内实例分割

3D Indoor Instance Segmentation in an Open-World

Humans have a natural instinct to identify unknown object instances in their
environments. The intrinsic curiosity about these unknown instances aids in
learning about them, when the corresponding knowledge is eventually available.
This motivates us to propose a novel computer vision problem called: `Open
World Object Detection', where a model is tasked to: 1) identify objects that
have not been introduced to it as `unknown', without explicit supervision to do
so, and 2) incrementally learn these identified unknown categories without
forgetting previously learned classes, when the corresponding labels are
progressively received. We formulate the problem, introduce a strong evaluation
protocol and provide a novel solution, which we call ORE: Open World Object
Detector, based on contrastive clustering and energy based unknown
identification. Our experimental evaluation and ablation studies analyze the
efficacy of ORE in achieving Open World objectives. As an interesting
by-product, we find that identifying and characterizing unknown instances helps
to reduce confusion in an incremental object detection setting, where we
achieve state-of-the-art performance, with no extra methodological effort. We
hope that our work will attract further research into this newly identified,
yet crucial research direction.

本文介绍了一个新的计算机视觉问题：“开放领域物体检测”，其中模型的任务是在未经明确监督的情况下识别未引入的对象，并在逐步接收相应标签时增量学习这些已识别的未知类别。阐述了该问题，并提供了一种名为 ORE 的解决方案，基于对比聚类和基于能源的未知物体识别。我们的实验评估和消融研究分析了 ORE 在实现开放世界目标方面的功效。同时，我们发现确定和表征未知实例有助于减少增量对象检测设置中的混淆，我们在没有额外的方法论努力下实现了最先进的性能。希望我们的工作能吸引更多关于此全新但至关重要的研究方向的研究。