This work addresses the task of open world semantic segmentation using RGBD
sensing to discover new semantic classes over time. Although there are many
types of objects in the real-word, current semantic segmentation methods make a
closed world assumption and are trained only to segment a limited number of
object classes. Towards a more open world approach, we propose a novel method
that incrementally learns new classes for image segmentation. The proposed
system first segments each RGBD frame using both color and geometric
information, and then aggregates that information to build a single segmented
dense 3D map of the environment. The segmented 3D map representation is a key
component of our approach as it is used to discover new object classes by
identifying coherent regions in the 3D map that have no semantic label. The use
of coherent region in the 3D map as a primitive element, rather than
traditional elements such as surfels or voxels, also significantly reduces the
computational complexity and memory use of our method. It thus leads to
semi-real-time performance at {10.7}Hz when incrementally updating the dense 3D
map at every frame. Through experiments on the NYUDv2 dataset, we demonstrate
that the proposed method is able to correctly cluster objects of both known and
unseen classes. We also show the quantitative comparison with the
state-of-the-art supervised methods, the processing time of each step, and the
influences of each component.

本文提出了一种使用 RGBD sensing 的增量学习方法来进行开放式世界的语义分割，使用三维地图表示方法来标记没有语义标签的区域，实现对新的物体类别的识别及聚类。通过实验验证，该方法能够正确聚类已知和未知类别的对象，并且比现有的监督式方法具有更快的处理时间及更高的性能。

基于 RGBD 感知的语义分割增量式类别发现

Incremental Class Discovery for Semantic Segmentation with RGBD Sensing

The task of a visual landmark recognition system is to identify photographed
buildings or objects in query photos and to provide the user with relevant
information on them. With their increasing coverage of the world's landmark
buildings and objects, Internet photo collections are now being used as a
source for building such systems in a fully automatic fashion. This process
typically consists of three steps: clustering large amounts of images by the
objects they depict; determining object names from user-provided tags; and
building a robust, compact, and efficient recognition index. To this date,
however, there is little empirical information on how well current approaches
for those steps perform in a large-scale open-set mining and recognition task.
Furthermore, there is little empirical information on how recognition
performance varies for different types of landmark objects and where there is
still potential for improvement. With this paper, we intend to fill these gaps.
Using a dataset of 500k images from Paris, we analyze each component of the
landmark recognition pipeline in order to answer the following questions: How
many and what kinds of objects can be discovered automatically? How can we best
use the resulting image clusters to recognize the object in a query? How can
the object be efficiently represented in memory for recognition? How reliably
can semantic information be extracted? And finally: What are the limiting
factors in the resulting pipeline from query to semantics? We evaluate how
different choices of methods and parameters for the individual pipeline steps
affect overall system performance and examine their effects for different query
categories such as buildings, paintings or sculptures.

探讨如何在全自动化的情况下，通过图片聚类，用户提供的标签以及建立高效的识别索引，对地标建筑和物体进行识别。通过对 500k 张巴黎图片数据集的分析，我们评估了不同的识别结果和方法的系统性能，以及他们对建筑、绘画或雕塑等不同类型的查询所产生的影响。