We present an open-source, real-time implementation of SemanticPaint, a system for geometric reconstruction, object-class segmentation and learning of 3D scenes. Using our system, a user can walk into a room wearing a depth camera and a virtual reality headset, and both densely reconstruct the 3D scene and interactively segment the environment into object classes such as 'chair', 'floor' and 'table'. The user interacts physically with the real-world scene, touching objects and using voice commands to assign them appropriate labels. These user-generated labels are leveraged by an online random forest-based machine learning algorithm, which is used to predict labels for previously unseen parts of the scene. The entire pipeline runs in real time, and the user stays 'in the loop' throughout the process, receiving immediate feedback about the progress of the labelling and interacting with the scene as necessary to refine the predicted segmentation.

本文介绍了SemanticPaint的实时开源实现，它能够实现几何重建、对象类别分割和3D场景学习；用户可以在佩戴深度相机和虚拟现实头戴设备的情况下，与真实场景进行物理交互并为场景中的对象分配标签，使用在线随机森林机器学习算法对先前未见过的场景部分进行预测，整个过程在实时运行中实现。

SemanticPaint：一个用于交互式分割三维场景的框架