We propose a novel approach to robot-operated active understanding of unknown
indoor scenes, based on online RGBD reconstruction with semantic segmentation.
In our method, the exploratory robot scanning is both driven by and targeting
at the recognition and segmentation of semantic objects from the scene. Our
algorithm is built on top of the volumetric depth fusion framework (e.g.,
KinectFusion) and performs real-time voxel-based semantic labeling over the
online reconstructed volume. The robot is guided by an online estimated
discrete viewing score field (VSF) parameterized over the 3D space of 2D
location and azimuth rotation. VSF stores for each grid the score of the
corresponding view, which measures how much it reduces the uncertainty
(entropy) of both geometric reconstruction and semantic labeling. Based on VSF,
we select the next best views (NBV) as the target for each time step. We then
jointly optimize the traverse path and camera trajectory between two adjacent
NBVs, through maximizing the integral viewing score (information gain) along
path and trajectory. Through extensive evaluation, we show that our method
achieves efficient and accurate online scene parsing during exploratory
scanning.

提出一种基于 RGBD 重建和语义分割的机器人操作主动理解未知室内场景的新方法，利用离线估计的离散视野分数场来驱动机器人的勘探扫描，结合基于体素的实时语义标注，优化遍历路径和相机轨迹，从而实现高效准确的在线场景解析。