3D scene understanding is important for robots to interact with the 3D world
in a meaningful way. Most previous works on 3D scene understanding focus on
recognizing geometrical or semantic properties of the scene independently. In
this work, we introduce Data Associated Recurrent Neural Networks (DA-RNNs), a
novel framework for joint 3D scene mapping and semantic labeling. DA-RNNs use a
new recurrent neural network architecture for semantic labeling on RGB-D
videos. The output of the network is integrated with mapping techniques such as
KinectFusion in order to inject semantic information into the reconstructed 3D
scene. Experiments conducted on a real world dataset and a synthetic dataset
with RGB-D videos demonstrate the ability of our method in semantic 3D scene
mapping.

本文提出了一种数据关联递归神经网络（DA-RNNs）的新框架，它可以将 RGB-D 视频的语义标记与 KinectFusion 等拓扑映射技术相结合，从而将 RGB-D 视频的语义信息注入重建的 3D 场景中，实现了对三维场景地图与语义标记的联合理解。实验结果表明，这种方法在语义 3D 场景映射方面具有良好的性能。