In this paper we study the problem of object detection for RGB-D images using semantically rich image and depth features. We propose a new geocentric embedding for depth images that encodes height above ground and angle with gravity for each pixel in addition to the horizontal disparity. We demonstrate that this geocentric embedding works better than using raw depth images for learning feature representations with convolutional neural networks. Our final object detection system achieves an average precision of 37.3%, which is a 56% relative improvement over existing methods. We then focus on the task of instance segmentation where we label pixels belonging to object instances found by our detector. For this task, we propose a decision forest approach that classifies pixels in the detection window as foreground or background using a family of unary and binary tests that query shape and geocentric pose features. Finally, we use the output from our object detectors in an existing superpixel classification framework for semantic scene segmentation and achieve a 24% relative improvement over current state-of-the-art for the object categories that we study. We believe advances such as those represented in this paper will facilitate the use of perception in fields like robotics.

本文研究使用语义丰富的图像和深度特征进行RGB-D图像目标检测问题。我们提出了一种新的地心嵌入深度图像的方法，该方法编码了每个像素相对于地面的高度和重力角度，以及水平视差。我们证明了这种地心嵌入方法比使用原始深度图像更适用于使用卷积神经网络进行特征表示学习。最后，我们使用对象检测器的输出在现有的超像素分类框架中进行语义场景分割，并在我们研究的对象类别中实现了24％的相对改进。

基于 RGB-D 图像学习丰富特征的物体检测和分割