Convolutional neural networks (CNN) are limited by the lack of capability to
handle geometric information due to the fixed grid kernel structure. The
availability of depth data enables progress in RGB-D semantic segmentation with
CNNs. State-of-the-art methods either use depth as additional images or process
spatial information in 3D volumes or point clouds. These methods suffer from
high computation and memory cost. To address these issues, we present
Depth-aware CNN by introducing two intuitive, flexible and effective
operations: depth-aware convolution and depth-aware average pooling. By
leveraging depth similarity between pixels in the process of information
propagation, geometry is seamlessly incorporated into CNN. Without introducing
any additional parameters, both operators can be easily integrated into
existing CNNs. Extensive experiments and ablation studies on challenging RGB-D
semantic segmentation benchmarks validate the effectiveness and flexibility of
our approach.

本文提出一种 Depth-aware CNN 方法，通过引入深度感知卷积和深度感知平均池化两种直观、灵活、有效的操作，将几何信息无缝地融入 CNN 中，从而解决了 CNN 在处理几何信息上的限制问题，并且在具有挑战性的 RGB-D 语义分割基准测试上进行的广泛实验和消融研究证实了该方法的有效性和灵活性。