Depth data provide geometric information that can bring progress in RGB-D
scene parsing tasks. Several recent works propose RGB-D convolution operators
that construct receptive fields along the depth-axis to handle 3D neighborhood
relations between pixels. However, these methods pre-define depth receptive
fields by hyperparameters, making them rely on parameter selection. In this
paper, we propose a novel operator called malleable 2.5D convolution to learn
the receptive field along the depth-axis. A malleable 2.5D convolution has one
or more 2D convolution kernels. Our method assigns each pixel to one of the
kernels or none of them according to their relative depth differences, and the
assigning process is formulated as a differentiable form so that it can be
learnt by gradient descent. The proposed operator runs on standard 2D feature
maps and can be seamlessly incorporated into pre-trained CNNs. We conduct
extensive experiments on two challenging RGB-D semantic segmentation dataset
NYUDv2 and Cityscapes to validate the effectiveness and the generalization
ability of our method.

该研究提出了可变形 2.5D 卷积算子，以学习沿深度轴的感受野，可以有效地进行 RGB-D 语义分割，并具有很好的泛化能力。