Depth completion aims to recover a dense depth map from a sparse depth map
with the corresponding color image as input. Recent approaches mainly formulate
depth completion as a one-stage end-to-end learning task, which outputs dense
depth maps directly. However, the feature extraction and supervision in
one-stage frameworks are insufficient, limiting the performance of these
approaches. To address this problem, we propose a novel end-to-end residual
learning framework, which formulates the depth completion as a two-stage
learning task, i.e., a sparse-to-coarse stage and a coarse-to-fine stage.
First, a coarse dense depth map is obtained by a simple CNN framework. Then, a
refined depth map is further obtained using a residual learning strategy in the
coarse-to-fine stage with a coarse depth map and color image as input.
Specially, in the coarse-to-fine stage, a channel shuffle extraction operation
is utilized to extract more representative features from the color image and
coarse depth map, and an energy based fusion operation is exploited to
effectively fuse these features obtained by channel shuffle operation, thus
leading to more accurate and refined depth maps. We achieve SoTA performance in
RMSE on KITTI benchmark. Extensive experiments on other datasets future
demonstrate the superiority of our approach over current state-of-the-art depth
completion approaches.

提出了一种基于残差学习的端到端深度完成框架，其将深度完成分为两个阶段进行训练，即粗 - 精阶段，并利用颜色图像和粗深度图提取了表示特征和进行了能量融合操作，将 KITTI 基准下的平均 RMSE 性能提高至当前最先进技术水平以上。