The raw depth image captured by the indoor depth sensor usually has an
extensive range of missing depth values due to inherent limitations such as the
inability to perceive transparent objects and limited distance range. The
incomplete depth map burdens many downstream vision tasks, and a rising number
of depth completion methods have been proposed to alleviate this issue. While
most existing methods can generate accurate dense depth maps from sparse and
uniformly sampled depth maps, they are not suitable for complementing the large
contiguous regions of missing depth values, which is common and critical. In
this paper, we design a novel two-branch end-to-end fusion network, which takes
a pair of RGB and incomplete depth images as input to predict a dense and
completed depth map. The first branch employs an encoder-decoder structure to
regress the local dense depth values from the raw depth map, with the help of
local guidance information extracted from the RGB image. In the other branch,
we propose an RGB-depth fusion GAN to transfer the RGB image to the
fine-grained textured depth map. We adopt adaptive fusion modules named W-AdaIN
to propagate the features across the two branches, and we append a confidence
fusion head to fuse the two outputs of the branches for the final depth map.
Extensive experiments on NYU-Depth V2 and SUN RGB-D demonstrate that our
proposed method clearly improves the depth completion performance, especially
in a more realistic setting of indoor environments with the help of the pseudo
depth map.

本文提出了一种基于深度信息补全技术，使用 RGB 图像和不完整的深度图像进行输入，设计了一种新的两支分支端对端融合网络，其中一个支路使用编码器 - 解码器结构从原始深度图中对局部密集深度值进行回归，另一个分支提出了一种 RGB 深度融合 GAN，通过自适应融合模块和置信度融合头来融合这两个分支的输出，从而实现了对室内环境中丢失深度数据的精确补全。