Lidar depth completion is a new and hot topic of depth estimation. In this
task, it is the key and difficult point to fuse the features of color space and
depth space. In this paper, we migrate the classic LSTM and Transformer modules
from NLP to depth completion and redesign them appropriately. Specifically, we
use Forget gate, Update gate, Output gate, and Skip gate to achieve the
efficient fusion of color and depth features and perform loop optimization at
multiple scales. Finally, we further fuse the deep features through the
Transformer multi-head attention mechanism. Experimental results show that
without repetitive network structure and post-processing steps, our method can
achieve state-of-the-art performance by adding our modules to a simple
encoder-decoder network structure. Our method ranks first on the current
mainstream autonomous driving KITTI benchmark dataset. It can also be regarded
as a backbone network for other methods, which likewise achieves
state-of-the-art performance.

激光雷达深度补全是深度估计的一个新的热门话题，在此任务中，颜色空间和深度空间的特征融合是关键和难点。本文将经典的 LSTM 和 Transformer 模块从自然语言处理迁移到深度补全中并进行相应的重新设计，通过使用遗忘门、更新门、输出门和跳跃门来实现颜色和深度特征的高效融合，并在多个尺度上进行循环优化，最后通过 Transformer 的多头注意力机制进一步融合深度特征。实验证明，我们的方法无需重复的网络结构和后处理步骤，只需在简单的编码器 - 解码器网络结构上添加我们的模块，就能达到最先进的性能。我们的方法在当前主流的自动驾驶 KITTI 基准数据集上排名第一，也可作为其他方法的主干网络，同样能达到最先进的性能。