Depth completion is a crucial task in autonomous driving, aiming to convert a
sparse depth map into a dense depth prediction. Due to its potentially rich
semantic information, RGB image is commonly fused to enhance the completion
effect. Image-guided depth completion involves three key challenges: 1) how to
effectively fuse the two modalities; 2) how to better recover depth
information; and 3) how to achieve real-time prediction for practical
autonomous driving. To solve the above problems, we propose a concise but
effective network, named CENet, to achieve high-performance depth completion
with a simple and elegant structure. Firstly, we use a fast guidance module to
fuse the two sensor features, utilizing abundant auxiliary features extracted
from the color space. Unlike other commonly used complicated guidance modules,
our approach is intuitive and low-cost. In addition, we find and analyze the
optimization inconsistency problem for observed and unobserved positions, and a
decoupled depth prediction head is proposed to alleviate the issue. The
proposed decoupled head can better output the depth of valid and invalid
positions with very few extra inference time. Based on the simple structure of
dual-encoder and single-decoder, our CENet can achieve superior balance between
accuracy and efficiency. In the KITTI depth completion benchmark, our CENet
attains competitive performance and inference speed compared with the
state-of-the-art methods. To validate the generalization of our method, we also
evaluate on indoor NYUv2 dataset, and our CENet still achieve impressive
results. The code of this work will be available at
this https URL

深度补全是自动驾驶中关键的任务之一，旨在将稀疏的深度图转化为密集的深度预测。通过基于 RGB 图像的融合，我们提出了一种简洁而有效的网络结构 CENet，实现了高性能深度补全，具有优秀的准确性和效率。

自动驾驶中基于图像导向的深度补全的简明有效网络

A Concise but Effective Network for Image Guided Depth Completion in  Autonomous Driving

In this paper, we tackle the problem of depth completion from RGBD data.
Towards this goal, we design a simple yet effective neural network block that
learns to extract joint 2D and 3D features. Specifically, the block consists of
two domain-specific sub-networks that apply 2D convolution on image pixels and
continuous convolution on 3D points, with their output features fused in image
space. We build the depth completion network simply by stacking the proposed
block, which has the advantage of learning hierarchical representations that
are fully fused between 2D and 3D spaces at multiple levels. We demonstrate the
effectiveness of our approach on the challenging KITTI depth completion
benchmark and show that our approach outperforms the state-of-the-art.

本文介绍了一种从 RGBD 数据中完成深度完成的简单且有效的神经网络块，该块能够学习提取联合的 2D 和 3D 特征，并成功地在 KITTI 深度完成基准测试中取得了优异的表现。

学习联合 2D-3D 表示进行深度补全

Learning Joint 2D-3D Representations for Depth Completion

We describe a method to infer dense depth from camera motion and sparse depth
as estimated using a visual-inertial odometry system. Unlike other scenarios
using point clouds from lidar or structured light sensors, we have few hundreds
to few thousand points, insufficient to inform the topology of the scene. Our
method first constructs a piecewise planar scaffolding of the scene, and then
uses it to infer dense depth using the image along with the sparse points. We
use a predictive cross-modal criterion, akin to `self-supervision,' measuring
photometric consistency across time, forward-backward pose consistency, and
geometric compatibility with the sparse point cloud. We also launch the first
visual-inertial + depth dataset, which we hope will foster additional
exploration into combining the complementary strengths of visual and inertial
sensors. To compare our method to prior work, we adopt the unsupervised KITTI
depth completion benchmark, and show state-of-the-art performance on it. Code
available at:
this https URL

本文提出一种利用相机运动和视觉惯性测距系统估算的稀疏深度来推断密集深度的方法，采用预测性跨模态标准，通过构建场景的分段平面脚手架来推断密集深度，并在 KITTI 深度完成基准上表现出最先进的性能。