Image-guided depth completion aims at generating a dense depth map from
sparse LiDAR data and RGB image. Recent methods have shown promising
performance by reformulating it as a classification problem with two sub-tasks:
depth discretization and probability prediction. They divide the depth range
into several discrete depth values as depth categories, serving as priors for
scene depth distributions. However, previous depth discretization methods are
easy to be impacted by depth distribution variations across different scenes,
resulting in suboptimal scene depth distribution priors. To address the above
problem, we propose a progressive depth decoupling and modulating network,
which incrementally decouples the depth range into bins and adaptively
generates multi-scale dense depth maps in multiple stages. Specifically, we
first design a Bins Initializing Module (BIM) to construct the seed bins by
exploring the depth distribution information within a sparse depth map,
adapting variations of depth distribution. Then, we devise an incremental depth
decoupling branch to progressively refine the depth distribution information
from global to local. Meanwhile, an adaptive depth modulating branch is
developed to progressively improve the probability representation from
coarse-grained to fine-grained. And the bi-directional information interactions
are proposed to strengthen the information interaction between those two
branches (sub-tasks) for promoting information complementation in each branch.
Further, we introduce a multi-scale supervision mechanism to learn the depth
distribution information in latent features and enhance the adaptation
capability across different scenes. Experimental results on public datasets
demonstrate that our method outperforms the state-of-the-art methods. The code
will be open-sourced at [this https URL](this https URL).

提出了一种渐进的深度解耦和调制网络，通过将深度范围分解为多个范围，并自适应地生成多尺度稠密深度图，以解决现有深度离散化方法在不同场景中受深度分布变化影响导致子优化场景深度分布先验的问题。

灵活深度完成的渐进解耦与调制

Progressive Depth Decoupling and Modulating for Flexible Depth  Completion

Depth completion is a crucial task in autonomous driving, aiming to convert a
sparse depth map into a dense depth prediction. Due to its potentially rich
semantic information, RGB image is commonly fused to enhance the completion
effect. Image-guided depth completion involves three key challenges: 1) how to
effectively fuse the two modalities; 2) how to better recover depth
information; and 3) how to achieve real-time prediction for practical
autonomous driving. To solve the above problems, we propose a concise but
effective network, named CENet, to achieve high-performance depth completion
with a simple and elegant structure. Firstly, we use a fast guidance module to
fuse the two sensor features, utilizing abundant auxiliary features extracted
from the color space. Unlike other commonly used complicated guidance modules,
our approach is intuitive and low-cost. In addition, we find and analyze the
optimization inconsistency problem for observed and unobserved positions, and a
decoupled depth prediction head is proposed to alleviate the issue. The
proposed decoupled head can better output the depth of valid and invalid
positions with very few extra inference time. Based on the simple structure of
dual-encoder and single-decoder, our CENet can achieve superior balance between
accuracy and efficiency. In the KITTI depth completion benchmark, our CENet
attains competitive performance and inference speed compared with the
state-of-the-art methods. To validate the generalization of our method, we also
evaluate on indoor NYUv2 dataset, and our CENet still achieve impressive
results. The code of this work will be available at
this https URL

深度补全是自动驾驶中关键的任务之一，旨在将稀疏的深度图转化为密集的深度预测。通过基于 RGB 图像的融合，我们提出了一种简洁而有效的网络结构 CENet，实现了高性能深度补全，具有优秀的准确性和效率。

自动驾驶中基于图像导向的深度补全的简明有效网络

A Concise but Effective Network for Image Guided Depth Completion in  Autonomous Driving

In this paper we consider the task of image-guided depth completion where our
system must infer the depth at every pixel of an input image based on the image
content and a sparse set of depth measurements. We propose a novel approach
that builds upon the strengths of modern deep learning techniques and classical
optimization algorithms and significantly improves performance. The proposed
method replaces the final $1\times 1$ convolutional layer employed in most
depth completion networks with a least squares fitting module which computes
weights by fitting the implicit depth bases to the given sparse depth
measurements. In addition, we show how our proposed method can be naturally
extended to a multi-scale formulation for improved self-supervised training. We
demonstrate through extensive experiments on various datasets that our approach
achieves consistent improvements over state-of-the-art baseline methods with
small computational overhead.

本研究提出了一种图像引导深度补全的方法，利用深度学习的技术和经典的优化算法，将常见深度补全网络中的最后一层 $1 	imes 1$ 卷积替换成最小二乘拟合模块，将隐式深度基于给定的稀疏深度测量进行权重计算，同时我们还将该方法自然地推广到多尺度形式以提高自监督训练的性能，在多个数据集上的实验证实相比基准方法，本方法能够在小型计算量的情况下实现良好性能。