Image-guided depth completion aims at generating a dense depth map from
sparse LiDAR data and RGB image. Recent methods have shown promising
performance by reformulating it as a classification problem with two sub-tasks:
depth discretization and probability prediction. They divide the depth range
into several discrete depth values as depth categories, serving as priors for
scene depth distributions. However, previous depth discretization methods are
easy to be impacted by depth distribution variations across different scenes,
resulting in suboptimal scene depth distribution priors. To address the above
problem, we propose a progressive depth decoupling and modulating network,
which incrementally decouples the depth range into bins and adaptively
generates multi-scale dense depth maps in multiple stages. Specifically, we
first design a Bins Initializing Module (BIM) to construct the seed bins by
exploring the depth distribution information within a sparse depth map,
adapting variations of depth distribution. Then, we devise an incremental depth
decoupling branch to progressively refine the depth distribution information
from global to local. Meanwhile, an adaptive depth modulating branch is
developed to progressively improve the probability representation from
coarse-grained to fine-grained. And the bi-directional information interactions
are proposed to strengthen the information interaction between those two
branches (sub-tasks) for promoting information complementation in each branch.
Further, we introduce a multi-scale supervision mechanism to learn the depth
distribution information in latent features and enhance the adaptation
capability across different scenes. Experimental results on public datasets
demonstrate that our method outperforms the state-of-the-art methods. The code
will be open-sourced at [this https URL](this https URL).

提出了一种渐进的深度解耦和调制网络，通过将深度范围分解为多个范围，并自适应地生成多尺度稠密深度图，以解决现有深度离散化方法在不同场景中受深度分布变化影响导致子优化场景深度分布先验的问题。

灵活深度完成的渐进解耦与调制

Progressive Depth Decoupling and Modulating for Flexible Depth  Completion

We tackle the problem of using 3D information in convolutional neural
networks for down-stream recognition tasks. Using depth as an additional
channel alongside the RGB input has the scale variance problem present in image
convolution based approaches. On the other hand, 3D convolution wastes a large
amount of memory on mostly unoccupied 3D space, which consists of only the
surface visible to the sensor. Instead, we propose SurfConv, which "slides"
compact 2D filters along the visible 3D surface. SurfConv is formulated as a
simple depth-aware multi-scale 2D convolution, through a new Data-Driven Depth
Discretization (D4) scheme. We demonstrate the effectiveness of our method on
indoor and outdoor 3D semantic segmentation datasets. Our method achieves
state-of-the-art performance with less than 30% parameters used by the 3D
convolution-based approaches.

本文提出使用 SurfConv 作为一种新型的深度学习算法，可以有效地解决在 3D 图像中使用卷积神经网络在尺度上变化和浪费空间等问题，通过新的数据驱动深度离散化方案（D4），在室内和室外 3D 语义分割数据集上获得了超越 3D 卷积方法并使用不到 30％的参数的最先进性能。