Depth completion has attracted extensive attention recently due to the
development of autonomous driving, which aims to recover dense depth map from
sparse depth measurements. Convolutional spatial propagation network (CSPN) is
one of the state-of-the-art methods in this task, which adopt a linear
propagation model to refine coarse depth maps with local context. However, the
propagation of each pixel occurs in a fixed receptive field. This may not be
the optimal for refinement since different pixel needs different local context.
To tackle this issue, in this paper, we propose a deformable spatial
propagation network (DSPN) to adaptively generates different receptive field
and affinity matrix for each pixel. It allows the network obtain information
with much fewer but more relevant pixels for propagation. Experimental results
on KITTI depth completion benchmark demonstrate that our proposed method
achieves the state-of-the-art performance.

本文提出了一种可变形空间传播网络，用于适应性地为每个像素生成不同的感受野和密合矩阵，从而更准确地从稀疏深度测量中恢复密集的深度地图，KITTI 深度完成基准实验结果表明我们提出的方法达到了最先进的性能。

可变形空间传播网络用于深度完成

Deformable spatial propagation network for depth completion

Depth Completion deals with the problem of converting a sparse depth map to a
dense one, given the corresponding color image. Convolutional spatial
propagation network (CSPN) is one of the state-of-the-art (SoTA) methods of
depth completion, which recovers structural details of the scene. In this
paper, we propose CSPN++, which further improves its effectiveness and
efficiency by learning adaptive convolutional kernel sizes and the number of
iterations for the propagation, thus the context and computational resources
needed at each pixel could be dynamically assigned upon requests. Specifically,
we formulate the learning of the two hyper-parameters as an architecture
selection problem where various configurations of kernel sizes and numbers of
iterations are first defined, and then a set of soft weighting parameters are
trained to either properly assemble or select from the pre-defined
configurations at each pixel. In our experiments, we find weighted assembling
can lead to significant accuracy improvements, which we referred to as
"context-aware CSPN", while weighted selection, "resource-aware CSPN" can
reduce the computational resource significantly with similar or better
accuracy. Besides, the resource needed for CSPN++ can be adjusted w.r.t. the
computational budget automatically. Finally, to avoid the side effects of noise
or inaccurate sparse depths, we embed a gated network inside CSPN++, which
further improves the performance. We demonstrate the effectiveness of CSPN++on
the KITTI depth completion benchmark, where it significantly improves over CSPN
and other SoTA methods.

本文提出 CSPN++ 方法，通过学习自适应卷积核大小和迭代传播次数的超参数，使得每个像素点都能动态分配上下文和计算资源，分别提出了 “上下文感知 CSPN” 和 “资源感知 CSPN”，在 KITTI 深度完成基准测试中取得了显著的精度和计算资源改进。

CSPN++：基于上下文和资源感知的卷积空间传播网络用于深度补全

CSPN++: Learning Context and Resource Aware Convolutional Spatial  Propagation Networks for Depth Completion

Depth prediction is one of the fundamental problems in computer vision. In
this paper, we propose a simple yet effective convolutional spatial propagation
network (CSPN) to learn the affinity matrix for various depth estimation tasks.
Specifically, it is an efficient linear propagation model, in which the
propagation is performed with a manner of recurrent convolutional operation,
and the affinity among neighboring pixels is learned through a deep
convolutional neural network (CNN). We can append this module to any output
from a state-of-the-art (SOTA) depth estimation networks to improve their
performances. In practice, we further extend CSPN in two aspects: 1) take
sparse depth map as additional input, which is useful for the task of depth
completion; 2) similar to commonly used 3D convolution operation in CNNs, we
propose 3D CSPN to handle features with one additional dimension, which is
effective in the task of stereo matching using 3D cost volume. For the tasks of
sparse to dense, a.k.a depth completion. We experimented the proposed CPSN
conjunct algorithms over the popular NYU v2 and KITTI datasets, where we show
that our proposed algorithms not only produce high quality (e.g., 30% more
reduction in depth error), but also run faster (e.g., 2 to 5x faster) than
previous SOTA spatial propagation network. We also evaluated our stereo
matching algorithm on the Scene Flow and KITTI Stereo datasets, and rank 1st on
both the KITTI Stereo 2012 and 2015 benchmarks, which demonstrates the
effectiveness of the proposed module. The code of CSPN proposed in this work
will be released at this https URL

这篇研究提出了使用卷积空间传播网络 (CSPN) 进行深度预测的方法，并通过实验展示了其在深度补全和立体匹配任务中的有效性。

使用卷积空间传播网络学习深度

Learning Depth with Convolutional Spatial Propagation Network

Depth estimation from a single image is a fundamental problem in computer
vision. In this paper, we propose a simple yet effective convolutional spatial
propagation network (CSPN) to learn the affinity matrix for depth prediction.
Specifically, we adopt an efficient linear propagation model, where the
propagation is performed with a manner of recurrent convolutional operation,
and the affinity among neighboring pixels is learned through a deep
convolutional neural network (CNN). We apply the designed CSPN to two depth
estimation tasks given a single image: (1) To refine the depth output from
state-of-the-art (SOTA) existing methods; and (2) to convert sparse depth
samples to a dense depth map by embedding the depth samples within the
propagation procedure. The second task is inspired by the availability of
LIDARs that provides sparse but accurate depth measurements. We experimented
the proposed CSPN over two popular benchmarks for depth estimation, i.e. NYU v2
and KITTI, where we show that our proposed approach improves in not only
quality (e.g., 30% more reduction in depth error), but also speed (e.g., 2 to 5
times faster) than prior SOTA methods.

提出了一种简单而有效的卷积空间传播网络（CSPN），通过递归卷积操作和深度卷积神经网络学习邻域像素之间的相似性，从单张图像中进行深度估计，可以在深度输出的精度和速度方面改进现有的先进技术方法。