In this paper we propose a convolutional neural network that is designed to
upsample a series of sparse range measurements based on the contextual cues
gleaned from a high resolution intensity image. Our approach draws inspiration
from related work on super-resolution and in-painting. We propose a novel
architecture that seeks to pull contextual cues separately from the intensity
image and the depth features and then fuse them later in the network. We argue
that this approach effectively exploits the relationship between the two
modalities and produces accurate results while respecting salient image
structures. We present experimental results to demonstrate that our approach is
comparable with state of the art methods and generalizes well across multiple
datasets.

本文提出了一种卷积神经网络，它可以基于高分辨率强度图像得出上采样一系列稀疏范围测量的上下文线索，并且在网络中分离、融合两种模态的上下文线索，旨在利用两种模态之间的关系来产生准确的结果，同时尊重显著的图像结构。通过实验结果表明，该方法与当前最先进的方法相媲美，并可在多个数据集上进行有效泛化。

DFuseNet: RGB 和稀疏深度信息的深度融合，用于图像引导的密集深度完成

DFuseNet: Deep Fusion of RGB and Sparse Depth Information for Image  Guided Dense Depth Completion

We present an approach to depth estimation that fuses information from a
stereo pair with sparse range measurements derived from a LIDAR sensor or a
range camera. The goal of this work is to exploit the complementary strengths
of the two sensor modalities, the accurate but sparse range measurements and
the ambiguous but dense stereo information. These two sources are effectively
and efficiently fused by combining ideas from anisotropic diffusion and
semi-global matching.
We evaluate our approach on the KITTI 2015 and Middlebury 2014 datasets,
using randomly sampled ground truth range measurements as our sparse depth
input. We achieve significant performance improvements with a small fraction of
range measurements on both datasets. We also provide qualitative results from
our platform using the PMDTec Monstar sensor. Our entire pipeline runs on an
NVIDIA TX-2 platform at 5Hz on 1280x1024 stereo images with 128 disparity
levels.

本研究提出了一种深度估计方法，利用来自双目立体视觉和稀疏激光雷达或深度相机的测量信息。通过融合各传感器的优势，即精确但稀疏的测量信息和模糊但密集的立体信息，并采用各向异性扩散和半全局匹配技术进行有效地融合。该方法在 KITTI 2015 和 Middlebury 2014 数据集上评估，表现显著，而且使用 PMDTec Monstar 传感器提供了定量结果。