We propose a deep neural network architecture to infer dense depth from an
image and a sparse point cloud. It is trained using a video stream and
corresponding synchronized sparse point cloud, as obtained from a LIDAR or
other range sensor, along with the intrinsic calibration parameters of the
camera. At inference time, the calibration of the camera, which can be
different than the one used for training, is fed as an input to the network
along with the sparse point cloud and a single image. A Calibrated
Backprojection Layer backprojects each pixel in the image to three-dimensional
space using the calibration matrix and a depth feature descriptor. The
resulting 3D positional encoding is concatenated with the image descriptor and
the previous layer output to yield the input to the next layer of the encoder.
A decoder, exploiting skip-connections, produces a dense depth map. The
resulting Calibrated Backprojection Network, or KBNet, is trained without
supervision by minimizing the photometric reprojection error. KBNet imputes
missing depth value based on the training set, rather than on generic
regularization. We test KBNet on public depth completion benchmarks, where it
outperforms the state of the art by 30.5% indoor and 8.8% outdoor when the same
camera is used for training and testing. When the test camera is different, the
improvement reaches 62%. Code available at:
this https URL

本研究使用卷积神经网络，引入校准回投影层来实现从图像和稀疏点云中推断密集深度图像，并通过最小化光度再投影误差来训练该神经网络，实验表明该网络在缺失值插值上比传统方法和目前状态的深度估计算法表现要好。

使用校准反投影层的无监督深度完成

Unsupervised Depth Completion with Calibrated Backprojection Layers

In this paper we present DOT (Dynamic Object Tracking), a front-end that
added to existing SLAM systems can significantly improve their robustness and
accuracy in highly dynamic environments. DOT combines instance segmentation and
multi-view geometry to generate masks for dynamic objects in order to allow
SLAM systems based on rigid scene models to avoid such image areas in their
optimizations.
To determine which objects are actually moving, DOT segments first instances
of potentially dynamic objects and then, with the estimated camera motion,
tracks such objects by minimizing the photometric reprojection error. This
short-term tracking improves the accuracy of the segmentation with respect to
other approaches. In the end, only actually dynamic masks are generated. We
have evaluated DOT with ORB-SLAM 2 in three public datasets. Our results show
that our approach improves significantly the accuracy and robustness of
ORB-SLAM 2, especially in highly dynamic scenes.

本文介绍了一种动态物体跟踪的方法，称为 DOT (Dynamic Object Tracking)，该方法结合了实例分割和多视角几何来生成动态物体的掩模，可以显著提高现有 SLAM 系统在高度动态环境下的鲁棒性和准确性。我们通过在三个公共数据集上对 ORB-SLAM 2 进行评估，结果表明我们的方法显著提高了 ORB-SLAM 2 的准确性和鲁棒性，特别是在高度动态的场景下。