In this paper, we present LidarStereoNet, the first unsupervised Lidar-stereo
fusion network, which can be trained in an end-to-end manner without the need
of ground truth depth maps. By introducing a novel "Feedback Loop'' to connect
the network input with output, LidarStereoNet could tackle both noisy Lidar
points and misalignment between sensors that have been ignored in existing
Lidar-stereo fusion studies. Besides, we propose to incorporate a piecewise
planar model into network learning to further constrain depths to conform to
the underlying 3D geometry. Extensive quantitative and qualitative evaluations
on both real and synthetic datasets demonstrate the superiority of our method,
which outperforms state-of-the-art stereo matching, depth completion and
Lidar-Stereo fusion approaches significantly.

本文提出了 LidarStereoNet，是第一个可以在端到端的未经监督训练的融合网络，无需地面真实深度图。我们通过引入新颖的 “反馈循环” 来连接网络输入与输出，并进一步将分段平面模型结合到网络学习中，以进一步约束深度与底层 3D 几何形状相一致，从而在现实和合成数据集上表现出显著的优越性，比现有的立体匹配、深度完成和 Lidar-Stereo 融合方法取得了更好的效果。