Stereo-LiDAR fusion is a promising task in that we can utilize two different
types of 3D perceptions for practical usage -- dense 3D information (stereo
cameras) and highly-accurate sparse point clouds (LiDAR). However, due to their
different modalities and structures, the method of aligning sensor data is the
key for successful sensor fusion. To this end, we propose a geometry-aware
stereo-LiDAR fusion network for long-range depth estimation, called volumetric
propagation network. The key idea of our network is to exploit sparse and
accurate point clouds as a cue for guiding correspondences of stereo images in
a unified 3D volume space. Unlike existing fusion strategies, we directly embed
point clouds into the volume, which enables us to propagate valid information
into nearby voxels in the volume, and to reduce the uncertainty of
correspondences. Thus, it allows us to fuse two different input modalities
seamlessly and regress a long-range depth map. Our fusion is further enhanced
by a newly proposed feature extraction layer for point clouds guided by images:
FusionConv. FusionConv extracts point cloud features that consider both
semantic (2D image domain) and geometric (3D domain) relations and aid fusion
at the volume. Our network achieves state-of-the-art performance on the KITTI
and the Virtual-KITTI datasets among recent stereo-LiDAR fusion methods.

本文提出了一种几何感知的立体 - LiDAR 融合网络，通过在三维空间中统一传入稀疏而准确的点云系作为指导进行对应，借鉴 2D 图像领域的语义和 3D 领域的几何关系辅助于点云的特征提取，进而成功的融合了两种不同的输入方式并回归出一个长距离深度图。 该网络可用于 KITTI 和 Virtual-KITTI 数据集的立体 - LiDAR 融合任务中，取得了最先进的性能表现。