LiDAR point clouds have become the most common data source in autonomous
driving. However, due to the sparsity of point clouds, accurate and reliable
detection cannot be achieved in specific scenarios. Because of their
complementarity with point clouds, images are getting increasing attention.
Although with some success, existing fusion methods either perform hard fusion
or do not fuse in a direct manner. In this paper, we propose a generic 3D
detection framework called MMFusion, using multi-modal features. The framework
aims to achieve accurate fusion between LiDAR and images to improve 3D
detection in complex scenes. Our framework consists of two separate streams:
the LiDAR stream and the camera stream, which can be compatible with any
single-modal feature extraction network. The Voxel Local Perception Module in
the LiDAR stream enhances local feature representation, and then the
Multi-modal Feature Fusion Module selectively combines feature output from
different streams to achieve better fusion. Extensive experiments have shown
that our framework not only outperforms existing benchmarks but also improves
their detection, especially for detecting cyclists and pedestrians on KITTI
benchmarks, with strong robustness and generalization capabilities. Hopefully,
our work will stimulate more research into multi-modal fusion for autonomous
driving tasks.

本文提出了一种名为 MMFusion 的多模态 3D 检测框架，以在复杂场景中实现 LiDAR 和图像的准确融合，通过实验证明，该框架不仅优于现有基准，而且尤其适用于在 KITTI 基准上检测骑自行车和行人。

一个广义的多模型融合检测框架

A Generalized Multi-Modal Fusion Detection Framework

Recent work has shown that convolutional neural networks (CNNs) can be used
to estimate optical flow with high quality and fast runtime. This makes them
preferable for real-world applications. However, such networks require very
large training datasets. Engineering the training data is difficult and/or
laborious. This paper shows how to augment a network trained on an existing
synthetic dataset with large amounts of additional unlabelled data. In
particular, we introduce a selection mechanism to assemble from multiple
estimates a joint optical flow field, which outperforms that of all input
methods. The latter can be used as proxy-ground-truth to train a network on
real-world data and to adapt it to specific domains of interest. Our
experimental results show that the performance of networks improves
considerably, both, in cross-domain and in domain-specific scenarios. As a
consequence, we obtain state-of-the-art results on the KITTI benchmarks.

本研究介绍了如何使用合成数据集来增强预训练的卷积神经网络，从而提高其在实际领域中的性能，尤其在光流估计领域取得了最先进的成果。