We introduce a discriminative multimodal descriptor based on a pair of sensor
readings: a point cloud from a LiDAR and an image from an RGB camera. Our
descriptor, named MinkLoc++, can be used for place recognition, re-localization
and loop closure purposes in robotics or autonomous vehicles applications. We
use late fusion approach, where each modality is processed separately and fused
in the final part of the processing pipeline. The proposed method achieves
state-of-the-art performance on standard place recognition benchmarks. We also
identify dominating modality problem when training a multimodal descriptor. The
problem manifests itself when the network focuses on a modality with a larger
overfit to the training data. This drives the loss down during the training but
leads to suboptimal performance on the evaluation set. In this work we describe
how to detect and mitigate such risk when using a deep metric learning approach
to train a multimodal neural network. Our code is publicly available on the
project website: this https URL

本研究介绍了一种基于 LiDAR 和 RGB 相机图像的判别式多模态描述符 MinkLoc++，该描述符可用于机器人或自动驾驶应用的位置识别、重新定位和环路闭合，同时使用深度度量学习方法来减轻多模态描述符训练过程中可能存在的主导模态问题，该方法在标准的位置识别基准测试上实现了最先进的性能。