This paper addresses the problem of 3D referring expression comprehension
(REC) in autonomous driving scenario, which aims to ground a natural language
to the targeted region in LiDAR point clouds. Previous approaches for REC
usually focus on the 2D or 3D-indoor domain, which is not suitable for
accurately predicting the location of the queried 3D region in an autonomous
driving scene. In addition, the upper-bound limitation and the heavy
computation cost motivate us to explore a better solution. In this work, we
propose a new multi-modal visual grounding task, termed LiDAR Grounding. Then
we devise a Multi-modal Single Shot Grounding (MSSG) approach with an effective
token fusion strategy. It jointly learns the LiDAR-based object detector with
the language features and predicts the targeted region directly from the
detector without any post-processing. Moreover, the image feature can be
flexibly integrated into our approach to provide rich texture and color
information. The cross-modal learning enforces the detector to concentrate on
important regions in the point cloud by considering the informative language
expressions, thus leading to much better accuracy and efficiency. Extensive
experiments on the Talk2Car dataset demonstrate the effectiveness of the
proposed methods. Our work offers a deeper insight into the LiDAR-based
grounding task and we expect it presents a promising direction for the
autonomous driving community.

本文提出了一种新的多模态视觉定位任务，称为 LiDAR Grounding，并通过设计一种名为 MSSG 的多模态单次定位方法来实现，通过跨模态学习来提高定位精度和效率。