We tackle the problem of 3D point cloud localization based on a few natural
linguistic descriptions and introduce a novel neural network, Text2Loc, that
fully interprets the semantic relationship between points and text. Text2Loc
follows a coarse-to-fine localization pipeline: text-submap global place
recognition, followed by fine localization. In global place recognition,
relational dynamics among each textual hint are captured in a hierarchical
transformer with max-pooling (HTM), whereas a balance between positive and
negative pairs is maintained using text-submap contrastive learning. Moreover,
we propose a novel matching-free fine localization method to further refine the
location predictions, which completely removes the need for complicated
text-instance matching and is lighter, faster, and more accurate than previous
methods. Extensive experiments show that Text2Loc improves the localization
accuracy by up to $2\times$ over the state-of-the-art on the KITTI360Pose
dataset. We will make the code publicly available.

我们提出了一种新型的神经网络 Text2Loc，通过几个自然语言描述实现了基于 3D 点云定位，并完全解释了点与文本之间的语义关系，实现了粗 - 细定位的本地化过程，文本子图全局位置识别中，用分层 Transformer 和最大池化捕获了每个文本提示之间的关系动态，而使用文本子图对比学习保持了正负样本之间的平衡；此外，我们还提出了一种新颖的无匹配精细定位方法，进一步提升了位置预测的准确性，摆脱了复杂的文本实例匹配需求，并且比之前的方法更轻量化、更快速、更准确。大量实验证明，Text2Loc 在 KITTI360Pose 数据集上超过了当前最先进技术，定位准确度提升了 2 倍。我们将公开提供代码。