This study introduces a novel approach to online embedding of multi-scale
CLIP (Contrastive Language-Image Pre-Training) features into 3D maps. By
harnessing CLIP, this methodology surpasses the constraints of conventional
vocabulary-limited methods and enables the incorporation of semantic
information into the resultant maps. While recent approaches have explored the
embedding of multi-modal features in maps, they often impose significant
computational costs, lacking practicality for exploring unfamiliar environments
in real time. Our approach tackles these challenges by efficiently computing
and embedding multi-scale CLIP features, thereby facilitating the exploration
of unfamiliar environments through real-time map generation. Moreover, the
embedding CLIP features into the resultant maps makes offline retrieval via
linguistic queries feasible. In essence, our approach simultaneously achieves
real-time object search and mapping of unfamiliar environments. Additionally,
we propose a zero-shot object-goal navigation system based on our mapping
approach, and we validate its efficacy through object-goal navigation, offline
object retrieval, and multi-object-goal navigation in both simulated
environments and real robot experiments. The findings demonstrate that our
method not only exhibits swifter performance than state-of-the-art mapping
methods but also surpasses them in terms of the success rate of object-goal
navigation tasks.

本研究介绍了一种新的方法，将多尺度的 CLIP (对比性语言 - 图像预训练) 特征在线嵌入到 3D 地图中，通过利用 CLIP，该方法克服了传统的词汇有限方法的限制，并将语义信息融入到生成的地图中，从而高效地计算和嵌入多尺度 CLIP 特征，并且通过将 CLIP 特征嵌入到结果地图中，实现了离线检索和实时对象搜索，进一步提出了一种基于地图方法的零样本对象 - 目标导航系统，并通过对象 - 目标导航、离线对象检索和多对象 - 目标导航在模拟环境和真实机器人实验中验证了其有效性，结果表明，我们的方法不仅在地图生成方面表现出更快的性能，而且在对象 - 目标导航任务的成功率方面也超过了现有的最先进方法。