Environment maps endowed with sophisticated semantics are pivotal for
facilitating seamless interaction between robots and humans, enabling them to
effectively carry out various tasks. Open-vocabulary maps, powered by
Visual-Language models (VLMs), possess inherent advantages, including
multimodal retrieval and open-set classes. However, existing open-vocabulary
maps are constrained to closed indoor scenarios and VLM features, thereby
diminishing their usability and inference capabilities. Moreover, the absence
of topological relationships further complicates the accurate querying of
specific instances. In this work, we propose OpenGraph, a representation of
open-vocabulary hierarchical graph structure designed for large-scale outdoor
environments. OpenGraph initially extracts instances and their captions from
visual images using 2D foundation models, encoding the captions with features
to enhance textual reasoning. Subsequently, 3D incremental panoramic mapping
with feature embedding is achieved by projecting images onto LiDAR point
clouds. Finally, the environment is segmented based on lane graph connectivity
to construct a hierarchical graph. Validation results from real public dataset
SemanticKITTI demonstrate that, even without fine-tuning the models, OpenGraph
exhibits the ability to generalize to novel semantic classes and achieve the
highest segmentation and query accuracy. The source code of OpenGraph is
publicly available at this https URL

本文提出了 OpenGraph，一种适用于大规模户外环境的开放词汇层次图结构表示，该结构兼具图像和文字语义，并通过 2D 模型提取实例和标题，以增强文本推理能力，并通过将图像投影到 LiDAR 点云上实现了 3D 增量全景映射和特征嵌入，最终根据车道图连通性对环境进行分割构建了一个层次化图。验证结果显示，OpenGraph 能够在没有微调模型的情况下推广到新的语义类别，并取得最高的分割和查询精度。