We present an Open-Vocabulary 3D Scene Graph (OVSG), a formal framework for
grounding a variety of entities, such as object instances, agents, and regions,
with free-form text-based queries. Unlike conventional semantic-based object
localization approaches, our system facilitates context-aware entity
localization, allowing for queries such as ``pick up a cup on a kitchen table"
or ``navigate to a sofa on which someone is sitting". In contrast to existing
research on 3D scene graphs, OVSG supports free-form text input and
open-vocabulary querying. Through a series of comparative experiments using the
ScanNet dataset and a self-collected dataset, we demonstrate that our proposed
approach significantly surpasses the performance of previous semantic-based
localization techniques. Moreover, we highlight the practical application of
OVSG in real-world robot navigation and manipulation experiments.

我们介绍了一种开放词汇的 3D 场景图（OVSG），它是一个形式化的框架，用于将各种实体，如物体实例、代理和区域，与自由文本查询进行关联。与传统的基于语义的物体定位方法不同，我们的系统支持上下文感知的实体定位，允许查询，如 ' 在厨房桌子上拿起一个杯子 ' 或' 导航到一个有人坐在上面的沙发 '。与现有的 3D 场景图研究相比，OVSG 支持自由文本输入和开放词汇查询。通过使用 ScanNet 数据集和自采集数据集进行的一系列比较实验证明，我们提出的方法明显超越了以前基于语义的定位技术的性能。此外，我们突出了 OVSG 在真实世界的机器人导航和操作实验中的实际应用。