Recent open-vocabulary robot mapping methods enrich dense geometric maps with
pre-trained visual-language features. While these maps allow for the prediction
of point-wise saliency maps when queried for a certain language concept,
large-scale environments and abstract queries beyond the object level still
pose a considerable hurdle, ultimately limiting language-grounded robotic
navigation. In this work, we present HOV-SG, a hierarchical open-vocabulary 3D
scene graph mapping approach for language-grounded robot navigation. Leveraging
open-vocabulary vision foundation models, we first obtain state-of-the-art
open-vocabulary segment-level maps in 3D and subsequently construct a 3D scene
graph hierarchy consisting of floor, room, and object concepts, each enriched
with open-vocabulary features. Our approach is able to represent multi-story
buildings and allows robotic traversal of those using a cross-floor Voronoi
graph. HOV-SG is evaluated on three distinct datasets and surpasses previous
baselines in open-vocabulary semantic accuracy on the object, room, and floor
level while producing a 75% reduction in representation size compared to dense
open-vocabulary maps. In order to prove the efficacy and generalization
capabilities of HOV-SG, we showcase successful long-horizon
language-conditioned robot navigation within real-world multi-storage
environments. We provide code and trial video data at this http URL

本研究致力于解决大规模环境下对象层次之外的抽象查询对于基于语言引导的机器人导航的难题，通过构建一个层次化的开放词汇三维场景图映射方法（HOV-SG），将先进的开放词汇分割级别地图与自由导航相结合，并在覆盖面积上实现了大小的减少，并且在物体、房间和楼层级别上产生了超过之前基线的开放词汇语义准确度。