We present a visually grounded hierarchical planning algorithm for
long-horizon manipulation tasks. Our algorithm offers a joint framework of
neuro-symbolic task planning and low-level motion generation conditioned on the
specified goal. At the core of our approach is a two-level scene graph
representation, namely geometric scene graph and symbolic scene graph. This
hierarchical representation serves as a structured, object-centric abstraction
of manipulation scenes. Our model uses graph neural networks to process these
scene graphs for predicting high-level task plans and low-level motions. We
demonstrate that our method scales to long-horizon tasks and generalizes well
to novel task goals. We validate our method in a kitchen storage task in both
physical simulation and the real world. Our experiments show that our method
achieved over 70% success rate and nearly 90% of subgoal completion rate on the
real robot while being four orders of magnitude faster in computation time
compared to standard search-based task-and-motion planner.

我们提出了一种基于视觉的分层规划算法，利用神经符号任务规划和基于低级运动产生的联合框架，以指定目标为条件。该算法使用两级场景图形式来表示操作场景，即几何场景图和符号场景图，并使用图形神经网络来为高层任务规划和低层动作生成处理这些场景图。我们在物理模拟和现实世界中的厨房储物任务中验证了我们的方法，并证明我们的方法比标准的搜索任务和运动规划器运行速度高四个数量级。