This paper studies the challenge of developing robots capable of
understanding under-specified instructions for creating functional object
arrangements, such as "set up a dining table for two"; previous arrangement
approaches have focused on much more explicit instructions, such as "put object
A on the table." We introduce a framework, SetItUp, for learning to interpret
under-specified instructions. SetItUp takes a small number of training examples
and a human-crafted program sketch to uncover arrangement rules for specific
scene types. By leveraging an intermediate graph-like representation of
abstract spatial relationships among objects, SetItUp decomposes the
arrangement problem into two subproblems: i) learning the arrangement patterns
from limited data and ii) grounding these abstract relationships into object
poses. SetItUp leverages large language models (LLMs) to propose the abstract
spatial relationships among objects in novel scenes as the constraints to be
satisfied; then, it composes a library of diffusion models associated with
these abstract relationships to find object poses that satisfy the constraints.
We validate our framework on a dataset comprising study desks, dining tables,
and coffee tables, with the results showing superior performance in generating
physically plausible, functional, and aesthetically pleasing object
arrangements compared to existing models.

本研究通过引入 SetItUp 框架，研究了开发能够理解模糊指令以创建功能性物体布局的机器人的挑战。SetItUp 利用少量的训练示例和人类设计的程序草图来揭示特定场景类型的布局规则，并利用抽象的空间对象之间的图形表示将布局问题分解为两个子问题：学习来自有限数据的布局模式，以及将这些抽象关系与对象姿势相连接。通过利用大型语言模型（LLMs）提出新场景中的物体之间的抽象空间关系作为要满足的约束，并组合与这些抽象关系相关的扩散模型库来找到满足约束的物体姿势，验证了我们的框架在包含学习桌、餐桌和咖啡桌的数据集上，生成物理上合理、功能性且美观的物体布局的卓越性能。