We focus on the task of language-conditioned object placement, in which a robot should generate placements that satisfy all the spatial relational constraints in language instructions. Previous works based on rule-based language parsing or scene-centric visual representation have restrictions on the form of instructions and reference objects or require large amounts of training data. We propose an object-centric framework that leverages foundation models to ground the reference objects and spatial relations for placement, which is more sample efficient and generalizable. Experiments indicate that our model can achieve a 97.75% success rate of placement with only ~0.26M trainable parameters. Besides, our method generalizes better to both unseen objects and instructions. Moreover, with only 25% training data, we still outperform the top competing approach.

我们提出了一种对象为中心的框架，利用基础模型来对参考对象和空间关系进行基础建模，使得机器人可以生成满足语言指令中所有空间关系限制的放置位置，该模型可提高训练数据的利用效率并具有更强的泛化性能，在只有25%的训练数据下，我们的模型仍然表现优于目前最佳方法，且可以达到97.75%的放置成功率。

面向语言驱动空间布局的物体中心推断:一种基于基础模型的方法