Robust and generalized tool manipulation requires an understanding of the
properties and affordances of different tools. We investigate whether
linguistic information about a tool (e.g., its geometry, common uses) can help
control policies adapt faster to new tools for a given task. We obtain diverse
descriptions of various tools in natural language and use pre-trained language
models to generate their feature representations. We then perform
language-conditioned meta-learning to learn policies that can efficiently adapt
to new tools given their corresponding text descriptions. Our results
demonstrate that combining linguistic information and meta-learning
significantly accelerates tool learning in several manipulation tasks including
pushing, lifting, sweeping, and hammering.

本文研究了在不同工具的属性和可利用性方面进行强大而广泛的工具操作所需的信息。通过获取各种工具的自然语言描述并使用预先训练的语言模型生成它们的要素表示，我们进行了语言条件的元学习，从而学习可以根据相应文本描述高效适应新工具的策略。我们的结果表明，结合语言信息和元学习可以显著加速推、举、扫和锤击等多个操纵任务中的工具学习。

利用语言加速学习工具操作

Leveraging Language for Accelerated Learning of Tool Manipulation

Tool manipulation is vital for facilitating robots to complete challenging
task goals. It requires reasoning about the desired effect of the task and thus
properly grasping and manipulating the tool to achieve the task. Task-agnostic
grasping optimizes for grasp robustness while ignoring crucial task-specific
constraints. In this paper, we propose the Task-Oriented Grasping Network
(TOG-Net) to jointly optimize both task-oriented grasping of a tool and the
manipulation policy for that tool. The training process of the model is based
on large-scale simulated self-supervision with procedurally generated tool
objects. We perform both simulated and real-world experiments on two tool-based
manipulation tasks: sweeping and hammering. Our model achieves overall 71.1%
task success rate for sweeping and 80.0% task success rate for hammering.
Supplementary material is available at: bit.ly/task-oriented-grasp

本文提出了一个任务导向抓取网络 (TOG-Net)，通过大规模模拟自我监督与程序生成的工具对象，同时优化工具的任务导向抓取和操纵策略。模型在打扫和敲打两个任务中表现出较高的成功率，并取得了显著的效果。