Autonomous systems that efficiently utilize tools can assist humans in
completing many common tasks such as cooking and cleaning. However, current
systems fall short of matching human-level of intelligence in terms of adapting
to novel tools. Prior works based on affordance often make strong assumptions
about the environments and cannot scale to more complex, contact-rich tasks. In
this work, we tackle this challenge and explore how agents can learn to use
previously unseen tools to manipulate deformable objects. We propose to learn a
generative model of the tool-use trajectories as a sequence of point clouds,
which generalizes to different tool shapes. Given any novel tool, we first
generate a tool-use trajectory and then optimize the sequence of tool poses to
align with the generated trajectory. We train a single model for four different
challenging deformable object manipulation tasks. Our model is trained with
demonstration data from just a single tool for each task and is able to
generalize to various novel tools, significantly outperforming baselines.
Additional materials can be found on our project website:
this https URL

通过生成模型学习工具使用轨迹和对应工具位姿的序列，以实现对变形物体的操纵，进而提高自治系统在常见任务（如烹饪和清洁）中的智能水平。