We study the problem of generalizable task learning from human demonstration
videos without extra training on the robot or pre-recorded robot motions. Given
a set of human demonstration videos showing a task with different objects/tools
(categorical objects), we aim to learn a representation of visual observation
that generalizes to categorical objects and enables efficient controller
design. We propose to introduce a geometric task structure to the
representation learning problem that geometrically encodes the task
specification from human demonstration videos, and that enables generalization
by building task specification correspondence between categorical objects.
Specifically, we propose CoVGS-IL, which uses a graph-structured task function
to learn task representations under structural constraints. Our method enables
task generalization by selecting geometric features from different objects
whose inner connection relationships define the same task in geometric
constraints. The learned task representation is then transferred to a robot
controller using uncalibrated visual servoing (UVS); thus, the need for extra
robot training or pre-recorded robot motions is removed.

本文提出了一种利用几何任务结构进行表示学习的方法，通过人类演示视频学习视觉观察的表征，并通过建立不同类别物体之间的任务说明对应来实现任务的泛化，从而不需要额外的机器人训练或预先录制的机器人运动来实现在机器人上的任务学习。