Generalization remains one of the most important desiderata for robust robot
learning systems. While recently proposed approaches show promise in
generalization to novel objects, semantic concepts, or visual distribution
shifts, generalization to new tasks remains challenging. For example, a
language-conditioned policy trained on pick-and-place tasks will not be able to
generalize to a folding task, even if the arm trajectory of folding is similar
to pick-and-place. Our key insight is that this kind of generalization becomes
feasible if we represent the task through rough trajectory sketches. We propose
a policy conditioning method using such rough trajectory sketches, which we
call RT-Trajectory, that is practical, easy to specify, and allows the policy
to effectively perform new tasks that would otherwise be challenging to
perform. We find that trajectory sketches strike a balance between being
detailed enough to express low-level motion-centric guidance while being coarse
enough to allow the learned policy to interpret the trajectory sketch in the
context of situational visual observations. In addition, we show how trajectory
sketches can provide a useful interface to communicate with robotic policies:
they can be specified through simple human inputs like drawings or videos, or
through automated methods such as modern image-generating or
waypoint-generating methods. We evaluate RT-Trajectory at scale on a variety of
real-world robotic tasks, and find that RT-Trajectory is able to perform a
wider range of tasks compared to language-conditioned and goal-conditioned
policies, when provided the same training data.

该研究论文探讨了使用粗略轨迹草图作为任务表示的方法，称为 RT-Trajectory，该方法通过有效地执行新任务，克服了现有语言条件和目标条件策略的局限性，具有广泛的泛化能力。

RT - 轨迹：通过事后轨迹草图实现机器人任务的泛化

RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory  Sketches

One of the great promises of robot learning systems is that they will be able
to learn from their mistakes and continuously adapt to ever-changing
environments. Despite this potential, most of the robot learning systems today
are deployed as a fixed policy and they are not being adapted after their
deployment. Can we efficiently adapt previously learned behaviors to new
environments, objects and percepts in the real world? In this paper, we present
a method and empirical evidence towards a robot learning framework that
facilitates continuous adaption. In particular, we demonstrate how to adapt
vision-based robotic manipulation policies to new variations by fine-tuning via
off-policy reinforcement learning, including changes in background, object
shape and appearance, lighting conditions, and robot morphology. Further, this
adaptation uses less than 0.2% of the data necessary to learn the task from
scratch. We find that our approach of adapting pre-trained policies leads to
substantial performance gains over the course of fine-tuning, and that
pre-training via RL is essential: training from scratch or adapting from
supervised ImageNet features are both unsuccessful with such small amounts of
data. We also find that these positive results hold in a limited continual
learning setting, in which we repeatedly fine-tune a single lineage of policies
using data from a succession of new tasks. Our empirical conclusions are
consistently supported by experiments on simulated manipulation tasks, and by
52 unique fine-tuning experiments on a real robotic grasping system pre-trained
on 580,000 grasps.

本论文提出了一种通过强化学习进行增量式 fine-tuning 的方法，可以有效地将图像为基础的机器人操作策略适应到新的环境、物体和感知中，在不到数据学习任务的 0.2％的情况下实现适应，这种方式可以大幅提高任务的性能表现，并且在连续学习场景下仍保持一致稳定。