Transformer-based methods have exhibited significant generalization ability
when prompted with target-domain demonstrations or example solutions during
inference. Although demonstrations, as a way of task specification, can capture
rich information that may be hard to specify by language, it remains unclear
what information is extracted from the demonstrations to help generalization.
Moreover, assuming access to demonstrations of an unseen task is impractical or
unreasonable in many real-world scenarios, especially in robotics applications.
These questions motivate us to explore what the minimally sufficient prompt
could be to elicit the same level of generalization ability as the
demonstrations. We study this problem in the contextural RL setting which
allows for quantitative measurement of generalization and is commonly adopted
by meta-RL and multi-task RL benchmarks. In this setting, the training and test
Markov Decision Processes (MDPs) only differ in certain properties, which we
refer to as task parameters. We show that conditioning a decision transformer
on these task parameters alone can enable zero-shot generalization on par with
or better than its demonstration-conditioned counterpart. This suggests that
task parameters are essential for the generalization and DT models are trying
to recover it from the demonstration prompt. To extract the remaining
generalizable information from the supervision, we introduce an additional
learnable prompt which is demonstrated to further boost zero-shot
generalization across a range of robotic control, manipulation, and navigation
benchmark tasks.

通过在任务参数上进行调节，基于 Transformer 的模型展现出了与依赖演示任务相当甚至更好的零样本泛化能力。