We investigate the use of transformer sequence models as dynamics models
(TDMs) for control. In a number of experiments in the DeepMind control suite,
we find that first, TDMs perform well in a single-environment learning setting
when compared to baseline models. Second, TDMs exhibit strong generalization
capabilities to unseen environments, both in a few-shot setting, where a
generalist model is fine-tuned with small amounts of data from the target
environment, and in a zero-shot setting, where a generalist model is applied to
an unseen environment without any further training. We further demonstrate that
generalizing system dynamics can work much better than generalizing optimal
behavior directly as a policy. This makes TDMs a promising ingredient for a
foundation model of control.

调查使用 Transformer 序列模型作为控制器动力学模型，发现它们不仅在单个环境下的学习表现优异，而且在少量数据或无数据的情况下能够很好地适应未见过的环境，由此可以成为控制器基础模型

控制的通用动力学模型

A Generalist Dynamics Model for Control

Standard deep network models lack the inductive biases needed to generalize
compositionally in tasks like semantic parsing, translation, and question
answering. A large body of work in natural language processing seeks to
overcome this limitation with new model architectures that enforce a
compositional process of sentence interpretation. In this paper, we present a
domain-general framework for compositional modeling that instead formulates
compositionality as a constraint on data distributions. We prove that for any
task factorizable into a lexicon and a composition function, there exists a
family of data transformation functions that are guaranteed to produce new,
well-formed examples when applied to training data. We further show that it is
possible to identify these data transformations even when the composition
function is unknown (e.g. when we do not know how to write or infer a symbolic
grammar). Using these transformation functions to perform data augmentation for
ordinary RNN and transformer sequence models, we obtain state-of-the-art
results on the CLEVR-CoGenT visual question answering dataset, and results
comparable to specialized model architectures on the COGS semantic parsing
dataset.

该文章提出了一种基于数据分布的组合建模通用框架，证明了针对具有词汇表和组合函数的任何可分解任务都存在一族数据转换函数，可在训练数据上产生新的、格式正确的例子，进而说明即使在未知组合函数的情况下 (例如无法编写或推断符号语法), 也可以识别这些数据转换并对普通 RNN 和转换器序列模型进行数据增强，在 CLEVR-CoGenT 视觉问答数据集上获得了最先进的结果，在 COGS 语义分析数据集上获得了与专用模型架构相当的结果。