Dataset distillation aims to condense large datasets into a small number of
synthetic examples that can be used as drop-in replacements when training new
models. It has applications to interpretability, neural architecture search,
privacy, and continual learning. Despite strong successes in supervised
domains, such methods have not yet been extended to reinforcement learning,
where the lack of a fixed dataset renders most distillation methods unusable.
Filling the gap, we formalize behaviour distillation, a setting that aims to
discover and then condense the information required for training an expert
policy into a synthetic dataset of state-action pairs, without access to expert
data. We then introduce Hallucinating Datasets with Evolution Strategies
(HaDES), a method for behaviour distillation that can discover datasets of just
four state-action pairs which, under supervised learning, train agents to
competitive performance levels in continuous control tasks. We show that these
datasets generalize out of distribution to training policies with a wide range
of architectures and hyperparameters. We also demonstrate application to a
downstream task, namely training multi-task agents in a zero-shot fashion.
Beyond behaviour distillation, HaDES provides significant improvements in
neuroevolution for RL over previous approaches and achieves SoTA results on one
standard supervised dataset distillation task. Finally, we show that
visualizing the synthetic datasets can provide human-interpretable task
insights.

行为压缩是一种旨在将大型数据集压缩为少数合成示例的方法，以用作在训练新模型时可直接替换使用。本文提出行为压缩的概念，并介绍了 HaDES 方法，它可以发现仅需包含四个状态 - 动作对的数据集，通过监督学习训练出在连续控制任务中表现竞争水平的智能体，同时 HaDES 在神经进化强化学习领域表现出显著改进， 并在标准的监督数据集压缩任务上取得了最先进的结果，最后，该方法的可视化结果可提供人类可解释的任务洞见。