In this paper, we explore an approach to auxiliary task discovery in
reinforcement learning based on ideas from representation learning. Auxiliary
tasks tend to improve data efficiency by forcing the agent to learn auxiliary
prediction and control objectives in addition to the main task of maximizing
reward, and thus producing better representations. Typically these tasks are
designed by people. Meta-learning offers a promising avenue for automatic task
discovery; however, these methods are computationally expensive and challenging
to tune in practice. In this paper, we explore a complementary approach to the
auxiliary task discovery: continually generating new auxiliary tasks and
preserving only those with high utility. We also introduce a new measure of
auxiliary tasks usefulness based on how useful the features induced by them are
for the main task. Our discovery algorithm significantly outperforms random
tasks, hand-designed tasks, and learning without auxiliary tasks across a suite
of environments.

本文介绍了一种基于表示学习思想的强化学习辅助任务发现方法，通过不断生成新的辅助任务并保留具有较高效用性的任务来提高数据效率，并引入了一种反映辅助任务效用的衡量标准。实现的算法在多种环境下显著优于随机任务和手动设计的任务。