We present Reinforcement Learning via Auxiliary Task Distillation
(AuxDistill), a new method that enables reinforcement learning (RL) to perform
long-horizon robot control problems by distilling behaviors from auxiliary RL
tasks. AuxDistill achieves this by concurrently carrying out multi-task RL with
auxiliary tasks, which are easier to learn and relevant to the main task. A
weighted distillation loss transfers behaviors from these auxiliary tasks to
solve the main task. We demonstrate that AuxDistill can learn a
pixels-to-actions policy for a challenging multi-stage embodied object
rearrangement task from the environment reward without demonstrations, a
learning curriculum, or pre-trained skills. AuxDistill achieves $2.3 \times$
higher success than the previous state-of-the-art baseline in the Habitat
Object Rearrangement benchmark and outperforms methods that use pre-trained
skills and expert demonstrations.

通过辅助任务蒸馏，我们提出了一种增强学习（RL）方法，使其能够通过从辅助 RL 任务中提取行为来解决长期规划的机器人控制问题。AuxDistill 通过并行进行辅助任务的多任务 RL，并通过加权蒸馏损失将这些辅助任务中的行为转移到主任务上，实现了这一目标。我们证明了 AuxDistill 能够从环境奖励中学习一个对挑战性的多阶段物体重新排列任务进行像素到动作策略的学习，无需演示、学习课程或预训练技能。AuxDistill 在 Habitat 物体重新排列基准测试中取得了比之前最先进的基准线高 2.3 倍的成功率，并超过使用预训练技能和专家演示的方法。