Curriculum design for reinforcement learning (RL) can speed up an agent's
learning process and help it learn to perform well on complex tasks. However,
existing techniques typically require domain-specific hyperparameter tuning,
involve expensive optimization procedures for task selection, or are suitable
only for specific learning objectives. In this work, we consider curriculum
design in contextual multi-task settings where the agent's final performance is
measured w.r.t. a target distribution over complex tasks. We base our
curriculum design on the Zone of Proximal Development concept, which has proven
to be effective in accelerating the learning process of RL agents for uniform
distribution over all tasks. We propose a novel curriculum, ProCuRL-Target,
that effectively balances the need for selecting tasks that are not too
difficult for the agent while progressing the agent's learning toward the
target distribution via leveraging task correlations. We theoretically justify
the task selection strategy of ProCuRL-Target by analyzing a simple learning
setting with REINFORCE learner model. Our experimental results across various
domains with challenging target task distributions affirm the effectiveness of
our curriculum strategy over state-of-the-art baselines in accelerating the
training process of deep RL agents.

基于近发展区概念，我们提出了一种名为 ProCuRL-Target 的新型课程计划，它可以在复杂任务的目标分布上平衡选择任务的需要，通过利用任务之间的相关性推动代理的学习，从而加速深度强化学习代理的训练过程。