A significant bottleneck in applying current reinforcement learning
algorithms to real-world scenarios is the need to reset the environment between
every episode. This reset process demands substantial human intervention,
making it difficult for the agent to learn continuously and autonomously.
Several recent works have introduced autonomous reinforcement learning (ARL)
algorithms that generate curricula for jointly training reset and forward
policies. While their curricula can reduce the number of required manual resets
by taking into account the agent's learning progress, they rely on
task-specific knowledge, such as predefined initial states or reset reward
functions. In this paper, we propose a novel ARL algorithm that can generate a
curriculum adaptive to the agent's learning progress without task-specific
knowledge. Our curriculum empowers the agent to autonomously reset to diverse
and informative initial states. To achieve this, we introduce a success
discriminator that estimates the success probability from each initial state
when the agent follows the forward policy. The success discriminator is trained
with relabeled transitions in a self-supervised manner. Our experimental
results demonstrate that our ARL algorithm can generate an adaptive curriculum
and enable the agent to efficiently bootstrap to solve sparse-reward maze
navigation tasks, outperforming baselines with significantly fewer manual
resets.

提出了一种新颖的自主强化学习算法，能够根据智能体的学习进展生成自适应课程，使智能体能够高效地解决稀疏奖励迷宫导航任务，同时减少了手动复位。

无任务特定知识的自主强化学习自我监督课程生成

Self-Supervised Curriculum Generation for Autonomous Reinforcement  Learning without Task-Specific Knowledge

Continual learning (CL) -- the ability to continuously learn, building on
previously acquired knowledge -- is a natural requirement for long-lived
autonomous reinforcement learning (RL) agents. While building such agents, one
needs to balance opposing desiderata, such as constraints on capacity and
compute, the ability to not catastrophically forget, and to exhibit positive
transfer on new tasks. Understanding the right trade-off is conceptually and
computationally challenging, which we argue has led the community to overly
focus on catastrophic forgetting. In response to these issues, we advocate for
the need to prioritize forward transfer and propose Continual World, a
benchmark consisting of realistic and meaningfully diverse robotic tasks built
on top of Meta-World as a testbed. Following an in-depth empirical evaluation
of existing CL methods, we pinpoint their limitations and highlight unique
algorithmic challenges in the RL setting. Our benchmark aims to provide a
meaningful and computationally inexpensive challenge for the community and thus
help better understand the performance of existing and future solutions.
Information about the benchmark, including the open-source code, is available
at this https URL

提出了 Continual World 基准，并进行了基于机器人任务的方法评估，该评估针对 Continual learning 中的前向迁移和 Catastrophic forgetting 进行优化，以提高方法在 RL 中的性能。