A lot of recent machine learning research papers have "Open-ended learning"
in their title. But very few of them attempt to define what they mean when
using the term. Even worse, when looking more closely there seems to be no
consensus on what distinguishes open-ended learning from related concepts such
as continual learning, lifelong learning or autotelic learning. In this paper,
we contribute to fixing this situation. After illustrating the genealogy of the
concept and more recent perspectives about what it truly means, we outline that
open-ended learning is generally conceived as a composite notion encompassing a
set of diverse properties. In contrast with these previous approaches, we
propose to isolate a key elementary property of open-ended processes, which is
to always produce novel elements from time to time over an infinite horizon.
From there, we build the notion of open-ended learning problems and focus in
particular on the subset of open-ended goal-conditioned reinforcement learning
problems, as this framework facilitates the definition of learning a growing
repertoire of skills. Finally, we highlight the work that remains to be
performed to fill the gap between our elementary definition and the more
involved notions of open-ended learning that developmental AI researchers may
have in mind.

开放式学习是一个复合概念，其核心特性是在无限时间内不断产生新元素，并以开放式目标条件下的强化学习问题为重点研究领域。该研究旨在定义开放式学习的基本属性，并探讨其与相关概念的区别与联系，以填补目前在发展性人工智能研究中的知识空白。

目标条件智能体开放式学习问题定义

A Definition of Open-Ended Learning Problems for Goal-Conditioned Agents

Humans learn to master open-ended repertoires of skills by imagining and
practicing their own goals. This autotelic learning process, literally the
pursuit of self-generated (auto) goals (telos), becomes more and more
open-ended as the goals become more diverse, abstract and creative. The
resulting exploration of the space of possible skills is supported by an
inter-individual exploration: goal representations are culturally evolved and
transmitted across individuals, in particular using language. Current
artificial agents mostly rely on predefined goal representations corresponding
to goal spaces that are either bounded (e.g. list of instructions), or
unbounded (e.g. the space of possible visual inputs) but are rarely endowed
with the ability to reshape their goal representations, to form new
abstractions or to imagine creative goals. In this paper, we introduce a
language model augmented autotelic agent (LMA3) that leverages a pretrained
language model (LM) to support the representation, generation and learning of
diverse, abstract, human-relevant goals. The LM is used as an imperfect model
of human cultural transmission; an attempt to capture aspects of humans'
common-sense, intuitive physics and overall interests. Specifically, it
supports three key components of the autotelic architecture: 1)~a relabeler
that describes the goals achieved in the agent's trajectories, 2)~a goal
generator that suggests new high-level goals along with their decomposition
into subgoals the agent already masters, and 3)~reward functions for each of
these goals. Without relying on any hand-coded goal representations, reward
functions or curriculum, we show that LMA3 agents learn to master a large
diversity of skills in a task-agnostic text-based environment.

本研究介绍了一种使用预训练语言模型（LM）的语言模型增强的自我目标学习环境，它支持自动生成并学习具有多样性、抽象性、与人类相关的目标 —— 而非手动编码的目标表示、回报函数或课程，该系统可以在基于文本的任务无关环境中学习掌握各种广泛的技能。