Intrinsically motivated spontaneous exploration is a key enabler of
autonomous developmental learning in human children. It enables the discovery
of skill repertoires through autotelic learning, i.e. the self-generation,
self-selection, self-ordering and self-experimentation of learning goals. We
present an algorithmic approach called Intrinsically Motivated Goal Exploration
Processes (IMGEP) to enable similar properties of autonomous learning in
machines. The IMGEP architecture relies on several principles: 1)
self-generation of goals, generalized as parameterized fitness functions; 2)
selection of goals based on intrinsic rewards; 3) exploration with incremental
goal-parameterized policy search and exploitation with a batch learning
algorithm; 4) systematic reuse of information acquired when targeting a goal
for improving towards other goals. We present a particularly efficient form of
IMGEP, called AMB, that uses a population-based policy and an object-centered
spatio-temporal modularity. We provide several implementations of this
architecture and demonstrate their ability to automatically generate a learning
curriculum within several experimental setups. One of these experiments
includes a real humanoid robot exploring multiple spaces of goals with several
hundred continuous dimensions and with distractors. While no particular target
goal is provided to these autotelic agents, this curriculum allows the
discovery of diverse skills that act as stepping stones for learning more
complex skills, e.g. nested tool use.

本论文提出了一种基于内在动机的目标探索算法，结合自主学习过程和增量式目标策略搜寻探索，可以自动生成任务序列并发掘多种技能，无需特定的目标指令，适用于机器人等人工智能设备的自主发展学习。