Autonomous open-ended learning (OEL) robots are able to cumulatively acquire
new skills and knowledge through direct interaction with the environment, for
example relying on the guidance of intrinsic motivations and self-generated
goals. OEL robots have a high relevance for applications as they can use the
autonomously acquired knowledge to accomplish tasks relevant for their human
users. OEL robots, however, encounter an important limitation: this may lead to
the acquisition of knowledge that is not so much relevant to accomplish the
users' tasks. This work analyses a possible solution to this problem that
pivots on the novel concept of `purpose'. Purposes indicate what the designers
and/or users want from the robot. The robot should use internal representations
of purposes, called here `desires', to focus its open-ended exploration towards
the acquisition of knowledge relevant to accomplish them. This work contributes
to develop a computational framework on purpose in two ways. First, it
formalises a framework on purpose based on a three-level motivational hierarchy
involving: (a) the purposes; (b) the desires, which are domain independent; (c)
specific domain dependent state-goals. Second, the work highlights key
challenges highlighted by the framework such as: the `purpose-desire alignment
problem', the `purpose-goal grounding problem', and the `arbitration between
desires'. Overall, the approach enables OEL robots to learn in an autonomous
way but also to focus on acquiring goals and skills that meet the purposes of
the designers and users.

通过内在动机和自主生成的目标的引导，自主开放式学习（OEL）机器人能够通过与环境的直接交互累积地获得新的技能和知识。本研究针对 OEL 机器人在获得的知识与用户任务的相关性之间存在的问题，提出了基于 'purpose' 的概念的解决方案，并开发了一个具有三层动机层次结构的计算框架，从而使机器人能够自主学习并专注于获取与设计师和用户目的相关的目标和技能。

开放式学习机器人的目标：一个计算分类、定义和操作化

Purpose for Open-Ended Learning Robots: A Computational Taxonomy,  Definition, and Operationalisation

Autonomous open-ended learning is a relevant approach in machine learning and
robotics, allowing the design of artificial agents able to acquire goals and
motor skills without the necessity of user assigned tasks. A crucial issue for
this approach is to develop strategies to ensure that agents can maximise their
competence on as many tasks as possible in the shortest possible time.
Intrinsic motivations have proven to generate a task-agnostic signal to
properly allocate the training time amongst goals. While the majority of works
in the field of intrinsically motivated open-ended learning focus on scenarios
where goals are independent from each other, only few of them studied the
autonomous acquisition of interdependent tasks, and even fewer tackled
scenarios where goals involve non-stationary interdependencies. Building on
previous works, we tackle these crucial issues at the level of decision making
(i.e., building strategies to properly select between goals), and we propose a
hierarchical architecture that treating sub-tasks selection as a Markov
Decision Process is able to properly learn interdependent skills on the basis
of intrinsically generated motivations. In particular, we first deepen the
analysis of a previous system, showing the importance of incorporating
information about the relationships between tasks at a higher level of the
architecture (that of goal selection). Then we introduce H-GRAIL, a new system
that extends the previous one by adding a new learning layer to store the
autonomously acquired sequences of tasks to be able to modify them in case the
interdependencies are non-stationary. All systems are tested in a real robotic
scenario, with a Baxter robot performing multiple interdependent reaching
tasks.

该论文提出了一种基于马尔可夫决策过程的分层架构，使用内在动机最大化机器人学习多个具有关联性目标的能力，并提出了一种新的系统 H-GRAIL 来记录自主获取的任务序列，以能够在非稳态情况下修改它们。