Open-ended and AI-generating algorithms aim to continuously generate and
solve increasingly complex tasks indefinitely, offering a promising path toward
more general intelligence. To accomplish this grand vision, learning must occur
within a vast array of potential tasks. Existing approaches to automatically
generating environments are constrained within manually predefined, often
narrow distributions of environment, limiting their ability to create any
learning environment. To address this limitation, we introduce a novel
framework, OMNI-EPIC, that augments previous work in Open-endedness via Models
of human Notions of Interestingness (OMNI) with Environments Programmed in Code
(EPIC). OMNI-EPIC leverages foundation models to autonomously generate code
specifying the next learnable (i.e., not too easy or difficult for the agent's
current skill set) and interesting (e.g., worthwhile and novel) tasks.
OMNI-EPIC generates both environments (e.g., an obstacle course) and reward
functions (e.g., progress through the obstacle course quickly without touching
red objects), enabling it, in principle, to create any simulatable learning
task. We showcase the explosive creativity of OMNI-EPIC, which continuously
innovates to suggest new, interesting learning challenges. We also highlight
how OMNI-EPIC can adapt to reinforcement learning agents' learning progress,
generating tasks that are of suitable difficulty. Overall, OMNI-EPIC can
endlessly create learnable and interesting environments, further propelling the
development of self-improving AI systems and AI-Generating Algorithms. Project
website with videos: this https URL

本研究介绍了一种新颖的框架 OMNI-EPIC，通过在人类有趣概念模型（OMNI）中加入编程环境（EPIC），使用基础模型自动生成学习任务的代码和环境，从而创造各种可学习和有趣的环境，进一步推动自我进化的 AI 系统和 AI 生成算法的发展。

OMNI-EPIC: 通过编程环境中的人类有趣观念模型实现的开放性

OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness  with Environments Programmed in Code

Reinforcement learning (RL) offers a capable and intuitive structure for the
fundamental sequential decision-making problem. Despite impressive
breakthroughs, it can still be difficult to employ RL in practice in many
simple applications. In this paper, we try to address this issue by introducing
a method for designing the components of the RL environment for a given,
user-intended application. We provide an initial formalization for the problem
of RL component design, that concentrates on designing a good representation
for observation and action space. We propose a method named DeLF: Designing
Learning Environments with Foundation Models, that employs large language
models to design and codify the user's intended learning scenario. By testing
our method on four different learning environments, we demonstrate that DeLF
can obtain executable environment codes for the corresponding RL problems.

通过使用大语言模型设计和编码用户预期的学习场景，我们提出了一种名为 DeLF 的方法，用于设计强化学习环境的组件，以解决在实践中应用 RL 在许多简单应用中仍然困难的问题。我们通过在四个不同的学习环境上测试我们的方法，证明 DeLF 能够为相应的 RL 问题获得可执行的环境代码。

DeLF: 使用基础模型设计学习环境

DeLF: Designing Learning Environments with Foundation Models

Recent advances in artificial intelligence have been driven by the presence
of increasingly realistic and complex simulated environments. However, many of
the existing environments provide either unrealistic visuals, inaccurate
physics, low task complexity, restricted agent perspective, or a limited
capacity for interaction among artificial agents. Furthermore, many platforms
lack the ability to flexibly configure the simulation, making the simulated
environment a black-box from the perspective of the learning system. In this
work, we propose a novel taxonomy of existing simulation platforms and discuss
the highest level class of general platforms which enable the development of
learning environments that are rich in visual, physical, task, and social
complexity. We argue that modern game engines are uniquely suited to act as
general platforms and as a case study examine the Unity engine and open source
Unity ML-Agents Toolkit. We then survey the research enabled by Unity and the
Unity ML-Agents Toolkit, discussing the kinds of research a flexible,
interactive and easily configurable general platform can facilitate.

该论文提出了一个现有模拟平台的新分类法，并探讨了通用平台的最高级别类，这些平台可以开发具有丰富的可视化、物理、任务和社交复杂性的学习环境。作者认为，现代游戏引擎非常适合作为通用平台，并以 Unity 引擎和开放源代码 Unity ML-Agents Toolkit 为案例研究。接着，作者调查了 Unity 和 Unity ML-Agents Toolkit 所支持的研究，讨论了灵活、交互和易配置的通用平台可以促进哪些研究。