Open-ended algorithms aim to learn new, interesting behaviors forever. That
requires a vast environment search space, but there are thus infinitely many
possible tasks. Even after filtering for tasks the current agent can learn
(i.e., learning progress), countless learnable yet uninteresting tasks remain
(e.g., minor variations of previously learned tasks). An Achilles Heel of
open-endedness research is the inability to quantify (and thus prioritize)
tasks that are not just learnable, but also $\textit{interesting}$ (e.g.,
worthwhile and novel). We propose solving this problem by
$\textit{Open-endedness via Models of human Notions of Interestingness}$
(OMNI). The insight is that we can utilize large (language) models (LMs) as a
model of interestingness (MoI), because they $\textit{already}$ internalize
human concepts of interestingness from training on vast amounts of
human-generated data, where humans naturally write about what they find
interesting or boring. We show that LM-based MoIs improve open-ended learning
by focusing on tasks that are both learnable $\textit{and interesting}$,
outperforming baselines based on uniform task sampling or learning progress
alone. This approach has the potential to dramatically advance the ability to
intelligently select which tasks to focus on next (i.e., auto-curricula), and
could be seen as AI selecting its own next task to learn, facilitating
self-improving AI and AI-Generating Algorithms.

利用基于语言模型的有趣性模型（MoI）改进无止境机器学习，集中于可学习且有趣的任务，实现自我优化和自我生成算法。

OMNI：基于人类有趣概念模型的开放性问题解决方案

OMNI: Open-endedness via Models of human Notions of Interestingness

Most real-world domains can be formulated as multi-agent (MA) systems.
Intentionality sharing agents can solve more complex tasks by collaborating,
possibly in less time. True cooperative actions are beneficial for egoistic and
collective reasons. However, teaching individual agents to sacrifice egoistic
benefits for a better collective performance seems challenging. We build on a
recently proposed Multi-Agent Reinforcement Learning (MARL) mechanism with a
Graph Neural Network (GNN) communication layer. Rarely chosen communication
actions were marginally beneficial. Here we propose a MARL system in which
agents can help collaborators perform better while risking low individual
performance. We conduct our study in the context of resource distribution for
wildfire management. Communicating environmental features and partially
observable fire occurrence help the agent collective to pre-emptively
distribute resources. Furthermore, we introduce a procedural training
environment accommodating auto-curricula and open-endedness towards better
generalizability. Our MA communication proposal outperforms a Greedy Heuristic
Baseline and a Single-Agent (SA) setup. We further demonstrate how
auto-curricula and openendedness improves generalizability of our MA proposal.

提出一种基于多智能体强化学习和图神经网络通讯层的方法，在火灾管理资源分配中，通过通信环境特征和部分可见火灾，实现协作分配资源，优于贪心启发式基线和单智能体设置，并展示了自动课程和开放性对其泛化能力的改善。

野火管理资源分配的图神经网络通信层协作自适应多智能体强化学习

Collaborative Auto-Curricula Multi-Agent Reinforcement Learning with Graph Neural Network Communication Layer for Open-ended Wildfire-Management Resource Distribution