The sustainable foraging problem is a dynamic environment testbed for
exploring the forms of agent cognition in dealing with social dilemmas in a
multi-agent setting. The agents need to resist the temptation of individual
rewards through foraging and choose the collective long-term goal of
sustainability. We investigate methods of online learning in Neuro-Evolution
and Deep Recurrent Q-Networks to enable agents to attempt the problem one-shot
as is often required by wicked social problems. We further explore if learning
temporal dependencies with Long Short-Term Memory may be able to aid the agents
in developing sustainable foraging strategies in the long term. It was found
that the integration of Long Short-Term Memory assisted agents in developing
sustainable strategies for a single agent, however failed to assist agents in
managing the social dilemma that arises in the multi-agent scenario.

可持续觅食问题是一个多智能体环境试验平台，探索智能体处理社会困境中的认知形式，需要通过觅食抵制个体奖励的诱惑并选择集体的长期可持续目标。我们研究了神经进化和深度递归 Q 网络在线学习的方法，使智能体可以像所需的恶性社会问题一样尝试解决问题。我们进一步探讨了利用长短期记忆学习时间依赖性是否能够帮助智能体开发长期的可持续觅食策略。研究发现，长短期记忆的整合帮助智能体开发了单个智能体的可持续策略，但未能帮助智能体处理多智能体场景中出现的社会困境。

可持续觅食问题的在线学习时间依赖性

Online Learning of Temporal Dependencies for Sustainable Foraging  Problem

The mutual relationship between evolution and learning is a controversial
argument among the artificial intelligence and neuro-evolution communities.
After more than three decades, there is still no common agreement on the
matter. In this paper the author investigates whether combining learning and
evolution permits to find better solutions than those discovered by evolution
alone. More specifically, the author presents a series of empirical studies
that highlight some specific conditions determining the success of such a
combination, like the introduction of noise during the learning and selection
processes. Results are obtained in two qualitatively different domains, where
agent/environment interactions are minimal or absent.

这篇论文通过一系列实证研究探讨结合学习和进化是否比单独使用进化更好，结果表明引入噪声可以改善此过程。

学习与进化：影响有效结合的因素

Learning and evolution: factors influencing an effective combination

A fascinating aspect of nature lies in its ability to produce a collection of
organisms that are all high-performing in their niche. Quality-Diversity (QD)
methods are evolutionary algorithms inspired by this observation, that obtained
great results in many applications, from wing design to robot adaptation.
Recently, several works demonstrated that these methods could be applied to
perform neuro-evolution to solve control problems in large search spaces. In
such problems, diversity can be a target in itself. Diversity can also be a way
to enhance exploration in tasks exhibiting deceptive reward signals. While the
first aspect has been studied in depth in the QD community, the latter remains
scarcer in the literature. Exploration is at the heart of several domains
trying to solve control problems such as Reinforcement Learning and QD methods
are promising candidates to overcome the challenges associated. Therefore, we
believe that standardized benchmarks exhibiting control problems in high
dimension with exploration difficulties are of interest to the QD community. In
this paper, we highlight three candidate benchmarks and explain why they appear
relevant for systematic evaluation of QD algorithms. We also provide
open-source implementations in Jax allowing practitioners to run fast and
numerous experiments on few compute resources.

本篇研究提出了三个基准测试，旨在通过质量多样性方法的系统性评估来解决控制问题和探索难题。

评估优质多样性神经进化算法在难度较高的探索问题中的表现

Assessing Quality-Diversity Neuro-Evolution Algorithms Performance in Hard Exploration Problems

An important goal for the machine learning (ML) community is to create
approaches that can learn solutions with human-level capability. One domain
where humans have held a significant advantage is visual processing. A
significant approach to addressing this gap has been machine learning
approaches that are inspired from the natural systems, such as artificial
neural networks (ANNs), evolutionary computation (EC), and generative and
developmental systems (GDS). Research into deep learning has demonstrated that
such architectures can achieve performance competitive with humans on some
visual tasks; however, these systems have been primarily trained through
supervised and unsupervised learning algorithms. Alternatively, research is
showing that evolution may have a significant role in the development of visual
systems. Thus this paper investigates the role neuro-evolution (NE) can take in
deep learning. In particular, the Hypercube-based NeuroEvolution of Augmenting
Topologies is a NE approach that can effectively learn large neural structures
by training an indirect encoding that compresses the ANN weight pattern as a
function of geometry. The results show that HyperNEAT struggles with performing
image classification by itself, but can be effective in training a feature
extractor that other ML approaches can learn from. Thus NeuroEvolution combined
with other ML methods provides an intriguing area of research that can
replicate the processes in nature.

本篇论文探讨神经进化在深度学习中的作用，其中 HyperNeat 通过训练一种间接编码来实现学习大型神经结构，并且结合其他机器学习方法可以有效提取特征。