We present DARLEI, a framework that combines evolutionary algorithms with
parallelized reinforcement learning for efficiently training and evolving
populations of UNIMAL agents. Our approach utilizes Proximal Policy
Optimization (PPO) for individual agent learning and pairs it with a tournament
selection-based generational learning mechanism to foster morphological
evolution. By building on Nvidia's Isaac Gym, DARLEI leverages GPU accelerated
simulation to achieve over 20x speedup using just a single workstation,
compared to previous work which required large distributed CPU clusters. We
systematically characterize DARLEI's performance under various conditions,
revealing factors impacting diversity of evolved morphologies. For example, by
enabling inter-agent collisions within the simulator, we find that we can
simulate some multi-agent interactions between the same morphology, and see how
it influences individual agent capabilities and long-term evolutionary
adaptation. While current results demonstrate limited diversity across
generations, we hope to extend DARLEI in future work to include interactions
between diverse morphologies in richer environments, and create a platform that
allows for coevolving populations and investigating emergent behaviours in
them. Our source code is also made publicly at
this https URL

我们提出了一个框架 DARLEI，结合了进化算法和并行强化学习，用于高效训练和演化 UNIMAL 智能体的种群。我们的方法利用 Proximal Policy Optimization (PPO) 进行个体智能体的学习，并与基于锦标赛选择的生成式学习机制相结合，促进形态进化。通过建立在 Nvidia 的 Isaac Gym 上，DARLEI 利用 GPU 加速模拟，在仅使用单个工作站的情况下，实现了超过 20 倍的加速比，而之前的方法需要大规模的分布式 CPU 集群。我们系统地特征化了 DARLEI 在各种条件下的性能，揭示了影响进化形态多样性的因素。当前的结果显示世代间的多样性有限，我们希望将来能够扩展 DARLEI，以在更丰富的环境中包括不同形态之间的相互作用，并创建一个允许共同演化种群和研究其中新兴行为的平台。我们的源代码也公开在此 https URL。