With the development of hardware accelerators and their corresponding tools,
evaluations have become more affordable through fast and massively parallel
evaluations in some applications. This advancement has drastically sped up the
runtime of evolution-inspired algorithms such as Quality-Diversity
optimization, creating tremendous potential for algorithmic innovation through
scale. In this work, we propose MAP-Elites-Multi-ES (MEMES), a novel QD
algorithm based on Evolution Strategies (ES) designed for fast parallel
evaluations. ME-Multi-ES builds on top of the existing MAP-Elites-ES algorithm,
scaling it by maintaining multiple independent ES threads with massive
parallelization. We also introduce a new dynamic reset procedure for the
lifespan of the independent ES to autonomously maximize the improvement of the
QD population. We show experimentally that MEMES outperforms existing
gradient-based and objective-agnostic QD algorithms when compared in terms of
generations. We perform this comparison on both black-box optimization and
QD-Reinforcement Learning tasks, demonstrating the benefit of our approach
across different problems and domains. Finally, we also find that our approach
intrinsically enables optimization of fitness locally around a niche, a
phenomenon not observed in other QD algorithms.

本文提出了一种基于进化策略的快速并行评估，名为 MAP-Elites-Multi-ES（MEMES）的新型 QD 算法，该算法通过保持多个独立的进化策略线程来扩展现有的 MAP-Elites-ES 算法，并引入了一种新的动态重置过程来自主地最大化 QD 种群的改进。实验结果表明，MEMES 在黑盒优化和 QD-Reinforcement 学习任务方面表现优越，并具有本地优化适应性的内在优势。