A prevalent limitation of optimizing over a single objective is that it can
be misguided, becoming trapped in local optimum. This can be rectified by
Quality-Diversity (QD) algorithms, where a population of high-quality and
diverse solutions to a problem is preferred. Most conventional QD approaches,
for example, MAP-Elites, explicitly manage a behavioral archive where solutions
are broken down into predefined niches. In this work, we show that a diverse
population of solutions can be found without the limitation of needing an
archive or defining the range of behaviors in advance. Instead, we break down
solutions into independently evolving species and use unsupervised skill
discovery to learn diverse, high-performing solutions. We show that this can be
done through gradient-based mutations that take on an information theoretic
perspective of jointly maximizing mutual information and performance. We
propose Diverse Quality Species (DQS) as an alternative to archive-based QD
algorithms. We evaluate it over several simulated robotic environments and show
that it can learn a diverse set of solutions from varying species. Furthermore,
our results show that DQS is more sample-efficient and performant when compared
to other QD algorithms. Relevant code and hyper-parameters are available at:
this https URL

本文提出了一种名为 Diverse Quality Species (DQS) 的新型质量 - 多样性算法，能够在不需要存档或事先定义行为范围的情况下，将解决方案分解为独立进化种类，并利用无监督技能发现来学习多样化而高性能的解决方案，在多个仿真机器人环境中进行评估，结果表明 DQS 比其他 QD 算法更具样本效率和性能。

通过多样质量种群实现高效的质量多样性优化

Efficient Quality-Diversity Optimization through Diverse Quality Species

Quality-Diversity (QD) algorithms are designed to generate collections of
high-performing solutions while maximizing their diversity in a given
descriptor space. However, in the presence of unpredictable noise, the fitness
and descriptor of the same solution can differ significantly from one
evaluation to another, leading to uncertainty in the estimation of such values.
Given the elitist nature of QD algorithms, they commonly end up with many
degenerate solutions in such noisy settings. In this work, we introduce Archive
Reproducibility Improvement Algorithm (ARIA); a plug-and-play approach that
improves the reproducibility of the solutions present in an archive. We propose
it as a separate optimization module, relying on natural evolution strategies,
that can be executed on top of any QD algorithm. Our module mutates solutions
to (1) optimize their probability of belonging to their niche, and (2) maximize
their fitness. The performance of our method is evaluated on various tasks,
including a classical optimization problem and two high-dimensional control
tasks in simulated robotic environments. We show that our algorithm enhances
the quality and descriptor space coverage of any given archive by at least 50%.

本文提出了一种 Archive Reproducibility Improvement Algorithm（ARIA）的方法，基于自然进化策略，用于改善质量 - 多样性（Quality-Diversity）算法在噪声环境下的可重现性，并对其在优化问题和模拟机器人环境中进行了测试，得出其可以提高给定存档的质量和描述符空间覆盖率至少 50% 的结论。