Quality-Diversity (QD) algorithms have emerged as a powerful optimization
paradigm with the aim of generating a set of high-quality and diverse
solutions. To achieve such a challenging goal, QD algorithms require
maintaining a large archive and a large population in each iteration, which
brings two main issues, sample and resource efficiency. Most advanced QD
algorithms focus on improving the sample efficiency, while the resource
efficiency is overlooked to some extent. Particularly, the resource overhead
during the training process has not been touched yet, hindering the wider
application of QD algorithms. In this paper, we highlight this important
research question, i.e., how to efficiently train QD algorithms with limited
resources, and propose a novel and effective method called RefQD to address it.
RefQD decomposes a neural network into representation and decision parts, and
shares the representation part with all decision parts in the archive to reduce
the resource overhead. It also employs a series of strategies to address the
mismatch issue between the old decision parts and the newly updated
representation part. Experiments on different types of tasks from small to
large resource consumption demonstrate the excellent performance of RefQD: it
not only uses significantly fewer resources (e.g., 16\% GPU memories on QDax
and 3.7\% on Atari) but also achieves comparable or better performance compared
to sample-efficient QD algorithms. Our code is available at
https://github.com/lamda-bbo/RefQD.

如何用有限资源高效训练品质多样性算法（QD algorithms），这篇论文提出了一种名为 RefQD 的新方法，通过将神经网络分解为表示部分和决策部分，并在存档中共享表示部分，以减少资源开销。RefQD 在各种资源消耗大小的任务上进行的实验表明了其出色的性能。