Optimizing functions without access to gradients is the remit of black-box
methods such as evolution strategies. While highly general, their learning
dynamics are often times heuristic and inflexible - exactly the limitations
that meta-learning can address. Hence, we propose to discover effective update
rules for evolution strategies via meta-learning. Concretely, our approach
employs a search strategy parametrized by a self-attention-based architecture,
which guarantees the update rule is invariant to the ordering of the candidate
solutions. We show that meta-evolving this system on a small set of
representative low-dimensional analytic optimization problems is sufficient to
discover new evolution strategies capable of generalizing to unseen
optimization problems, population sizes and optimization horizons. Furthermore,
the same learned evolution strategy can outperform established neuroevolution
baselines on supervised and continuous control tasks. As additional
contributions, we ablate the individual neural network components of our
method; reverse engineer the learned strategy into an explicit heuristic form,
which remains highly competitive; and show that it is possible to
self-referentially train an evolution strategy from scratch, with the learned
update rule used to drive the outer meta-learning loop.

该研究采用元学习来探寻对进化策略更有效的更新规则，并通过自我关注机制加以参数化，其结果显示新进化策略推广到未知的优化问题、种群大小和优化目标上表现出良好的性能。研究结果还表明，所学习的进化策略可以在监督学习和连续控制任务中优于传统的神经进化算法。

通过元黑盒优化发现进化策略

Discovering Evolution Strategies via Meta-Black-Box Optimization

We present TraDE, a self-attention-based architecture for auto-regressive
density estimation with continuous and discrete valued data. Our model is
trained using a penalized maximum likelihood objective, which ensures that
samples from the density estimate resemble the training data distribution. The
use of self-attention means that the model need not retain conditional
sufficient statistics during the auto-regressive process beyond what is needed
for each covariate. On standard tabular and image data benchmarks, TraDE
produces significantly better density estimates than existing approaches such
as normalizing flow estimators and recurrent auto-regressive models. However
log-likelihood on held-out data only partially reflects how useful these
estimates are in real-world applications. In order to systematically evaluate
density estimators, we present a suite of tasks such as regression using
generated samples, out-of-distribution detection, and robustness to noise in
the training data and demonstrate that TraDE works well in these scenarios.

TraDE 是一种自注意力的架构，用于连续和离散值数据的自回归密度估计，使用罚最大似然目标进行训练，可以产生比现有方法更好的密度估计，如归一化流估计器和循环自回归模型。