We present a non-convex optimization algorithm metaheuristic, based on the
training of a deep generative network, which enables effective searching within
continuous, ultra-high dimensional landscapes. During network training,
populations of sampled local gradients are utilized within a customized loss
function to evolve the network output distribution function towards one peak at
high-performing optima. The deep network architecture is tailored to support
progressive growth over the course of training, which allows the algorithm to
manage the curse of dimensionality characteristic of high-dimensional
landscapes. We apply our concept to a range of standard optimization problems
with dimensions as high as one thousand and show that our method performs
better with fewer function evaluations compared to state-of-the-art algorithm
benchmarks. We also discuss the role of deep network over-parameterization,
loss function engineering, and proper network architecture selection in
optimization, and why the required batch size of sampled local gradients is
independent of problem dimension. These concepts form the foundation for a new
class of algorithms that utilize customizable and expressive deep generative
networks to solve non-convex optimization problems.

我们提出了一种非凸优化算法元启发式方法，基于深度生成网络的训练，能在连续、超高维度的空间中实现有效搜索。通过网络训练，利用采样的局部梯度种群在定制的损失函数内，将网络输出分布函数演化朝向高性能最优点的一个峰值。深度网络架构根据训练过程进行渐进增长，使得该算法能够处理高维空间的维度困境。我们将这一概念应用于一系列标准优化问题，维度高达一千，证明我们的方法在较少函数评估下表现更好，超过了现有算法基准。我们还讨论了深度网络超参数化、损失函数设计和适当网络架构选择在优化中的作用，以及采样的局部梯度所需的批量大小与问题维度无关。这些概念为一类利用可自定义和表达性强的深度生成网络来解决非凸优化问题的算法奠定了基础。