Natural gradient methods have been used to optimise the parameters of
probability distributions in a variety of settings, often resulting in
fast-converging procedures. Unfortunately, for many distributions of interest,
computing the natural gradient has a number of challenges. In this work we
propose a novel technique for tackling such issues, which involves reframing
the optimisation as one with respect to the parameters of a surrogate
distribution, for which computing the natural gradient is easy. We give several
examples of existing methods that can be interpreted as applying this
technique, and propose a new method for applying it to a wide variety of
problems. Our method expands the set of distributions that can be efficiently
targeted with natural gradients. Furthermore, it is fast, easy to understand,
simple to implement using standard autodiff software, and does not require
lengthy model-specific derivations. We demonstrate our method on maximum
likelihood estimation and variational inference tasks.

通过引入替代分布参数的优化，我们提出了一种用于克服自然梯度计算困难的新技术，该方法适用于多种问题，并且扩展了可以高效利用自然梯度的分布集合，可用于最大似然估计和变分推断任务。

优化自然梯度替代品的分布

Optimising Distributions with Natural Gradient Surrogates

Finding efficient optimization methods plays an important role for quantum
optimization and quantum machine learning on near-term quantum computers. While
backpropagation on classical computers is computationally efficient, obtaining
gradients on quantum computers is not, because the computational complexity
usually scales with the number of parameters and measurements. In this paper,
we connect Koopman operator theory, which has been successful in predicting
nonlinear dynamics, with natural gradient methods in quantum optimization. We
propose a data-driven approach using Koopman operator learning to accelerate
quantum optimization and quantum machine learning. We develop two new families
of methods: the sliding window dynamic mode decomposition (DMD) and the neural
DMD for efficiently updating parameters on quantum computers. We show that our
methods can predict gradient dynamics on quantum computers and accelerate the
variational quantum eigensolver used in quantum optimization, as well as
quantum machine learning. We further implement our Koopman operator learning
algorithm on a real IBM quantum computer and demonstrate their practical
effectiveness.

该论文使用 Koopman 算子学习通过动态模态分解和神经动态模态分解的两种新方法加速量子优化和量子机器学习，并在实际 IBM 量子计算机上进行了实现和测试。

用 Koopman 运算符学习加速量子优化和机器学习

Koopman Operator learning for Accelerating Quantum Optimization and Machine Learning

Deep feedforward neural networks (DFNNs) are a powerful tool for functional
approximation. We describe flexible versions of generalized linear and
generalized linear mixed models incorporating basis functions formed by a DFNN.
The consideration of neural networks with random effects is not widely used in
the literature, perhaps because of the computational challenges of
incorporating subject specific parameters into already complex models.
Efficient computational methods for high-dimensional Bayesian inference are
developed using Gaussian variational approximation, with a parsimonious but
flexible factor parametrization of the covariance matrix. We implement natural
gradient methods for the optimization, exploiting the factor structure of the
variational covariance matrix in computation of the natural gradient. Our
flexible DFNN models and Bayesian inference approach lead to a regression and
classification method that has a high prediction accuracy, and is able to
quantify the prediction uncertainty in a principled and convenient way. We also
describe how to perform variable selection in our deep learning method. The
proposed methods are illustrated in a wide range of simulated and real-data
examples, and the results compare favourably to a state of the art flexible
regression and classification method in the statistical literature, the
Bayesian additive regression trees (BART) method. User-friendly software
packages in Matlab, R and Python implementing the proposed methods are
available at this https URL

使用深度前馈神经网络、广义线性模型和贝叶斯推断等方法，提出一种高精度的回归和分类方法，能够在原有方法中量化预测不确定性并进行变量选择，并在多种模拟和真实数据示例中得到了令人满意的结果。