Gaussian process (GP) hyperparameter optimization requires repeatedly solving
linear systems with $n \times n$ kernel matrices. To address the prohibitive
$\mathcal{O}(n^3)$ time complexity, recent work has employed fast iterative
numerical methods, like conjugate gradients (CG). However, as datasets increase
in magnitude, the corresponding kernel matrices become increasingly
ill-conditioned and still require $\mathcal{O}(n^2)$ space without
partitioning. Thus, while CG increases the size of datasets GPs can be trained
on, modern datasets reach scales beyond its applicability. In this work, we
propose an iterative method which only accesses subblocks of the kernel matrix,
effectively enabling \emph{mini-batching}. Our algorithm, based on alternating
projection, has $\mathcal{O}(n)$ per-iteration time and space complexity,
solving many of the practical challenges of scaling GPs to very large datasets.
Theoretically, we prove our method enjoys linear convergence and empirically we
demonstrate its robustness to ill-conditioning. On large-scale benchmark
datasets up to four million datapoints our approach accelerates training by a
factor of 2$\times$ to 27$\times$ compared to CG.

通过基于交替投影的迭代算法，我们提出了一种能够有效进行小批量处理的方法，在解决大规模高斯过程训练的实际挑战中，获得了线性收敛并具有良好的鲁棒性，实验证明在大规模基准数据集上，相比于共轭梯度方法，我们的方法加速了 2 倍到 27 倍。

大规模高斯过程的交替投影

Large-Scale Gaussian Processes via Alternating Projection

We design a novel algorithm for optimal transport by drawing from the
entropic optimal transport, mirror descent and conjugate gradients literatures.
Our algorithm is able to compute optimal transport costs with arbitrary
accuracy without running into numerical stability issues. The algorithm is
implemented efficiently on GPUs and is shown empirically to converge more
quickly than traditional algorithms such as Sinkhorn's Algorithm both in terms
of number of iterations and wall-clock time in many cases. We pay particular
attention to the entropy of marginal distributions and show that high entropy
marginals make for harder optimal transport problems, for which our algorithm
is a good fit. We provide a careful ablation analysis with respect to algorithm
and problem parameters, and present benchmarking over the MNIST dataset. The
results suggest that our algorithm can be a useful addition to the
practitioner's optimal transport toolkit. Our code is open-sourced at
this https URL .

我们设计了一种新的算法来计算最优运输成本，该算法结合了熵最优运输、镜像下降和共轭梯度文献，能够高效地在 GPU 上实现，表现出更快的迭代收敛速度，并能适应高熵边际分布的复杂优化问题，我们在 MNIST 数据集上进行了实验，结果显示该算法是实践者最佳运输工具包中有用的补充。

使用镜像下降与共轭梯度的高效准确最优输运

Efficient and Accurate Optimal Transport with Mirror Descent and  Conjugate Gradients

We propose a lower bound on the log marginal likelihood of Gaussian process
regression models that can be computed without matrix factorisation of the full
kernel matrix. We show that approximate maximum likelihood learning of model
parameters by maximising our lower bound retains many of the sparse variational
approach benefits while reducing the bias introduced into parameter learning.
The basis of our bound is a more careful analysis of the log-determinant term
appearing in the log marginal likelihood, as well as using the method of
conjugate gradients to derive tight lower bounds on the term involving a
quadratic form. Our approach is a step forward in unifying methods relying on
lower bound maximisation (e.g. variational methods) and iterative approaches
based on conjugate gradients for training Gaussian processes. In experiments,
we show improved predictive performance with our model for a comparable amount
of training time compared to other conjugate gradient based approaches.

本文提出一种无需完全核矩阵的矩阵分解即可计算的高斯过程回归模型的对数边际似然的下界。我们通过最大化我们的下界来学习模型参数的近似最大似然方法保留了许多稀疏变分方法的优点，同时减少了参数学习中引入的偏差。我们的方法通过对出现在对数边际似然中的对数行列式项进行更仔细的分析，以及使用共轭梯度法导出涉及二次形式的项的紧凑下界，从而在统一依赖下界最大化的方法和基于共轭梯度的迭代方法的训练高斯过程方面迈出了一步。实验结果表明，相对于其他基于共轭梯度的方法，在相当的训练时间内，我们的模型具有更好的预测性能。