Gaussian Processes (GPs) are Bayesian models that provide uncertainty
estimates associated to the predictions made. They are also very flexible due
to their non-parametric nature. Nevertheless, GPs suffer from poor scalability
as the number of training instances N increases. More precisely, they have a
cubic cost with respect to $N$. To overcome this problem, sparse GP
approximations are often used, where a set of $M \ll N$ inducing points is
introduced during training. The location of the inducing points is learned by
considering them as parameters of an approximate posterior distribution $q$.
Sparse GPs, combined with variational inference for inferring $q$, reduce the
training cost of GPs to $\mathcal{O}(M^3)$. Critically, the inducing points
determine the flexibility of the model and they are often located in regions of
the input space where the latent function changes. A limitation is, however,
that for some learning tasks a large number of inducing points may be required
to obtain a good prediction performance. To address this limitation, we propose
here to amortize the computation of the inducing points locations, as well as
the parameters of the variational posterior approximation q. For this, we use a
neural network that receives the observed data as an input and outputs the
inducing points locations and the parameters of $q$. We evaluate our method in
several experiments, showing that it performs similar or better than other
state-of-the-art sparse variational GP approaches. However, with our method the
number of inducing points is reduced drastically due to their dependency on the
input data. This makes our method scale to larger datasets and have faster
training and prediction times.

使用神经网络进行影子点位置计算和变分后验分布参数估计可提高 Gaussian Processes 的可扩展性。

输入相关的稀疏高斯过程

Input Dependent Sparse Gaussian Processes

Large, multi-dimensional spatio-temporal datasets are omnipresent in modern
science and engineering. An effective framework for handling such data are
Gaussian process deep generative models (GP-DGMs), which employ GP priors over
the latent variables of DGMs. Existing approaches for performing inference in
GP-DGMs do not support sparse GP approximations based on inducing points, which
are essential for the computational efficiency of GPs, nor do they handle
missing data -- a natural occurrence in many spatio-temporal datasets -- in a
principled manner. We address these shortcomings with the development of the
sparse Gaussian process variational autoencoder (SGP-VAE), characterised by the
use of partial inference networks for parameterising sparse GP approximations.
Leveraging the benefits of amortised variational inference, the SGP-VAE enables
inference in multi-output sparse GPs on previously unobserved data with no
additional training. The SGP-VAE is evaluated in a variety of experiments where
it outperforms alternative approaches including multi-output GPs and structured
VAEs.

研究了高斯过程深度生成模型中的氐变量的稀疏高斯过程近似的问题，并提出了一种基于部分推理网络的稀疏高斯过程变分自编码器，从而使得稀疏高斯过程能处理多维度的时空数据中缺失的数据，并提高模型的计算效率。