While much research effort has been dedicated to scaling up sparse Gaussian process (GP) models based on inducing variables for big data, little attention is afforded to the other less explored class of low-rank GP approximations that exploit the sparse spectral representation of a GP kernel. This paper presents such an effort to advance the state of the art of sparse spectrum GP models to achieve competitive predictive performance for massive datasets. Our generalized framework of stochastic variational Bayesian sparse spectrum GP (sVBSSGP) models addresses their shortcomings by adopting a Bayesian treatment of the spectral frequencies to avoid overfitting, modeling these frequencies jointly in its variational distribution to enable their interaction a posteriori, and exploiting local data for boosting the predictive performance. However, such structural improvements result in a variational lower bound that is intractable to be optimized. To resolve this, we exploit a variational parameterization trick to make it amenable to stochastic optimization. Interestingly, the resulting stochastic gradient has a linearly decomposable structure that can be exploited to refine our stochastic optimization method to incur constant time per iteration while preserving its property of being an unbiased estimator of the exact gradient of the variational lower bound. Empirical evaluation on real-world datasets shows that sVBSSGP outperforms state-of-the-art stochastic implementations of sparse GP models.

本文针对大数据的稀疏高斯过程模型的一类不同于现有方法的低秩GP逼近模型展开研究，提出了基于稀疏频谱GP模型的随机变分贝叶斯框架，该框架结合贝叶斯方法处理频率谱避免过拟合，利用本地数据提高预测性能，并利用参数化技巧使得所得到的随机梯度具有线性结构，从而提高了稀疏GP模型的性能。实验结果表明，我们的模型优于现有的SGP模型的实现方法。

一种用于稀疏谱高斯过程回归的广义随机变分贝叶斯超参数学习框架