Gaussian processes (GPs) are powerful non-parametric function estimators.
However, their applications are largely limited by the expensive computational
cost of the inference procedures. Existing stochastic or distributed
synchronous variational inferences, although have alleviated this issue by
scaling up GPs to millions of samples, are still far from satisfactory for
real-world large applications, where the data sizes are often orders of
magnitudes larger, say, billions. To solve this problem, we propose ADVGP, the
first Asynchronous Distributed Variational Gaussian Process inference for
regression, on the recent large-scale machine learning platform,
PARAMETERSERVER. ADVGP uses a novel, flexible variational framework based on a
weight space augmentation, and implements the highly efficient, asynchronous
proximal gradient optimization. While maintaining comparable or better
predictive performance, ADVGP greatly improves upon the efficiency of the
existing variational methods. With ADVGP, we effortlessly scale up GP
regression to a real-world application with billions of samples and demonstrate
an excellent, superior prediction accuracy to the popular linear models.

提出了一种基于参数服务器的异步分布式变分高斯过程回归方法 ADVGP 解决了高斯过程的计算成本问题，提高了可伸缩性、预测性能和回归精度。