We address the problem of continual learning in multi-task Gaussian process (GP) models for handling sequential input-output observations. Our approach extends the existing prior-posterior recursion of online Bayesian inference, i.e.\ past posterior discoveries become future prior beliefs, to the infinite functional space setting of GP. For a reason of scalability, we introduce variational inference together with an sparse approximation based on inducing inputs. As a consequence, we obtain tractable continual lower-bounds where two novel Kullback-Leibler (KL) divergences intervene in a natural way. The key technical property of our method is the recursive reconstruction of conditional GP priors conditioned on the variational parameters learned so far. To achieve this goal, we introduce a novel factorization of past variational distributions, where the predictive GP equation propagates the posterior uncertainty forward. We then demonstrate that it is possible to derive GP models over many types of sequential observations, either discrete or continuous and amenable to stochastic optimization. The continual inference approach is also applicable to scenarios where potential multi-channel or heterogeneous observations might appear. Extensive experiments demonstrate that the method is fully scalable, shows a reliable performance and is robust to uncertainty error propagation over a plenty of synthetic and real-world datasets.

本篇论文研究了在多任务高斯过程模型中如何持续不断地学习处理顺序输入输出观测值的问题，提出了基于在线贝叶斯推断先验——后验递归的方法，并结合诱导输入的稀疏近似引入了变分推断。通过该方法得到了可行的无限条件下界，并在自然情况下引入两个新的KL散度。本研究方法的关键技术是基于条件GP先验的递归重构，实现了对迄今为止学习到的变分参数的条件GP先验的递归重构。同时，该方法适用于许多类型的连续或离散的顺序观测，并且适用于可能出现潜在的多通道或异构观测值的场景。广泛的实验证明该方法具有可扩展性，表现可靠，并且对合成和真实世界的大量数据集的误差传播具有鲁棒性。

连续多任务高斯过程