Through sequential construction of posteriors on observing data online,
Bayes' theorem provides a natural framework for continual learning. We develop
Variational Auto-Regressive Gaussian Processes (VAR-GPs), a principled
posterior updating mechanism to solve sequential tasks in continual learning.
By relying on sparse inducing point approximations for scalable posteriors, we
propose a novel auto-regressive variational distribution which reveals two
fruitful connections to existing results in Bayesian inference, expectation
propagation and orthogonal inducing points. Mean predictive entropy estimates
show VAR-GPs prevent catastrophic forgetting, which is empirically supported by
strong performance on modern continual learning benchmarks against competitive
baselines. A thorough ablation study demonstrates the efficacy of our modeling
choices.

本研究提出了一种名为 VAR-GPs 的方法，采用贝叶斯定理作为框架，在通过观察数据在线连续学习过程中解决后验更新的问题，并使用稀疏诱导点逼近可扩展后验概率分布，从而避免了灾难性遗忘的问题。经过实验验证，在现代连续学习基准测试中，VAR-GPs 表现优异，证明了作者们建模选择的功效。