We introduce the Kronecker factored online Laplace approximation for
overcoming catastrophic forgetting in neural networks. The method is grounded
in a Bayesian online learning framework, where we recursively approximate the
posterior after every task with a Gaussian, leading to a quadratic penalty on
changes to the weights. The Laplace approximation requires calculating the
Hessian around a mode, which is typically intractable for modern architectures.
In order to make our method scalable, we leverage recent block-diagonal
Kronecker factored approximations to the curvature. Our algorithm achieves over
90% test accuracy across a sequence of 50 instantiations of the permuted MNIST
dataset, substantially outperforming related methods for overcoming
catastrophic forgetting.

介绍了 Kronecker 因式分解的在线 Laplace 近似方法，用于克服神经网络中的灾难性遗忘问题，该方法建立在贝叶斯在线学习框架的基础上，以高斯分布递归逼近后验概率，通过计算模牛顿法到达二阶导近似，利用 K-FAC 近似方法实现可扩展性，50 个 MNIST 图像数字识别测试中达到 90% 的准确率，明显优于其他相关方法。