This paper introduces kernel continual learning, a simple but effective variant of continual learning that leverages the non-parametric nature of kernel methods to tackle catastrophic forgetting. We deploy an episodic memory unit that stores a subset of samples for each task to learn task-specific classifiers based on kernel ridge regression. This does not require memory replay and systematically avoids task interference in the classifiers. We further introduce variational random features to learn a data-driven kernel for each task. To do so, we formulate kernel continual learning as a variational inference problem, where a random Fourier basis is incorporated as the latent variable. The variational posterior distribution over the random Fourier basis is inferred from the coreset of each task. In this way, we are able to generate more informative kernels specific to each task, and, more importantly, the coreset size can be reduced to achieve more compact memory, resulting in more efficient continual learning based on episodic memory. Extensive evaluation on four benchmarks demonstrates the effectiveness and promise of kernels for continual learning.

本文介绍基于kernel方法的kernel continual learning模型，通过采用episodic memory存储子集样本的方式，利用kernel ridge regression算法学习任务特定的分类器，避免了内存重放以及分类器中存在任务干扰的问题。进一步，引入variational random features方法学习数据驱动的kernel，而不需要提前选择kernel函数。在四个基准测试上的结果表明了kernel在continual learning任务中的有效性和灵活性。

核心连续学习