In the present era of deep learning, continual learning research is mainly
focused on mitigating forgetting when training a neural network with stochastic
gradient descent on a non-stationary stream of data. On the other hand, in the
more classical literature of statistical machine learning, many models have
sequential Bayesian update rules that yield the same learning outcome as the
batch training, i.e., they are completely immune to catastrophic forgetting.
However, they are often overly simple to model complex real-world data. In this
work, we adopt the meta-learning paradigm to combine the strong
representational power of neural networks and simple statistical models'
robustness to forgetting. In our novel meta-continual learning framework,
continual learning takes place only in statistical models via ideal sequential
Bayesian update rules, while neural networks are meta-learned to bridge the raw
data and the statistical models. Since the neural networks remain fixed during
continual learning, they are protected from catastrophic forgetting. This
approach not only achieves significantly improved performance but also exhibits
excellent scalability. Since our approach is domain-agnostic and
model-agnostic, it can be applied to a wide range of problems and easily
integrated with existing model architectures.

在当前深度学习时代，为了在训练神经网络时使用随机梯度下降处理非平稳流数据时减轻遗忘现象，我们采用元学习范式来结合神经网络的强表示能力与简单统计模型对遗忘现象的抵抗能力，在我们的新颖元持续学习框架中，持续学习仅在统计模型中进行，而神经网络在原始数据和统计模型之间进行元学习，由于神经网络在持续学习过程中保持不变，它们免受遗忘现象的影响。该方法不仅实现了显著提高的性能，而且具有出色的可伸缩性，由于我们的方法与领域和模型无关，因此可以应用于各种问题，并与现有模型架构轻松集成。