In this work, we aim to establish a strong connection between two significant
bodies of machine learning research: continual learning and sequence modeling.
That is, we propose to formulate continual learning as a sequence modeling
problem, allowing advanced sequence models to be utilized for continual
learning. Under this formulation, the continual learning process becomes the
forward pass of a sequence model. By adopting the meta-continual learning (MCL)
framework, we can train the sequence model at the meta-level, on multiple
continual learning episodes. As a specific example of our new formulation, we
demonstrate the application of Transformers and their efficient variants as MCL
methods. Our experiments on seven benchmarks, covering both classification and
regression, show that sequence models can be an attractive solution for general
MCL.

我们在这项工作中旨在建立起机器学习研究的两个重要领域之间的紧密联系：连续学习和序列建模。也就是说，我们提出将连续学习作为序列建模问题进行建模，使得先进的序列模型可以用于连续学习。通过采用元连续学习（MCL）框架，我们可以在元级别上对序列模型进行训练，应用于多个连续学习实验。作为我们新建模的具体示例，我们演示了将 Transformers 及其高效变体用作 MCL 方法的应用。我们在七个基准测试上进行了实验，涵盖了分类和回归问题，结果表明序列模型可以成为通用连续学习的一个有吸引力的解决方案。