General-purpose learning systems should improve themselves in open-ended
fashion in ever-changing environments. Conventional learning algorithms for
neural networks, however, suffer from catastrophic forgetting (CF) --
previously acquired skills are forgotten when a new task is learned. Instead of
hand-crafting new algorithms for avoiding CF, we propose Automated Continual
Learning (ACL) to train self-referential neural networks to meta-learn their
own in-context continual (meta-)learning algorithms. ACL encodes all desiderata
-- good performance on both old and new tasks -- into its meta-learning
objectives. Our experiments demonstrate that ACL effectively solves "in-context
catastrophic forgetting"; our ACL-learned algorithms outperform hand-crafted
ones, e.g., on the Split-MNIST benchmark in the replay-free setting, and
enables continual learning of diverse tasks consisting of multiple few-shot and
standard image classification datasets.

通用学习系统应在不断变化的环境中以开放式方式不断改进自己。本文提出一种自动连续学习 (ACL) 的方法，通过训练自指神经网络来元学习其上下文中的连续学习算法，以解决传统神经网络学习算法中的 “上下文灾难性遗忘” 问题。ACL 有效地解决了这个问题，并且其学习到的算法表现优于手工设计的算法，在无回放设置下，在 Split-MNIST 基准测试上实现了持续学习多个少样本和标准图像分类数据集的目标。