Neural networks suffer from catastrophic forgetting and are unable to
sequentially learn new tasks without guaranteed stationarity in data
distribution. Continual learning could be achieved via replay -- by
concurrently training externally stored old data while learning a new task.
However, replay becomes less effective when each past task is allocated with
less memory. To overcome this difficulty, we supplemented replay mechanics with
meta-learning for rapid knowledge acquisition. By employing a meta-learner,
which \textit{learns a learning rate per parameter per past task}, we found
that base learners produced strong results when less memory was available.
Additionally, our approach inherited several meta-learning advantages for
continual learning: it demonstrated strong robustness to continually learn
under the presence of noises and yielded base learners to higher accuracy in
less updates.

本文研究神经网络存在的遗忘问题以及连续学习的解决方案，通过重放机制和元学习的结合，发现引入元学习可以解决传统重放机制在每个任务分配的内存有限时容易导致失效的问题，并在保证学习效率和准确性方面具有优势。