Supervised Continual learning involves updating a deep neural network (DNN)
from an ever-growing stream of labeled data. While most work has focused on
overcoming catastrophic forgetting, one of the major motivations behind
continual learning is being able to efficiently update a network with new
information, rather than retraining from scratch on the training dataset as it
grows over time. Despite recent continual learning methods largely solving the
catastrophic forgetting problem, there has been little attention paid to the
efficiency of these algorithms. Here, we study recent methods for incremental
class learning and illustrate that many are highly inefficient in terms of
compute, memory, and storage. Some methods even require more compute than
training from scratch! We argue that for continual learning to have real-world
applicability, the research community cannot ignore the resources used by these
algorithms. There is more to continual learning than mitigating catastrophic
forgetting.

本文研究了增量类学习的最新方法，并指出很多算法在计算、存储和内存方面效率极低，有些甚至比从头开始训练的方法更费时，作者认为在实际应用中，不能忽视这些算法的资源消耗，持续学习不仅仅只是为了减轻灾难性遗忘。