We study the relationship between catastrophic forgetting and properties of task sequences. In particular, given a sequence of tasks, we would like to understand which properties of this sequence influence the error rates of continual learning algorithms trained on the sequence. To this end, we propose a new procedure that makes use of recent developments in task space modeling as well as correlation analysis to specify and analyze the properties we are interested in. As an application, we apply our procedure to study two properties of a task sequence: (1) total complexity and (2) sequential heterogeneity. We show that error rates are strongly and positively correlated to a task sequence's total complexity for some state-of-the-art algorithms. We also show that, surprisingly, the error rates have no or even negative correlations in some cases to sequential heterogeneity. Our findings suggest directions for improving continual learning benchmarks and methods.

本文研究了灾难性遗忘和任务序列属性之间的关系，尤其是给定任务序列，我们希望了解这个序列的哪些属性会影响在该序列上训练的永续学习算法的错误率。我们提出了一种新的程序，利用最近在任务空间建模和相关性分析方面的发展来指定和分析我们感兴趣的属性。作为一个应用，我们将我们的程序应用于研究任务序列的两个属性：“总复杂度”和“顺序异质性”。我们发现，对于一些最先进的算法，错误率与任务序列的总复杂度强烈且正相关；令人惊讶的是，在某些情况下，错误率与顺序异质性没有或甚至具有负相关性。我们的发现为改进永续学习基准和方法提供了方向。

持续学习中灾难性遗忘的理解