It was recently shown that architectural, regularization and rehearsal
strategies can be used to train deep models sequentially on a number of
disjoint tasks without forgetting previously acquired knowledge. However, these
strategies are still unsatisfactory if the tasks are not disjoint but
constitute a single incremental task (e.g., class-incremental learning). In
this paper we point out the differences between multi-task and
single-incremental-task scenarios and show that well-known approaches such as
LWF, EWC and SI are not ideal for incremental task scenarios. A new approach,
denoted as AR1, combining architectural and regularization strategies is then
specifically proposed. AR1 overhead (in term of memory and computation) is very
small thus making it suitable for online learning. When tested on CORe50 and
iCIFAR-100, AR1 outperformed existing regularization strategies by a good
margin.

本文讨论了单增量任务情况下深度学习的策略和现有方法的不足，提出了一种名为 AR1 的新方法，结合了架构和规则化策略，具有较小的内存和计算开销，适合在线学习。在 CORe50 和 iCIFAR-100 上的实验证明，AR1 比现有的规则化策略表现更好。