This paper proposes a simple but highly efficient expansion-based model for
continual learning. The recent feature transformation, masking and
factorization-based methods are efficient, but they grow the model only over
the global or shared parameter. Therefore, these approaches do not fully
utilize the previously learned information because the same task-specific
parameter forgets the earlier knowledge. Thus, these approaches show limited
transfer learning ability. Moreover, most of these models have constant
parameter growth for all tasks, irrespective of the task complexity. Our work
proposes a simple filter and channel expansion based method that grows the
model over the previous task parameters and not just over the global parameter.
Therefore, it fully utilizes all the previously learned information without
forgetting, which results in better knowledge transfer. The growth rate in our
proposed model is a function of task complexity; therefore for a simple task,
the model has a smaller parameter growth while for complex tasks, the model
requires more parameters to adapt to the current task. Recent expansion based
models show promising results for task incremental learning (TIL). However, for
class incremental learning (CIL), prediction of task id is a crucial challenge;
hence, their results degrade rapidly as the number of tasks increase. In this
work, we propose a robust task prediction method that leverages entropy
weighted data augmentations and the models gradient using pseudo labels. We
evaluate our model on various datasets and architectures in the TIL, CIL and
generative continual learning settings. The proposed approach shows
state-of-the-art results in all these settings. Our extensive ablation studies
show the efficacy of the proposed components.

我们提出了一个基于扩展的模型，用于持续学习，通过在先前任务的参数上增加模型来充分利用以前学到的信息，从而实现更好的知识传递。同时，我们还提出了一种强大的任务预测方法，通过使用熵加权的数据增强和伪标签来评估模型性能，并在各种数据集和架构设置中展示了最先进的结果。