Continual learning (CL) learns a sequence of tasks incrementally. There are two popular CL settings, class incremental learning (CIL) and task incremental learning (TIL). A major challenge of CL is catastrophic forgetting (CF). While a number of techniques are already available to effectively overcome CF for TIL, CIL remains to be highly challenging. So far, little theoretical study has been done to provide a principled guidance on how to solve the CIL problem. This paper performs such a study. It first shows that probabilistically, the CIL problem can be decomposed into two sub-problems: Within-task Prediction (WP) and Task-id Prediction (TP). It further proves that TP is correlated with out-of-distribution (OOD) detection, which connects CIL and OOD detection. The key conclusion of this study is that regardless of whether WP and TP or OOD detection are defined explicitly or implicitly by a CIL algorithm, good WP and good TP or OOD detection are necessary and sufficient for good CIL performances. Additionally, TIL is simply WP. Based on the theoretical result, new CIL methods are also designed, which outperform strong baselines in both CIL and TIL settings by a large margin.

研究表明，类增量学习(CIL)的问题可以分解为两个子问题：任务内预测(WP)和任务ID预测(TP)，其中TP与分布外检测(OOD)相关联。该研究的关键结论是，无论是否通过CIL算法显式或隐式定义WP和TP或OOD检测，优秀的WP和TP或OOD检测都对于良好的CIL表现是必要且充分的，其中TIL就是WP。基于理论结果，还设计了新的CIL方法，其在CIL和TIL两个设置中性能均优于强基线。

解决不间断学习问题的理论研究