In this paper, we investigate continual learning performance metrics used in class incremental learning strategies for continual learning (CL) using some high performing methods. We investigate especially mean task accuracy. First, we show that it lacks of expressiveness through some simple experiments to capture performance. We show that monitoring average tasks performance is over optimistic and can lead to misleading conclusions for future real life industrial uses. Then, we propose first a simple metric, Minimal Incremental Class Accuracy (MICA) which gives a fair and more useful evaluation of different continual learning methods. Moreover, in order to provide a simple way to easily compare different methods performance in continual learning, we derive another single scalar metric that take into account the learning performance variation as well as our newly introduced metric.

本研究通过实验证明了现有的基于任务准确率的持续学习性能度量方法的局限性，并提出了一个公平且更有用的评估指标Minimal Incremental Class Accuracy（MICA），以及一个综合考虑学习性能变化的单一标量度量方法。

走向持续学习的工业应用：类增量学习的新度量提案