The dilemma between plasticity and stability presents a significant challenge
in Incremental Learning (IL), especially in the exemplar-free scenario where
accessing old-task samples is strictly prohibited during the learning of a new
task. A straightforward solution to this issue is learning and storing an
independent model for each task, known as Single Task Learning (STL). Despite
the linear growth in model storage with the number of tasks in STL, we
empirically discover that averaging these model parameters can potentially
preserve knowledge across all tasks. Inspired by this observation, we propose a
Dual-Learner framework with Cumulative Parameter Averaging (DLCPA). DLCPA
employs a dual-learner design: a plastic learner focused on acquiring new-task
knowledge and a stable learner responsible for accumulating all learned
knowledge. The knowledge from the plastic learner is transferred to the stable
learner via cumulative parameter averaging. Additionally, several task-specific
classifiers work in cooperation with the stable learner to yield the final
prediction. Specifically, when learning a new task, these modules are updated
in a cyclic manner: i) the plastic learner is initially optimized using a
self-supervised loss besides the supervised loss to enhance the feature
extraction robustness; ii) the stable learner is then updated with respect to
the plastic learner in a cumulative parameter averaging manner to maintain its
task-wise generalization; iii) the task-specific classifier is accordingly
optimized to align with the stable learner. Experimental results on CIFAR-100
and Tiny-ImageNet show that DLCPA outperforms several state-of-the-art
exemplar-free baselines in both Task-IL and Class-IL settings.

在增量学习中，我们提出了一个具有累计参数平均的双学习者框架，适用于无例子的场景，并通过任务特定分类器提高预测性能。通过在 CIFAR-100 和 Tiny-ImageNet 上的实验证明，该方法在任务增量学习和类别增量学习中优于多种先进方法。