Continual learning aims to provide intelligent agents that are capable of
learning continually a sequence of tasks, building on previously learned
knowledge. A key challenge in this learning paradigm is catastrophically
forgetting previously learned tasks when the agent faces a new one. Current
rehearsal-based methods show their success in mitigating the catastrophic
forgetting problem by replaying samples from previous tasks during learning a
new one. However, these methods are infeasible when the data of previous tasks
is not accessible. In this work, we propose a new pseudo-rehearsal-based
method, named learning Invariant Representation for Continual Learning (IRCL),
in which class-invariant representation is disentangled from a conditional
generative model and jointly used with class-specific representation to learn
the sequential tasks. Disentangling the shared invariant representation helps
to learn continually a sequence of tasks, while being more robust to forgetting
and having better knowledge transfer. We focus on class incremental learning
where there is no knowledge about task identity during inference. We
empirically evaluate our proposed method on two well-known benchmarks for
continual learning: split MNIST and split Fashion MNIST. The experimental
results show that our proposed method outperforms regularization-based methods
by a big margin and is better than the state-of-the-art pseudo-rehearsal-based
method. Finally, we analyze the role of the shared invariant representation in
mitigating the forgetting problem especially when the number of replayed
samples for each previous task is small.

本文提出了一种名为 IRCL 的伪排练式学习方法，其中通过将类不变表示与条件生成模型分离并与类特定表示共同使用，以学习顺序任务。该方法证明了在两个著名的连续学习基准上都比基于正则化和基于伪排练的方法更好，并成功地解决了灾难性遗忘问题。