Catastrophic forgetting has been a major challenge in continual learning,
where the model needs to learn new tasks with limited or no access to data from
previously seen tasks. To tackle this challenge, methods based on knowledge
distillation in feature space have been proposed and shown to reduce
forgetting. However, most feature distillation methods directly constrain the
new features to match the old ones, overlooking the need for plasticity. To
achieve a better stability-plasticity trade-off, we propose Backward Feature
Projection (BFP), a method for continual learning that allows the new features
to change up to a learnable linear transformation of the old features. BFP
preserves the linear separability of the old classes while allowing the
emergence of new feature directions to accommodate new classes. BFP can be
integrated with existing experience replay methods and boost performance by a
significant margin. We also demonstrate that BFP helps learn a better
representation space, in which linear separability is well preserved during
continual learning and linear probing achieves high classification accuracy.

提出了 Backward Feature Projection (BFP) 方法用于解决持续学习中的灾难性遗忘问题，并通过学习线性变换来让新特征更具可塑性，从而保持了旧课程的线性可分性并允许新类的出现，实验证明 BFP 方法极大提高了模型性能和表达空间的线性可分性。