Intrinsic capability to continuously learn a changing data stream is a
desideratum of deep neural networks (DNNs). However, current DNNs suffer from
catastrophic forgetting, which hinders remembering past knowledge. To mitigate
this issue, existing Continual Learning (CL) approaches either retain exemplars
for replay, regularize learning, or allocate dedicated capacity for new tasks.
This paper investigates an unexplored CL direction for incremental learning
called Incremental Latent Rectification or ILR. In a nutshell, ILR learns to
propagate with correction (or rectify) the representation from the current
trained DNN backward to the representation space of the old task, where
performing predictive decisions is easier. This rectification process only
employs a chain of small representation mapping networks, called rectifier
units. Empirical experiments on several continual learning benchmarks,
including CIFAR10, CIFAR100, and Tiny ImageNet, demonstrate the effectiveness
and potential of this novel CL direction compared to existing representative CL
methods.

本文研究了一种名为增量潜在矫正（Incremental Latent Rectification，ILR）的连续学习方法，通过使用一系列小的表示映射网络（称为矫正器单元），使当前训练的深度神经网络的表示向过去任务的表示空间进行校正，从而有效缓解当前深度神经网络面临的灾难性遗忘问题。实证实验在多个连续学习基准数据集上，包括 CIFAR10、CIFAR100 和 Tiny ImageNet，证明了增量潜在矫正方法相较于已有的连续学习方法的有效性和潜力。