We propose a general, yet simple patch that can be applied to existing
regularization-based continual learning methods called classifier-projection
regularization (CPR). Inspired by both recent results on neural networks with
wide local minima and information theory, CPR adds an additional regularization
term that maximizes the entropy of a classifier's output probability. We
demonstrate that this additional term can be interpreted as a projection of the
conditional probability given by a classifier's output to the uniform
distribution. By applying the Pythagorean theorem for KL divergence, we then
prove that this projection may (in theory) improve the performance of continual
learning methods. In our extensive experimental results, we apply CPR to
several state-of-the-art regularization-based continual learning methods and
benchmark performance on popular image recognition datasets. Our results
demonstrate that CPR indeed promotes a wide local minima and significantly
improves both accuracy and plasticity while simultaneously mitigating the
catastrophic forgetting of baseline continual learning methods. The codes and
scripts for this work are available at this https URL

该研究提出了分类器投影正则化（CPR）的概念，通过最大化分类器输出的概率熵作为一种正则化项来投影分类器输出的条件概率至均匀分布上，从而改善基于正则化的持续学习方法中的性能，减轻灾难性遗忘并提高准确度。