Catastrophic forgetting is a critical challenge in training deep neural
networks. Although continual learning has been investigated as a countermeasure
to the problem, it often suffers from the requirements of additional network
components and the limited scalability to a large number of tasks. We propose a
novel approach to continual learning by approximating a true loss function
using an asymmetric quadratic function with one of its sides overestimated. Our
algorithm is motivated by the empirical observation that the network parameter
updates affect the target loss functions asymmetrically. In the proposed
continual learning framework, we estimate an asymmetric loss function for the
tasks considered in the past through a proper overestimation of its unobserved
sides in training new tasks, while deriving the accurate model parameter for
the observable sides. In contrast to existing approaches, our method is free
from the side effects and achieves the state-of-the-art accuracy that is even
close to the upper-bound performance on several challenging benchmark datasets.

本文提出了一种利用非对称二次函数逼近真实损失函数的方法来解决深度神经网络中的灾难性遗忘问题和大规模任务可扩展性问题的连续学习框架，并通过多个挑战性基准数据集的实验结果表明该方法优于现有方法，且可达到接近上限性能。