Continual Learning (CL) focuses on learning from dynamic and changing data
distributions while retaining previously acquired knowledge. Various methods
have been developed to address the challenge of catastrophic forgetting,
including regularization-based, Bayesian-based, and memory-replay-based
techniques. However, these methods lack a unified framework and common
terminology for describing their approaches. This research aims to bridge this
gap by introducing a comprehensive and overarching framework that encompasses
and reconciles these existing methodologies. Notably, this new framework is
capable of encompassing established CL approaches as special instances within a
unified and general optimization objective. An intriguing finding is that
despite their diverse origins, these methods share common mathematical
structures. This observation highlights the compatibility of these seemingly
distinct techniques, revealing their interconnectedness through a shared
underlying optimization objective. Moreover, the proposed general framework
introduces an innovative concept called refresh learning, specifically designed
to enhance the CL performance. This novel approach draws inspiration from
neuroscience, where the human brain often sheds outdated information to improve
the retention of crucial knowledge and facilitate the acquisition of new
information. In essence, refresh learning operates by initially unlearning
current data and subsequently relearning it. It serves as a versatile plug-in
that seamlessly integrates with existing CL methods, offering an adaptable and
effective enhancement to the learning process. Extensive experiments on CL
benchmarks and theoretical analysis demonstrate the effectiveness of the
proposed refresh learning. Code is available at
https://github.com/joey-wang123/CL-refresh-learning.

该研究提出了一个统一框架来解决连续学习中的灾难性遗忘问题，并引入了一种创新的概念 —— 刷新学习，通过重新学习当前数据来提高学习性能，在连续学习基准测试和理论分析中取得了有效的结果。

连续学习的统一通用框架

A Unified and General Framework for Continual Learning

Continual learning (CL) is one of the most promising trends in recent machine
learning research. Its goal is to go beyond classical assumptions in machine
learning and develop models and learning strategies that present high
robustness in dynamic environments. The landscape of CL research is fragmented
into several learning evaluation protocols, comprising different learning
tasks, datasets, and evaluation metrics. Additionally, the benchmarks adopted
so far are still distant from the complexity of real-world scenarios, and are
usually tailored to highlight capabilities specific to certain strategies. In
such a landscape, it is hard to objectively assess strategies. In this work, we
fill this gap for CL on image data by introducing two novel CL benchmarks that
involve multiple heterogeneous tasks from six image datasets, with varying
levels of complexity and quality. Our aim is to fairly evaluate current
state-of-the-art CL strategies on a common ground that is closer to complex
real-world scenarios. We additionally structure our benchmarks so that tasks
are presented in increasing and decreasing order of complexity -- according to
a curriculum -- in order to evaluate if current CL models are able to exploit
structure across tasks. We devote particular emphasis to providing the CL
community with a rigorous and reproducible evaluation protocol for measuring
the ability of a model to generalize and not to forget while learning.
Furthermore, we provide an extensive experimental evaluation showing that
popular CL strategies, when challenged with our benchmarks, yield sub-par
performance, high levels of forgetting, and present a limited ability to
effectively leverage curriculum task ordering. We believe that these results
highlight the need for rigorous comparisons in future CL works as well as pave
the way to design new CL strategies that are able to deal with more complex
scenarios.

本研究针对机器学习中的不断学习提出了两个新的基准，该基准涉及来自六个图像数据集的多个异构任务，其目的是为了更好地评估当前最先进的 CL 策略，并显示出当前 CL 模型在真实世界场景中表现较差的能力，高水平遗忘并限制了课程任务顺序。