Catastrophic forgetting(CF) is a significant challenge in continual learning
(CL). In regularization-based approaches to mitigate CF, modifications to
important training parameters are penalized in subsequent tasks using an
appropriate loss function. We propose the RTRA, a modification to the widely
used Elastic Weight Consolidation (EWC) regularization scheme, using the
Natural Gradient for loss function optimization. Our approach improves the
training of regularization-based methods without sacrificing test-data
performance. We compare the proposed RTRA approach against EWC using the
iFood251 dataset. We show that RTRA has a clear edge over the state-of-the-art
approaches.

在连续学习中，灾难性遗忘是一个重大挑战。我们提出了 RTRA 方法，它是 Elastic Weight Consolidation（EWC）正则化方案的修改版本，使用自然梯度进行损失函数优化。我们通过与 iFood251 数据集比较，证明 RTRA 方法在性能上明显优于最先进的方法。

RTRA：持续学习中基于正则化方法的快速训练

RTRA: Rapid Training of Regularization-based Approaches in Continual  Learning

This primer is an attempt to provide a detailed summary of the different
facets of lifelong learning. We start with Chapter 2 which provides a
high-level overview of lifelong learning systems. In this chapter, we discuss
prominent scenarios in lifelong learning (Section 2.4), provide 8 Introduction
a high-level organization of different lifelong learning approaches (Section
2.5), enumerate the desiderata for an ideal lifelong learning system (Section
2.6), discuss how lifelong learning is related to other learning paradigms
(Section 2.7), describe common metrics used to evaluate lifelong learning
systems (Section 2.8). This chapter is more useful for readers who are new to
lifelong learning and want to get introduced to the field without focusing on
specific approaches or benchmarks. The remaining chapters focus on specific
aspects (either learning algorithms or benchmarks) and are more useful for
readers who are looking for specific approaches or benchmarks. Chapter 3
focuses on regularization-based approaches that do not assume access to any
data from previous tasks. Chapter 4 discusses memory-based approaches that
typically use a replay buffer or an episodic memory to save subset of data
across different tasks. Chapter 5 focuses on different architecture families
(and their instantiations) that have been proposed for training lifelong
learning systems. Following these different classes of learning algorithms, we
discuss the commonly used evaluation benchmarks and metrics for lifelong
learning (Chapter 6) and wrap up with a discussion of future challenges and
important research directions in Chapter 7.

这篇论文详细总结了终身学习的不同方面，包括早期的概述、不同的学习算法、评估标准以及未来的研究方向等。

终身监督学习简介

An Introduction to Lifelong Supervised Learning

Standard artificial neural networks suffer from the well-known issue of
catastrophic forgetting, making continual or lifelong learning difficult for
machine learning. In recent years, numerous methods have been proposed for
continual learning, but due to differences in evaluation protocols it is
difficult to directly compare their performance. To enable more structured
comparisons, we describe three continual learning scenarios based on whether at
test time task identity is provided and--in case it is not--whether it must be
inferred. Any sequence of well-defined tasks can be performed according to each
scenario. Using the split and permuted MNIST task protocols, for each scenario
we carry out an extensive comparison of recently proposed continual learning
methods. We demonstrate substantial differences between the three scenarios in
terms of difficulty and in terms of how efficient different methods are. In
particular, when task identity must be inferred (i.e., class incremental
learning), we find that regularization-based approaches (e.g., elastic weight
consolidation) fail and that replaying representations of previous experiences
seems required for solving this scenario.

本文提出三种不同的持续学习场景，探讨不同的评估方法以及其性能差异；通过研究表明，针对识别任务需要推断类别（即类别增量学习）的场景中，基于正则化的方法失败，需要回放以前经验的表示来解决此问题。