We introduce a lifelong language learning setup where a model needs to learn
from a stream of text examples without any dataset identifier. We propose an
episodic memory model that performs sparse experience replay and local
adaptation to mitigate catastrophic forgetting in this setup. Experiments on
text classification and question answering demonstrate the complementary
benefits of sparse experience replay and local adaptation to allow the model to
continuously learn from new datasets. We also show that the space complexity of
the episodic memory module can be reduced significantly (~50-90%) by randomly
choosing which examples to store in memory with a minimal decrease in
performance. We consider an episodic memory component as a crucial building
block of general linguistic intelligence and see our model as a first step in
that direction.

提出一种生命化语言学习的模型，其中采用了 “稀疏经验回放” 和 “本地适应” 以减轻灾难性遗忘，并将这种情况应用于文本分类和问题回答中，证明了这种模型的优越性，同时表明通过随机选择存储在内存中的样本可以大大减少经验记忆模块的空间复杂度，并认为经验记忆组件是构建通用语言智能的重要组成部分。