Replay in neural networks involves training on sequential data with memorized
samples, which counteracts forgetting of previous behavior caused by
non-stationarity. We present a method where these auxiliary samples are
generated on the fly, given only the model that is being trained for the
assessed objective, without extraneous buffers or generator networks. Instead
the implicit memory of learned samples within the assessed model itself is
exploited. Furthermore, whereas existing work focuses on reinforcing the full
seen data distribution, we show that optimizing for not forgetting calls for
the generation of samples that are specialized to each real training batch,
which is more efficient and scalable. We consider high-level parallels with the
brain, notably the use of a single model for inference and recall, the
dependency of recalled samples on the current environment batch, top-down
modulation of activations and learning, abstract recall, and the dependency
between the degree to which a task is learned and the degree to which it is
recalled. These characteristics emerge naturally from the method without being
controlled for.

本篇文章提出了一种新的方法来缓解训练神经网络时非稳态带来的行为遗忘现象，利用模型内部的内隐记忆来进行实时样本生成，并侧重于优化实际的训练数据批次，从而高效和可扩展的进行训练。作者探讨了该方法与大脑模型的对应之处，并发现这种方法的高效特性是自然而然地产生的。