Lack of performance when it comes to continual learning over non-stationary
distributions of data remains a major challenge in scaling neural network
learning to more human realistic settings. In this work we propose a new
conceptualization of the continual learning problem in terms of a temporally
symmetric trade-off between transfer and interference that can be optimized by
enforcing gradient alignment across examples. We then propose a new algorithm,
Meta-Experience Replay (MER), that directly exploits this view by combining
experience replay with optimization based meta-learning. This method learns
parameters that make interference based on future gradients less likely and
transfer based on future gradients more likely. We conduct experiments across
continual lifelong supervised learning benchmarks and non-stationary
reinforcement learning environments demonstrating that our approach
consistently outperforms recently proposed baselines for continual learning.
Our experiments show that the gap between the performance of MER and baseline
algorithms grows both as the environment gets more non-stationary and as the
fraction of the total experiences stored gets smaller.

通过 Meta-Experience Replay 算法，在经验回放的基础上进行基于元学习的优化，实现了对未来渐变更少的干扰和更多的知识迁移。在多个领域的实验中，该方法优于目前的基准算法。