Deep representation learning methods struggle with continual learning, suffering from both catastrophic forgetting of useful units and loss of plasticity, often due to rigid and unuseful units. While many methods address these two issues separately, only a few currently deal with both simultaneously. In this paper, we introduce Utility-based Perturbed Gradient Descent (UPGD) as a novel approach for the continual learning of representations. UPGD combines gradient updates with perturbations, where it applies smaller modifications to more useful units, protecting them from forgetting, and larger modifications to less useful units, rejuvenating their plasticity. We use a challenging streaming learning setup where continual learning problems have hundreds of non-stationarities and unknown task boundaries. We show that many existing methods suffer from at least one of the issues, predominantly manifested by their decreasing accuracy over tasks. On the other hand, UPGD continues to improve performance and surpasses or is competitive with all methods in all problems. Finally, in extended reinforcement learning experiments with PPO, we show that while Adam exhibits a performance drop after initial learning, UPGD avoids it by addressing both continual learning issues.

深度表示学习方法在持续学习中面临着有用单元的灾难性遗忘和可塑性损失的困扰。本文介绍了基于效用的扰动梯度下降（UPGD）作为一种新的表示持续学习方法，通过梯度更新和扰动相结合的方式，在保护有用单元免受遗忘的同时，对不太有用的单元施加更大的修改来恢复其可塑性。在具有数百个非静态性和未知任务边界的连续学习问题中，我们证明了现有的许多方法都存在至少一种问题，主要表现为在任务上的准确性下降。相反，UPGD在所有问题上继续提高性能，并超越或与所有方法竞争。最后，通过使用PPO进行扩展的强化学习实验，我们证明了在初始学习后Adam表现出的性能下降，而UPGD通过解决连续学习的两个问题来避免这种下降。

应对连续学习中的可塑性丧失和灾难性遗忘