Humans and most animals can learn new tasks without forgetting old ones.
However, training artificial neural networks (ANNs) on new tasks typically
cause it to forget previously learned tasks. This phenomenon is the result of
"catastrophic forgetting", in which training an ANN disrupts connection weights
that were important for solving previous tasks, degrading task performance.
Several recent studies have proposed methods to stabilize connection weights of
ANNs that are deemed most important for solving a task, which helps alleviate
catastrophic forgetting. Here, drawing inspiration from algorithms that are
believed to be implemented in vivo, we propose a complementary method: adding a
context-dependent gating signal, such that only sparse, mostly non-overlapping
patterns of units are active for any one task. This method is easy to
implement, requires little computational overhead, and allows ANNs to maintain
high performance across large numbers of sequentially presented tasks when
combined with weight stabilization. This work provides another example of how
neuroscience-inspired algorithms can benefit ANN design and capability.

通过添加上下文依赖的门控信号，使得只有稀疏且大多非重叠的单元模式在任何一个任务中处于活动状态，从而稳定 ANN 的连接权重，以减轻灾难性遗忘。该方法易于实施，计算开销小，并且在与权重稳定相结合情况下能够使 ANN 在大量顺序呈现的任务中保持高性能，这些工作提供了另一个神经科学启发的算法如何有益于 ANN 设计和能力的例子。