Catastrophic forgetting remains a challenge for neural networks, especially
in lifelong learning scenarios. In this study, we introduce MEtaplasticity from
Synaptic Uncertainty (MESU), inspired by metaplasticity and Bayesian inference
principles. MESU harnesses synaptic uncertainty to retain information over
time, with its update rule closely approximating the diagonal Newton's method
for synaptic updates. Through continual learning experiments on permuted MNIST
tasks, we demonstrate MESU's remarkable capability to maintain learning
performance across 100 tasks without the need of explicit task boundaries.

通过利用突触的不确定性，基于元可塑性和贝叶斯推断原理，我们引入了 MESU（MEtaplasticity from Synaptic Uncertainty），其近似于对角牛顿方法的更新规则，保持信息长期记忆，并在连续学习实验中展示了处理 100 个任务时无需明确任务边界的出色性能。

贝叶斯元可塑性与突触不确定性

Bayesian Metaplasticity from Synaptic Uncertainty

Deep learning has made remarkable progress in various tasks, surpassing human
performance in some cases. However, one drawback of neural networks is
catastrophic forgetting, where a network trained on one task forgets the
solution when learning a new one. To address this issue, recent works have
proposed solutions based on Binarized Neural Networks (BNNs) incorporating
metaplasticity. In this work, we extend this solution to quantized neural
networks (QNNs) and present a memristor-based hardware solution for
implementing metaplasticity during both inference and training. We propose a
hardware architecture that integrates quantized weights in memristor devices
programmed in an analog multi-level fashion with a digital processing unit for
high-precision metaplastic storage. We validated our approach using a combined
software framework and memristor based crossbar array for in-memory computing
fabricated in 130 nm CMOS technology. Our experimental results show that a
two-layer perceptron achieves 97% and 86% accuracy on consecutive training of
MNIST and Fashion-MNIST, equal to software baseline. This result demonstrates
immunity to catastrophic forgetting and the resilience to analog device
imperfections of the proposed solution. Moreover, our architecture is
compatible with the memristor limited endurance and has a 15x reduction in
memory

本研究提出了一种基于 memristor 的硬件架构，实现了 QNNs 的元可塑性，并验证了其在 MNIST 和 Fashion-MNIST 上的有效性和鲁棒性。