Online learning holds the promise of enabling efficient long-term credit
assignment in recurrent neural networks. However, current algorithms fall short
of offline backpropagation by either not being scalable or failing to learn
long-range dependencies. Here we present a high-performance online learning
algorithm that merely doubles the memory and computational requirements of a
single inference pass. We achieve this by leveraging independent recurrent
modules in multi-layer networks, an architectural motif that has recently been
shown to be particularly powerful. Experiments on synthetic memory problems and
on the challenging long-range arena benchmark suite reveal that our algorithm
performs competitively, establishing a new standard for what can be achieved
through online learning. This ability to learn long-range dependencies offers a
new perspective on learning in the brain and opens a promising avenue in
neuromorphic computing.

本研究通过利用多层网络中的独立递归模块，提出了一种可行的高性能在线学习算法，能够有效地解决递归神经网络中长期信用分配的问题，并通过在合成记忆问题和长距离竞技场基准测试中的表现明确了该算法的有效性，为大脑学习和神经计算提供了新的思路。