The Backpropagation algorithm, widely used to train neural networks, has
often been criticised for its lack of biological realism. In an attempt to find
a more biologically plausible alternative, and avoid to back-propagate
gradients in favour of using local learning rules, the recently introduced
Forward-Forward algorithm replaces the traditional forward and backward passes
of Backpropagation with two forward passes. In this work, we show that internal
representations obtained with the Forward-Forward algorithm organize into
robust, category-specific ensembles, composed by an extremely low number of
active units (high sparsity). This is remarkably similar to what is observed in
cortical representations during sensory processing. While not found in models
trained with standard Backpropagation, sparsity emerges also in networks
optimized by Backpropagation, on the same training objective of
Forward-Forward. These results suggest that the learning procedure proposed by
Forward-Forward may be superior to Backpropagation in modelling learning in the
cortex, even when a backward pass is used.

本研究中，作者探讨了一种名为 Forward-Forward 的训练神经网络的方法，与传统的 Backpropagation 背向算法相比，它采用了本地学习规则。作者通过实验发现，使用 Forward-Forward 算法进行训练所得到的神经网络内部表示可以组织成强健的、类别特定的合奏，并且由极少量的活跃单元（高稀疏度）组成，这与感官处理期间皮层表示的观察结果非常相似。这表明在模拟皮层学习方面，Forward-Forward 算法提出的学习过程比 Backpropagation 更优秀。