The Backpropagation algorithm, widely used to train neural networks, has often been criticised for its lack of biological realism. In an attempt to find a more biologically plausible alternative, and avoid to back-propagate gradients in favour of using local learning rules, the recently introduced Forward-Forward algorithm replaces the traditional forward and backward passes of Backpropagation with two forward passes. In this work, we show that internal representations obtained with the Forward-Forward algorithm organize into robust, category-specific ensembles, composed by an extremely low number of active units (high sparsity). This is remarkably similar to what is observed in cortical representations during sensory processing. While not found in models trained with standard Backpropagation, sparsity emerges also in networks optimized by Backpropagation, on the same training objective of Forward-Forward. These results suggest that the learning procedure proposed by Forward-Forward may be superior to Backpropagation in modelling learning in the cortex, even when a backward pass is used.

本研究中，作者探讨了一种名为Forward-Forward的训练神经网络的方法，与传统的Backpropagation背向算法相比，它采用了本地学习规则。作者通过实验发现，使用Forward-Forward算法进行训练所得到的神经网络内部表示可以组织成强健的、类别特定的合奏，并且由极少量的活跃单元（高稀疏度）组成，这与感官处理期间皮层表示的观察结果非常相似。这表明在模拟皮层学习方面，Forward-Forward算法提出的学习过程比Backpropagation更优秀。

使用前向前向算法训练的网络中的紧急表示