The Forward Forward algorithm, proposed by Geoffrey Hinton in November 2022,
is a novel method for training neural networks as an alternative to
backpropagation. In this project, we replicate Hinton's experiments on the
MNIST dataset, and subsequently extend the scope of the method with two
significant contributions. First, we establish a baseline performance for the
Forward Forward network on the IMDb movie reviews dataset. As far as we know,
our results on this sentiment analysis task marks the first instance of the
algorithm's extension beyond computer vision. Second, we introduce a novel
pyramidal optimization strategy for the loss threshold - a hyperparameter
specific to the Forward Forward method. Our pyramidal approach shows that a
good thresholding strategy causes a difference of upto 8% in test error. 1
Lastly, we perform visualizations of the trained parameters and derived several
significant insights, such as a notably larger (10-20x) mean and variance in
the weights acquired by the Forward Forward network.

该研究论文介绍了一种名为 “Forward Forward 算法” 的新颖神经网络训练方法，通过在 MNIST 和 IMDb 数据集上的实验拓展了该方法的应用范围，并提出了一种使用金字塔优化策略的阈值设定方法。实验结果表明，该方法在电影评论情感分析任务中取得了令人满意的性能，并通过可视化参数的分析揭示了一些重要洞察，例如，Forward Forward 网络所获得的权重的平均值和方差明显较大。