Recent works have examined theoretical and empirical properties of wide
neural networks trained in the Neural Tangent Kernel (NTK) regime. Given that
biological neural networks are much wider than their artificial counterparts,
we consider NTK regime wide neural networks as a possible model of biological
neural networks. Leveraging NTK theory, we show theoretically that gradient
descent drives layerwise weight updates that are aligned with their input
activity correlations weighted by error, and demonstrate empirically that the
result also holds in finite-width wide networks. The alignment result allows us
to formulate a family of biologically-motivated, backpropagation-free learning
rules that are theoretically equivalent to backpropagation in infinite-width
networks. We test these learning rules on benchmark problems in feedforward and
recurrent neural networks and demonstrate, in wide networks, comparable
performance to backpropagation. The proposed rules are particularly effective
in low data regimes, which are common in biological learning settings.

本文通过考虑神经元的宽度，利用神经切向核学习理论研究了神经网络在生物学中的可能模型，并提出了一种基于输入活动相关性的反向传播算法，该方法在低数据环境中表现出与反向传播相当的性能。