Building on the model proposed in Lillicrap et. al. we show that deep networks can be trained using biologically plausible Hebbian rules yielding similar performance to ordinary back-propagation. To overcome the unrealistic symmetry in connections between layers, implicit in back-propagation, the feedback weights are separate from the feedforward weights. But, in contrast to Lillicrap et. al., they are also updated with a local rule - a weight is updated solely based on the activity of the units it connects. With fixed feedback weights the performance degrades quickly as the depth of the network increases, when they are updated the performance is comparable to regular back-propagation. We also propose a cost function whose derivative can be represented as a local update rule on the last layer. Convolutional layers are updated with tied weights across space, which is not biologically plausible. We show that similar performance is achieved with sparse layers corresponding to the connectivity implied by the convolutional layers, but where weights are untied and updated separately. In the linear case we show theoretically that the convergence of the error to zero is accelerated by the update of the feedback weights.

本论文提出一种使用Hebbian更新训练深度网络的方法，反馈权重与前馈权重分离以克服反向传播中不真实的对称性，即使在权重不同的情况下，其性能也与常规反向传播相当，同时提出了一种可以表示为最后一层局部Hebbian更新的成本函数。

具有不对称连接和Hebbian更新 的深度学习

具有不对称连接和Hebbian更新的深度学习