The backpropagation of error algorithm (backprop) has been instrumental in
the recent success of deep learning. However, a key question remains as to
whether backprop can be formulated in a manner suitable for implementation in
neural circuitry. The primary challenge is to ensure that any candidate
formulation uses only local information, rather than relying on global signals
as in standard backprop. Recently several algorithms for approximating backprop
using only local signals have been proposed. However, these algorithms
typically impose other requirements which challenge biological plausibility:
for example, requiring complex and precise connectivity schemes, or multiple
sequential backwards phases with information being stored across phases. Here,
we propose a novel algorithm, Activation Relaxation (AR), which is motivated by
constructing the backpropagation gradient as the equilibrium point of a
dynamical system. Our algorithm converges rapidly and robustly to the correct
backpropagation gradients, requires only a single type of computational unit,
utilises only a single parallel backwards relaxation phase, and can operate on
arbitrary computation graphs. We illustrate these properties by training deep
neural networks on visual classification tasks, and describe simplifications to
the algorithm which remove further obstacles to neurobiological implementation
(for example, the weight-transport problem, and the use of nonlinear
derivatives), while preserving performance.

提出了 Activation Relaxation (AR) 算法，利用构建反向传播梯度的动力系统平衡点实现只利用局部信号的反向传播，可以在任意计算图上收敛，能在视觉分类任务上训练深度神经网络并且进行神经生物学实现的简化。