The backpropagation of error algorithm used to train deep neural networks has
been fundamental to the successes of deep learning. However, it requires
sequential backward updates and non-local computations, which make it
challenging to parallelize at scale and is unlike how learning works in the
brain. Neuroscience-inspired learning algorithms, however, such as
\emph{predictive coding}, which utilize local learning, have the potential to
overcome these limitations and advance beyond current deep learning
technologies. While predictive coding originated in theoretical neuroscience as
a model of information processing in the cortex, recent work has developed the
idea into a general-purpose algorithm able to train neural networks using only
local computations. In this survey, we review works that have contributed to
this perspective and demonstrate the close theoretical connections between
predictive coding and backpropagation, as well as works that highlight the
multiple advantages of using predictive coding models over
backpropagation-trained neural networks. Specifically, we show the
substantially greater flexibility of predictive coding networks against
equivalent deep neural networks, which can function as classifiers, generators,
and associative memories simultaneously, and can be defined on arbitrary graph
topologies. Finally, we review direct benchmarks of predictive coding networks
on machine learning classification tasks, as well as its close connections to
control theory and applications in robotics.

回传误差算法已经成为深度学习成功的基础之一，而预测编码则利用局部学习，具有潜力克服其限制并在当前深度学习技术之上发展。本文综述了有关这个问题的各种作品及其理论联系，并展示了预测编码网络相对于深度神经网络具有更高的灵活性及其在机器学习分类任务上的潜在优势以及与控制理论的紧密联系和在机器人领域的应用。

预测编码：走向超越反向传播的深度学习未来？

Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation?

In this paper, we propose a fully distributed algorithm for frequency offsets
estimation in decentralized systems. With the proposed algorithm, each node
estimates its frequency offsets by local computations and limited exchange of
information with its direct neighbors. Such algorithm does not require any
centralized information processing or knowledge of global network topology. It
is shown analytically that the proposed algorithm always converges to the
optimal estimates regardless of network topology. Simulation results
demonstrate the fast convergence of the algorithm and show that estimation
mean-squared-error at each node touches the centralized Cram\'{e}r-Rao bound
within a few {iterations of message exchange}. Therefore, the proposed method
has low overhead and is scalable with network size.

本文提出了一种分布式算法，用于分散系统中频率偏移的估计，该算法通过本地计算和与直接相邻的节点进行有限的信息交换来估计每个节点的频率偏移，无需任何集中式信息处理或全球网络拓扑知识，通过模拟结果，表明这种算法具有快速的收敛速度和低延迟，是一种具有可扩展性的方法，可以达到最佳估计。