Predictive coding (PC) is an influential theory in computational
neuroscience, which argues that the cortex forms unsupervised world models by
implementing a hierarchical process of prediction error minimization. PC
networks (PCNs) are trained in two phases. First, neural activities are updated
to optimize the network's response to external stimuli. Second, synaptic
weights are updated to consolidate this change in activity -- an algorithm
called \emph{prospective configuration}. While previous work has shown how in
various limits, PCNs can be found to approximate backpropagation (BP), recent
work has demonstrated that PCNs operating in this standard regime, which does
not approximate BP, nevertheless obtain competitive training and generalization
performance to BP-trained networks while outperforming them on tasks such as
online, few-shot, and continual learning, where brains are known to excel.
Despite this promising empirical performance, little is understood
theoretically about the properties and dynamics of PCNs in this regime. In this
paper, we provide a comprehensive theoretical analysis of the properties of
PCNs trained with prospective configuration. We first derive analytical results
concerning the inference equilibrium for PCNs and a previously unknown close
connection relationship to target propagation (TP). Secondly, we provide a
theoretical analysis of learning in PCNs as a variant of generalized
expectation-maximization and use that to prove the convergence of PCNs to
critical points of the BP loss function, thus showing that deep PCNs can, in
theory, achieve the same generalization performance as BP, while maintaining
their unique advantages.

本文主要针对预测编码理论进行理论分析，展示了其与目标传播算法的紧密联系以及在一定情况下可以取得与反向传播算法相同的泛化性能，同时保持其独特的优点。