In theoretical neuroscience, recent work leverages deep learning tools to
explore how some network attributes critically influence its learning dynamics.
Notably, initial weight distributions with small (resp. large) variance may
yield a rich (resp. lazy) regime, where significant (resp. minor) changes to
network states and representation are observed over the course of learning.
However, in biology, neural circuit connectivity generally has a low-rank
structure and therefore differs markedly from the random initializations
generally used for these studies. As such, here we investigate how the
structure of the initial weights, in particular their effective rank,
influences the network learning regime. Through both empirical and theoretical
analyses, we discover that high-rank initializations typically yield smaller
network changes indicative of lazier learning, a finding we also confirm with
experimentally-driven initial connectivity in recurrent neural networks.
Conversely, low-rank initialization biases learning towards richer learning.
Importantly, however, as an exception to this rule, we find lazier learning can
still occur with a low-rank initialization that aligns with task and data
statistics. Our research highlights the pivotal role of initial weight
structures in shaping learning regimes, with implications for metabolic costs
of plasticity and risks of catastrophic forgetting.

通过理论和实证分析，我们研究发现，初始权重的结构对网络学习机制起着重要影响，具有高秩的初始权重表现出更懒惰的学习，低秩的初始权重倾向于更丰富的学习，同时我们还发现与任务和数据统计信息一致的低秩初始权重仍然可能导致懒散学习。这一研究突出了初始权重结构在塑性代谢成本和灾难性遗忘风险方面的关键作用。

神经电路中连通性结构对丰富与懒散学习的塑造

How connectivity structure shapes rich and lazy learning in neural  circuits

Predictive coding offers a potentially unifying account of cortical function
-- postulating that the core function of the brain is to minimize prediction
errors with respect to a generative model of the world. The theory is closely
related to the Bayesian brain framework and, over the last two decades, has
gained substantial influence in the fields of theoretical and cognitive
neuroscience. A large body of research has arisen based on both empirically
testing improved and extended theoretical and mathematical models of predictive
coding, as well as in evaluating their potential biological plausibility for
implementation in the brain and the concrete neurophysiological and
psychological predictions made by the theory. Despite this enduring popularity,
however, no comprehensive review of predictive coding theory, and especially of
recent developments in this field, exists. Here, we provide a comprehensive
review both of the core mathematical structure and logic of predictive coding,
thus complementing recent tutorials in the literature. We also review a wide
range of classic and recent work within the framework, ranging from the
neurobiologically realistic microcircuits that could implement predictive
coding, to the close relationship between predictive coding and the widely-used
backpropagation of error algorithm, as well as surveying the close
relationships between predictive coding and modern machine learning techniques.

该论文提供了对预测编码理论的全面评估，包括其核心数学结构和逻辑以及在神经科学领域的实现和应用。该理论适用于皮层功能的统一解释，主张大脑的核心功能是通过一个世界的生成模型来减少与预测误差。