We advance an information-theoretic model of human language processing in the
brain, in which incoming linguistic input is processed at two levels, in terms
of a heuristic interpretation and in terms of error correction. We propose that
these two kinds of information processing have distinct electroencephalographic
signatures, corresponding to the well-documented N400 and P600 components of
language-related event-related potentials (ERPs). Formally, we show that the
information content (surprisal) of a word in context can be decomposed into two
quantities: (A) heuristic surprise, which signals processing difficulty of word
given its inferred context, and corresponds with the N400 signal; and (B)
discrepancy signal, which reflects divergence between the true context and the
inferred context, and corresponds to the P600 signal. Both of these quantities
can be estimated using modern NLP techniques. We validate our theory by
successfully simulating ERP patterns elicited by a variety of linguistic
manipulations in previously-reported experimental data from Ryskin et al.
(2021). Our theory is in principle compatible with traditional cognitive
theories assuming a `good-enough' heuristic interpretation stage, but with
precise information-theoretic formulation.

该文提出了一个基于信息理论的人类语言处理模型，观点是内在的语言输入在启发式解释和错误校正的两个层次上处理，由此提出两种信息处理的脑电信号（N400 和 P600）和由现代自然语言处理技术获得的 surprisal 量。该理论通过成功模拟实验证据，可以为传统的认知理论提供精确的信息理论解释。

一种统一的信息论模型：人类语言处理的脑电图标记

A unified information-theoretic model of EEG signatures of human language processing

Throughout the cognitive-science literature, there is widespread agreement
that decision-making agents operating in the real world do so under limited
information-processing capabilities and without access to unbounded cognitive
or computational resources. Prior work has drawn inspiration from this fact and
leveraged an information-theoretic model of such behaviors or policies as
communication channels operating under a bounded rate constraint. Meanwhile, a
parallel line of work also capitalizes on the same principles from
rate-distortion theory to formalize capacity-limited decision making through
the notion of a learning target, which facilitates Bayesian regret bounds for
provably-efficient learning algorithms. In this paper, we aim to elucidate this
latter perspective by presenting a brief survey of these information-theoretic
models of capacity-limited decision making in biological and artificial agents.

本文介绍了一种信息理论模型，它使用一个有限传输速率的通信通道作为决策代理行为的描述，并在此基础上研究了一种基于学习目标的容量有限的决策制定方法， 为实现高效的学习算法提供了贝叶斯遗憾界的保证。

容量有限认知及强化学习的速率失真理论

On Rate-Distortion Theory in Capacity-Limited Cognition & Reinforcement Learning

The goal of meta-learning is to train a model on a variety of learning tasks,
such that it can adapt to new problems within only a few iterations. Here we
propose a principled information-theoretic model that optimally partitions the
underlying problem space such that specialized expert decision-makers solve the
resulting sub-problems. To drive this specialization we impose the same kind of
information processing constraints both on the partitioning and the expert
decision-makers. We argue that this specialization leads to efficient
adaptation to new tasks. To demonstrate the generality of our approach we
evaluate three meta-learning domains: image classification, regression, and
reinforcement learning.

该论文提出了一种信息论的元学习模型，通过优化划分学习任务，使得专业的决策者解决子问题，从而达到高效的适应新任务的效果，在图像分类、回归和强化学习三个元学习领域进行验证。