In recent years, large Transformer-based Pre-trained Language Models (PLM)
have changed the Natural Language Processing (NLP) landscape, by pushing the
performance boundaries of the state-of-the-art on a wide variety of tasks.
However, this performance gain goes along with an increase in complexity, and
as a result, the size of such models (up to billions of parameters) represents
a constraint for their deployment on embedded devices or short-inference time
tasks. To cope with this situation, compressed models emerged (e.g.
DistilBERT), democratizing their usage in a growing number of applications that
impact our daily lives. A crucial issue is the fairness of the predictions made
by both PLMs and their distilled counterparts. In this paper, we propose an
empirical exploration of this problem by formalizing two questions: (1) Can we
identify the neural mechanism(s) responsible for gender bias in BERT (and by
extension DistilBERT)? (2) Does distillation tend to accentuate or mitigate
gender bias (e.g. is DistilBERT more prone to gender bias than its uncompressed
version, BERT)? Our findings are the following: (I) one cannot identify a
specific layer that produces bias; (II) every attention head uniformly encodes
bias; except in the context of underrepresented classes with a high imbalance
of the sensitive attribute; (III) this subset of heads is different as we
re-fine tune the network; (IV) bias is more homogeneously produced by the heads
in the distilled model.

基于大型 Transformer 预训练语言模型（PLM）的研究已经改变了自然语言处理（NLP）领域，但这种性能提升伴随着复杂性的增加，以及模型规模的限制。本文通过对两个问题的实证研究探索了 BERT 和 DistilBERT 中性别偏见的神经机制和蒸馏过程对性别偏见的影响，发现无法确定特定的层次产生偏见，除少数特定情况外，每一个注意力头都均匀地编码偏见，而蒸馏模型则更均衡地产生偏见。

BERT 和 DistilBERT 中导致性别偏见的结构研究

An investigation of structures responsible for gender bias in BERT and  DistilBERT

We report the presence of a simple neural mechanism that represents an
input-output function as a vector within autoregressive transformer language
models (LMs). Using causal mediation analysis on a diverse range of
in-context-learning (ICL) tasks, we find that a small number attention heads
transport a compact representation of the demonstrated task, which we call a
function vector (FV). FVs are robust to changes in context, i.e., they trigger
execution of the task on inputs such as zero-shot and natural text settings
that do not resemble the ICL contexts from which they are collected. We test
FVs across a range of tasks, models, and layers and find strong causal effects
across settings in middle layers. We investigate the internal structure of FVs
and find while that they often contain information that encodes the output
space of the function, this information alone is not sufficient to reconstruct
an FV. Finally, we test semantic vector composition in FVs, and find that to
some extent they can be summed to create vectors that trigger new complex
tasks. Taken together, our findings suggest that LLMs contain internal
abstractions of general-purpose functions that can be invoked in a variety of
contexts.

通过对多样的上下文学习任务的因果中介分析，我们发现一种简单的神经机制，在自回归 Transformer 语言模型中将输入输出函数表示为向量，这种向量被我们称为函数向量（Function Vector）。函数向量（FV）对上下文的变化具有鲁棒性，在与上下文不相似的输入（如零 - shot 和自然文本设置）上触发任务的执行。我们测试了函数向量在多种任务、模型和层次上的效果，并发现在中间层次的不同设置中具有强大的因果效应。我们研究了函数向量的内部结构，并发现虽然它们经常包含编码函数的输出空间的信息，但仅有这些信息并不足以重构一个函数向量。最后，我们测试了函数向量中语义向量的组合，并发现在一定程度上它们可以被相加以创建触发新复杂任务的向量。综上所述，我们的研究结果表明，语言模型中包含了内部的抽象的通用函数，可以在各种不同的上下文中被调用。

大型语言模型中的功能向量

Function Vectors in Large Language Models

The neural implementation of operant conditioning with few trials is unclear.
We propose a Hippocampus-Inspired Cognitive Architecture (HICA) as a neural
mechanism for operant conditioning. HICA explains a learning mechanism in which
agents can learn a new behavior policy in a few trials, as mammals do in
operant conditioning experiments. HICA is composed of two different types of
modules. One is a universal learning module type that represents a cortical
column in the neocortex gray matter. The working principle is modeled as
Modulated Heterarchical Prediction Memory (mHPM). In mHPM, each module learns
to predict a succeeding input vector given the sequence of the input vectors
from lower layers and the context vectors from higher layers. The prediction is
fed into the lower layers as a context signal (top-down feedback signaling),
and into the higher layers as an input signal (bottom-up feedforward
signaling). Rewards modulate the learning rate in those modules to memorize
meaningful sequences effectively. In mHPM, each module updates in a local and
distributed way compared to conventional end-to-end learning with
backpropagation of the single objective loss. This local structure enables the
heterarchical network of modules. The second type is an innate, special-purpose
module representing various organs of the brain's subcortical system. Modules
modeling organs such as the amygdala, hippocampus, and reward center are
pre-programmed to enable instinctive behaviors. The hippocampus plays the role
of the simulator. It is an autoregressive prediction model of the top-most
level signal with a loop structure of memory, while cortical columns are lower
layers that provide detailed information to the simulation. The simulation
becomes the basis for learning with few trials and the deliberate planning
required for operant conditioning.

本文提出了一种大脑启发的认知架构 —— 海马体启发认知架构 (HICA)，通过模拟海马等器官，使用 Modulated Heterarchical Prediction Memory (mHPM) 模块，实现了哺乳动物在少量试验中学习新行为政策的学习机制，为操作性条件反射的学习提供了一种新的神经机制。

基于海马体启发的操作性条件作用认知架构（HICA）

Hippocampus-Inspired Cognitive Architecture (HICA) for Operant Conditioning

The key to high-level cognition is believed to be the ability to
systematically manipulate and compose knowledge pieces. While token-like
structured knowledge representations are naturally provided in text, it is
elusive how to obtain them for unstructured modalities such as scene images. In
this paper, we propose a neural mechanism called Neural Systematic Binder or
SysBinder for constructing a novel structured representation called Block-Slot
Representation. In Block-Slot Representation, object-centric representations
known as slots are constructed by composing a set of independent factor
representations called blocks, to facilitate systematic generalization.
SysBinder obtains this structure in an unsupervised way by alternatingly
applying two different binding principles: spatial binding for spatial
modularity across the full scene and factor binding for factor modularity
within an object. SysBinder is a simple, deterministic, and general-purpose
layer that can be applied as a drop-in module in any arbitrary neural network
and on any modality. In experiments, we find that SysBinder provides
significantly better factor disentanglement within the slots than the
conventional object-centric methods, including, for the first time, in visually
complex scene images such as CLEVR-Tex. Furthermore, we demonstrate
factor-level systematicity in controlled scene generation by decoding unseen
factor combinations.

本文提出了一种名为 Neural Systematic Binder 或 SysBinder 的神经机制，用于构建一种新的结构化表示，即块槽表示，用于促进系统性推广。SysBinder 以无监督的方式获得这种结构，其可以应用于任何模态，并可以作为任意神经网络的调入模块。实验表明，SysBinder 在 CLEVR-Tex 等视觉复杂场景图像中提供了更好的因素解缠，同时在受控场景生成中演示了因素级系统性。