Human cognition excels at symbolic reasoning, deducing abstract rules from
limited samples. This has been explained using symbolic and connectionist
approaches, inspiring the development of a neuro-symbolic architecture that
combines both paradigms. In parallel, recent studies have proposed the use of a
"relational bottleneck" that separates object-level features from abstract
rules, allowing learning from limited amounts of data . While powerful, it is
vulnerable to the curse of compositionality meaning that object representations
with similar features tend to interfere with each other. In this paper, we
leverage hyperdimensional computing, which is inherently robust to such
interference to build a compositional architecture. We adapt the "relational
bottleneck" strategy to a high-dimensional space, incorporating explicit vector
binding operations between symbols and relational representations.
Additionally, we design a novel high-dimensional attention mechanism that
leverages this relational representation. Our system benefits from the low
overhead of operations in hyperdimensional space, making it significantly more
efficient than the state of the art when evaluated on a variety of test
datasets, while maintaining higher or equal accuracy.

人类认知在符号推理方面表现出色，从有限样本中推导出抽象规则。本文通过结合符号和连接主义方法，提出了一种神经符号化结构；同时，在学习有限数据量时，采用了将对象级特征与抽象规则分离的 “关系瓶颈” 策略。此外，本文借助于鲁棒性更强的高维计算方法来构建具有组合性的体系结构，并设计了一种新颖的高维注意机制。相较于现有技术，在各种测试数据集上，本系统具有更高的效率且保持相同或更高的准确性。

LARS-VSA：用于学习抽象规则的矢量符号架构

LARS-VSA: A Vector Symbolic Architecture For Learning with Abstract  Rules

The human cognitive system exhibits remarkable flexibility and generalization
capabilities, partly due to its ability to form low-dimensional, compositional
representations of the environment. In contrast, standard neural network
architectures often struggle with abstract reasoning tasks, overfitting, and
requiring extensive data for training. This paper investigates the impact of
the relational bottleneck -- a mechanism that focuses processing on relations
among inputs -- on the learning of factorized representations conducive to
compositional coding and the attendant flexibility of processing. We
demonstrate that such a bottleneck not only improves generalization and
learning efficiency, but also aligns network performance with human-like
behavioral biases. Networks trained with the relational bottleneck developed
orthogonal representations of feature dimensions latent in the dataset,
reflecting the factorized structure thought to underlie human cognitive
flexibility. Moreover, the relational network mimics human biases towards
regularity without pre-specified symbolic primitives, suggesting that the
bottleneck fosters the emergence of abstract representations that confer
flexibility akin to symbols.

研究关系瓶颈对因式表示学习和处理灵活性的影响，并证明该瓶颈不仅改善了泛化和学习效率，而且与人类行为偏差相一致，从而促进了抽象表示的产生，使处理灵活性类似于符号。

神经网络中的维度抽象的关系归纳偏好

A Relational Inductive Bias for Dimensional Abstraction in Neural  Networks

A central challenge for cognitive science is to explain how abstract concepts
are acquired from limited experience. This effort has often been framed in
terms of a dichotomy between empiricist and nativist approaches, most recently
embodied by debates concerning deep neural networks and symbolic cognitive
models. Here, we highlight a recently emerging line of work that suggests a
novel reconciliation of these approaches, by exploiting an inductive bias that
we term the relational bottleneck. We review a family of models that employ
this approach to induce abstractions in a data-efficient manner, emphasizing
their potential as candidate models for the acquisition of abstract concepts in
the human mind and brain.

认知科学的一个关键挑战是解释如何从有限经验中获得抽象概念，本文讨论了一种最近出现的观点，该观点提出了一个称为关系瓶颈的归纳偏置，通过利用这种方法在数据高效的方式下诱导抽象，强调其作为人类思维和大脑中抽象概念获取的候选模型的潜力。