Emerging from the monolithic pairwise attention mechanism in conventional
Transformer models, there is a growing interest in leveraging sparse
interactions that align more closely with biological principles. Approaches
including the Set Transformer and the Perceiver employ cross-attention
consolidated with a latent space that forms an attention bottleneck with
limited capacity. Building upon recent neuroscience studies of Global Workspace
Theory and associative memory, we propose the Associative Transformer (AiT).
AiT induces low-rank explicit memory that serves as both priors to guide
bottleneck attention in the shared workspace and attractors within associative
memory of a Hopfield network. Through joint end-to-end training, these priors
naturally develop module specialization, each contributing a distinct inductive
bias to form attention bottlenecks. A bottleneck can foster competition among
inputs for writing information into the memory. We show that AiT is a sparse
representation learner, learning distinct priors through the bottlenecks that
are complexity-invariant to input quantities and dimensions. AiT demonstrates
its superiority over methods such as the Set Transformer, Vision Transformer,
and Coordination in various vision tasks.

建立在生物学原理的基础上，我们提出了关联变压器（AiT），它通过引入显式记忆形成注意力瓶颈，并在共享工作空间和关联记忆中形成吸引子，从而在不同视觉任务中展示出了比 Set Transformer、Vision Transformer 和 Coordination 等方法更好的性能。