"Perception is reality". Human perception plays a vital role in forming
beliefs and understanding reality. Exploring how the human brain works in the
visual system facilitates bridging the gap between human visual perception and
computer vision models. However, neuroscientists study the brain via
Neuroimaging, i.e., Functional Magnetic Resonance Imaging (fMRI), to discover
the brain's functions. These approaches face interpretation challenges where
fMRI data can be complex and require expertise. Therefore, neuroscientists make
inferences about cognitive processes based on patterns of brain activities,
which can lead to potential misinterpretation or limited functional
understanding. In this work, we first present a simple yet effective
Brainformer approach, a novel Transformer-based framework, to analyze the
patterns of fMRI in the human perception system from the machine learning
perspective. Secondly, we introduce a novel mechanism incorporating fMRI, which
represents the human brain activities, as the supervision for the machine
vision model. This work also introduces a novel perspective on transferring
knowledge from human perception to neural networks. Through our experiments, we
demonstrated that by leveraging fMRI information, the machine vision model can
achieve potential results compared to the current State-of-the-art methods in
various image recognition tasks.

通过引入新颖的基于 Transformer 的 Brainformer 方法，分析人类感知系统中 fMRI 模式，并将其作为监督机制应用于机器视觉模型，通过实验证明，在各种图像识别任务中，利用 fMRI 信息可以实现与当前最先进方法相比的潜在结果。

脑形式化：将 MRI 脑功能建模到机器视觉中

Brainformer: Modeling MRI Brain Functions to Machine Vision

Transformers are central to recent successes in natural language processing
and computer vision. Transformers have a mostly uniform backbone where layers
alternate between feed-forward and self-attention in order to build a deep
network. Here we investigate this design choice and find that more complex
blocks that have different permutations of layer primitives can be more
efficient. Using this insight, we develop a complex block, named Brainformer,
that consists of a diverse sets of layers such as sparsely gated feed-forward
layers, dense feed-forward layers, attention layers, and various forms of layer
normalization and activation functions. Brainformer consistently outperforms
the state-of-the-art dense and sparse Transformers, in terms of both quality
and efficiency. A Brainformer model with 8 billion activated parameters per
token demonstrates 2x faster training convergence and 5x faster step time
compared to its GLaM counterpart. In downstream task evaluation, Brainformer
also demonstrates a 3% higher SuperGLUE score with fine-tuning compared to GLaM
with a similar number of activated parameters. Finally, Brainformer largely
outperforms a Primer dense model derived with NAS with similar computation per
token on fewshot evaluations.

本文介绍了一种名为 Brainformer 的复合深度神经网络模型，其中包括稀疏门控前馈层、紧密前馈层、自注意层以及不同形式的层规范化和激活函数。 Brainformer 模型表现优异且更为高效，比当前最先进的密集和稀疏的 Transformers 模型快 5 倍。在下游任务评估中，Brainformer 模型与 GLaM 模型相比，细调后 SuperGLUE 得分高出 3％。