Image-to-fMRI encoding is important for both neuroscience research and
practical applications. However, such "Brain-Encoders" have been typically
trained per-subject and per fMRI-dataset, thus restricted to very limited
training data. In this paper we propose a Universal Brain-Encoder, which can be
trained jointly on data from many different subjects/datasets/machines. What
makes this possible is our new voxel-centric Encoder architecture, which learns
a unique "voxel-embedding" per brain-voxel. Our Encoder trains to predict the
response of each brain-voxel on every image, by directly computing the
cross-attention between the brain-voxel embedding and multi-level deep image
features. This voxel-centric architecture allows the functional role of each
brain-voxel to naturally emerge from the voxel-image cross-attention. We show
the power of this approach to (i) combine data from multiple different subjects
(a "Crowd of Brains") to improve each individual brain-encoding, (ii) quick &
effective Transfer-Learning across subjects, datasets, and machines (e.g.,
3-Tesla, 7-Tesla), with few training examples, and (iii) use the learned
voxel-embeddings as a powerful tool to explore brain functionality (e.g., what
is encoded where in the brain).

我们提出了一种通用脑编码器，能够联合训练来自多个不同主体 / 数据集 / 机器的数据，通过直接计算脑体素嵌入和多级深度图像特征之间的交叉注意力来预测每个脑体素对每个图像的响应，并且利用学习到的体素嵌入作为探索脑功能的强大工具。