Brain decoding, a pivotal field in neuroscience, aims to reconstruct stimuli
from acquired brain signals, primarily utilizing functional magnetic resonance
imaging (fMRI). Currently, brain decoding is confined to a
per-subject-per-model paradigm, limiting its applicability to the same
individual for whom the decoding model is trained. This constraint stems from
three key challenges: 1) the inherent variability in input dimensions across
subjects due to differences in brain size; 2) the unique intrinsic neural
patterns, influencing how different individuals perceive and process sensory
information; 3) limited data availability for new subjects in real-world
scenarios hampers the performance of decoding models. In this paper, we present
a novel approach, MindBridge, that achieves cross-subject brain decoding by
employing only one model. Our proposed framework establishes a generic paradigm
capable of addressing these challenges by introducing biological-inspired
aggregation function and novel cyclic fMRI reconstruction mechanism for
subject-invariant representation learning. Notably, by cycle reconstruction of
fMRI, MindBridge can enable novel fMRI synthesis, which also can serve as
pseudo data augmentation. Within the framework, we also devise a novel
reset-tuning method for adapting a pretrained model to a new subject.
Experimental results demonstrate MindBridge's ability to reconstruct images for
multiple subjects, which is competitive with dedicated subject-specific models.
Furthermore, with limited data for a new subject, we achieve a high level of
decoding accuracy, surpassing that of subject-specific models. This advancement
in cross-subject brain decoding suggests promising directions for wider
applications in neuroscience and indicates potential for more efficient
utilization of limited fMRI data in real-world scenarios. Project page:
this https URL

通过生物启发的聚合函数和新颖的循环 fMRI 重建机制，提出了一种新的跨被试脑解码方法 MindBridge，能够实现跨被试脑解码，并能通过循环重建的 fMRI 实现新颖的 fMRI 合成和伪数据增强。实验证明 MindBridge 能够竞争性地为多个被试重建图像，并且在有限的新被试数据中达到较高的解码准确性，超过专门针对被试的模型。这种跨被试脑解码的进展为神经科学领域提供了更广泛的应用方向，并表明在现实场景中更高效地利用有限的 fMRI 数据的潜力。