Brain decoding, which aims at reconstructing visual stimuli from brain
signals, primarily utilizing functional magnetic resonance imaging (fMRI), has
recently made positive progress. However, it is impeded by significant
challenges such as the difficulty of acquiring fMRI-image pairs and the
variability of individuals, etc. Most methods have to adopt the
per-subject-per-model paradigm, greatly limiting their applications. To
alleviate this problem, we introduce a new and meaningful task, few-shot brain
decoding, while it will face two inherent difficulties: 1) the scarcity of
fMRI-image pairs and the noisy signals can easily lead to overfitting; 2) the
inadequate guidance complicates the training of a robust encoder. Therefore, a
novel framework named MindShot, is proposed to achieve effective few-shot brain
decoding by leveraging cross-subject prior knowledge. Firstly, inspired by the
hemodynamic response function (HRF), the HRF adapter is applied to eliminate
unexplainable cognitive differences between subjects with small trainable
parameters. Secondly, a Fourier-based cross-subject supervision method is
presented to extract additional high-level and low-level biological guidance
information from signals of other subjects. Under the MindShot, new subjects
and pretrained individuals only need to view images of the same semantic class,
significantly expanding the model's applicability. Experimental results
demonstrate MindShot's ability of reconstructing semantically faithful images
in few-shot scenarios and outperforms methods based on the
per-subject-per-model paradigm. The promising results of the proposed method
not only validate the feasibility of few-shot brain decoding but also provide
the possibility for the learning of large models under the condition of
reducing data dependence.

提出一种名为 MindShot 的新框架，通过利用跨主体的先验知识实现有效的少样本脑解码，能够在少样本场景中重建语义准确的图像，并且在性能上优于基于每个主体每个模型范式的方法。