The reconstruction of human visual inputs from brain activity, particularly
through functional Magnetic Resonance Imaging (fMRI), holds promising avenues
for unraveling the mechanisms of the human visual system. Despite the
significant strides made by deep learning methods in improving the quality and
interpretability of visual reconstruction, there remains a substantial demand
for high-quality, long-duration, subject-specific 7-Tesla fMRI experiments. The
challenge arises in integrating diverse smaller 3-Tesla datasets or
accommodating new subjects with brief and low-quality fMRI scans. In response
to these constraints, we propose a novel framework that generates enhanced 3T
fMRI data through an unsupervised Generative Adversarial Network (GAN),
leveraging unpaired training across two distinct fMRI datasets in 7T and 3T,
respectively. This approach aims to overcome the limitations of the scarcity of
high-quality 7-Tesla data and the challenges associated with brief and
low-quality scans in 3-Tesla experiments. In this paper, we demonstrate the
reconstruction capabilities of the enhanced 3T fMRI data, highlighting its
proficiency in generating superior input visual images compared to
data-intensive methods trained and tested on a single subject.

通过无监督生成对抗网络 (GAN) 结合 7T 和 3T 两个不同的 fMRI 数据集，本文提出了一种新的框架来生成改进的 3T fMRI 数据，克服高质量 7-Tesla 数据的稀缺性和 3-Tesla 实验中简短和低质量扫描的挑战，证明了该方法在生成优质输入视觉图像方面相对于在单个主题上进行训练和测试的数据密集型方法的能力。

基于 3T fMRI 数据和无监督学习的视网膜视觉图像重建

Reconstructing Retinal Visual Images from 3T fMRI Data Enhanced by  Unsupervised Learning

In the past five years, the use of generative and foundational AI systems has
greatly improved the decoding of brain activity. Visual perception, in
particular, can now be decoded from functional Magnetic Resonance Imaging
(fMRI) with remarkable fidelity. This neuroimaging technique, however, suffers
from a limited temporal resolution ($\approx$0.5 Hz) and thus fundamentally
constrains its real-time usage. Here, we propose an alternative approach based
on magnetoencephalography (MEG), a neuroimaging device capable of measuring
brain activity with high temporal resolution ($\approx$5,000 Hz). For this, we
develop an MEG decoding model trained with both contrastive and regression
objectives and consisting of three modules: i) pretrained embeddings obtained
from the image, ii) an MEG module trained end-to-end and iii) a pretrained
image generator. Our results are threefold: Firstly, our MEG decoder shows a 7X
improvement of image-retrieval over classic linear decoders. Second, late brain
responses to images are best decoded with DINOv2, a recent foundational image
model. Third, image retrievals and generations both suggest that MEG signals
primarily contain high-level visual features, whereas the same approach applied
to 7T fMRI also recovers low-level features. Overall, these results provide an
important step towards the decoding - in real time - of the visual processes
continuously unfolding within the human brain.

通过基于高时间分辨率的 MEG 测量脑活动，利用生成型和基础型人工智能系统解码大脑活动，特别是通过功能性磁共振成像 (fMRI) 解码视觉感知，从而在实时应用方面取得了重大进展。

脑解码：朝着实时重建视觉感知迈进

Brain decoding: toward real-time reconstruction of visual perception

Frontotemporal Dementia (FTD) diagnosis has been successfully progress using
deep learning techniques. However, current FTD identification methods suffer
from two limitations. Firstly, they do not exploit the potential of multi-view
functional magnetic resonance imaging (fMRI) for classifying FTD. Secondly,
they do not consider the reliability of the multi-view FTD diagnosis. To
address these limitations, we propose a reliable multi-view impartial decision
network (MID-Net) for FTD diagnosis in fMRI. Our MID-Net provides confidence
for each view and generates a reliable prediction without any conflict. To
achieve this, we employ multiple expert models to extract evidence from the
abundant neural network information contained in fMRI images. We then introduce
the Dirichlet Distribution to characterize the expert class probability
distribution from an evidence level. Additionally, a novel Impartial Decision
Maker (IDer) is proposed to combine the different opinions inductively to
arrive at an unbiased prediction without additional computation cost. Overall,
our MID-Net dynamically integrates the decisions of different experts on FTD
disease, especially when dealing with multi-view high-conflict cases. Extensive
experiments on a high-quality FTD fMRI dataset demonstrate that our model
outperforms previous methods and provides high uncertainty for hard-to-classify
examples. We believe that our approach represents a significant step toward the
deployment of reliable FTD decision-making under multi-expert conditions. We
will release the codes for reproduction after acceptance.

我们提出了一种可靠的多视角公正决策网络 (MID-Net)，用于 fMRI 中的 FTD 诊断。我们的 MID-Net 从 fMRI 图像中提取信息，利用多个专家模型来进行证据提取，并引入狄利克雷分布来表征每个专家的类概率分布。我们还提出了一种新颖的公正决策制定者 (IDer)，以在感性地结合不同意见的同时，不增加计算成本地得出一个无偏预测。我们的实验结果表明，我们的模型优于之前的方法，并为难以分类的例子提供高不确定性。