In the past five years, the use of generative and foundational AI systems has
greatly improved the decoding of brain activity. Visual perception, in
particular, can now be decoded from functional Magnetic Resonance Imaging
(fMRI) with remarkable fidelity. This neuroimaging technique, however, suffers
from a limited temporal resolution ($\approx$0.5 Hz) and thus fundamentally
constrains its real-time usage. Here, we propose an alternative approach based
on magnetoencephalography (MEG), a neuroimaging device capable of measuring
brain activity with high temporal resolution ($\approx$5,000 Hz). For this, we
develop an MEG decoding model trained with both contrastive and regression
objectives and consisting of three modules: i) pretrained embeddings obtained
from the image, ii) an MEG module trained end-to-end and iii) a pretrained
image generator. Our results are threefold: Firstly, our MEG decoder shows a 7X
improvement of image-retrieval over classic linear decoders. Second, late brain
responses to images are best decoded with DINOv2, a recent foundational image
model. Third, image retrievals and generations both suggest that MEG signals
primarily contain high-level visual features, whereas the same approach applied
to 7T fMRI also recovers low-level features. Overall, these results provide an
important step towards the decoding - in real time - of the visual processes
continuously unfolding within the human brain.

通过基于高时间分辨率的 MEG 测量脑活动，利用生成型和基础型人工智能系统解码大脑活动，特别是通过功能性磁共振成像 (fMRI) 解码视觉感知，从而在实时应用方面取得了重大进展。